Robot control method and robot control system

Abstract

The invention relates to a robot control system and a method of controlling a robot, wherein the robot control system (12) comprises: a human-machine interface (16; 30); a real robot control environment (26) comprising a real robot (38) and a real robot controller (36) controlling the real robot (38); and a cloud-hosted robot control environment (14) comprising a virtual robot controller (20), the virtual robot controller (20) being a replica of the real robot controller, wherein the human-machine interface (16; 30) is configured to transfer robot change instructions (RCI) from a user to the cloud-hosted robot control environment (14), the virtual robot controller (20) is configured to validate the robot change instructions (RCI), and the real robot controller (36) is configured to apply the validated robot change instructions when controlling the real robot (38).

Description

Technical Field

[0001] This disclosure relates to a robot control method and a robot control system. Background Technology

[0002] With the introduction of cloud computing, robot control software can be hosted in the cloud. This is discussed, for example, in US2019 / 0022856. Users can also modify the software relatively easily in the cloud, thus reducing the need for localized processing.

[0003] Digital twins for robot controllers are also known. In this case, the digital twin is a virtual robot controller, a replica of the real robot controller. By providing a digital twin that can be delivered in the cloud, simulations can be performed without requiring significant software and local processing resources. For example, Li et al. described placing digital twins in the cloud in their paper "Digital twin-based control approach for industrial cloud robotics" presented at the 14th International Conference on Manufacturing Science and Engineering (MSEC2019), ASME 2019, June 10-14, 2019.

[0004] However, while simplifying software development as described above is important, it is equally important not to introduce software changes hastily into real robot controllers. Real robots can exert considerable mechanical forces, so it is crucial that any changes result in the actions expected.

[0005] Therefore, it is necessary to enable changes to robot control software in a user-friendly manner while ensuring that operation is as expected. Summary of the Invention

[0006] Therefore, one object of the present invention is to allow software changes to robot controller software in a user-friendly manner while ensuring that operation is as expected.

[0007] According to the first aspect, a control method for a real robot is provided, the method comprising:

[0008] - The robot change instructions are transmitted from the user to the cloud-hosted robot control environment through the human-machine interface. The cloud-hosted robot control environment includes a virtual robot controller, which is a replica of the real robot controller in the real robot control environment.

[0009] - Verify robot change commands via virtual robot controller; and

[0010] - When controlling an associated real robot, apply validated robot change instructions through the real robot controller.

[0011] According to the second aspect, a robot control system is provided, comprising:

[0012] Human-machine interface;

[0013] A realistic robot control environment, including a real robot and a realistic robot controller that controls the real robot; and

[0014] A cloud-hosted robot control environment, including a virtual robot controller that is a replica of a real robot controller.

[0015] The human-machine interface is configured to transmit robot change instructions from the user to the cloud-hosted robot control environment. The virtual robot controller is configured to verify the robot change instructions, and the real robot controller is configured to apply the verified robot change instructions when controlling the real robot.

[0016] The human-machine interface (HMI) can be an interface between the user and the cloud-hosted robot control environment. It can also be provided within the cloud-hosted robot control environment. Alternatively, the HMI can be an interface between the user and the actual robot control environment, which can also be provided within the actual robot control system.

[0017] Human-machine interfaces can also be web-based. As an alternative, it can be provided by a robot application running on a user terminal, in which case it is not provided in a cloud-hosted robot control environment or a physical robot control environment.

[0018] When the virtual robot controller receives or acquires a robot change instruction, it can automatically perform verification. Verification may include investigating the altered robot operation caused by the robot change instruction. Verification may also include comparing the altered robot operation caused by the robot change instruction with quality criteria, and determining that the altered robot operation is satisfactory if the quality criteria are met. If the quality criteria are met, the virtual robot controller can thereby verify the robot change instruction. The quality criteria can also be safety criteria, in which case the safety of the robot operator is considered.

[0019] Investigations into altered robot operations may also include: the execution of operations commanded or affected by robot modification instructions, and an assessment of whether the operations are satisfactory. Commands to the operations may be direct instructions to the robot controller to perform operations (such as moving the robot), and instructions affecting the operations may be parameter or boundary settings that influence the operations. Additionally, verification may include investigating operational profitability. Therefore, verification can be a verification of whether the actual robot can and does indeed operate as planned. Verification may include, for example, investigating whether the robot arm remains within the robot cell and / or does not collide with any obstacles while moving within the robot cell. Verification may also include investigating whether input / output (I / O) signals are correctly set and whether the actual robot has entered a singularity.

[0020] More specifically, the comparison of the modified robot operation with quality criteria may include comparing virtual physical quantities of the robot arm with corresponding physical quantity thresholds, and verifying the robot change command while keeping the threshold on the same side as the threshold before the operation change caused by the robot change command, where the physical quantity can be position. However, other physical quantities such as velocity, acceleration, force, or momentum are also possible.

[0021] The robot's change instructions can be transmitted to the virtual robot controller in the cloud-hosted robot control environment through the human-machine interface.

[0022] Additionally, a cloud-hosted robot control environment may include a version control database, which may contain different versions of robot control software used to operate real robot control software and virtual robot control software.

[0023] When a version control database exists in a cloud-hosted robot control environment, robot change instructions can be transmitted to this database via a human-machine interface to be included in the versions of the robot controller software used by both the real and virtual robot controllers. Furthermore, this version can be a newer version that replaces the current version currently being run by the real robot controller. Additionally, validated robot change instructions can be applied by using a real robot controller with the newer version of the robot controller software.

[0024] According to the first aspect, the method may further include: after the virtual robot controller verifies the modified robot operation, receiving the verified robot modification instructions from the cloud-hosted robot control environment through the real robot controller in the real robot control environment for application when controlling the real robot.

[0025] According to the second aspect, the real robot controller can be configured to receive verified robot change instructions from the cloud-hosted robot control environment after the modified robot operation has been verified by the virtual robot controller, for application when controlling the real robot.

[0026] When the robot change instruction is transmitted to the real robot control environment through the user's human-machine interface, the method according to the first aspect may further include: sending the received robot change instruction to the real robot controller through the human-machine interface for application when controlling the real robot; and synchronizing the operation of the real robot controller with the operation of the virtual robot controller after the robot change instruction is verified.

[0027] In the same case in the robot control system according to the second aspect, the human-machine interface can be configured to send the received robot change instructions to the real robot controller for application when controlling the real robot, and the real robot controller can be configured to synchronize its operation with the operation of the virtual robot controller after verification of the robot change instructions.

[0028] The real robot control environment may include a communication entity that provides communication between the cloud-hosted robot control environment and the real robot control environment.

[0029] When robot change commands are transmitted to the real robot control environment via the user's human-machine interface, this communication entity can be used to transmit robot change commands to the cloud-hosted robot control environment. It can also be used to synchronize the operation of the real robot controller with the operation of the virtual robot controller.

[0030] When the robot changes its instructions and transmits them to the cloud-hosted robot control environment through the user's human-machine interface, the real robot controller can receive verified robot control instructions from the cloud-hosted robot control environment through a communication entity.

[0031] The real robot control environment may include a private data network to which the real robot controller is connected. If the human-machine interface is the interface to the real robot control environment, it can also be an interface for the user to the private data network.

[0032] The communication entity can be a dedicated communication interface for a real robot controller, configured to provide direct communication between the real robot controller and the cloud-hosted robot control environment. The dedicated communication interface can be a wireless interface, such as an interface for Wi-Fi or mobile communication. Alternatively, the dedicated communication interface can be a wired interface. The dedicated communication interface may be part of the real robot controller.

[0033] Alternatively, the communication entity can be a shared communication interface, such as a gateway, configured to provide communication between the cloud-hosted robot control environment and the real robot control environment. Furthermore, the gateway can connect to a private data network. The gateway can also exist within the real robot control environment along with one or more dedicated communication interfaces. The gateway can be an edge gateway or a gateway that communicates with the cloud-hosted robot control environment via Wi-Fi or mobile communication.

[0034] Various entities in a cloud-hosted robot control environment (such as virtual robot controllers, version-controlled databases, and / or human-machine interfaces) can be implemented using data center resources (such as processing resources and data storage resources).

[0035] A real robot controller can also be implemented using a processor and a data storage device with computer program instructions, which, when executed by the processor, enable the operation of the real robot controller.

[0036] Human-machine interfaces can also be implemented using a processor and a data storage device with computer program instructions, which are executed by the processor to implement the human-machine interface.

[0037] Generally, unless otherwise expressly defined herein, all terms used in the claims shall be interpreted according to their ordinary meaning in the art. Unless otherwise expressly stated, all references to “a / an / the said element, device, component, means, step, etc.” shall be interpreted as referring to at least one instance of an element, device, component, means, step, etc. Unless expressly stated otherwise, the steps of any method disclosed herein need not be performed in the exact order disclosed. Attached Figure Description

[0038] Now, aspects and embodiments are described by way of example with reference to the accompanying drawings, wherein

[0039] Figure 1 The diagram schematically illustrates a robot control system, which includes a cloud-hosted robot control environment with a first human-machine interface and a virtual robot controller, and a real robot control environment with a second human-machine interface and a real robot controller.

[0040] Figure 2 It is a schematic diagram illustrating some components of the second human-machine interface;

[0041] Figure 3 It is a schematic diagram illustrating some components of a real robot controller;

[0042] Figure 4 It is a flowchart that schematically illustrates the control method of a real robot;

[0043] Figure 5 This is a schematic diagram illustrating the verification of changes to a virtual robot controller before they are applied to a real robot.

[0044] Figure 6 This is a schematic diagram illustrating the storage of changes in a version control database before verification of those changes is performed.

[0045] Figure 7 This is a schematic diagram illustrating the push to the actual robot controller via an edge gateway; and

[0046] Figure 8 This diagram schematically illustrates the synchronization of changes to the real robot control environment and changes to the cloud-hosted robot control environment after the verification of changes to the virtual robot controller. Detailed Implementation

[0047] The various aspects of this disclosure are described more fully below with reference to the accompanying drawings, in which certain embodiments of the invention are illustrated.

[0048] However, these aspects can be embodied in many different forms and should not be construed as limiting; rather, these embodiments are provided by way of example, and therefore this disclosure will be thorough and complete, fully conveying the scope of all aspects of the invention to those skilled in the art. Throughout the description, the same numbers refer to the same elements.

[0049] Figure 1The diagram schematically illustrates a robot control system RCS12, which includes a cloud-hosted robot control environment CHRCE 14 and a real robot control environment RRCE 26. To enable user access to the cloud-hosted robot control environment 14, the robot control system 12 includes a first human-machine interface (HMI) 16, which is provided as an example within the cloud-hosted robot control environment 14. To allow user access to the real robot control environment 26, the robot control system 12 includes a second human-machine interface (HMI) 2 30, which is also placed within the real robot control environment 26 in this case. These HMIs 16 and 30 are provided as web-based interfaces in this example. Therefore, they can be reached using a web browser on a user terminal, for example, by using a URL (Uniform Resource Locator). This is exemplified by the first user terminal UT1 10 reaching the first HMI1 16 and the second user terminal UT2 24 reaching the second HMI2 30, wherein in these cases, HMIs 16 and 30 are therefore accessed via a web browser (not shown) in user terminals 10 and 24. Thus, user terminals 10 and 24 can be laptops or desktop computers, or even mobile terminals such as smartphones and phablets. It should also be recognized that in the robot control system 12, there may be only one HMI, either the first HMI 16 or the second HMI 30.

[0050] The real robot control environment 26 includes a real robot controller 36 (RRC) and a real robot 38 (RR), wherein the real robot controller 36 controls the real robot 38. The real robot control environment 26 also includes at least one entity or device for communicating with the cloud-hosted robot control environment 14. Figure 1In the example given, it includes two such communication entities: one is a dedicated communication interface (DCI) 34 for the real robot controller 36, and the other is a shared communication interface in the form of a gateway GW 32. The dedicated communication interface 34 provides its own link to the cloud-hosted robot control environment 14 for processing entities such as the real robot controller 36. It can be a wireless interface (such as a Wi-Fi interface, i.e., an interface operating according to the IEEE 802.11 standard) or a mobile communication interface (such as an interface to a fifth-generation mobile communication system (5G)). Such a wireless interface can be implemented via a radio circuit with antennas(multiple). Alternatively, the dedicated communication interface 34 can be a wired interface. On the other hand, the gateway 32 does provide a shared link from the real robot control environment 26 to the cloud-hosted robot control environment 14. The gateway 32 can be a gateway using wireless communication (such as Wi-Fi or 5G) for communicating with the cloud-hosted robot control environment. Alternatively, it can be a wired gateway, such as an edge gateway or edge controller.

[0051] Depending on the number of processing entities in the real robot control environment 26, the real robot control system 26 may also include a private data network, such as a local area network (LAN) or a wireless local area network (WLAN). Figure 1 In the example given, the second human-machine interface 30, the real robot controller 36, and the gateway 32, which acts as an edge gateway, are connected to the private data network PDN 28, while a dedicated communication interface 34, which serves as a wireless interface, is connected to the real robot controller. It should be understood that the dedicated communication interface 34 can actually be part of the real robot controller 36.

[0052] It should also be recognized that there may be several real robot controllers in the real robot control environment 26. In this case, each such robot controller may be connected to a private data network and / or equipped with a dedicated communication interface. If there are several real robot controllers, the real robots controlled by them may also be placed in different robot lines, and one or more real programmable logic controllers (PLCs) connected to the private data network 28 may additionally exist, each configured as a real robot controller to control the robot lines.

[0053] The second human-machine interface 30 can be implemented based on a processing entity, such as a processor that acts on computer instructions (e.g., computer instructions in program memory). Thus, it can be implemented as a separate computer connected to the private data network 28. Alternatively, it can be implemented as part of another processing entity connected to the private data network 28. For example, it can be implemented as part of a real robot controller 36 or gateway 32, where the processor of the processing entity acts on computer instructions that implement the second human-machine interface 30, and as discussed previously, the processing entity and the computer program code implementing the human-machine interface can be invoked by the previously mentioned URL.

[0054] Figure 1 The cloud-hosted robot control environment 14 in the example shown includes a virtual robot controller (VRC) 20, which is a copy of the real robot controller 36. Additionally, a virtual robot may also be present. The virtual robot can be implemented using dynamic and kinematic models of the real robot 38. These models can be advantageously combined with the virtual robot controller 20. Furthermore, an optional version control database (VCDB) 22 exists, which includes different versions of robot control software used by the real robot controller 36 to control the real robot 38 and copied by the virtual robot controller 20. If several real robot controllers exist in the real robot control environment 26, more virtual robot controllers may also exist in the cloud-hosted robot control environment 14, each corresponding to a real robot controller. If the real robot control environment 26 includes a real PLC, the cloud-hosted robot control environment 14 may also include a corresponding virtual PLC.

[0055] Here, it should be recognized that human-machine interfaces can be provided in applications rather than on a network, in which case they can be provided through robot applications running on user terminals 10 and 24.

[0056] In addition, Figure 1 The diagram shows a robot change instruction (RCI) sent from a first user terminal 10 to a first human-machine interface 16, and from there to a virtual robot controller 20 and a version control database 22. The RCI is also shown being sent from a second user terminal 24 to a second human-machine interface 30. The final verified robot change instruction (VRCI) is shown being sent from the virtual robot controller 20 to a gateway 32 and a dedicated communication interface 34, and from the version control database 22 to a gateway 320 and a dedicated communication interface 34. These RCIs and VRCIs will be described in more detail later.

[0057] The cloud-hosted robot control environment 14 can be implemented using resources in the data center provided by the cloud provider, which may include processing or computing resources and data storage resources.

[0058] Figure 2 This diagram schematically illustrates a simplified implementation of the second human-machine interface 30. A processor 40 and a data storage device 44 with computer program instructions 46, which, when executed by the processor 40, implement the second human-machine interface 30, are present. An input / output (I / O) interface 42 is also present for communication with user terminals, other entities on the private data network 28, and possibly also with the cloud-hosted robot control environment 14.

[0059] Figure 3 This diagram schematically illustrates a simplified implementation of a real robot controller 36. A processor 50 and a data storage device 54 containing computer program instructions 56, which, when executed by the processor 50, implement the operation of the real robot controller 36, are present. An input / output (I / O) interface 52 also communicates with a private data network 28.

[0060] Robot controllers have various HMI applications. Traditionally, these applications were developed as standalone applications and could only run on specific computer platforms using operating systems developed for fixed computers. Furthermore, such traditional interfaces typically required a physical network connection to the actual robot controller.

[0061] Such a traditional interface is also the interface for a virtual robot controller running on a desktop PC, which is a digital twin of the real robot controller.

[0062] Today, many handheld devices capable of displaying human-machine interfaces (HMIs) utilize wireless communication technologies. While connecting a robot controller to a wireless network to access the HMI is technically feasible, thin devices such as smartphones or tablets may not be able to run traditional HMI applications because they typically use different types of operating systems specific to the wireless device. The solution for this situation is either to create a web-based HMI directly on the actual robot controller or to create a dedicated application for each target HMI platform. However, connecting all HMI devices to the actual robot control environment is not necessarily a good idea due to cybersecurity risks. It is generally desirable to minimize the attack vectors of the actual robot control environment. Allowing any device with a generic fixed or wireless operating system to access the actual robot control environment used to display the HMI may be a less than ideal alternative.

[0063] As mentioned before, Figure 1A robot control system 12 is schematically shown, comprising a cloud-hosted robot control environment 14 and a physical robot control environment 26. In many cases of interest, all robot controllers in the physical robot control environment 26 are allowed to connect to the cloud-hosted robot control environment 14. This connection can be implemented using either a shared communication interface or a dedicated communication interface, where the dedicated interface provides a direct connection to the cloud point. The connection can be secure and encrypted, and network traffic can avoid interfering with other traffic on a private data network 28.

[0064] Because robot controllers may have cloud connectivity, a network interface can be created in the cloud, providing controller access to anyone with a network-enabled device (such as a mobile phone, tablet, augmented reality device, virtual reality device, or personal computer (PC)). Through this network access, the operator of the robot control system can be an online user, thus eliminating the need for an offline user who is directly connected to the robot control system without internet access.

[0065] Furthermore, any device can communicate with and control the web-hosted digital virtual robot controller, thus providing a similar human-machine interface experience across devices.

[0066] By providing a cloud-hosted robot control environment, multiple users can be allowed to access the robot control environment (potentially simultaneously) to collaborate on the same robot project.

[0067] As mentioned earlier, the virtual robot controller 20 can be an exact replica of the real robot controller 36. It can be used in a variety of applications for offline robot programming, simulation, education, and many other tasks.

[0068] The virtual robot controller 20 can be built on the same codebase as the real robot controller 36, thus ensuring the correctness of the results.

[0069] Just like a real robot controller, a virtual robot controller will run code written by the end user to perform its tasks. This code can be written in a fully functional proprietary language, such as the RAPID language developed by ABB.

[0070] Therefore, virtual robot controllers can run arbitrary code. Just like on a real robot controller, this code can access private data networks and any locally mounted disks that the user can access. Therefore, insecure code (such as malicious code or code that leads to unexpected or unsafe operation) can be placed in a virtual robot controller.

[0071] Therefore, it is necessary to allow users to easily change the code used in the robot control system while ensuring operational quality.

[0072] To address this issue, various aspects of the present invention propose the use of a cloud-hosted robot control environment, in which a user can run one or more virtual robot controllers and, after verification in the cloud-hosted robot control environment, implement changes only in a real robot control environment.

[0073] Now, for reference Figure 4 This describes the different operating modes of the robot control system that achieve the objectives mentioned above. Figure 4 A flowchart illustrating the method for controlling a real robot 38 is shown.

[0074] The real robot controller 36 can initially operate the real robot 38 using an older or current version of the robot control software. Then, the user may want to implement changes to the robot control software, which could be a single change to the robot control software or a change to a new or subsequent version. More specifically, this change could be a change to the robot configuration or the robot's operation.

[0075] Therefore, the method begins by transmitting a robot change instruction (RCI) from the user (S100) to the cloud-hosted robot control environment (14) via a human-machine interface. The robot change instruction (RCI) can be an instruction that causes a change in the operation of the real robot (38) relative to its current operation (such as a change in the movement of the real robot (38) relative to its current movement), where the current movement can be caused by the current version of the robot control software. The robot change instruction (RCI) can be a change to the robot program (operational data), such as a change to instructions for moving the robot. However, it can also be robot configuration, such as the configuration of I / O parameters, including firewall and user-managed networking, all of which are used for robot debugging.

[0076] The transmission or delivery of Robot Change Instruction (RCI) can be performed via either the first HMI 16 or the second HMI 30. When transmitted via the first HMI 16, it can receive the RCI from the first user terminal 10 and forward it to another entity in the cloud-hosted robot control environment 14, such as the virtual robot controller 20 or the version control database 22. When transmitted via the second HMI 30, it can receive the RCI from the second user terminal 24 and forward it to both the real robot controller 36 and the cloud-hosted robot control environment 14. Regarding forwarding to the cloud-hosted robot control environment 14, the second HMI 30 can more specifically forward the RCI to the version control database 22 and / or the virtual robot controller 20. Forwarding can additionally be performed via a communication entity in the real robot control environment 26, which can be a gateway 32 or a dedicated communication interface 34.

[0077] After the robot change instruction (RCI) has been delivered to the cloud-hosted robot control environment 14, the virtual robot controller 20 verifies the S110 robot change instruction (RCI). To do this, if the virtual robot controller 20 has not yet received the robot change instruction (RCI) from the human-machine interface 16 or 30, the virtual robot controller can obtain the robot change instruction (RCI) from the version control database 22.

[0078] When the virtual robot controller receives or acquires a Robot Change Instruction (RCI), it can automatically perform verification. Verification may include investigating the changes in robot operation caused by the robot change instruction. This can include running the operation commanded or affected by the robot change instruction and evaluating whether the operation is satisfactory, where the command for the operation can be an instruction that directly commands the robot controller to perform an operation (such as moving the robot), and the instruction affecting the operation can be a parameter or boundary setting that affects the operation. Verification may also include comparing the operational profitability. Verification may also verify that the real robot can and does indeed operate as planned. Verification may include, for example, investigating whether the robot arm remains within the robot cell or does not collide with any obstacles. It may also include investigating whether the input / output (I / O) signals are correctly set and whether the real robot enters a mechanical singularity. The changes in robot operation caused by the robot change instruction can also be presented to the user to allow the user to confirm the verification of the robot change instruction.

[0079] A virtual robot controller can compare modified robot actions with quality criteria, and determine that the modified robot actions are satisfactory if the quality criteria are met. Robot actions can be, for example, the movement of a robot arm along a robot path, and a quality criterion can be that the robot arm does not occupy prohibited positions along the path, such as positions corresponding to the arm colliding with other objects or the boundaries of units where the robot is placed. The comparison of modified robot actions with quality criteria can more specifically include: comparing virtual physical quantities of the robot arm with corresponding physical quantity thresholds; and verifying the robot change instruction while the threshold remains on the same side as the threshold before the action change caused by the robot change instruction. Thus, the threshold is not crossed, but the physical quantity remains above or below the threshold. The physical quantity can be position. However, other physical quantities are also possible, such as velocity, acceleration, force, or momentum. The quality criterion can also be a safety criterion, in which case the safety of the robot operator is considered.

[0080] After verification, the virtual robot controller 20 can make robot change commands that have been verified in the version control database 22.

[0081] After verification, when operating the relevant real robot 38, the real robot controller 36 also applies the verified robot change instruction VRCI from S120.

[0082] Once the virtual robot controller 20 verifies the Robot Change Instruction (RCI) received via the first human-machine interface 16, the verified RCI can be sent from the cloud-hosted robot control environment 14 to the real robot control environment 26 via gateway 32 or dedicated communication interface 34. The verified RCI can then be sent from the virtual robot controller 20 or from the version control database 22. The verified RCI sent from the version control database 22 can then be sent as part of a new or next version of the robot control software to be used by the real robot controller 36. Therefore, the real robot controller 36 can replace the current robot control software version with a new or next version, wherein all or at least a portion of the new or next version has been verified.

[0083] Conversely, if the second human-machine interface 30 receives a robot change instruction (RCI), the virtual robot controller 20 or the version control database 22 can send a message to the real robot controller 36 stating that the RCI, already present in the real robot controller, has been verified. The real robot controller 36 can then continue to apply the thus verified robot change instruction under the control of the real robot 38, which can be accomplished by synchronizing the operational changes of the real robot controller with those of the virtual robot controller 20. In all cases, the operational changes caused by the RCI are not used for the control of the real robot 38 until the virtual robot controller 20 has verified the RCI, which may include determining that the operation caused by the RCI meets quality criteria.

[0084] Therefore, it can be seen that users can easily modify robot control software while ensuring satisfactory operation in a real robot control environment, for example, operation that matches the user's intent.

[0085] Therefore, users can quickly make changes within the secure virtual robot control environment provided by the cloud-hosted robot control environment, and these changes can be pushed to the real robot control environment after verification. Additionally, virtual controllers from any platform can be used.

[0086] Now, several examples of how to perform verification will be given.

[0087] Figure 5 This demonstrates the verification of changes to the virtual robot controller 20 before applying robot change instructions received via the first HMI 16 to the real robot 38, and without using a version control database. Three examples of online users are shown: a maintenance administrator connecting to the first HMI 16 using a web browser, a debugging engineer connecting to the first HMI 16 using a web browser, and an operator connecting to the first HMI using a web browser, where the web browser used by the operator can be part of a teach pendant (TPU) application. In this case, the first HMI 16 can also be a front-end network HMI when using a web browser. It can be seen that the sequence of events is as follows:

[0088] 1. The user makes changes to the robot configuration / program in the first human-machine interface 16. 2. The changes are sent to the virtual robot controller 20 in the cloud-hosted robot control environment. 3. The changes are verified via the virtual robot controller 20. 4. The verified changes are sent from the virtual robot controller 20 to the real robot controller 36 via a dedicated communication interface 34 and applied to the real robot 38, where in this example, the dedicated communication interface is part of the real robot controller.

[0089] Figure 6 The verification of changes to the virtual robot controller 20 before applying robot change instructions received via the first human-machine interface 16 to the real robot 38 is shown, where the robot change instructions are part of a software version to be implemented in the real robot controller 36. Three identical example users are also shown. It can be seen that the sequence of events is as follows:

[0090] 1. The user makes changes to the robot configuration / program in the first human-machine interface 16. 2. The changes are sent to the version control database 22 to be included in the new or next version of the robot control software. 3. The changes are verified via the virtual robot controller 20. Here, all changes in the new version may be verified compared to the current or older version. 4. The verified changes are then sent from the virtual robot controller 20 to the real robot controller 36 in the real robot control environment via a dedicated communication interface 34, which may be part of the real robot controller 36. The real robot controller 36 then applies these changes to the real robot 38. The entire new version may be verified first and then applied. In other words, no changes are made in the real robot controller 36 until the entire new software version is verified.

[0091] Figure 7 This illustrates the verification of changes to the virtual robot controller before applying robot change instructions received via the first human-machine interface 16 to the real robot 38, where the robot change instructions are again part of a new or next software version to be implemented in the real robot controller 36, and gateway 32 is used for communication between the two robot control environments. The three exemplary users remain unchanged. It can be seen that the sequence of events is as follows:

[0092] 1. The user makes changes to the robot configuration / program in the first human-machine interface 16. 2. The changes are sent to the version control database 22 to be included in a new or next version of the robot control software. 3. The changes are verified via the virtual robot controller 20. 4. The verified changes are then sent from the virtual robot controller 20 to the real robot controller 36 via gateway 32. The real robot controller 36 then applies the changes to the real robot 38, where the verified robot change instructions can either be sent from the version control database 22 or the entire new version can be verified first and then applied.

[0093] Figure 8 The verification of changes to the virtual robot controller 20 before applying robot change instructions received via the second human-machine interface 30 to the real robot 38 is shown. The three users remain unchanged. The second human-machine interface 30 is, in this case, a local web user interface (UI). Furthermore, in this case, the second human-machine interface 30 can be part of the real robot controller 36, just as the dedicated communication interface 34 is. It can be seen that the sequence of events is as follows:

[0094] 1. The user makes changes to the robot configuration / program of the real robot controller 36 through the second human-machine interface 30. 2. The changes are sent to the version control database 22 (2a) or the virtual robot controller 20 (2b) via a dedicated communication interface 34 for the real robot controller 36. If the changes are sent to the version control database 22, the virtual robot controller can obtain them from the database 22. 3. The changes are then verified by the virtual robot controller 20. 4. After verification, the operation of the real robot controller 36 is synchronized with the operation of the virtual robot controller 20. Verification can also be notified to the real robot controller 36 via the virtual robot controller 20 or the version control database 22. It should be understood that changes and notifications can be sent via the gateway 32 instead of the dedicated communication interface 34.

[0095] In addition to the advantages already mentioned, the present invention has many other advantages.

[0096] The entire robot controller does not run in the cloud infrastructure; only the robot's configuration and programs are hosted in the cloud, and a web-based HMI can be provided to them. All robot motion control and sensor data collection, as well as all execution databases, are still hosted in the physical or virtual robot. The virtual robot can simulate what the physical robot will do, but the simulation results cannot be directly transmitted to the physical machine.

[0097] No process templates are required. You can start from an empty robot program or automatically generate programs based on computer-aided design (CAD) templates.

[0098] HMIs for robot controllers can be web-based or cloud-hosted. Because HMIs can be cloud-hosted, user experience and security improvements / features can be easily deployed without impacting the actual robot controller; that is, they only affect the cloud platform until validation.

[0099] The HMI of a robot controller can be consistent across devices, independent of shape factor, and also independent of operating system platform.

[0100] No physical cable connection is required to access features via the robot controller.

[0101] The virtual robot controller shares the same interface as the real robot controller, and because it is cloud-hosted, it can be accessed at any time for simulation and offloading computational tasks from the real robot controller. Therefore, digital twins can be simulated and analyzed. These simulations and analyses can also be performed on historical versions of the virtual robot controller using historical versions of the robot control software.

[0102] Existing applications can be modified to benefit from cloud-provided content, including digital twins in the cloud, which also allows for the provision of always-up-to-date virtual replicas of production equipment.

[0103] The virtual robot controller 20 can be sandboxed (isolated) from the real robot controller 36 and optional other virtual robot controllers in the cloud-hosted robot control environment 14 and their corresponding real robot controllers in the real robot control environment 26. This protects end users from data breaches. However, it also protects the real robot controller 36 from running potentially malicious code. Therefore, it is possible to securely host virtual robot controllers that still allow potentially malicious code to execute within them, as well as virtual devices that were not originally designed to be hosted in a shared cloud environment.

[0104] Isolation can be at the tenant level (a single organization that purchases the service) or at the individual user level.

[0105] Virtual robot controllers can be created as replicas of real robot controllers and continuously updated. Synchronization can be achieved using a dedicated communication interface or directly from the real robot controller to the cloud-hosted virtual robot controller via a gateway on a private data network, which in turn communicates with the real robot controller.

[0106] The virtual robot controller can be accessed from multiple platforms, such as existing simulation and robot control applications, mobile applications, or web applications.

[0107] The cloud-hosted robot control environment can be extended to support other virtual devices, such as virtual PLCs, where changes to the real PLC can be verified by the virtual PLC before being implemented by the real PLC.

[0108] Vertical replicas can be used as non-destructive environments to alter the real production environment formed by the actual robotic control environment, allowing users to test changes before pushing them to production. By integrating a version control system, changes can be tracked at any time, and rollbacks are permitted.

[0109] Virtual replicas can be easily cloned to run long-term experiments, machine learning, or other optimization algorithms within a dedicated virtual replica of a production environment formed by a real robotic control environment. Another benefit is the ability to run comparative studies between the current state of the production environment and previous versions, thereby measuring, for example, cycle time degradation.

[0110] The cloud can also host numerous analytics and optimization services, which can be easily integrated into the cloud and enrich the user experience. Updates to cloud content can be made without affecting the software on the actual robot controller, enabling "continuous deployment" of new user experiences and safety features, and also allowing for limited deployments (such as beta solutions for a select group of users and region-based updates).

[0111] Even if the first HMI is used, the second HMI can still be hosted in the actual robot control environment. In the event of troubleshooting or loss of cellular or wireless network connectivity, the second HMI can be provided within the actual robot controller.

[0112] Existing human-machine interface applications can be modified to utilize cloud-delivered content, including digital twins in the cloud. This could make these applications more powerful while remaining connected. Alternatively, users can be gradually moved towards cloud platforms, rather than undergoing a "big bang" replacement of human-machine interfaces.

[0113] The foregoing description has primarily referenced some embodiments and examples thereof to illustrate various aspects of this disclosure. However, as will be readily apparent to those skilled in the art, other embodiments besides those disclosed above are also possible within the scope of this invention, as defined in the appended claims.

Claims

1. A method for controlling a real robot, the method comprising: - Robot change instructions (RCI) are transmitted from the user (S100) to the cloud-hosted robot control environment (14) via a human-machine interface (16; 30), the cloud-hosted robot control environment (14) including a virtual robot controller (20), the virtual robot controller (20) being a copy of the real robot controller (36) in the real robot control environment (26); - Verify (S110) the robot change command via the virtual robot controller (20); and - When controlling the associated real robot (38), the verified robot change instruction (VRCI) is applied (S120) through the real robot controller (36).

2. The method according to claim 1, wherein the verification includes: The modified robot operation caused by the robot change instruction is compared with a quality criterion, and if the quality criterion is met, the modified robot operation is determined to be satisfactory, thereby obtaining the verified robot change instruction (VRCI).

3. The method according to claim 1 or 2, wherein the transmission of the robot change instruction (RCI) is performed to the virtual robot controller (20) of the cloud-hosted robot control environment (14) via the human-machine interface (16; 30) (S100).

4. The method according to claim 1 or 2, wherein the transmission of the robot change instruction (RCI) is performed (S100) to the version control database (22) in the cloud-hosted robot control environment (14) via the human-machine interface (16) so that the robot change instruction (RCI) is included in the version of the robot controller software for the real robot controller (36) and the virtual robot controller (20).

5. The method according to claim 2, further comprising: After the modified robot operation is verified by the virtual robot controller (20), the verified robot change instruction (VRCI) is received from the cloud-hosted robot control environment in the real robot control environment (26) by the real robot controller (36) for application when controlling the real robot (38).

6. The method of claim 5, wherein the real robot control environment (26) includes a private data network (28), the real robot controller (36) is connected to the private data network (28), and the receiving is performed via a gateway (32) of the real robot control environment (26) connected to the private data network (28).

7. The method according to claim 1 or 2, further comprising: The received Robot Change Instruction (RCI) is sent to the real robot controller (36) via the human-machine interface (16; 30) for application when controlling the real robot (38); and after the verification of the Robot Change Instruction, the operation of the real robot controller (36) is synchronized with the operation of the virtual robot controller (20).

8. A robot control system (12), comprising: Human-machine interface (16; 30); A real robot control environment (26) includes a real robot (38) and a real robot controller (36) that controls the real robot; and The cloud-hosted robot control environment (14) includes a virtual robot controller (20), which is a replica of the real robot controller (36). The human-machine interface (16; 30) is configured to transmit robot change instructions (RCI) from the user to the cloud-hosted robot control environment (14), the virtual robot controller (20) is configured to verify the robot change instructions (RCI), and the real robot controller (36) is configured to apply the verified robot change instructions (VRCI) when controlling the real robot (38).

9. The robot control system (12) according to claim 8, wherein the cloud-hosted robot control environment (14) includes a version control database (22), and the human-machine interface (16; 30) is configured to transmit the robot change instruction (RCI) to the version control database (22) such that the robot change instruction is included in a version of the robot controller software for the real robot controller (36) and the virtual robot controller (20), and instructs the virtual robot controller (20) to verify the robot change instruction (RCI) in the verification of the robot controller software version.

10. The robot control system (12) according to claim 8 or 9, wherein the human-machine interface (16; 30) is a network-based human-machine interface.

11. The robot control system (12) according to claim 8 or 9, wherein the human-machine interface is provided by a robot application running on a user terminal.

12. The robot control system (12) according to claim 8 or 9, wherein the human-machine interface (16) is an interface for a user to the cloud-hosted robot control environment (14).

13. The robot control system (12) according to claim 8 or 9, wherein the real robot control environment includes a private data network (28), the real robot controller (36) is connected to the private data network (28), and the human-machine interface (30) is an interface for the user to the private data network (28).

14. The robot control system (12) according to claim 8 or 9 further includes a dedicated communication interface (34) for the real robot controller (36) and is configured to provide communication between the real robot controller (36) and the cloud-hosted robot control environment (14).

15. The robot control system (12) according to claim 8 or 9 further includes a gateway (32) configured to provide communication between the cloud-hosted robot control environment (14) and the real robot control environment.

Images