Robot controller and client device for commanding an industrial robot

Abstract

A robot controller (120) for use with a robot manipulator (110) of an industrial robot (100), comprising: a machine interface (124) configured to apply motion references (151) to actuators (115) in the robot manipulator, each motion reference representing a setpoint value for synchronous execution by the actuators; an external programming interface (121) configured to receive a robot program (128) comprising motion instructions, which have a higher level of abstraction than the motion references; processing circuitry (125), which implements a path planner (126) configured to input a robot program and to generate a timed sequence of motion references which realizes the robot program; and an external instruction interface (122) configured to receive a stream (130) with one or more motion instructions (131) for execution as soon as practicable. Further disclosed is a client device (180) with an operator interface (181), an output buffer (182), and a robot interface (183) operable to transfer motion instructions from the output buffer to the external instruction interface (122) of the described robot controller (120).

Description

CLIENT DEVICE FOR COMMANDING AN INDUSTRIAL ROBOT TECHNICAL FIELD

[0001] The present disclosure relates to the field of robotics and in particular to external commanding of industrial robots. In particular, it describes a robot controller equipped with an external instruction interface for receiving from a client device a data stream with one or more motion instructions intended for execution as soon as practicable.BACKGROUND

[0002] Industrial robots typically receive commands from external devices in two different ways:i) a control-level signal protocol that controls the robot in terms of immediately executed motion references in joint space or cartesian space, andii) an interface through which an operator may upload robot programs, which thereby become available for execution by the robot controller at a point in time the operator decides.Some robot controllers manufactured by the applicant realize option i) by their Externally Guided Motion (EGM) interface, and option ii) is made possible by the Robot Web Service (RWS) interface.

[0003] EGM is described in Application manual. Externally Guided Motion. RobotWare 6.15.08, ABB Document ID 3HAC073319-001, revision H (2024-09-26). Section 1.3 in this manual cautions operators that the instruction stream received through EGM goes directly into the motor reference generation, i.e. the robot controller does not carry out any path planning. Since the path planning is bypassed by EGM in the robot controller, the robot path is exactly as the operator input defines it to be. It is therefore important to make sure that the stream of position references sent to the controller is as smooth as possible. The robot will react quickly to all position references sent to the controller, also faulty ones. All this means that the operator cannot order a movement to a pose target and expect a linear movement. Further, it is not possible either to order a movement with a specified speed or order a movement that is supposed to take a specified time. For ordering such movements, path planning is needed and the standard movement instructions in the RAPID™language should be used, i.e. MoveL, MoveJ, etc., which may include inherent safeguards, smoothing and similar logic. EGM is restricted to the set of RAPID™ instructions according to section 5.1 in the manual.

[0004] Through EGM, robot operators can send control-level signals to either the joints or the tool center point (TCP), it being understood that the operators normally use equipment that pre-calculates the robot motion in detail and stream real-time commands to the robot. There are imaginable use cases where the following characteristics of EGM may not be of benefit:- EGM must be initiated from the robot side. The industrial robot, and more particularly the robot controller, is always the master.- EGM does not control any other non-motion related functionality.- EGM requires the user to have knowledge of the dynamic properties of the robot for optimal motion performance.- EGM does not offer error recovery through the external interface.With RWS, on the other hand, the operator can interact with the robot controller in a way that is more complete, but not necessarily fast or responsive. RWS was not designed to be a real-time protocol. Although operators have access to all aspects of the robot parameters and programs, they cannot affect the robot program execution during runtime. Some characteristics of the RWS interface, which may be felt as limitations in some situations, are:

[0005] Robot programs uploaded via RWS must use a programming language compatible with the industrial robot (or ABB's PC Software Development Kit, PCSDK).- Once the robot controller begins the execution of an uploaded program, the external device (e.g., a PC) from which it was uploaded cannot control the program flow.- The typical communication latency of RWS is of the order of 100 milliseconds.

[0006] EP4474112A1 discloses a control system for industrial robots comprising a robot controller and an external computation module (with a user interface) configured to communicate with the robot controller. The system supports various network protocols for communication between the robot and external devices. Therobot controller includes a first memory, in which state information of the robot is stored. The computation module includes a second memory synchronized with the first memory and a processor for executing an application that performs a computation related to the control of the robot based on the content of the second memory. Further, a memory manager in the computation module is configured to queue a plurality of requests from one or more applications, sequentially dequeue the requests and write data corresponding to the dequeued requests in the second memory.

[0007] In the terminology of EP4474112A1, an operation program includes a plurality of time-series operation commands, where an operation command defines a target position and target speed, like the instructions MoveL, MoveJ in RAPID™ do. Further, an application (in the computation module) is a program constructed in advance so as to perform a computation related to the control of the robot, and it is by executing the application that the computation module generates control data to operate the robot. An operation program is started by an execution command. In other words, the computation module interacts with the robot controller in a way similar to option ii).

[0008] This review shows that robot commanding according to option i) may require extensive preprocessing, such as preprocessing based on detailed knowledge of the machinery. Option ii) for its part may lack the responsiveness that some use cases require. It would be desirable to enable a novel robot commanding modality which is, at a time, responsive and easy to use.SUMMARY

[0009] One objective of the present disclosure is to make available a technical solution for operator-initiated real-time commanding in terms of motion instructions. A further objective is to make at least some of the motion instructions which operators use when writing robot programs (or equivalents of these motion instructions) available for real-time execution. Such real-time execution shall be initiated from a client device which is external to the robot controller, and possibly located remotely. A further objective is to reduce the risk of machinery damage and excessive machinery wear in connection with real-time robot commanding. A further objective is to propose a robot controller which accepts motion instructions from aclient device in real time, and which is configured to process the motion instructions into a timed sequence of motion references before execution. A further objective is to propose a client device suitable for use with the robot controller. A further objective is to propose elements of a communication protocol that enables the robot controller and the client device to interact smoothly and safely in connection with real-time robot commanding.

[0010] At least some of these objectives are achieved by the invention as defined by the independent claims. The dependent claims are directed to currently preferred embodiments.[oon] In a first aspect of the present disclosure, there is provided a robot controller for use with a robot manipulator of an industrial robot. The robot controller comprises a machine interface configured to apply motion references to actuators in the robot manipulator, each motion reference representing a setpoint value for synchronous execution by the actuators; an external programming interface configured to receive a robot program comprising motion instructions, which have a higher level of abstraction than the motion references; and processing circuitry, which implements a path planner configured to input a robot program and to generate a timed sequence of motion references which realizes the robot program. According to the first aspect, the robot controller further has an external instruction interface configured to receive a stream with one or more motion instructions for execution as soon as practicable.

[0012] Thanks to the external instruction interface, the robot controller according to the first aspect accepts motion instructions from a client device in real time. The client device need not be co-located with the robot controller but it can be any kind of external device, especially a remote device. The motion instructions are of the same general type as those appearing in robot programs received through the robot controller’s external programming interface. The inventors have realized that any needed buffering, queuing and / or prioritization at the input side of the robot controller can be provided by means of the external instruction interface. With reference to the previously discussed EGM, it is noted that the high-level movement instructions cannot be added to the EGM instruction set simply by a mere design choice, rather this requires the novel technical solution presented herein or an equivalent technical solution.

[0013] By providing the robot controller with the external instruction interface, it becomes possible for an external client device to operate the industrial robot without capabilities of real-time control-signal generation, trajectory generation and communication. Instead, the robot controller carries out processing to ensure good motion performance and any necessary real-time event handling. Further, the operator who uses the client device can utilize available software libraries, which are standardized or otherwise suitable for interoperability, which saves the operator the effort of learning a native programming interface specific to the robot controller at hand. For example, an operator accustomed to the RAPID language may utilize the external instruction interface to execute RAPID-like motion instructions in real time.

[0014] In some embodiments, the robot controller further comprises an input buffer to support asynchronous execution of the received motion instructions. This way, if the robot controller is engaged with the execution of a previous motion instruction, a later motion instruction can be temporarily stored in the buffer until the robot controller becomes available for the next instruction.

[0015] In a second aspect of the present disclosure, there is provided a client device for use with a robot controller with the characteristics just described. The client device comprises an operator interface enabling an operator to select motion instructions from a predefined set of motion instructions; an output buffer for temporarily storing selected motion instructions; and a robot interface operable to transfer a burst of one or more motion instructions from the output buffer to the external instruction interface of the robot controller.

[0016] The presence of the output buffer allows the operator to select the motion instructions at his or her preferred pace, and sensibly independently of the status of the robot. The motion instructions are forwarded one by one, or in groups, according to a protocol acceptable to the robot controller. Unlike some existing technical solutions within option i), the client device operates in a lean fashion in that it may forward the motion instructions without necessarily applying path planning, path smoothing or comprehensive fault checking.

[0017] In some embodiments, the output buffer of the client device comprises one higher-priority buffer and one lower-priority buffer, and the robot interface is configured to transfer motions instructions to the robot controller while respecting these priorities. The combination of higher- and lower-priority buffers may aid theresponsiveness of event-based programming. A command execution module maybe responsible for forwarding the operator’s selected motion instruction to the higher-priority buffer or the lower-priority buffer based on preconfigured rules. Specifically, the command execution module may be further configured to control the inflow of the selected motion instructions into the output buffer in accordance with a configurable lookahead parameter.

[0018] In some embodiments, the robot controller and the client device communicate in accordance with a communication protocol with functionalities supporting the real-time robot commanding. The communication protocol may in particular be a remote procedure calling (RPC) protocol.

[0019] In some embodiments, the external instruction interface in the robot controller may be configured to output responses which indicate a current state of the industrial robot. The robot controller maybe configured to output such responses in reaction to receiving a motion instruction or a non-motion instruction in the stream from the client device. Optionally, the responses may further indicate execution statuses of previously received motion instructions. This behavior by which the external instruction interface sends reports on request empowers the client device to update itself about the state of the industrial robot, or to monitor that its previous motion instructions have been executed as expected. It also gives the client device the option of using an empty instruction (special case of non-motion instruction) as a status inquiry.

[0020] As used herein, executing a motion instruction “as soon as practicable” signifies that, after the motion instruction has been received (e.g., by the robot controller), the execution of the motion instruction is not dependent on receiving any further approval or input (e.g., a program execution command) from an operator. Rather, the execution is to happen as early as possible, while respecting any applicable limitations of the robot concerning speed, acceleration, torque, motor power etc. Execution “as soon as practicable” does not exclude asynchronous execution. For example, it is understood that technical circumstances prevailing when a new motion instruction is received may prevent a literally immediate execution of the new motion instruction; notably, if the robot manipulator is busy executing an earlier motion instruction at the time of receipt, then the new motion instruction may need to be buffered until the robot manipulator becomes available.

[0021] Generally, all terms used in the claims are to be interpreted according to their ordinary meaning in the technical field, unless explicitly defined otherwise herein. All references to “a / an / the element, apparatus, component, means, step, etc.” are to be interpreted openly as referring to at least one instance of the element, apparatus, component, means, step, etc., unless explicitly stated otherwise. The steps of any method disclosed herein do not have to be performed in the exact order disclosed, unless this is explicitly stated.BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Aspects and embodiments are now described, by way of example, with reference to the accompanying drawings, on which:figure 1 shows an industrial robot and a client device;figure 2 illustrates the interaction between the client device and the robot controller at runtime in an example situation where the client device has received a sequence of seven motion instructions;figure 3 shows contents of instruction queues maintained by the client device in said example situation;figure 4 is a flowchart of a process of adding instructions to queues; andfigure 5 is a flowchart of a process of consuming instructions from queues.DETAILED DESCRIPTION

[0023] The aspects of the present disclosure will now be described more fully hereinafter with reference to the accompanying drawings, on which certain embodiments of the invention are shown. These aspects may, however, be embodied in many different forms and should not be construed as limiting; rather, these embodiments are provided by way of example so that this disclosure will be thorough and complete, and to fully convey the scope of all aspects of the invention to those skilled in the art. Like numbers refer to like elements throughout the description.

[0024] Figure 1 shows an industrial robot 100 that comprises a multi-joint robot manipulator no and a robot controller 120 with a communication link between themselves. In the terminology of the present disclosure, “industrial robot” is used in a broad sense, to cover in particular manufacturing robots, material-handling robots,assembly robots, cutting / welding robots, service robots, collaborative robots, hygiene robots, industrial robot tracks, industrial robot positioners. An industrial robot may be a stationary robot or a mobile robot. A mobile robot, where generally speaking the robot controller 120 forms an integral part, maybe an automated guided vehicle (AGV), an autonomous mobile robot (AMR) or an autonomous mobile manipulator robot (AMMR). The term industrial robot covers the full range from lightweight robots designed to replace human manual work, over collaborative robots for supporting a human worker, all the way up to heavy-duty robots.

[0025] The example robot manipulator no in figure 1 has the shape of an arm that extends from a base 112. The arm includes a plurality of structural elements 114, which are connected by linear or rotary joints 113, which are movable by action of actuators 115, such as motors. The distal end of the robot manipulator no may carry a tool 111, such as a gripper in the depicted example. The tool 111 allows the robot manipulator 110 to interact with objects in the form of various workpieces, which are present in a work area of the robot manipulator 110. The workpieces are subject to manufacture, processing or handling by the robot manipulator 110. The work area may further include non-workpiece objects, such as containers, fixtures, robot positioners (which are optionally under the control of the robot controller 120), separators, thermal or electric insulators, supports etc. Apart from wear, staining etc., they are generally in the same condition at the beginning and the end of a work cycle. The non-workpiece objects could also be consumables or containers for consumables.

[0026] Sensors 116 are arranged in or at the robot manipulator no to capture measurements of mechanical or kinematic quantities (e.g., linear or angular position, speed, force, torque, strain), internal states (e.g., temperature, pressure, current) of the robot manipulator no and its supporting systems, or even sound and video from the work area. The sensor values 152 are reported from the robot manipulator 110 to a machine interface 124 in the robot controller 120 over a communicative connection therebetween, and in the opposite direction control signals 151 maybe conveyed towards the actuators 115. The control signals 151 comprise motion references 151 which the machine interface 124 applies in this manner to the actuators 115 in therobot manipulator no. Each motion reference represents a setpoint value for synchronous execution by the actuators 115. The motion reference may specify:- position in joint space or cartesian space,- velocity in joint space or cartesian space,- acceleration in joint space or cartesian space,- jerk in joint space or cartesian space,- force,- torque,- pose,- motor current,or combinations of two or more of these quantities. With synchronous execution, it is meant that the actuator 115 is expected to move, without delay, into a position or state which is consistent with the motion reference. Alternatively, synchronous execution could mean that the actuator 115 is expected to move into a position or state which is consistent with the motion reference as soon as practicable. To illustrate this behavior in the special case where the motion reference includes a position reference (e.g., the robot is instructed to go to its home position), the actuators 115 accelerate, move and then decelerate the manipulator no in such manner as to bring it to a position equal to the position reference as early as possible, while respecting any applicable limitations on speed, acceleration, torque, motor power etc.

[0027] In addition to the machine interface 124 towards the robot manipulator no, the robot controller 120 comprises processing circuitry 125 and a memory 127 suitable for storing system information, such as an operating system, global configuration settings, event logs and the like. The system information may include the applicant’s software RobotWare™ and / or the Robot Operating System (ROS). The memory 127 may further store one or more robot programs 128 that are executable on the processing circuitry 125.

[0028] A robot program 128 may contain a plurality of commands, such as commands compliant with the RAPID™ robot programming language. The commands may include motion instructions which express movements in a fixed reference frame (e.g., reference frame with its origin at the robot base 112) or a moving referenceframe (e.g., reference frame with origin in a workpiece or in a non-workpiece object). The motion instructions may be selected from a predefined set of robot motion instructions executable by the robot controller 120, such as the motion instructions defined by the RAPID™ robot programming language, including MoveL and MoveJ. The commands contained in a robot program 128 may further include non-motion instructions, which specify a change of reference frame, an input / output setting, a current state (e.g., enabled, disabled, teaching) of the robot manipulator no, or the like. It is noted that a robot program 128 maybe a compiled executable (binary) or a script.

[0029] The processing circuitry 125 implements a path planner 126 configured to generate motion references from motion instructions. In particular, the path planner 126 maybe utilized such that it inputs a robot program 128 and generates a timed sequence of motion references which realizes the robot program. The path planner 126 may be configured to generate the motion references by applying robot-specific logic, which could include one or more of: enforcing a machine-level limitation of the robot manipulator no (e.g., avoid singular joint configurations of the robot manipulator 110), enforcing a path smoothness condition (e.g., a differentiability class), enforcing an operational safeguard, or the like. The robot-specific logic may be specific to robots in general, specific to the robot type / robot category that the robot manipulator 110 belongs to, or specific to the particular robot deployment that the robot controller 120 is part of. Because the motion instructions are executed by the robot manipulator 110 after the path planner 126 transforms them into motion references, the motion instructions maybe said to have a higher level of abstraction than the motion references.

[0030] The path planner 126 may further be responsible for runtime event handling based on the sensor signals 152. The runtime event handling could include collision avoidance, checks that the robot operates in nominal conditions (e.g., nominal operating ranges, overheating protection), and compute motion references using the robot manipulator’s no current pose as initial value etc.

[0031] On the user side, the robot controller 120 comprises:- an external programming interface 121 configured to receive a robot program 128 containing motion instructions, which have a higher level of abstraction than the motion references,- an external motion-reference interface 123 configured to receive motion references for execution as soon as practicable, and- an external instruction interface 122 configured to receive a stream 130, which includes one or more motion instructions 131 for execution as soon as practicable. The external instruction interface 122 also accepts a stream 130 that additionally contains one or more non-motion instructions 132, in particular empty instructions 133. Empty instructions 133 maybe given to the external instruction interface 122 to trigger a status report, as further detailed below.Each of these interfaces 121, 122, 123 can be adapted for use by a human operator, by an executing software process or a device, or by any of these. It is instructive to note in figure 1 that the motion references received from the external motion-reference interface 123 are passed directly to the machine interface 124, where motor reference generation takes place. By contrast, the motion instructions - whether received in a robot program 128 or through the external instruction interface 122 - are first fed to the path planner 126, which generates motion references suitable for being forwarded via the machine interface 124 to the robot manipulator no. From the perspective of the external instruction interface 122, it is noted that the external instruction interface 122 could be configured to initiate execution of a motion instruction by causing the path planner 126 to generate a timed sequence of motion references 151 which realizes the motion instruction, and by further causing the machine interface 124 to apply said timed sequence of motion references to the actuators 115 of the robot manipulator no. It is appreciated that although slight lexical and / or syntactical differences may exist between the motion instructions 131 as used in a robot program 128 and the motion instructions 131 fed to the external instruction interface 122, the respective instruction sets are equivalent in effect on the industrial robot 100. In particular, instructions in the respective instruction sets may be in a one-to-one relationship to each other.

[0032] The RWS and EGM interfaces in some of the applicant’s robot controller products may be considered to be, respectively, an external programming interface 121 and an external motion-reference interface 123 in this sense.

[0033] Optionally, as shown in figure 1, the external instruction interface 122 is associated with an input buffer 129. The input buffer 129 supports asynchronousexecution of received motion instructions (and any non-motion instructions that are included in the same received stream), in the sense that the motion instructions are not necessarily executed as soon as they are received. Instead, one or more motion instructions may be temporarily stored in the input buffer 129 until it is possible to execute them. This gives the external instruction interface 122 a certain latitude to delay execution of received motion instructions as needed. In some embodiments, the external instruction interface 122 is configured to determine when to initiate execution of a buffered motion instruction based on one or more of the following:- a current status of the robot manipulator, such as busy, free, in maintenance mode;- a motion instruction for which execution is ongoing, such as by accommodating an expected duration of this execution;- relative priorities of two or more motion instructions, e.g., to schedule execution of high-priority motion instructions first;- any conflicts between two or more motion instructions which limit contemporaneous execution, it being understood that multiple non-conflicting instructions can be active concurrently (i.e., executed contemporaneously) provided sufficient processing and machinery resources are available.

[0034] In the example use case illustrated in figure 1, the external instruction interface 122 interacts with a robot interface 183 of a client device 180. The client device 180 may for example have the overall appearance of a handheld device, a tablet, a smartphone, a personal computer (PC), a robot programming device, a pendant or a teach pendant. The client device 180 maybe a dedicated robotcommanding device or it may be a multifunctional device, such as a PC on which suitable robot-commanding software is installed.

[0035] The client device 180 additionally comprises processing circuitry 184, an operator interface 181 and an output buffer 182. One functionality of the operator interface 181 is to let the operator select motion instructions (and non-motion instructions, if any) from a predefined set of motion instructions. The selection may be enabled by displaying menus with preconfigured options which the operator can select. Alternatively, the operator interface 181 may include an input prompt or an editor, through which the operator can input text that is interpreted on the basis ofpreconfigured syntax, such as a preconfigured dictionary of allowed motion instructions. In some implementations, the operator can use a programming language of their choice to define how and when the motion instructions are executed. The operator interface 181 maybe configured to compile the operator’s input into an executable binary, or to interpret the input.

[0036] The output buffer 182 is included in the depicted embodiment but is not an essential component of the client device 180. When present, the output buffer 182 may enable event-driven responsiveness from the industrial robot 100 to the operator’s program in the operator interface 181. The output buffer 182 is suitable for temporarily storing a motion instruction that the operator has selected via the operator interface 181, wherein the storage may persist until a point in time when the motion instruction is transferred to the robot controller 120 (consumed), or until the robot controller 120 confirms that the motion instruction has been executed.

[0037] The transfer of motion instructions proceeds via the robot interface 183. More precisely, the robot interface 183 is configured to transfer motion instructions 131 to the robot controller’s 120 external instruction interface 122 over a wired or wireless connection, and it may receive responses 141 from the external instruction interface 122 in the opposite direction. For interoperability, the robot interface 183 and external instruction interface 122 maybe compatible with a common communication protocol, such as a remote procedure calling (RPC) protocol, Recommended Standard 232 (RS-232) protocol, Universal Serial Bus (USB) protocol, Ethernet, or an OPC Unified Architecture (OPC-UA) protocol over Transmission Control Protocol (TCP).

[0038] The motion instructions 131 traveling from the robot interface 183 to the external instruction interface 122 maybe said to constitute a data stream 130. The motion instructions 131 maybe transferred periodically. For instance, the communication protocol may specify a repeating time frame that is synchronous between the robot controller 120 and the client device 180, and at a pre-agreed location therein for sending the motion instructions 131. The repeating time frame may have a duration of tens of milliseconds, such as 10 ms. The robot interface 183 may be configured to transfer a burst with a pre-agreed number Nt> 1 of motion instructions in each such time slot. Another option may be to transfer the motioninstructions 131 in an event-triggered fashion, or by a hybrid scheme by which a periodic transfer schedule is turned on and off based on pre-agreed events.

[0039] The data stream 130 may further contain non-motion instructions of the type introduced above. In figure 1, reference number 132 denotes non-motion instructions which order a change in the industrial robot 100 (e.g., reference frame, input / output, operational state), and reference number 133 denotes an empty instruction. The non-motion instructions 132, 133 maybe multiplexed with the motion instructions 131 into a common sequence, and transferred to the robot controller 120 in the manner described, i.e., according to a periodic, event-triggered or hybrid scheme. Each motion instructions 131 or non-motion instructions 132, 133, or each group of Ntsuch instructions, may be transmitted as a message or packet within the data stream 130.

[0040] The responses 141 from the external instruction interface 122 towards the robot interface 183 maybe said to constitute a further data stream 140. At least some of the responses 141 indicate a current state of the industrial robot 100, such as a current pose of the robot manipulator no, or whether the industrial robot 100 has executed particular motion instructions that it has received previously from the client device 180 (execution status). The responses 141 could also indicate whether the industrial robot 100 has executed particular non-motion instructions, e.g., whether it has changed into a requested reference frame, input / output setting, operational state, or the like. In some embodiments, information from the responses 141 is exposed to the operator through a client library (see below) which is displayed by the operator interface 181. The operator may run an application in a programming language of their choice which consumes this information, such as a C++ or Python application.

[0041] The external instruction interface 122 may be configured to transfer the responses 141 in reaction to the instructions 131, 132, 133. For example, the external instruction interface 122 maybe configured to trigger a transmission of a response 141 whenever a motion instruction 131 or a non-motion instruction 132 (such as an empty instruction 133) arrives. In this sense, each transaction between the robot controller 120 and the client device 180 is initiated by message transfer by the client device 180. The responses 141 maybe transferred in messages with a format that includes- a status of the industrial robot 100, and- execution statuses of one or more previously received motion instructions, and optionally including that or those motion instructions which triggered the response 141 at hand.In this response message format, it may for example be pre-agreed between the external instruction interface 122 and the robot interface 183 that the external instruction interface 122 shall only transfer an execution status for those motion instructions that have been executed (positive execution status). From this, it can be deduced implicitly that the remaining received motion instructions are yet to be executed (negative execution status).

[0042] The flow of event-triggered responses 141 from the external instruction interface 122 to the robot interface 183 enables the client device 180 to observe the industrial robot’s 100 state in near real time, if necessary by polling the robot controller 120 using an empty instruction 133. In particular, the client device 180 can monitor that the motion instructions it has transferred are executed by the industrial robot 100 as expected. To comply with a periodic transfer scheme, the robot interface 183 maybe configured to extract Nt> 1 motion instructions 131 from the output buffer 182 as long as the output buffer 182 is nonempty, and transfer it or them to the external instruction interface 122. When the output buffer 182 is empty, the robot interface 183 transfers non-motion instructions 132 - or empty instructions 133 in particular - according to the periodic scheme. This serves to ensure that the robot interface 183 continues to receive responses 141 with status information even when the operator is not selecting further motion instructions. To save resources, the client device 180 may in some embodiments be configured such that the robot interface 183 suspends its transfer of non-motion instructions 132 when all transferred motion instructions thus far have been executed. In other embodiments, the client device 180 keeps transferring non-motion instructions 132 - or empty instructions 133 in particular - to the robot controller 120 according to the periodic scheme by way of heartbeats. The robot controller 120 may deduce from such successfully received heartbeats that (i) the client device 180 is still connected, and (ii) the client device 180 is still functioning correctly. Although the first conclusion could be figured out on a network level, this is not possible with a scenario where an application on the client device 180 has crashed but the communication channel to the robot controller 120 remains open.

[0043] The client device 180 may include a command execution module 185 which is responsible for processing the responses 141 received from the robot controller 120. The command execution module 185 maybe implemented as a software process (calling thread) executing on the processing circuitry 184.

[0044] In some embodiments, the command execution module 185 is configured to maintain a record 185.1 of previously transferred motion instructions (which optionally includes previously transferred non-motion instructions) as well, and to erase a motion instruction from said record when a response 141 indicates that the motion instruction has been executed. If a motion instruction which the robot controller 120 has buffered remains in the record 185.1 in the client device 180 for more than a threshold duration, this could be a sign that the robot controller 120 has encountered some form of error. In reaction, the command handling on the robot controller 120 may be halted and optionally resumed when the operator has acknowledged the error by an input action to this effect. On the other hand, if a motion instruction 131 is lost during its transfer to the robot controller 120, the motion instruction will never enter the buffer 129 in the robot controller 120. The underlying communication protocol (e.g., RPC) will inform the client device 180 of the communication loss and the client device 180 will throw an exception for the operator to handle according to the operator’s configured error handling logic. In particular, an emergency policy defined by the operator or a system owner may be executed.

[0045] Additionally or alternatively, the command execution module 185 is configured to review the received responses 141 and update the state of the robot. The operator can modify the program flow based on the updated robot state. Also motion instructions 131 previously transferred to the robot controller 120 (in record 185.1, see below) maybe updated. For example, the current pose of the robot manipulator no or the currently used reference frame may be incompatible with a particular motion instruction. In this event, the command execution module 185 is configured to make any necessary updates to the temporarily stored motion instructions in the output buffer 182, such that they are transmitted in a form that suits the current state of the industrial robot 100. In the event that an incompatible motion instruction cannot be salvaged by updating, a further option may be to erase the motion instruction from the output buffer 182.

[0046] In some embodiments, the output buffer 182 in the client device 180 comprises one higher-priority buffer 182.1 and one lower-priority buffer 182.2. In general terms, the buffers 182.1, 182.2 make it possible to discriminate motion instructions selected by the operator into more important and less important motion instructions, so as to make sure that safety-critical instructions are and / or the expected program flow is respected. Concretely, when one higher-priority buffer 182.1 and one lower-priority buffer 182.2 are provided, the robot interface 183 can be configured to consume motion instructions for (i.e., transfer them to) the robot controller 120 from the higher-priority buffer before the robot interface 183 transfers motions instructions from the lower-priority buffer 182.2. Such a prioritization rule maybe rigid (e.g., never consume motions instructions from the lower-priority buffer 182.2 until the higher-priority buffer 182.1 is empty), or there could be exceptions to the effect that motions instructions from the lower-priority buffer 182.2 are allowed to constitute up to a preconfigured percentage (e.g., 20%) of the data stream 130, and / or that motions instructions which have remained in the lower-priority buffer 182.2 for more than a preconfigured duration shall be consumed even though the higher-priority buffer 182.1 is not yet empty.

[0047] The flowchart in figure 5 summarizes an example set of rules for consuming motion instructions from two queues with different priority levels in the client device 180.

[0048] The already introduced command execution module 185 may be responsible for controlling whether a motion instruction selected by the operator is to be stored in the higher-priority buffer 182.1 or in the lower-priority buffer 182.2 of the output buffer 182. The command execution module’s 185 routing of motion instructions into the higher-priority buffer 182.1 or the lower-priority buffer 182.2 may follow an execution logic defined by a system owner or the operator, e.g., a logic defined by means of the operator interface 181. For example, the motion instructions may be routed based on the application they originate from and / or based on their type. Further, the higher-priority buffer 182.1 maybe used for disable robot commands instructed by the client device 180. The command execution module 185 maybe configured to populate the buffers 182.1, 182.2 and to do so asynchronously relative to the operator’s selecting of the motion instructions.

[0049] The command execution module 185 may further be responsible for controlling the inflow of the selected motion instructions into the output buffer 182. The command execution module 185 may be configured to ensure that the motion instructions are fed to the output buffer 182 at a rate dictated, at least in part, by a configurable lookahead parameter. The lookahead parameter may be said to define a lookahead motion window in the sense that all non-sequential non-motion instructions are consumed in the order they were selected by the operator and as early as possible (provided that the calling thread is not blocked by the sequential motion instructions), then pushed to a corresponding queue 182.1, 182.2 and is eventually transferred within the data stream 130 to the robot controller 120. It is appreciated that sequential commands are commands where the order of execution is honored, whereas non-sequential instructions either run in parallel or instantaneously. Conversely, only a predefined number of motion instructions are present in the output buffer 182 and the record 185.1. After this, new instructions are only added as soon as a previously queued instruction is completed and removed from the record 185.1. This setup can be used both to hide the communication latency between the robot interface 183 and the external command instruction interface 122, and to allow the robot controller 120 to perform interpolation between subsequent motion instructions if needed. Further, the operator has the option of overriding the sequence in which the motion instructions were selected, notably by requesting interruption (abortion) of motion instructions which have a high priority and are thus likely to leave room for later or lower-ranking motion instructions.

[0050] To illustrate the interaction between the client device 180 and the robot controller 120 at runtime, reference is made to figure 2, which shows the high-level architecture of the novel external robot commanding approach disclosed herein. In the functional view according to figure 2, the client device 180 comprises an orches-trator module 186 and a communication layer 187 in addition to the components already described. The orchestrator module 186 maybe an enabler of a configured rate of processing; as described above, the command execution module 185 may be configured to ensure that the motion instructions are fed to the output buffer 182 at a rate dictated by a configurable lookahead parameter, and this activity is supervised by the orchestrator module 186 according to some embodiments. It is understood that the client device 180 may further include the components shown in figure 1.

[0051] The term client library (or programming library) may be used to refer to the orchestrator module 186, the command execution module 185 and the communication layer 187 - all implemented within the processing circuitry 184 -together with the operator interface 181. In the detailed view of figure 2, furthermore, the robot controller 120 comprises a communication layer 122.1 associated with the external instruction interface 122. It is appreciated that a communication layer in this sense may be a protocol endpoint which maintains a wired or wireless communication link between the client device 180 and the robot controller 120.

[0052] For the purpose of illustration and not limitation, figure 2 shows an example of what the current state of the input buffer 129 could be. It is assumed that the operator interface 181 in the client device 180 has caused seven motion instructions (identified by a pair of a unique identifier UID and a COMMAND_TYPE) to be queued in the input buffer 129:- UIDi, ENABLE - Enable the robot with UID of 1 populated in the higher- priority queue 182.1.- UID2,P2P - P2P motion command with UID of 2 populated in the lower- priority queue 182.2.- UID3,LIN - LIN motion command with UID of 3 populated in the lower- priority queue 182.2.- UID4,CIRC - CIRC motion command with UID of 4 populated in the lower- priority queue 182.2.- UID5,ST0P - Stop motion command with UID of 5 populated in the higher- priority queue 182.1.- UID6. ENABLE - (Re)enable the robot with UID of 6 populated in the higher- priority queue 182.1.- UID7,ST0P - Stop motion command with UID of 7 populated in the higher- priority queue 182.1.As shown in figure 2, two of these motion instructions, enumerated below, have been placed in the input buffer 129 (active command buffer) by the command execution module 185:- UIDi, ENABLE.- UID5,ST0P.

[0053] From the perspective of the operator of the client device 180, the client library offers an abstracted application programming interface (API) to assist the operator in developing their application based on function calls in the native syntax of the programming language of choice (e.g., C++, Python). By calling one of the functions of the API, a motion instruction gets added to one of the pending queues visible in the client library (through interaction with the orchestrator module 186 and command execution module 185), waiting to be sent through the protocol (e.g., RPC protocol) to the robot controller 120. It is noted that the client device 180 and the robot controller 120 act, respectively, as client and server from the perspective of a RPC protocol.

[0054] Figure 3 illustrates a possible implementation of the command execution module 185, which is responsible for managing and sending commands into the queues of the respective buffers 182.1, 182.2. In this implementation, the command execution module 185 comprises a queue manager 185.2 and it maintains a record 185.1 of previously transferred motion instructions (active command buffer).

[0055] Whether to queue a selected motion instruction in the higher-priority buffer 182.1 or the lower-priority buffer 182.2 is decided internally by the execution logic of the queue manager 185.2. It maybe configured with the following rules:- Sequential motion instructions are considered by default to be of low priority. - Most other instructions (e.g., setting a manipulator speed, causing the industrial robot 100 to enter enabled mode) are considered high priority.

[0056] The flowchart in figure 4 presents one set of rules to be applied by an enduser application (calling thread) running in the operator interface 181, the orchestrator module 186 and the queue manager 185.2 in the client device 180.

[0057] The queue manager 185.2 consumes the queues, one item at a time, and transfers motion instructions 131 and non-motion instructions 132, 133 to the robot controller 120 through the communication layer 187 periodically. The following rules may be applied:The queue of the higher-priority buffer 182.1 is consumed (i.e., extracted from the queue and then transferred to the robot controller 120) first.- The queue of the lower-priority buffer 182.2 is consumed if the queue of the higher-priority buffer 182.1 is empty.- An empty instruction 133 (keep-alive) is generated if both queues are empty. - Consumed instructions are moved to the record 185.1 (active command buffer).- Execution states of the instructions are monitored by the command execution module 185, and they are updated according to the responses 141 from the robot controller 120.- Once an instruction reaches a completion state (i.e., it has been executed, as indicated by a completion message from the robot controller 120), the command execution module 185 removes that instruction from the record 185.1 (active command buffer).Regarding the consumption of queued instructions, further reference is made to figure 5.

[0058] The operator interface 181 or the command execution module 185 is configured to assign each command a UID, to enable identification in subsequent responses 141 from the robot controller 120. This may proceed as follows:- Each function call is associated with a unique user’s response and paired with the instruction that was generated and sent to the robot controller 120.- Whenever the robot controller 120 transfers an asynchronous response 141 to the client device 180 that affects the response which is exposed to the operator through the interface 181, then the response is updated to reflect the content of the response 141. This allows the operator to modify existing commands in the record 185.1 in a controlled manner (through function calls) without needing to expose a new response when the execution status of the commands change.

[0059] The robot controller 120 receives and parses the motion instructions 131 and non-motion instructions 132, 133 arriving in the external instruction interface 122. In detail, the processing may include the following steps:- A nonempty instruction 131, 132 is first validated and then added to the input buffer 129 of the robot controller 120.- If an interruption or abortion request relating to a motion instruction 131 is received from the client device 180, the input buffer 129 is cleared of all previously added motion instructions. The next new instruction (or an instruction contained in the interruption or abortion request) becomes an active instruction, which the industrial robot 100 is to execute next.

[0060] The input buffer 129 is queried, and the instructions therein are executed according to the robot controllers’ 120 internal clock (not shown). The execution of the instructions is handled completely by the robot controller’s 120 real-time operating system, ensuring optimal motion performance that is protected from latency spikes and / or incorrect calculation from the side of the client device 180. It is appreciated that multiple commands can be stored in the input buffer 129. Further, motion instructions 131 are executed in sequence or in an interrupting fashion, while nonempty non-motion instructions 132 can be running in parallel, such as input / output operations, modifying speed or acceleration, and the like.

[0061] A watchdog process (not shown) in the robot controller 120 may optionally be used to monitor the arrival of instructions and other requests in the robot controller 120. If at some point a communication drop is detected, an operator-defined policy can be executed to ensure the safe onward operation of the industrial robot 100.

[0062] To summarize, the present disclosure describes an advantageous combination of an application running on an external device, a communication interface and an application running on the robot controller with which the external device is communicating. By providing this interface, an external device can operate the industrial robot without requiring real-time control-signal generation and trajectory planning. The external device can for example apply motion instructions (e.g., MoveL, MoveJ) that are normally susceptible of use inside a robot program constructed in advance.

[0063] The aspects of the present disclosure have mainly been described above with reference to a few embodiments. However, as is readily appreciated by a person skilled in the art, other embodiments than the ones disclosed above are equally possible within the scope of the invention, as defined by the appended patent claims.

Claims

CLAIMS1. A robot controller (120) for use with a robot manipulator (110) of an industrial robot (100), the robot controller comprising:a machine interface (124) configured to apply motion references (151) to actuators (115) in the robot manipulator, each motion reference representing a setpoint value for synchronous execution by the actuators;an external programming interface (121) configured to receive a robot program (128) containing motion instructions, which have a higher level of abstraction than the motion references; andprocessing circuitry (125), which implements a path planner (126) configured to input a robot program and to generate a timed sequence of motion references which realizes the robot program,characterized by an external instruction interface (122) configured to receive a stream (130) with one or more motion instructions (131) for execution as soon as practicable.

2. The robot controller (120) of claim 1, wherein the external instruction interface (122) is further configured to output responses (141) which indicate a current state of the industrial robot (100).

3. The robot controller (120) of claim 2, wherein the external instruction interface (122) is configured to output a response (141) in reaction to receiving a motion instruction (131) or a non-motion instruction (132, 133) in the stream.

4. The robot controller (120) of any of the preceding claims, wherein the external instruction interface (122) is associated with an input buffer (129) which supports asynchronous execution of received motion instructions.

5. The robot controller (120) of claims 2 and 4, wherein the external instruction interface (122) is configured such that the responses (141) further indicate execution statuses of previously received motion instructions.

6. The robot controller (120) of claim 4 or 5, wherein the external instruction interface (122) is further configured to determine when to initiate execution of a buffered motion instruction based on one or more of the following:a current status of the robot manipulator,a motion instruction for which execution is ongoing,relative priorities of two or more motion instructions,any conflicts between two or more motion instructions which limit contemporaneous execution.

7. The robot controller (120) of any of the preceding claims, wherein the external instruction interface (122) is further configured to initiate execution of a motion instruction by:causing the path planner (126) to generate a timed sequence of motion references (151) which realizes the motion instruction; andcausing the machine interface (124) to apply said timed sequence of motion references to the actuators (115) of the robot manipulator (no).

8. The robot controller (120) of any of the preceding claims, further comprising an external motion-reference interface (123), which is distinct from the external instruction interface (122).

9. A client device (180) for use with a robot controller (120) of an industrial robot (100), the client device comprising:an operator interface (181) enabling an operator to select motion instructions (131) from a predefined set of motion instructions;an output buffer (182) for temporarily storing selected motion instructions; and a robot interface (183) operable to transfer a burst of one or more motion instructions from the output buffer to an external instruction interface (122) of the robot controller.

10. The client device (180) of claim 9, in which the output buffer (182) comprises one higher-priority buffer (182.1) and one lower-priority buffer (182.2),wherein the robot interface (183) is configured to transfer motions instructions to the robot controller (120) from the higher-priority buffer before the lower-priority buffer.

11. The client device (180) of claim 9 or 10, wherein the operator interface (181) is associated with a command execution module (185) configured to control whether aselected motion instruction is to be stored in the higher-priority buffer (182.1) or the lower-priority buffer (182.2) of the output buffer (182).

12. The client device (180) of claim 11, wherein the command execution module (185) is further configured to control the selected motion instructions’ inflow into the output buffer (182) in accordance with a configurable lookahead parameter.

13. The client device of any of claims 9 to 12, wherein the robot interface (183) is configured to transfer the bursts of motion instructions to the robot controller (120) periodically, and furthermore to transfer non-motion instructions while the output buffer (182) is empty.

14. The client device (180) of any of claims 9 to 13, wherein the operator interface (181) is associated with a command execution module (185) configured to: monitor responses (141), which are received from the robot controller (120) via the robot interface (183) and which indicate a current state of the industrial robot (100); andmake any necessary updates occasioned by the responses (141) to motion instructions (131) previously transferred to the robot controller (120).

15. The client device (180) of any of claims 9 to 14, wherein the operator interface (181) is associated with a command execution module (185) configured to: monitor responses (141), which are received from the robot controller (120) via the robot interface (183), and which indicate execution statuses of motion instructions (131) previously transferred to the robot controller (120);maintain a record (185.1) of previously transferred motion instructions; and erase a motion instruction from said record when a response indicates that it has been executed.

Images