Method and computer program for generating a graphical user interface of a manipulator program

Abstract

Computer program product comprising instructions which, when the program is executed by a computer, cause it to generate a graphical user interface (100) of a manipulator program for controlling a manipulator system (1), wherein the manipulator system (1) comprises at least one manipulator (30), wherein the manipulator program includes at least one insertion point (AP1 to AP5), and the user interface has (100): a graphical program progress indicator (150) that displays the current program progress of the manipulator program and the at least one starting point (AP1 to AP5) of the manipulator program, wherein at least one of the displayed setup points (AP1 to AP5) can be selected by a user; and wherein the manipulator program is configured to control the manipulator system (1) in such a way that, upon selection, it assumes a system state corresponding to the selected setup point (AP1 to AP5), wherein the computer program product includes instructions which, when the program is executed by a computer, cause the computer to instruct the manipulator program to automatically determine a path from a current position in the execution of the manipulator program to the selected starting point (AP1 to AP5), wherein the computer program product includes instructions which, when the program is executed by a computer, cause it to open an input dialog which allows the manual determination of a path if automatic determination is not possible.

Description

[0001] The invention relates to a method for navigating through a manipulator program using a computer program product and to a computer program product comprising instructions that, when the program is executed by a computer, cause the computer to generate a graphical user interface of a manipulator program for controlling a manipulator system. Furthermore, the invention relates to a control device for a manipulator system, which control device comprises at least one computer program product.

[0002] First, the background of the invention is briefly presented.

[0003] Manipulator systems typically comprise at least one manipulator configured to physically interact with its environment. For example, such a manipulator can be an industrial robot with at least three movable, freely programmable axes and an end effector, such as a gripper or a machining tool. Such manipulator systems are used in industrial applications, such as automotive manufacturing.

[0004] Manipulator systems are typically controlled by a manipulator program that specifies the system behavior for the specific application. The requirements for programming manipulator programs have changed, for example, due to the use of new sensor technology that opens up new fields of application. For example, new functionalities can be programmed through the use of force and / or torque sensors, such as those used in the LBR iiwa industrial robot from KUKA AG. These include, among other things, search movements with sensor technology, force-controlled tool operation, HRC (human-robot cooperation) capabilities, and sensitive gripping.

[0005] Today, manipulator programs are usually programmed using domain-specific programming languages, such as those based on BASIC or PASCAL. A large portion of programming involves error correction and process or parameter optimization, which occurs after the actual manipulator program has been programmed. Error detection and correction require considerable time and personnel resources, resulting in high costs. Furthermore, there is a risk of significant damage if an error goes undetected and leads, for example, to an interruption of series production.

[0006] Common debugging and error correction techniques typically use "Jump To Line" instructions. These "Jump To Line" instructions are used to undo previously executed code and / or continue the program at a new location. However, a common "Jump To Line" instruction cannot guarantee that the program will continue correctly from the new location, as it cannot reset all essential system parameters to achieve a system state that corresponds to the one that would have existed had the program reached the location in question as planned.

[0007] This can, for example, lead to undesired movements of a manipulator in the manipulator system. For example, a parameter that specifies the target manipulator speed cannot be reset to a previously valid value. Consequently, the manipulator would move at a different speed than originally specified after the "Jump To Line" command.

[0008] For example, DE 11 2006 000 785 T5 discloses a method and system for providing a user interface of an industrial control system. The interface partially comprises information from the control system, which is displayed in layers on one or more display screens of a control system. Furthermore, DE 10 2010 046 274 A1 discloses a method for checking a stored production sequence for one or more machines with a cyclical machine operating sequence, in which a production sequence for a cycle is stored using command functions.Furthermore, DE 10 2010 018 475 A1 discloses a method for displaying the programmable sequence for one or more machines with a cyclic machine sequence on a screen. A sequence is programmed or modified using commands, production parameters are specified, actual values ​​for the machine components are determined through cyclic operation, and the sequence with the individual process steps and their dependencies is displayed on the screen. DE 10 2009 051 808 A1 also discloses a method for direct and semi-automatic user support in creating, modifying, and testing a sequence program for a state-based machine control system, executed on an integrated development environment of an electronic data processing device.Also known is the control of a machine such as a machine tool, a production machine, or an automated handling device, as disclosed in DE 10 2004 062 852 A1. Finally, an automated manipulator system and methods for controlling the automated control system are known from US 2009 / 0105851 A1.

[0009] To circumvent such errors, such as unwanted movements of the manipulator in the manipulator system, debugging backwards-in-time methods have been developed. These methods record the execution history of a program, such as a manipulator program. These debugging backwards-in-time methods enable the manipulator program to be "rewound" and continue deterministically, as previously valid system parameters can always be used. Consequently, the manipulator program always behaves as if it were being executed for the first time. However, to fully capture and store the execution history of the manipulator program and thus every past system state, complex sensors and extensive memory and computing resources are required. In particular, such sensors capture parameters of the manipulator system.For example, cameras, joint angle sensors, force and / or torque sensors, or the like are used.

[0010] In addition to the high hardware requirements, debugging backwards-in-time methods have complex operating structures, which can potentially lead to further programming errors. Therefore, it is an object of the present invention to provide navigation through a manipulator program, particularly during program creation, using a graphical user interface. Such navigation can be particularly advantageous in the areas of error correction and parameter correction or optimization. Furthermore, it is an object of the present invention to at least partially eliminate the aforementioned disadvantages.

[0011] A detailed description of the invention follows.

[0012] The object is achieved by a computer program product according to claim 1, a method according to claim 14 and by a control device according to claim 16.

[0013] In particular, the object is achieved by a computer program product comprising instructions which, when the program is executed by a computer, cause the computer to generate a graphical user interface of a manipulator program for controlling a manipulator system, wherein the manipulator system comprises at least one manipulator, wherein the manipulator program comprises at least one touchdown point, and wherein the user interface comprises a graphical program progress display which displays the current program progress of the manipulator program and the at least one touchdown point of the manipulator program, wherein the displayed at least one touchdown point can be selected by a user. Furthermore, the manipulator program is configured to control the manipulator system such that, upon selection, it assumes a system state associated with the selected touchdown point.The computer program product further includes instructions that, when the program is executed by a computer, cause the manipulator program to automatically determine a path from a current point in the execution of the manipulator program to the selected starting point. Additionally, the computer program product includes instructions that, when the program is executed by a computer, cause the computer to open an input dialog that allows the manual determination of a path if automatic determination is not possible.

[0014] Typically, the graphical user interface is displayed on a display of the manipulator system. The display can be a display screen or an input screen, such as a touchscreen.

[0015] The manipulator program controls the manipulator system and in particular at least one manipulator. The control can, for example, provide a defined movement of the manipulator or the gripping of workpieces and the processing of workpieces by means of a manipulator. For this purpose, the manipulator program preferably comprises several operations, with a starting point forming the beginning and / or end of an operation. A defined system state of the manipulator system is preferably assigned to each starting point of the manipulator program. The system state is preferably consistent at the starting points, so that the manipulator program can be continued deterministically from a starting point. Consequently, an operation executed from a starting point always leads to the same result, regardless of the execution history of the manipulator program.

[0016] Preferably, a touchdown point is linked to at least one other touchdown point via at least one operation. The sequence of operations from one touchdown point to a second touchdown point is referred to as a path. The manipulator program can be represented in program code or in higher-level program models. A representation of the manipulator program in model form is also possible. The graphical program progress display shows the user (regardless of the selected representation of the manipulator program) how far the execution of the manipulator program has progressed. For example, the start of a new operation or the successful completion of an already performed operation can be displayed. The touchdown points that lie between two operations are also displayed on the program progress display.

[0017] If the manipulator program is not yet fully executable, i.e., if it still contains errors, programming-related errors may occur when running the program. These errors can lead, for example, to an interruption of the manipulator program. Alternatively, the operator can interrupt execution of the manipulator program if they detect an error. Typical errors are parameterization errors or an incorrect sequence of operations in the manipulator program sequence. For example, an incorrect movement may be programmed or a parameter threshold for a measured value may not be set high enough. Likewise, parameter values ​​may be incorrectly calculated or incorrectly transferred. Other errors, caused, for example, by unforeseen events or by the activation of a safety monitoring system, are also possible.

[0018] If such an error has occurred and been detected, a user can select a starting point on the program progress display that precedes the error in the manipulator program execution. In this case, the manipulator program is "rewound" so that it again assumes the parameter values ​​corresponding to the starting point. Furthermore, the manipulator system is returned to the system state associated with the selected starting point. Consequently, the manipulator program can subsequently be executed again from that starting point, achieving the same results regardless of the manipulator program's execution history.

[0019] Before restarting, detected errors can preferably be corrected and / or parameters adjusted. This eliminates the need to abort and restart the manipulator program to correct an error; instead, it can be continued directly from the selected starting point. Furthermore, it is not necessary to record all system parameters and the complete execution history, as defined starting points provide suitable starting points. The system state therefore only needs to be defined for one assigned starting point. However, the path to a selected starting point from the current point in the manipulator program's execution is calculated automatically.

[0020] Preferably, a starting point defines a point in the execution of the manipulator program to which a system state of the manipulator system is assigned. Assigning a system state of the manipulator system to a starting point enables the manipulator program and the manipulator system to be "rewound" to the system state from which the manipulator program can be continued.

[0021] The manipulator program preferably comprises a plurality of operations, wherein the at least one touchdown point forms the beginning and / or end of an operation and preferably the beginning and / or end of an operation structure. An operation structure preferably comprises a plurality of operations. Preferably, not every operation ends and / or begins at a touchdown point. Preferably, a plurality of operations are combined to form an operation structure, and these operation structures are linked to one another via touchdown points. This makes it possible to define touchdown points at those points in the execution of the manipulator program whose respectively assigned system state is characteristic and / or important for the correct execution of the manipulator program. For example, a touchdown point can be set after a successful movement of the manipulator along a defined trajectory.Likewise, processing steps performed by the manipulator can be limited by touchdown points.

[0022] If multiple operation structures are executed consecutively, the preceding operation structure is completed. However, a single first operation can initiate a second operation that extends beyond the execution time of the first operation. In this case, the context of the operation structure must be maintained until the last operation of the operation structure is completely executed. Thus, the operation structures do not influence each other. Consequently, operation structures executed in parallel produce the same results as serially executed operation structures. Furthermore, a state change of an executed operation structure is permanent.

[0023] As described above, the beginning and / or end of an operation structure can form a starting point. From this starting point, the manipulator program can continue regardless of its execution history, always achieving the same results.

[0024] Preferably, at least one reaction structure is assigned to an operation structure, wherein the at least one reaction structure contains reaction operations, during the execution of which the manipulator program controls the manipulator system in such a way that it is guided into a system state that corresponds to a touchdown point.

[0025] If an error occurs that is due to a programming error or an unforeseen event, such a starting point can be reached using a reaction structure, and the manipulator program can be continued from there. The starting point does not have to correspond to the last completed starting point. Likewise, a different operation structure than the one in which the error occurred can be executed from a starting point. This allows a process that has already begun to be corrected or continued with a new plan without having to abort or restart the manipulator program.

[0026] Furthermore, the direct assignment of operation structure and reaction structure enables a clear manipulator program, which can reduce programming errors and simplify troubleshooting.

[0027] The reaction structure preferably includes reaction operations that can undo individual operations if the respective operation structure describes a reversible process. If the respective operation structure describes an irreversible process, the reaction structure can contain reaction operations that allow continuation with a modified plan, i.e., that deviate from a mere reversal of executed operations.

[0028] For example, when setting a rivet using a manipulator, the rivet may become jammed in a hole. In this case, correct riveting is not possible. If the manipulator system detects this error, the "Riveting" operation structure can be aborted and an associated reaction structure initiated. In this example, a possible reaction structure could involve opening the riveting tool. Subsequently, pulling the jammed rivet out of the hole could be commanded. Finally, a new rivet could be picked up, and the "Riveting" operation structure executed again.

[0029] Additional reaction structures are also possible. For example, depending on the number of failed attempts to execute an operation structure, a reaction structure can command a different procedure. To continue with the previous example, after repeatedly executing the "Riveting" operation structure, which results in the same "jammed rivet" error, a different procedure could be commanded and an operator could be called. This operator could be asked to manually remove the jammed rivet.

[0030] An operation structure and an associated reaction structure are preferably implemented in a common semantic module. This ensures a fixed assignment of operation structure and reaction structure in the manipulator program, reducing complexity.

[0031] In addition to undoing an operation structure, a reaction structure can also command the aborting of the operation structure and / or the continuation of the manipulator program from a different starting point. Preferably, the system state of the manipulator system at a starting point and / or before and / or after the execution of an operation structure is not consistent in all system parameters. However, consistency must be maintained in the essential parameters so that the continuation or repetition of the manipulator program from this starting point is possible.

[0032] Such inconsistent system parameters can, for example, arise from irreversible processes. For example, if a welding process is controlled by the manipulator program, the "weld length" parameter is irreversible because, for example, a certain part of the weld has already been welded before the error occurs. If an error now occurs, the reaction structure cannot restore the same system state as before the welding process was executed. Therefore, in this case, the reaction structure must provide an appropriate reaction operation that allows the causal error of the welding process to be corrected (e.g., cleaning the welding tool) and then continuing at the point in the weld where the welding was interrupted.It is also possible that in this case, the reaction structure provides an alternative procedure, whereby the interrupted weld seam should not be fully welded but reworked in a manual step. In this case, the manipulator program can be continued at a different point.

[0033] Preferably, the execution of an operation structure is interrupted upon the occurrence of an error and subsequently continued with the execution of a reaction structure. Thus, an operation structure of the manipulator program can be repeated immediately after the occurrence of an error, or the manipulator program can be continued from a different starting point. This allows downtimes of the manipulator system to be minimized and, preferably, to continue automatically, i.e., without operator intervention. This can increase the productivity of the manipulator system.

[0034] Particularly preferably, not every operation of an operation structure is assigned its own reaction operation of a corresponding reaction structure. Likewise, not every operation structure needs to be assigned its own reaction structure. Preferably, a reaction structure is assigned to an operation structure, with the reaction structure preferably being assigned to at least one further operation structure. Thus, errors in different operation structures can be responded to with a reduced number of reaction structures. Preferably, a reaction structure is assigned to a plurality of operation structures if the operation structures have a common starting point.

[0035] Preferably, the reaction structure contains at least one reaction operation, the execution of which leads the manipulator system to a system state corresponding to the touchdown point, which touchdown point forms the beginning of the operation structure in which the error occurred. Such reaction structures enable the manipulator system to be "rewound." This allows the operation structure in which the error occurred to be repeated. Returning to the touchdown point does not have to correspond to a direct inversion of the operation structure, but can be carried out via an alternative path. For example, if a manipulator movement is to be reversed and the direct return path is blocked, the manipulator can be returned to the starting point of the movement via an alternative movement path.

[0036] Preferably, the at least one reaction operation reverses an operation of the operation structure. Thus, individual operations can be immediately reversed. This is particularly advantageous for reversible processes. For example, a movement performed by the manipulator can be immediately reversed.

[0037] Preferably, an operation of the manipulator program is defined by at least one parameter, wherein the at least one parameter can be changed by a reaction operation of the reaction structure. Preferably, the parameter can be adjusted using the graphical user interface.

[0038] A parameter can be a numeric parameter, such as a manipulator speed, or an instruction parameter that influences the sequence of operations within an operation structure or the selection of the subsequent operation structure. Thus, a reaction operation can cause the operation structure to continue with modified parameters (e.g., with a slower manipulator speed) or to repeat with modified parameters. It is also possible to achieve an alternative branching of the manipulator program.

[0039] Preferably, the reaction structure contains at least one reaction operation, the execution of which continues the interrupted operation structure with at least one modified parameter or re-executes the interrupted operation structure. This makes it possible to re-execute operation structures in which an error has occurred or, if necessary, with modified, preferably optimized parameters.

[0040] Preferably, the reaction structure includes at least one reaction operation, the execution of which interrupts the interrupted operation structure until an operator input, preferably by means of the graphical user interface, has been made.

[0041] This allows operator intervention to be requested if the error cannot be automatically corrected by the manipulator system. For example, the operator can be prompted to check a specific system status, remove or replace faulty parts or workpieces from the manipulator system, and / or similar tasks. If the operator confirms to the manipulator system that the task has been successfully completed, the system can continue with the reaction structure or an operation structure.

[0042] Preferably, different reaction operations of the reaction structure are executed depending on the type of error that has occurred. This is advantageous because it allows for individual responses to different error types. Preferably, different reaction structures are executed depending on the error that has occurred. For this purpose, several reaction structures are preferably assigned to an operation structure, which are executed depending on the error that has occurred.

[0043] If, for example, an operation structure is aborted because a safety mechanism of the manipulator system intervenes, it may be possible to simply repeat the operation structure if the reason for the safety device intervening was temporary. On the other hand, if an error occurs that cannot be corrected automatically, an operator can be called in, who can preferably provide input via the graphical user interface, to correct the error that has occurred. For example, it is possible that an error requires the manipulator program to be continued at a different starting point. For example, an operation structure can be skipped and the manipulator program continued at a different point. Thus, an incorrectly executed operation of an operation structure and / or an incorrectly executed operation structure can be followed by a corresponding reaction structure of the manipulator program.This enables error-dependent, differentiated error correction and continuation of the manipulator program from a suitable starting point.

[0044] Furthermore, parameters can preferably be changed differently depending on the error that has occurred. For example, an operation structure can be repeated with a reduced manipulator speed if a first error occurs, or with a changed force threshold of a manipulator force monitor if a second error occurs. Other changed parameters are also conceivable. Preferably, a starting point enables the manipulator program to be correctly continued from this point in the execution of the manipulator program, regardless of the execution history of the manipulator program. This ensures that the influence of an error correction or an optimized parameter value can be considered in isolation. If the error correction or parameter removal leads to the desired results when the manipulator program is continued.If a further iteration loop is necessary to correct errors or parameters, it is possible to return to the previous starting point and restart the manipulator program under the same conditions (e.g. same parameterization).

[0045] Preferably, the computer program product is configured to cause the manipulator program to automatically determine a path from a current point in the execution of the manipulator program to the selected starting point. If an error occurs in an operation of the manipulator program and this error has been detected, the program must be rewound to a previous starting point before the faulty operation is performed again. This is preferably done using at least one suitable reaction structure. For example, a manipulator must be moved back to the system state corresponding to the selected starting point.Because the computer program product causes the manipulator program to automatically determine a path from a current location in the manipulator program to the selected touchdown point, the program can be quickly continued or the faulty operation can be re-executed without operator intervention.

[0046] If automatic determination of the path is not possible, the computer program product is further configured to open an input dialog which enables the manual determination of a path. The manual determination of a path to a selected starting point makes it possible to continue debugging, error recovery or error correction and parameter optimization without having to abort and restart the manipulator program. The manual determination of a path can be carried out, for example, by selecting suitable operations, operation structures and / or reaction structures which undo operations already carried out or enable an alternative path to a selected starting point. The path can also be configured manually, for example by the user controlling the manipulator using a joystick or other input device.The manipulator system is brought into the system state associated with the selected touchdown point. Feedback to the operator indicating or simplifying the achievement of the system state can be displayed via the graphical user interface.

[0047] The graphical user interface preferably comprises at least one model representation of the manipulator program, wherein the program progress and / or the at least one starting point of the manipulator program and / or a selection of a starting point of the manipulator program can be displayed on the program progress display in the model representation of the manipulator program. The program progress can be displayed, for example, by a cursor, an arrow, a colored background, or the like. The model representation can be a semantic model, a flowchart, or another known graphically representable model of a manipulator program.

[0048] The touchdown points can be represented, for example, by appropriate highlighting between the individual operations of the manipulator program. The selection of a touchdown point of the manipulator program can also be indicated by a cursor or a colored or graphic highlight. Other display methods are also possible. This allows the user to see the current point in the manipulator program's sequence and, if a touchdown point is selected, which touchdown point is selected. Furthermore, the user can be shown which touchdown points are currently selectable, i.e., which touchdown points lie before the current point in the manipulator program's sequence, or which touchdown points can be reached via alternative paths.

[0049] Preferably, when navigating through the manipulator program using the graphical user interface, and in particular using the program progress display, the user's current selection is snapped to locations on the program progress display that correspond to a landing point, in order to simplify the selection of the landing point. For example, if a user selects a point near a landing point, the selection can automatically jump to the corresponding landing point. Likewise, if the user guides the selection via the program progress display, the selection can remain at a landing point, i.e., be snapped, in order to inform the user of the position of the landing point and simplify the selection of the landing point.

[0050] Preferably, the at least one displayed touchdown point is assigned a link, which can be used to open an input dialog box by means of which the manipulator program can be modified. For example, if an error occurs and a touchdown point is selected, a parameter list can be displayed in an input dialog box which is characteristic of the operations performed after the touchdown point. This allows parameter optimization to be carried out immediately. It is also possible to display an editor which can, for example, change the sequence or order of operations performed after the touchdown point. Further changes to the manipulator program can also be made using the input dialog box. This makes it possible to switch to error correction mode immediately after a touchdown point is selected, which enables errors to be corrected efficiently and quickly.Parameters can also be optimized quickly and easily.

[0051] The graphical user interface preferably comprises at least one graphical representation of the manipulator system, which shows the manipulator system in a system state corresponding to the at least one touchdown point, wherein the at least one graphical representation is linked to the displayed touchdown point. If, in addition to a progress of the manipulator program, the actual manipulator system is also displayed on the graphical user interface, it is easier for an operator to select touchdown points that correspond to a system state of the manipulator system. This simplifies the selection of a correct or favorable touchdown point. The graphical representation of the manipulator system can be an actual image of the manipulator system, which was created before, during, or after programming the manipulator program.Likewise, the graphical representation may be an abstracted representation, such as a line drawing or a representation created from a CAD model of the manipulator system.

[0052] Preferably, the program progress indicator may further display at least one of the following elements: - A link to further data, where further data includes graphical representations and / or model representations of the manipulator program; - a wait and / or synchronization instruction of the manipulator program; - Repetitions and / or branches of the manipulator program.

[0053] Graphical representations and model representations of the manipulator program simplify the assignment of a system state to a selectable starting point. A display of wait and / or synchronization instructions from the manipulator program allows the user to capture manipulator program processes on the graphical user interface that have no direct impact on the manipulator system. This allows the operator to capture instructions and operations that do not cause a perceptible change in the system state. Consequently, the operator can verify at any time whether the manipulator program is still executing or whether an error may have occurred.

[0054] Displaying repetitions and / or branches in the manipulator program enables a clear representation of the manipulator program sequence. For example, the manipulator program can branch at a starting point if different options for continuation exist at that starting point or at another point in the manipulator program. For example, a manipulator program can specify that once an object has been grasped, this object is first checked and, depending on the check, initiate further steps. These steps could include, for example, reprocessing the object, removing the object from a production process, or continuing processing of the object. Displaying repetitions also makes it possible to display recurring elements of the manipulator program in a space-saving manner.

[0055] Preferably, the graphical user interface comprises at least one of the following buttons: - a button by means of which the manipulator program can be stopped; - a button by means of which a new starting point can be defined at a selected location displayed on the program progress bar and / or by means of which a new path can be defined; - a button by means of which the manipulator program can be continued from a selected point displayed on the program progress display; - a button by means of which starting points can be selected that lie between the currently selected position of the program progress display and the position at which the manipulator system stopped; - a button by means of which individual operations of the manipulator program can be undone and / or touchdown points can be selected and / or - a button with which you can select previously passed landing points.

[0056] The buttons mentioned allow easy and intuitive navigation through the manipulator program or a model representation of the manipulator program. The button that triggers a stop of the manipulator program allows the manipulator program to be interrupted if the operator detects an error or wishes to optimize parameters. The button that can be used to define new touchdown points allows a system state to be recorded as a touchdown point that is optimal for a specific error correction or parameter optimization. For example, the distance between two adjacent touchdown points can be reduced by defining a new touchdown point between two existing touchdown points. This avoids repetitions of already stable operations and saves time.The button used to continue the manipulator program allows the program to be continued after an interruption and, if necessary, after an error has been corrected. The button for selecting start points allows the manipulator program to be "rewound" and the manipulator system to return to a system state that corresponds to the selected start point. The button used to undo individual operations of the manipulator program allows the program to be reversed independently of start points, for example, to immediately check the effect of a changed parameter. The button used to select start points that have already been passed through allows for quick selection of previous start points.Thus, the essential functions of the computer program product can be accessed directly from the graphical user interface via buttons.

[0057] Preferably, the graphical user interface comprises at least one second program progress indicator, which displays the current program progress of a manipulator program of a second manipulator. This is particularly advantageous when the manipulator system comprises multiple manipulators. For example, these manipulators can work together. If they are controlled individually, they can also assume different system states. It is therefore advantageous to indicate the point in the sequence at which the respective manipulator program is located.

[0058] Preferably, a timescale is assigned to the program progress display, with the timescale being zoomable. A zoomable timescale allows a smaller or larger section of the program progress display to be shown on the manipulator system's display. This allows both an overview of the entire manipulator program and details of the manipulator program to be displayed. Depending on the type of error to be corrected, either an overview (for example, to change the sequence of operations) or details of the manipulator program (for example, to optimize parameters) can be displayed.

[0059] Preferably, waiting times of the manipulator system are displayed in the program progress display on a condensed timescale, and even more preferably, hidden. Consequently, the timescale assigned to the program progress display does not need to be linear. Thus, periods in the execution of the manipulator program that do not provide any significant information to the user can be displayed in a condensed manner or even hidden, thereby increasing the clarity of relevant information.

[0060] Preferably, the program progress indicator is a program progress bar. Program progress bars have the advantage of being intuitive and quick to understand and provide the user with quick feedback on the current program progress.

[0061] Preferably, the computer program product comprising instructions for generating a graphical user interface is integrated into the manipulator program for controlling a manipulator system. It is also possible for the computer program product to be a standalone computer program product that can be applied to existing manipulator programs.

[0062] The object is further achieved by a method for navigating through a manipulator program by means of a computer program product as described above, the method comprising: - Providing a manipulator program; - Creation of the graphical user interface; - Stop the manipulator program; - Receiving a selection of a touch point on the program progress display of the graphical user interface, and - Leading the manipulator program to the selected touchdown point so that the manipulator program controls the manipulator system so that it assumes the system state assigned to the selected touchdown point.

[0063] The method makes it possible to stop a manipulator program using a graphical user interface if an error has been detected. The manipulator program can also be stopped if, for example, a breakpoint is set for debugging. A breakpoint specifies a point in the execution of the manipulator program at which the manipulator program is to be stopped or interrupted, for example in order to check or optimize parameters or the like. Once the manipulator program is stopped, changes can be made to the manipulator program. These changes can include adjusting parameters and / or changing a sequence and / or adding operations to the manipulator program and / or the like. Receiving a start point on the program progress display of the graphical user interface makes it possible to navigate to or from a selected start point.to establish a system state of the manipulator system that corresponds to the touchdown point. This allows the manipulator program to be "rewound," with the manipulator system assuming a system state corresponding to the touchdown point. The manipulator program can then be continued from the touchdown point.

[0064] The method further comprises automatically determining a path from a current point in the manipulator program to the selected landing point. If automatic determination is not possible, opening an input dialog that allows the manual determination of a path. This ensures that the selected landing point can be reached. This can be done either automatically or through operator intervention, as described above.

[0065] When guiding the manipulator program to the selected touchdown point, it is preferably possible to wait for an operator to interact with the manipulator system, and preferably, an interaction is displayed for an operator. For example, it may be possible that an error that occurs cannot be automatically corrected by the manipulator system. For example, a component may jam in a manufacturing process or a defective component may be present. If, for example, the manipulator system cannot automatically remove these components from the production cycle, operator intervention may be necessary. In this case, the described method can react to this error by calling in an operator to correct the error. The manipulator system or the graphical user interface preferably shows the operator which error was detected and / or how the error can be corrected.This means that even if a touchdown point cannot be reached automatically, it can be reached after operator intervention without the manipulator program having to be restarted.

[0066] The object is further achieved by a control device for a manipulator system comprising a computer program product as described above, wherein the graphical user interface is preferably generated on a handheld display device such as a tablet computer and / or smartphone. A control device for a manipulator system is typically assigned to the manipulator system and serves to control the manipulator system. Such control devices can themselves comprise a display on which the graphical user interface can be displayed. Such displays can be, for example, touchscreens or the like. It is also possible for the control device to display the graphical user display additionally or exclusively on a further display device. Display devices are known and can be, for example, tablet computers or smartphones.This is advantageous because tablet computers and smartphones are available in large quantities and can be purchased inexpensively.

[0067] The invention is described below with reference to the attached figures, which show Fig. 1 a schematic representation of a manipulator system; Fig. 2 a schematic model representation of a manipulator program; Fig. 3 a schematic representation of a display device comprising a graphical user interface; Fig. 4 a schematic representation of a display device comprising a further graphical user interface, and Fig. 5 a schematic representation of a program progress indicator.

[0068] In particular, Fig. 1 shows a manipulator system 1 comprising a manipulator 30, a control device 20, and a display device 10. The manipulator 30 can be, for example, an industrial robot. The display device 10 can be integrated into the control device 20 or embodied as a separate device. For example, the display device 10 can be a tablet computer or a smartphone that communicates wired or wirelessly with the control device 20 of the manipulator system 1. A computer program product for generating a graphical user interface of a manipulator program can be installed on the display device 10 and / or on the control device 20. The computer program product can also be integrated into an existing manipulator program.

[0069] Fig. 2 shows a graphical representation of a manipulator program 2. The graphical representation of the manipulator program comprises operations S200 to S261, which are represented by circles. The individual operations are linked to one another via the solid arrows. The operations of the manipulator program can be executed one after the other or can contain branches. Individual operations are assigned to starting points AP1 to AP5 (represented by squares). At these starting points AP1 to AP5, the system state of the manipulator system is consistent, i.e., starting from the starting point, operations lead to the same result, regardless of the execution history of the manipulator program. The dotted arrows indicate a possible sequence of the manipulator program. Starting at starting point AP1, operation S200 is executed first. The program then continues with operations S210 and S211 to reach operation S220.Starting with operation S220, which is assigned to the touchdown point AP3, operations S221, S222, S223, and S225 are executed. An error occurs in operation S225, as indicated by a lightning bolt.

[0070] The manipulator program is subsequently interrupted in operation S225. Now, for example, an operator selects touchdown point AP2 to continue manipulator program 2. The computer program product assigned to manipulator program 2 can now automatically determine a path to touchdown point 2 and have the manipulator program execute the corresponding operations. To do this, operation S250 is first executed. In S260, in order to reach touchdown point AP2, operation S262 must be selected as the next operation. This leads to touchdown point AP3. Touchdown point AP3 is linked to touchdown point AP2 via the path defined in operation S263. Consequently, it is possible to automatically return to touchdown point AP2 from the faulty operation S225. Depending on the operations, the manipulator system returns to the system state assigned to touchdown point AP2.

[0071] In addition, error correction can be performed so that when the manipulator program is re-executed starting from AP2, the manipulator program can be continued without having to be interrupted at operation S225. After that, for example, it can continue with operations S250, S260, and S261 to reach the starting point AP5.

[0072] Fig. 3 shows a graphical display device 10, which is, for example, a smartphone or a tablet computer. The display device can also be integrated into a control device of the manipulator system. The graphical user interface 100 is displayed on the display device 210. The graphical user interface 100 includes a program progress indicator 150, which is represented in the form of a program progress bar. The program progress bar 150 is, for example, a time bar that uses a cursor to indicate the current point in the execution of the manipulator program.

[0073] Block structures 210 to 225 are displayed on another part of the graphical user interface 100. These block structures 210 to 225 are a model representation of the manipulator program. For example, block 210 represents an operation that corresponds to operation S210. Similarly, block 220 represents an operation that corresponds to operation S220 from Fig. 2. Block 220 comprises further subordinate blocks corresponding to operations S221, S223, and S225. Operations S224, S226, and S227, which are not currently being executed in the manipulator program, are hidden in the selected model representation. This increases clarity.

[0074] In addition to the program progress bar 150, the current program progress can also be indicated in the model display, for example, by highlighting the current block in color. For example, block 223, which corresponds to operation S223, is highlighted in color. This indicates to the operator that the manipulator program is currently in operation S223.

[0075] Fig. Figure 4 shows another form of model representation of the manipulator program on the graphical user interface. In addition to the program progress indicator 150, a graphical model or the like can also be displayed on the graphical user interface 100. For example, the Fig. 2, a familiar graphical model of the manipulator program is displayed. This allows for a clear representation of all possible branches of the manipulator program. Operations are represented, for example, as circles and starting points as rectangles. The currently executing operation of the manipulator program is highlighted by a corresponding background. Additionally, an arrow or cursor 160 can indicate the current program progress.

[0076] The program progress indicator 150 is detailed in Fig.5. A time scale 151 is assigned to the program progress display 150. Furthermore, a sequence of start points AP1 to AP4 is shown on the program progress display. The cursor 155 indicates the point in the execution of the manipulator program. For example, if an error occurs after start point AP4 (dotted representation of the cursor 155), the program can be returned to start point AP3. To do this, the cursor is dragged to the corresponding position in the program progress bar and the corresponding start point is selected. Alternatively, a start point can also be selected directly. Buttons 160-165 enable simple operation of the graphical user interface. For example, the manipulator program can be interrupted using button 160.Button 161 can be used to define a starting point at the current location in the manipulator program flow and / or to define a new path. Button 162 allows the program to be continued, while button 163 causes the program to jump to a starting point that lies after the current location in the manipulator program flow. Button 164 can be used to undo individual operations of the manipulator program. Button 165 allows you to jump directly back to previous starting points. List of reference symbols 1 manipulator system 10 graphic display device 20 Control device 30 Manipulator AP1-AP5 touchdown points S200-S261 Operations 100 graphical user interface 210-225 blocks of a block diagram of the manipulator program 150 Program progress indicator 160 Arrow / Cursor 151 Timescale 155 cursors 160 to 165 buttons

Claims

[1] Computer program product comprising instructions which, when executed by a computer, cause the computer to generate a graphical user interface (100) of a manipulator program for controlling a manipulator system (1), wherein the manipulator system (1) comprises at least one manipulator (30), wherein the manipulator program comprises at least one touchdown point (AP1 to AP5), and wherein the user interface (100) comprises: a graphical program progress display (150) which displays the current program progress of the manipulator program and the at least one starting point (AP1 to AP5) of the manipulator program, wherein the displayed at least one touchdown point (AP1 to AP5) can be selected by a user; and wherein the manipulator program is designed to control the manipulator system (1) in such a way that, upon selection, it assumes a system state associated with the selected touchdown point (AP1 to AP5), wherein the computer program product comprises instructions which, when the program is executed by a computer, cause the computer to cause the manipulator program to automatically determine a path from a current point in the execution of the manipulator program to the selected starting point (AP1 to AP5), wherein the computer program product comprises instructions which, when the program is executed by a computer, cause the computer to open an input dialog which enables the manual determination of a path if automatic determination is not possible. [2] Computer program product according to claim 1, wherein a touchdown point (AP1 to AP5) defines a point in the execution of the manipulator program to which a system state of the manipulator system (1) is assigned. [3] Computer program product according to one of claims 1 or 2, wherein the manipulator program comprises a plurality of operations (O1 to On), wherein the at least one touchdown point (AP1 to AP5) forms the beginning and / or the end of an operation (O1 to On) and preferably the beginning and / or the end of an operation structure comprising a plurality of operations (O1 to On). [4] Computer program product according to one of the preceding claims, wherein a starting point (AP1 to AP5) enables the correct continuation of the manipulator program from this point in the execution of the manipulator program, regardless of the execution history of the manipulator program. [5] Computer program product according to one of the preceding claims, wherein the graphical user interface (100) comprises at least one model representation (2) of the manipulator program, wherein • the program progress and / or • at least one starting point (AP1 to AP5) of the manipulator program • and / or a selection of a starting point (AP1 to AP5) of the manipulator program can be displayed on the program progress display (150) in the model representation (2) of the manipulator program. [6] Computer program product according to one of the preceding claims, wherein the at least one displayed touchpoint (AP1 to AP5) is assigned a link with the aid of which an input dialog can be opened by means of which the manipulator program can be changed. [7] Computer program product according to one of the preceding claims, wherein the graphical user interface (100) comprises at least one graphical representation of the manipulator system (1) showing the manipulator system (1) in a system state corresponding to the at least one touchdown point (AP1 to AP5), wherein the at least one graphical representation is linked to the displayed touchdown point (AP1 to AP5). [8] A computer program product according to any one of the preceding claims, wherein the program progress indicator (150) may further display at least one of the following elements: • A link to further data, where further data includes graphical representations and / or model representations of the manipulator program; • A wait and / or synchronization instruction of the manipulator program; • Repetitions and / or branches of the manipulator program. [9] Computer program product according to one of the preceding claims, wherein the graphical user interface (100) comprises at least one of the following buttons: a button (160) by means of which the manipulator program can be stopped; a button (161) by means of which a new starting point can be defined at a selected location displayed on the program progress display and / or by means of which a new path can be defined; a button (162) by means of which the manipulator program can be continued from a selected point displayed on the program progress display; a button (163) by means of which starting points can be selected which lie between the currently selected position of the program progress display and the position at which a stop of the manipulator system has occurred; a button (164) by means of which individual operations of the manipulator program can be undone and / or touchdown points (AP1 to AP5) can be selected, and / or a button (165) by means of which previously passed landing points can be selected. [10] Computer program product according to one of the preceding claims, wherein the graphical user interface (100) comprises at least one second program progress indicator, which program progress indicator displays the current program progress of a manipulator program of at least one second manipulator. [11] Computer program product according to one of the preceding claims, wherein the program progress indicator (150) is associated with a time scale (151), wherein the time scale (151) is zoomable. [12] Computer program product according to one of the preceding claims, wherein the program progress indicator (150) displays and preferably hides waiting times of the manipulator system (1) in a condensed time scale. [13] Computer program product according to one of the preceding claims, wherein the program progress indicator (150) is a program progress bar. [14] A method for navigating through a manipulator program by means of a computer program product according to any one of the preceding claims, the method comprising: • Providing a manipulator program; • Creating the graphical user interface (100); • Stop the manipulator program; • Receiving a selection of a touchdown point (AP1 to AP5) on the program progress display (150) of the graphical user interface, and guiding the manipulator program to the selected touchdown point (AP1 to AP5) so that the manipulator program controls the manipulator system (1) to assume a system state associated with the selected touchdown point (AP1 to AP5), the method further comprising: • Automatic determination of a path from a current point in the manipulator program to the selected starting point (AP1 to AP5) and, if automatic determination is not possible, • Open an input dialog that allows you to manually specify a path. [15] Method according to claim 14, wherein when guiding the manipulator program to the selected touchdown point (AP1 to AP5), it is possible to wait for an interaction of an operator with the manipulator system (1), and preferably the interaction is displayed for an operator. [16] Control device (20) for a manipulator system (1), comprising a computer program product according to one of claims 1 to 13, wherein the graphical user interface (100) is preferably generated on a handheld display device (10), such as a tablet computer and / or a smartphone.

Images