HANDHELD DEVICE FOR TRAINING AT LEAST ONE MOVEMENT AND AT LEAST ONE ACTIVITY OF A MACHINE, SYSTEM AND METHOD

Description

[0001] Various embodiments relate to a handheld device for training at least one movement and at least one activity of a machine, a corresponding system and a corresponding method.

[0002] Both the programming of an industrial robot and the programming of the associated system control and / or tooling (also referred to as "tooling") are traditionally manufacturer- and robot-dependent. Programming is typically performed in the form of program code by one or more specially trained experts. This currently still applies to more than 96% of applications. A programmer manually writes the program code that allows the robot to perform the task autonomously. Therefore, programming is complex and expensive, especially for path-based or point-based applications (e.g., welding, bonding, painting).

[0003] The costs make automation using an industrial robot (also simply referred to as a robot) economically unattractive for small and medium-sized companies, as they typically don't have high-volume production with low production variability that could offset the costs. The same applies to other types of robots. For large companies, however, the limited flexibility of programming can be unattractive. Programming retooling is time-consuming, resulting in shorter and uneconomical production cycles.

[0004] The complexity of programming increases due to the integration of the industrial robot with its diverse components, such as an end effector (e.g. a glue gun), a sensor system (e.g. a camera) and a control system (e.g. a programmable logic controller - PLC).

[0005] An industrial robot can be programmed alternatively or additionally by an expert using CAD-based code generation. This involves creating a virtual representation of reality (also known as the virtual world), and programming the robot in the virtual world. This not only enables simulation but also makes it easier to access. However, this CAD-based code generation cannot easily be implemented by a non-technical person. Furthermore, the virtual world often deviates significantly from reality. Even small deviations can lead to significant discrepancies in the robot's work in reality. For this reason, the program code generated using code generation is usually also adapted by a programmer.

[0006] As an alternative to completely manual programming, a learning process (also known as training) is conventionally used.

[0007] For the training process, the robot can be controlled manually, for example. A sensitive robot (also known as a co-bot), for example, can also be hand-guided. With both mechanisms, the trajectory (i.e., the path along which the robot is to move) can be demonstrated. However, activities that the robot is to perform beyond the trajectory remain complex and are therefore traditionally not taken into account by the training process. This complexity lies, for example, in the integration of the various components of the robot, such as the end effector, the sensors, and the control system, into the process to be performed, which must therefore be programmed manually.

[0008] The training process can be performed alternatively or additionally via an interactive input device. Typically, a manufacturer-specific input device, such as a 6D mouse, is used for this purpose. Similar to manual or hand-held control, only the trajectory can be trained in this case. The integration of the various components of the robot is therefore performed manually via programming.

[0009] The learning process can be performed alternatively or additionally using sensor data processing. Various extensions for the end effector of a robot equipped for this purpose are available, which integrate a sensor system (e.g., a camera) directly into the robot controller. Due to technical limitations, this is currently only applicable to assembly applications (also known as pick-and-place applications).

[0010] In general, there is always some manual programming involved. These conventional methods have in common that implementation cannot be fully carried out by a non-technical person if the manual programming portion exceeds their capabilities. This is because the overall application is an interplay of diverse sub-problems (such as the trajectory, end-effector control, sensor data processing, and integration into the process control system). Simple teaching methods therefore focus on specifying the trajectory or recording path points. The teaching method with sensor data processing is based on sensors attached directly to the robot. However, the field of vision is often restricted by the end-effector and robot. In addition, changing lighting conditions or air particles (e.g., during painting) impair the sensors on the robot.

[0011] ES 2 668 930 A1 discloses the programming of a robot using a handheld device.

[0012] According to various embodiments, a handheld device for training at least one movement and at least one activity of a machine, a corresponding system and a corresponding method are provided which facilitate the automation of a process flow (e.g., one or more than one activity thereof).

[0013] According to various embodiments, handheld devices according to claims 1 and 14 are provided.

[0014] It shows Figure 1 shows a handheld device according to various embodiments in a schematic side view or cross-sectional view; Figures 2, 5, 7 to 9 each show a system according to various embodiments in various schematic views; Figures 3 and 4 each show a method according to various embodiments in various schematic views; Figure 6 shows a machine according to various embodiments in a schematic structure diagram; Figure 10 shows a handheld device according to various embodiments in a schematic structure diagram; Figure 11 shows a method according to various embodiments in a schematic flow diagram; Figure 12 shows a system in a method according to various embodiments in a communication diagram; and Figure 13 shows a trajectory determination mechanism of the system in a schematic communication diagram.

[0015] In the following detailed description, reference is made to the accompanying drawings, which form a part hereof, and in which is shown by way of illustration specific embodiments in which the invention may be practiced. In this regard, directional terminology such as "top," "bottom," "front," "back," "fore," "rear," etc., will be used with reference to the orientation of the described figure(s). Since components of embodiments may be positioned in a number of different orientations, the directional terminology is for purposes of illustration and is in no way limiting. It is to be understood that other embodiments may be utilized and structural or logical changes may be made without departing from the scope of the present invention.It is understood that the features of the various exemplary embodiments described herein may be combined with one another unless specifically stated otherwise. The following detailed description is therefore not to be taken in a limiting sense, and the scope of the present invention is defined by the appended claims.

[0016] Throughout this description, the terms "connected," "attached," and "coupled" are used to describe both a direct and an indirect connection (e.g., resistive and / or electrically conductive, e.g., an electrically conductive connection), a direct or indirect connection, and a direct or indirect coupling. In the figures, identical or similar elements are provided with identical reference numerals where appropriate.

[0017] According to various embodiments, the term "coupled" or "coupling" can be understood in the sense of a (e.g., mechanical, hydrostatic, thermal, and / or electrical), e.g., direct or indirect, connection and / or interaction. For example, several elements can be coupled to one another along an interaction chain, along which the interaction (e.g., a signal) can be transmitted. For example, two coupled elements can exchange an interaction with one another, e.g., a mechanical, hydrostatic, thermal, and / or electrical interaction. According to various embodiments, "coupled" can be understood in the sense of a mechanical (e.g., physical) coupling, e.g., by means of direct physical contact. A coupling can be configured to transmit a mechanical interaction (e.g., force, torque, etc.).

[0018] A network described herein can, for example, differentiated by range, comprise or be formed from a local area network (for example a local area network (LAN), a wireless LAN (WLAN), or a personal area network (PAN), such as a wireless PAN (WPAN), such as a Bluetooth network) or a non-local area network (such as a metropolitan area network (MAN), a wide area network (WAN) or a global area network (GAN)). The network can, for example, differentiated by transmission type, comprise or be formed from a radio network (e.g. a cellular network) or a wired network. The network can, for example, also comprise or be formed from a cellular radio network (e.g. a WLAN of the IEEE 802.11 type in ad hoc mode, a Bluetooth network or another cellular cellular network). The network can also comprise or be formed from a plurality of interconnected sub-networks of different types.

[0019] According to various embodiments, the transmission of information (information transfer) can take place according to a communication protocol (CP). The information transfer can involve generating and / or transmitting a message containing the information according to the communication protocol. The communication protocol can clearly describe an agreement according to which the information transfer takes place between two or more parties. In its simplest form, the communication protocol can be defined as a set of rules that determine the syntax, semantics, and synchronization of the information transfer. The communication protocol(s) used (e.g., one or more network protocols) can, in principle, be selected arbitrarily and can (but do not have to) be configured according to the OSI (Open System Interconnect) reference model.Any protocols can also be used in the respective protocol layers. For example, protocols according to Bluetooth or other radio-based communication protocols can be used. Thus, the transmission of information via Bluetooth can include generating and / or transmitting a message containing the information according to a Bluetooth communication protocol stack. The Bluetooth communication protocol stacks can optionally be configured according to a low-energy communication protocol stack, meaning the information can be transmitted via low-energy Bluetooth.

[0020] In the following, various steps and details of a method are described. It can be understood that the description (e.g. individual steps of the method) can be implemented analogously using hardware (such as .a hard-wired circuit) and / or software (e.g. code segments or an entire application). For example, an application (also referred to as a program) can be or will be provided which has corresponding code segments (e.g. program code) and which can be or will be executed on a processor and / or by means of a circuit which has the processor. The processor (or the circuit) can, for example, be part of a mobile radio device or a computing device. The computing device can, for example, have a plurality of processors which are arranged centrally within a physically connected network or which are connected to one another in a decentralized manner by means of a network (e.g. cellular or wired).In the same way, code segments or the application can be executed on the same processor or parts of it can be distributed across multiple processors that communicate with each other via the network (for example, cellular or wired).

[0021] The term "processor" can be understood as any type of entity that allows the processing of data or signals. The data or signals can, for example, be processed according to at least one (i.e., one or more) specific function performed by the processor. A processor can include or be formed from an analog circuit, a digital circuit, a mixed-signal circuit, a logic circuit, a microprocessor, a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), a programmable gate array (FPGA), an integrated circuit, or any combination thereof.Any other type of implementation of the respective functions, which are described in more detail below, can also be understood as a processor or logic circuit, for example virtual processors (or a virtual machine) or a plurality of decentralized processors, which are connected to one another, for example by means of a network, are spatially distributed as desired and / or have any share in the implementation of the respective functions (e.g. distribution of the computing load among the processors). The same generally applies to differently implemented logic for implementing the respective functions. It is understood that one or more of the method steps described in detail herein can be carried out (e.g. realized) by a processor, by one or more specific functions carried out by the processor.

[0022] The term "system" can be understood as a set of interacting entities. The set of interacting entities can, for example, comprise or be formed from at least one mechanical component, at least one electromechanical transducer (or other types of actuators), at least one electrical component, at least one instruction (e.g., encoded in a storage medium), and / or at least one control device. Multiple communicatively interconnected entities of the system can, for example, be managed by means of a common system management system. For example, the entities (e.g., the handset and / or a handset-external device) of the system can be or become registered in the system, e.g., by means of the system management system.

[0023] The term "actuator" (also referred to as actuator) can be understood as a component configured to influence a mechanism or process in response to a control signal. The actuator can convert instructions issued by the control device (the so-called control signal) into mechanical movements or changes in physical quantities such as pressure or temperature. The actuator, e.g., an electromechanical converter, can, for example, be configured to convert electrical energy into mechanical energy (e.g., through movement) in response to a control signal.

[0024] The term "control device" can be understood as any type of logic-implementing entity, which can, for example, have a circuit and / or a processor that can execute software stored in a storage medium, in firmware, or in a combination thereof, and can output instructions based thereon. The control device can, for example, be configured using code segments (e.g., software) to control the operation of a system (e.g., its operating point), e.g., a machine or a system, e.g., at least its kinematic chain. The operating point can describe the point in the characteristic map or on the characteristic curve of a technical device that is assumed due to the system properties and the external influences and parameters of the device. The operating point can vividly describe the operating state (i.e., actual state) of the device.The working point must be distinguished from the working location (i.e. the spatial location at which, for example, the effect of the machine occurs).

[0025] Open-loop control can be understood as the intentional influencing of a system. The state of the system can be changed according to a specified value using an actuator. Closed-loop control can be understood as open-loop control, whereby a change in the state of the system caused by disturbances is also counteracted. To illustrate this, the open-loop control can have a forward-facing control path and thus implement a sequential control that converts an input variable into an output variable. However, the control path can also be part of a control loop, thus implementing closed-loop control. In contrast to pure forward-facing control, closed-loop control has a continuous influence of the output variable on the input variable, which is brought about by the control loop (feedback).

[0026] A process activity (also referred to simply as an activity) can be understood as the sum of all operations (e.g. a temporal sequence of controlled events) that fulfill a predefined process task. The process activity can optionally be broken down into sub-processes, each of which takes into account at least one of the operations. A sub-process of the process activity can fulfill a sub-task (i.e. a part of the process task) or the entire process task. Depending on the type of process activity, several sub-processes can be interconnected and / or build on one another, e.g. occur in a strict sequence, and / or be independent of one another, e.g. be interchangeable.

[0027] To illustrate this, for each of the sub-processes, a process (e.g. a grinding process) and a location of the process can be taken into account, which together describe the sub-process. This makes it possible to take into account which action is to be carried out at which point on the workpiece (the so-called work location), e.g. where the workpiece is to be machined and in what way. To fulfil the process task, the machine can be moved, for example by adjusting the machine's kinematic chain and / or relocating the machine. Smaller movements of the machine can also be carried out, for example, in which the position of the end effector and / or the robot arm itself remains stationary and only the machine tool is moved. For example, a drill as the machine tool can be guided along the progressive material removal at the work location so that the machine drill is moved into the drill hole.

[0028] For this purpose, at least one vector of the process (e.g., a grinding process) can be assigned to a sub-process. The same can apply analogously to a large number of sub-processes or multiple processes per sub-process. A vector can generally be understood as an element of a vector space, which does not necessarily have to be only spatial or only three-dimensional. The vector can generally have associated parameters of a parameter set of the process activity, e.g., process parameters, location parameters, or input data. The parameters of the parameter set can, however, also be formulated other than by means of a vector. More generally, what has been described for the vector (or its components) can apply analogously to a more generally formulated parameter set.

[0029] The vector can define at least one position, its change, a spatial distribution and / or at least one direction of the process. The spatial details (e.g. about the process activity) described here in a simplified manner using a vector can also be more complex or detailed and are referred to more generally herein as spatial information. Optionally, the spatial information can be assigned temporal information (e.g. about the process activity), which defines, for example, the duration, the start, the end and / or a cycle of the process. In a simple example, the temporal information can also only order the processes chronologically, if this is desired.

[0030] If the process activity is carried out by a hand-held tool, the process activity can describe the sum of work operations, the spatial information the spatial sequence of work operations and the optional temporal information the chronology of work operations that are carried out using the tool in order to provide the corresponding effect on a workpiece to achieve the process task. The corresponding spatial information can describe where (i.e. with which spatial distribution) and / or in which direction the effect is to be provided, i.e. in which corresponding spatial location (i.e. position and / or orientation) the tool is located.

[0031] This information about the process activity (also referred to as process information), i.e. the type of activity, the spatial information and the optional temporal information, can also be provided and / or recorded using the handheld device described herein. For this purpose, an interchangeable attachment is coupled to the handheld device, which is configured according to the type of activity. The handheld device can be hand-held (i.e. manually guided by a person) in a similar way to the work device, as if the work process were actually taking place using the interchangeable attachment. In contrast to the work device, however, the handheld device is equipped with electrical components (e.g. sensors, interfaces, transmitters, etc.) that enable the process information to be recorded and provided as training data.The handling of the handheld device can reflect how the tool is guided and / or operated during a process, e.g., how it is held, how hard it is pressed, and / or how long a work process is performed. The training data can be recorded, for example, using a local computer system or remotely (e.g., in a cloud).

[0032] A model can be understood as a data-based (e.g., digital and / or virtual) representation of an original, e.g., a physical object (e.g., a machine) or a process (e.g., a control process or a process sequence). To create the model (the so-called model building, i.e., the mapping of the original to the model), the original can be abstracted, parameterized, and / or simplified. The model can, for example, contain physical information (e.g., length, distance, weight, volume, composition, etc.), movement-related information (e.g., position, orientation, direction of movement, acceleration, speed of movement, etc.), logical information (links, sequence, couplings, interrelationships, dependencies, etc.), time-related information (e.g., time, total duration, frequency, period, etc.), and / or functional information (e.g.,current, effect, characteristic field or characteristic curve, operating point space, force, degree of freedom, etc.) over the original.

[0033] A control model can accordingly refer to a formal representation of an automated control system. The control model can have a plurality of control instructions (e.g., to bring the machine to an operating point) and also criteria, the fulfillment of which triggers, ends, or maintains the instruction assigned to these. Furthermore, the control model can have control logic that logically links several criteria and / or several instructions and / or that implements a sequence (e.g., a sequence plan) according to which the control takes place. On the path to the control model, a machine-type-specific model can optionally be determined as an intermediate step, which represents a machine type (i.e., a type of identical machines).When mapping the machine type-specific model to the control model, type-specific deviations between machines of a type can be taken into account (also known as delta formation).

[0034] In an analogous manner, a process model can refer to a formal representation of a process flow. The process model can have a plurality of links between a process activity and the corresponding spatial information and can optionally assign corresponding process situations to the process activities, which, for example, are present during the process activity, condition it, or terminate it. Furthermore, the process model can have process logic that logically links several process situations and / or several sub-processes and / or which implements a process (e.g., a flow chart) according to which the process activity takes place. In general, a flow chart can have at least branches, jumps, and / or loops. The presence or absence of a process situation can generally be represented by means of at least one criterion, which is met, for example, when the process situation is present or absent.

[0035] Mapping can involve transferring elements of an initial set (also called an preimage) into a target set, with the elements of the target set then being the image (mapping) of the preimage. Mapping can assign at least one element of the mapping to each element of the preimage. The preimage does not necessarily have to contain all available elements, but can also be an application-specific selection from them. Mapping can involve applying operators, transformations, and / or operations to the elements of the initial set, for example. The elements can generally include: logical relationships, operations, information, properties, coordinates or the associated coordinate system, mathematical objects (such as formulas or numbers), processes, activities, etc.

[0036] A code generator can be understood as a computer program designed to convert a model, e.g., in a modeling language, into a programming language, e.g., the programming language of the machine's control device. Alternatively or in addition to the modeling language, e.g., a unified modeling language (UML), the model can also be in a markup language, a structure diagram, a decision table, or another formal language. The code generator creates code segments (also referred to as code generation) that can be combined with other optional program components to form a program.

[0037] Spatial position can be understood here as spatial information about the orientation and / or position of an object. The position can clearly describe the location (e.g. a point) in space and the orientation the respective orientation (e.g. a direction) of an object relative to the space. A trajectory can be understood as a series of spatial positions that are successively assumed by an object. The spatial position can optionally be time-dependent (i.e. movement-related, then also referred to as movement), according to a timing or speed, so that movement along the trajectory is taken into account. Analogously, the movement can optionally be time-dependent, so that acceleration along the trajectory is taken into account. In general, the spatial position or other spatial information in three-dimensional space can be described using Cartesian coordinates.However, any other coordinate system can also be used, e.g. cylindrical coordinates or the so-called joint space (also referred to as the machine-specific coordinate system), which is suitable for clearly describing the spatial information. The machine-specific coordinate system can be related to the kinematic chain of the machine and have a dimension for each degree of freedom of the kinematic chain. This makes it possible to represent not only the position of the end effector in space but also the state of each link (e.g. joint or other connecting link) of the kinematic chain, thus enabling more precise control of the machine. The machine-specific coordinate system can have a multitude of nested coordinate systems, each of which contains the degrees of freedom (e.g.up to 6 degrees of freedom, having up to 3 rotational degrees of freedom and / or up to 3 translational degrees of freedom) of a link in the kinematic chain are taken into account.

[0038] According to various embodiments, a handheld device (for training at least one movement and at least one activity) of a machine, a corresponding system and a corresponding method are provided, which facilitate the teaching (also referred to as training) of a machine. For example, a fully integrated program code can be generated which, when executed by a processor of the machine, is set up to autonomously carry out the activity by means of the machine. For example, training (up to the generation of the program code) can be carried out particularly quickly, e.g. by demonstrating the activity manually (i.e. by a person) using the handheld device. For example, it is possible to switch between different activities quickly and easily, e.g. by exchanging the interchangeable attachment coupled to the handheld device or by switching between the tools of an interchangeable attachment.For example, different types of machines can be trained using the same system, e.g., by determining a platform-independent process model as an intermediate step. For example, training can be particularly cost-effective, e.g., because fewer programmers are required to create the program code. For example, different interchangeable attachments can be assigned to the same activity. However, multiple activities can also be assigned to exactly one interchangeable attachment.

[0039] According to various embodiments, the handheld device can enable the recording of data representing the movement and / or at least one activity of the machine freely in space, e.g., without a (e.g., physical or virtual) fixed point and / or supporting axes. For example, the handheld device can be moved freely in space in order to perform the activity to be taught, e.g., on a workpiece. This can be achieved, for example, by the handheld device being free from suspension (i.e., support) on one or more physical axes.

[0040] This allows for an increase in the training space in which the handheld device can be moved for training, so that, for example, there is little or no restriction of the training space (at least not by the handheld device itself). For example, the handheld device can be handled overhead, lying down, or in other positions by a person (e.g., with one or two hands). This makes it easier to handle, for example, in built-up work spaces.

[0041] For example, the handset can be moved independently of the machine, meaning it can be free from any coupling with the machine (e.g., not coupled to it). A communicative coupling between the machine and the handset can be optionally provided, e.g., via a replaceable cable or wirelessly. Alternatively or additionally, the handset can be free from any articulated coupling or other bearing that is coupled to the machine, for example.

[0042] The recording of data representing the movement and / or at least one activity of the machine can, for example, be carried out in a coordinate system that is independent of the machine that is to be trained. The handheld device and / or its movement or the recording of the data can, for example, be independent of the machine (e.g., its workspace, its reference point or machine coordinate system, and / or its base). The reference point (also referred to as the tool center point, TCP) can, for example, be a fixed coordinate in the machine coordinate system.

[0043] The system can be used to determine trajectories with any number of waypoints. For each waypoint, one or more of the following properties can be parameterized: a type of movement (point-to-point, linear, circular, and polynomial), the trajectory, a speed or its change (i.e., acceleration), a force (e.g., its moment, i.e., a torque), one or more pieces of information about the tool, and / or a blending radius. Alternatively or additionally, one or more of the following properties can be parameterized for each waypoint: a functional state of the interchangeable attachment and / or sensor data (e.g., of the interchangeable attachment and / or the handset).

[0044] For example, the trajectory can be or become piecewise composed of polynomials. The resulting polynomial train can, for example, consist of polynomials of at most nth degree, so that an nth degree polynomial train (also called an nth degree spline) is provided. The points of the trajectory where two polynomials adjoin each other (the so-called nodes) can optionally be (n-1) times continuously differentiable. For example, using the polynomial train, a point-based trajectory (whose points define the nodes) can be converted into a path-based trajectory by searching for polynomials that connect the points of the point-based trajectory.

[0045] Different tool and / or workpiece coordinate systems can be configured. The trajectory can be described, for example, by specifying the successive waypoints (also referred to as a point-based trajectory), by specifying the traversed path curve (also referred to as a path-based trajectory), or by a combination of these specifications.

[0046] The point-based trajectory, for example, can leave open the path taken by the waypoints one after the other, meaning that the determination of the specific trajectory connecting the waypoints can be left to the control device or the capabilities of the machine. The point-based trajectory is suitable, for example, when the activity involves discrete work steps at discrete locations (e.g., pick-and-place).

[0047] The path-based trajectory enables a unique waypoint-time assignment and thus describes in detail which waypoint is occupied at which time. The path-based trajectory is suitable, for example, when the activity is to be carried out along a specific path or at a specific speed (e.g., spreading glue or painting).

[0048] According to various embodiments, the machine is trained using the training data. For this purpose, the training data can be recorded. For example, the recording of the training data can be started by entering the activation information. In principle, however, the recording of data can also take place before the actual training, e.g. as soon as the handset is switched on and / or registered in the system. In this case, the activation information can specify which part of the recorded data is used as training data. In other words, the recorded data that is used as training data can be filtered out using the activation information (e.g. time-resolved). For this purpose, the activation information can, for example, have a timestamp that is compared with the time-dependent recording of the data.Alternatively, the recorded data can also be filtered after recording has ended, e.g., manually using a graphical representation of the recorded data. In this case, the user can mark the data that should be fed into the machine as training data using the activation information (e.g., the beginning and end of a data sequence). Similarly, additional data can be added to the existing training data, or parts can be removed from the existing training data. Filtering can be performed, for example, using a device external to the handheld device.

[0049] Filtering out the training data (also referred to as filtering for short) can, for example, involve playing back the entire recorded movement sequence using the device external to the handheld device and examining its movements. The recorded movement sequence can be played back using the physical robot and / or a virtual representation of the robot. The recorded data can, for example, be played back on the robot once or more than once using remote access (e.g., an application program such as an iOS application), and the movements can be examined. Individual points in space, so-called keyframes, which are to be retained as training data, can be marked in the data and fine-tuned if necessary, e.g., using functions in the various coordinate systems.Additionally or alternatively, machine learning and mathematical optimization can be used to predict and filter selectable points and / or all or parts of the entire data used for training. Filtering may, for example, involve considering anomalies, movement quality and expressiveness, redundant information, and / or data reduction.

[0050] Additionally or alternatively, the robot's movement mode (also referred to as the motion type) between two points can be defined, for example, point-to-point, linear, or semicircular movement. Filtering can be repeated one or more times using user-specified information, e.g., until the data to be used for training is determined.

[0051] The sum of all marked points (keyframes) of the recorded motion sequence and its movement type can be used as training data, or at least as part of it. Optionally, the marked points can be Cartesian-spatially translated and / or the robot's end effector can be rotated around such a point.

[0052] Examples of filtering or filtering components can include: detecting and removing anomalies from the training data, smoothing the training data (e.g. to improve the quality of movement and its expressiveness), detecting and removing redundant parts of the training data (which are, for example, identical or at least offer little added value), data reduction while preserving the content.

[0053] Alternatively or in addition to filtering, parts of the training data can be grouped, e.g. as a representation of individual operations or activities of the more complex overall process.

[0054] Fig.1 illustrates a handheld device 100 according to various embodiments in a schematic side view or cross-sectional view. The handheld device 100 can have a longitudinal extension 101 (extension in the longitudinal direction 711a), which is, for example, in a range from approximately 10 cm (centimeters) to approximately 50 cm, e.g., in a range from approximately 15 cm to approximately 30 cm. Along the longitudinal direction 711a, the handheld device 100 can be delimited by two end faces 101a, 101b, of which a first end face 101a has the coupling structure 102. The coupling structure 102 (e.g., a coupling mechanism) can be configured for the releasable coupling of an interchangeable attachment, as will be described in more detail later.

[0055] The handle 104 can extend away from the coupling structure 102 along the longitudinal direction 711a. The handle 104 can, for example, be ergonomically shaped, e.g., rounded, tapered, and / or otherwise adapted to the shape of a hand. In general, the handle 104 can be configured to be grasped and / or held by one or more than one hand of a user. For example, various embodiments do not necessarily have to have a separate handle (extra handle), but can have one integrated. For this purpose, the handle 104 can, for example, have a longitudinal extension 101 that is, for example, in a range from approximately 5 cm (centimeters) to approximately 30 cm, e.g., in a range from approximately 10 cm to approximately 20 cm. Alternatively or additionally, the handle 104 can have a circumference (e.g.,along a self-contained path extending transversely to the longitudinal direction 711a), which is in a range of approximately 10 cm to approximately 30 cm.

[0056] The handle 104 can also be oriented differently relative to the coupling structure 102, e.g., extending obliquely to the coupling direction so that the handset 100 is angled. This can be more intuitive for some activities. What is described herein for the elongated handset 100 can of course apply analogously to a different configuration or orientation of the handle 104. Likewise, the handset 100 can have multiple handles (e.g., extending away from or across from each other) that enable secure handling of the handset 100. Analogously, the or each handle can have multiple grip points for multiple hands. One or more than one of the handles can be the handle of the handset. One or more than one of the handles can be detachably attached to the handset, for example, as a component of the interchangeable attachment or provided as a separate handle.One or more than one of the handles of the handset may be persistent or non-persistent.

[0057] Optionally, the handheld device 100 may have one or more user interfaces (e.g., user input interface and / or user output interface). Examples of the one or more user interfaces of the handheld device 100 may include a screen (also referred to as a display device), a physical keyboard, a speaker, a vibration device, one or more physical switches (e.g., buttons), one or more light sources, or the like. Alternatively or additionally, the handheld device 100 may have one or more sensors, as will be described in more detail later. The vibration device may generally include a vibration exciter, e.g., a membrane or a piezo element. The vibration exciter may be configured to convert an electrical signal coupled thereto into a mechanical vibration that can be detected haptically (also referred to as vibration) (e.g.,the frequency of the signal).

[0058] The input unit 108 can be provided, for example, by means of a user interface (then also referred to as a user input interface) which is configured for actuation by a user, so that the activation information can be entered by actuating the user interface. The activation information can generally be detected by means of a sensor, e.g., by means of a touch-sensitive sensor (which a touch-sensitive input unit 108 provides). Alternatively or additionally, the input unit can also have another sensor which can detect actuation of the input unit. The sensor of the input unit 108 can, for example, be a proximity sensor, a touch sensor, a physical switch (e.g., a button), or the like.

[0059] The input unit 108 can, for example, be part of the handle 104 or be adjacent to it. This facilitates operation.

[0060] The activation information can generally trigger the activation of the training of the machine. In a less complex example, the training can be activated when a user input (e.g., comprising a force and / or a touch) is detected by the input unit 108. For example, the switch can be actuated by means of the force, so that the training is activated. Similarly, the training can be deactivated, for example, when it is detected that another user input is given or the user input is interrupted. However, more complex user inputs can also be implemented. For example, a first input sequence (e.g., a long or repeated key press) can activate the training and / or a second input sequence can deactivate the training.

[0061] The input unit 108 enables, for example, the recording of information about the activity to be started and / or ended. However, the functions provided by the input unit 108 can also be provided in whole or in part by another device external to the handset, e.g., by a wireless switch and / or by a mobile computing device (e.g., a tablet) on which, for example, an application is executed that emulates the functions of the input unit 108.

[0062] More generally, a first activation information item input via input unit 108 may represent that the training should be activated and / or continued. Alternatively or additionally, a second activation information item input via the input unit may represent that the training should be deactivated and / or interrupted.

[0063] The input unit 108 can optionally be configured according to a near-field communication protocol, according to which the activation information is input. For example, NFC can be used to input the activation information.

[0064] The activation information can be output (e.g., communicated) by means of the output unit 110. The output unit 110 can, for example, be configured to output the activation information according to a communication protocol, e.g., according to a network communication protocol. The activation information can, for example, be transmitted wired (also referred to as cable-based) or wirelessly. In principle, any communication protocols can be used, standardized or proprietary. For ease of understanding, reference is made below to wireless communication of the output unit 110 (e.g., by radio). However, what is described can also apply analogously to wired communication of the output unit 110.

[0065] The output unit 110 may, for example, comprise a data transmission interface, e.g., a signal generator and an antenna. The signal generator may, for example, be configured to encode the activation information, e.g., according to the communication protocol, and supply a signal to the antenna in accordance with the encoded activation information.

[0066] The handle 104, the input unit 108, and / or the coupling structure 102 may, for example, be part of a housing 136 of the handset 100 or be held by the housing 136. The housing 136 may comprise a hollow body in which one or more electrical components of the handset 100 are arranged, e.g., the output unit 110, at least one sensor, a battery, etc.

[0067] To make the handset 100 easier to handle (e.g., move), the handset 100 can be lightweight, e.g., weighing less than approximately 5 kg (kilograms), e.g., less than approximately 2.5 kg, e.g., less than approximately 1 kg, e.g., less than approximately 0.5 kg. Alternatively or additionally, the handset 100 can be made of a lightweight material. For example, a housing 136 of the handset 100 can comprise or be formed from a plastic and / or light metal.

[0068] In general, the coupling structure 102 does not necessarily have to be arranged at the front. For example, the coupling structure 102 can also be arranged at any position of the end portion of the handset 100 that extends away from the handle 104, e.g., even laterally. A front-facing coupling structure 102 can be more intuitive and easier to use. A side-facing coupling structure 102 can enable a more complex coupling mechanism.

[0069] For example, several magnets can be arranged along an annular path on the outside of the end section.

[0070] For ease of understanding, reference is made below to a front-end coupling structure 102. The description may apply analogously to a differently arranged coupling structure 102, e.g., to a laterally arranged coupling structure 102.

[0071] Fig.2 illustrates a system 200 according to various embodiments in a schematic side view or cross-sectional view, wherein the system 200 comprises the handset 100 and an interchangeable attachment 210 which is releasably coupled to the coupling structure 102.

[0072] Detachable coupling can be understood as meaning that the interchangeable attachment can be non-destructively attached to and / or detached from the handset 200 by means of the coupling structure 102, for example, repeatedly and / or without the need for tools. For example, two interchangeable attachments can be interchanged.

[0073] For this purpose, the coupling structure 102 and the or each interchangeable attachment 210 can be configured to correspond to one another, so that, when brought into physical contact with one another, they can be connected to one another, e.g., by means of a relative movement of the two. In a similar manner, they can be separated again, so that the interchangeable attachment 210 can be moved away from the coupling structure 102 or replaced.

[0074] For example, the or each interchangeable attachment 210 can have a mating coupling structure 212 corresponding to the coupling structure 102, which can be connected to one another. For example, the connection can be achieved by means of a positive connection and / or a frictional connection. If the coupling structure 102 has a bayonet or thread, this can provide a positive connection. If the coupling structure 102 has an insertion section, this can provide a frictional connection by means of interlocking. This is cost-effective to implement and easy to handle.

[0075] For example, the connection can be achieved by means of a magnetic field, e.g., by the coupling structure 102 and / or the interchangeable attachment 210 having a magnet that provides the magnetic field. The magnet can be a permanent magnet or an electromagnet. The magnetic field can provide an attractive force between the coupling structure 102 and the interchangeable attachment 210, e.g., by means of a ferromagnetic material of the interchangeable attachment 210.

[0076] The interchangeable attachment 210 can be configured according to the activity to be trained. For example, the interchangeable attachment 210 can have a tool 214 for performing the activity. The tool 214 can generally represent a function corresponding to the activity, by means of which a workpiece is acted upon. The tool 214 can, for example, represent a primary forming tool, an inspection tool, a joining tool (e.g., a screwdriver, glue gun, or welding device), a relocation tool (e.g., a gripper), a cutting tool, or the like. The joining tool can, for example, have a coating tool (e.g., a paint spray gun, a powder coating gun), or be formed therefrom.However, the tool 214 of the interchangeable attachment 210 does not have to be functional or does not have to be fully functional, but it may be sufficient if the tool 214 of the interchangeable attachment 210 has the shape and / or contour of the real process tool and / or at least an image thereof (for example, analogous to a dummy element).

[0077] The system 200 can, for example, have several (e.g., at least 2, 3, 4, 5, 10, or at least 20) such interchangeable attachments 210, which differ from one another in their tool 214. Of the several interchangeable attachments 210, one interchangeable attachment can be coupled to the handheld device 100. If the tool is to be changed, a first interchangeable attachment 210 coupled to the handheld device, which has a first tool 214, can be exchanged for a second interchangeable attachment, which has a different second tool 214.

[0078] The system 200 may include additional components, as described in more detail later.

[0079] The pair of handheld device 100 and the interchangeable attachment 210 coupled thereto is also referred to below as the training device 302, and the tool of the interchangeable attachment 210 is referred to as the training tool 214. In general, an interchangeable attachment 210 can also have multiple training tools 214, which differ, for example, in the activity for which they are configured. The multiple training tools 214 of the interchangeable attachment 210 can, for example, be rotatably mounted on the counter-coupling structure 212.

[0080] For example, an electrical power of the or each training tool 214 may be smaller than that of the handheld device 100. This makes it easier to supply electrical power to the training tool 214 by means of the handheld device 100 and / or enables a passive training tool 214.

[0081] In various embodiments, the training tool 214 may be provided as a non-fully functional tool, for example.

[0082] For example, a weight of the or each training tool 214 may be smaller than that of the handheld device 100. This makes it easier to carry the training tool 214 by means of the handheld device 100 and simplifies its attachment.

[0083] A material of the or each training tool 214 can be, for example, plastic and / or light metal (e.g., aluminum). This reduces its weight.

[0084] A size (e.g., volume and / or extension along the longitudinal direction) of the or each training tool 214 may, for example, be smaller than that of the handheld device 100. This makes it easier to carry the training tool 214 using the handheld device 100 and simplifies its attachment. If less or no ease of carrying is acceptable, the training tool 214 may also be the same size as or larger than the handheld device 100.

[0085] The coupling structure 102 can facilitate the replacement of the interchangeable attachment 210, as will be described in more detail below. As an alternative to the coupling structure 102, the training tool 214 can also be permanently (i.e., non-detachably) attached to the handheld device 100 (so that it cannot be replaced non-destructively, without tools, i.e., without assembly). The training tool 214 can, for example, be fastened to the front side by means of screws or in some other way, e.g., welded or glued thereto. The training tool 214 can, for example, be embedded in the front side. Such a handheld device 100 with a permanently installed training tool 214 can then be configured to train precisely the one activity for which the training tool 214 is configured. What is described below for the handheld device 200 with the coupling structure 102 can also apply analogously to the handheld device with a permanently installed training tool 214.In various embodiments, the activity can be defined generally by the process task to be performed by it and not (e.g., only) by the specific operation for performing the process task. This allows, for example, variations in the performance of the same process task to be taken into account and thus allows more flexibility in designing the training of the machine. For example, the activity can also include variations or modifications of operations as well as similar or identical operations that perform the same process task.

[0086] Fig.3 illustrates a method 300 according to various embodiments in various schematic views.

[0087] Furthermore, in Fig.3 a machine 114 to be trained is shown. The machine 114 to be programmed can be a robot, e.g., an industrial robot for handling, assembling, or processing a workpiece, a collaborative robot (also referred to as a cobot), or a service robot (e.g., a cooking machine or a care machine). The method 300 enables, for example, end-user programming of the complete automation application (including process parameters and integration) by a technical layperson. The machine 114 to be trained can be a physical machine. What is described herein for this machine can also apply analogously to a virtual machine 114 to be trained.

[0088] For example, when using multiple physical machines 114 of the same type (e.g., by averaging the manufacturing inaccuracies of each of the machines), an idealized virtual model of the type (the machine-type-specific model) can be used and trained. Based on the trained virtual machine 114, the control information can be determined (illustratively, a mapping of the physical machine), e.g., in an automated manner.

[0089] The machine 114 to be trained can generally comprise a manipulator and a frame 114u on which the manipulator is supported. The term manipulator summarizes the entirety of the movable members 114v, 114g, 114w of the machine 114, the control of which enables physical interaction with the environment, e.g., to carry out a process activity. For control, the machine 114 can comprise a control device 702 (also referred to as machine controller 702), which is configured to implement the interaction with the environment according to a control program. The last member 114w of the manipulator (also referred to as end effector 114w) can comprise one or more than one tool 124w (also referred to as process tool 124w), which is configured according to the activity to be trained, e.g., to carry out the activity.The process tool 124w may, for example, comprise a welding torch, a gripping instrument, a glue gun, a painting device, or the like.

[0090] The manipulator may comprise at least one positioning device 114p, for example, a robot arm 114p (more generally also referred to as an articulated arm), to which the end effector 114w is attached. The robot arm 114p clearly provides a mechanical arm that can provide functions similar to or even beyond a human arm (e.g., multi-axis movements per joint or combined rotation and pivoting per joint). The robot arm 114p may, for example, have multiple (e.g., at least 2, 3, 4, 5, 10, or at least 20) joints, each of which can provide at least one (e.g., 2, 3, 4, 5, or 6) degree of freedom.

[0091] The manipulator can, of course, also be configured differently, e.g., in a gantry machine, the general type of multi-joint robot, or a delta robot. The machine can, for example, have an open or closed kinematic chain. In the following, reference is made to the more easily understood robot arm 114p. The description for the robot arm 114p can also apply analogously to a differently configured machine or manipulator.

[0092] The members of the positioning device 114p can, for example, be connecting members 114v and joint members 114g, wherein the connecting members 114v are interconnected by means of the joint members 114g. A joint member 114g can, for example, have one or more joints, each of which can provide the connected connecting members 114v with a rotational movement (i.e., turning movement) and / or a translational movement (i.e., displacement) relative to one another. The movement of the joint members 114g can be initiated by means of actuators controlled by the control device 702. One or more joints can also have or be formed from a ball joint.

[0093] At 301, a training device 302 (having the handheld device 100 and the interchangeable attachment 210 coupled thereto) can be provided. For training (for example, as soon as or as long as the training is activated), a person 106 (also referred to as the user) can demonstrate an activity for completing the process task using the training device 302 (e.g., painting, manufacturing, and / or assembling a component). The training tool 214 can, for example, represent any process tool 124w that a machine 114 can guide. For this purpose, the process tool 124w can interact with the workpiece.

[0094] For training, the training device 302 can transmit data to a device external to the handheld device, e.g., at least the training data. The training data can include or be formed from the activation information and optionally spatial information. Optionally, the training data can include one or more of the following information: one or more calibration values, quality information, sensor data, (e.g., aggregated, fused, and / or optimized) information, activation information. Quality information can, for example, indicate the reliability of a training data point, for example, clearly showing how good the data point is. The quality information can be assigned to each training data point, e.g., by means of the device external to the handheld device. The activation information can, for example, relate to the end effector of the machine, e.g., its gripper system and / or welding device.

[0095] The spatial information can, for example, represent a position and / or its change (i.e., a movement) of the training device 302 in space. The activation information can represent an input at the input unit 108. The movement can, for example, comprise a translation and / or rotation of the training device 302 and can be determined, for example, by measuring the acceleration and / or the speed.

[0096] The device external to the handheld device receives, for example, the time-dependent position 111 (i.e., location and / or orientation) of the training device 302 or its coordinate system 711 in space 701, 703, 705 (e.g., within a building 511). Based on this, the time-dependent position 111 of the training tool 214 can be determined. A plurality of positions that an object (e.g., the handheld device 100, its coordinate system 711, and / or the training tool 210) can assume can be represented by means of a trajectory 111 (also referred to as a training trajectory). Each point of the trajectory 111 can optionally be assigned a time and / or an orientation of the object. Each point of the trajectory 111 and / or the orientation can be specified by means of corresponding coordinates. In an analogous manner, the trajectory 111 may alternatively or additionally be related to the coordinate system of the handheld device 100 and / or to a work location.The space 701, 703, 705 (also called working space), in which the trajectory 111 is specified, can be spanned by a coordinate system which is stationary, ie has an invariant position relative to the earth's surface.

[0097] Each point (e.g. vector) of the trajectory 111 can optionally be assigned one or more than one activity-specific and / or tool-specific process parameter, e.g. a flow rate, a flow rate, an intensity, an electrical power, a keystroke, etc.

[0098] For this purpose, for example, a locating device 112 can be stationary and define (e.g., span) the coordinate system 711. The locating device 112 can, for example, have one or more sensors and / or one or more emitters, as will be described in more detail later. The portion of the training data provided by the locating device 112 can be synchronized by means of a timestamp and related to the portion of the training data provided by the training device 302.

[0099] The training data can optionally include task-specific process parameters, as described in more detail later. Task-specific process parameters can represent the parameters of the respective function and / or the operating point of the process tool, e.g., the volume flow of the paint spray gun.

[0100] Based on the training data and the optional activity-specific process parameters, a model 104m of the process activity (also referred to as process model 104m) can be determined in 303. This process model 104m clearly describes the movement to be performed by the process tool 124w to complete the process task. The process model 104m can optionally be examined and adapted by a person 106.

[0101] In one example, the incoming training data includes time-based movement data from the training device 302 operated by the person 106 and activation data from the input device 108. The time sequence of the training data is then broken down into sub-processes (e.g., approaching the starting point of the trajectory, assuming the starting position, beginning the painting process, painting, completing the process, departing from the end point of the trajectory), for example, using reference points and / or task-specific analytical algorithms. Optionally, the user can manually post-process the training data. An instance of a process model 104m is then generated, e.g., in the form of a metamodel. The metamodel describes the data types of the model instance and their possible relationships. A model in this case is, for example, a directed graph with typed nodes.Nodes have a data type (metamodel node) that describes the model parameters and their value ranges. The model instance is generated based on the training data using, for example, artificial neural networks. Artificial neural networks (kNN) can be trained using conventional training methods, such as the so-called backpropagation method. Alternatively or additionally, the training data can be optimized using mathematical optimization and / or machine learning methods. During training, the training vectors are selected according to the desired input parameters, such as spatial coordinates of the training device 302 (or their temporal change), associated time information, inputs to the training device 302, which can represent, for example, operating points and / or control points of the process tool, spatial orientation of the training device 302, etc.It should be noted that both the parameters contained in the input vector of the kNN and the parameters contained in the output vector of the kNN are highly application-dependent or process-dependent and are selected accordingly.

[0102] Furthermore, a specific hardware platform 114 (more generally also referred to as machine 114) can be selected (e.g., a specific robot type or end effector, etc.). The machine specifics (e.g., structure) of the machine 114 can be taken into account by means of a model 114m of the machine 114. The model 114m of the machine 114 can include machine-specific information of one or more different machines 114. The machine-specific information can include machine-specific parameters, such as positioning and repeatability accuracies, maximum range of motion, speeds, acceleration, etc. Alternatively or additionally, the machine-specific information can represent at least the process tool 124w (also referred to as machine tool 124w), which is attached, for example, to the positioning device 114p of the machine 114.

[0103] Based on the model 114m of the machine 114 and the process model 104m, a platform-specific model 116m (also referred to as a control model 116m) for the machine controller 702 can be generated in 305. The control model 116m can contain the respective control information that controls the movement and / or activity of the machine 114. For example, the machine-specific control information (e.g., volume flow at the paint end effector and / or movement sequences) that correspond to the activity-specific process parameters can be determined.

[0104] However, the process model 104m does not necessarily have to be determined separately. Based on the training data and the optional activity-specific process parameters, the control model 116m can also be determined directly in 305, for example, by mapping the trajectory 111 of the training tool 214 onto a trajectory 113 of the process tool 124w.

[0105] In 307, a program code 116 (e.g., source code) can optionally be generated based on the control model 116m using a code generator 412. The program code 116 can designate the respective code in which the control program 116 is written. Depending on the process task, information technology infrastructure, and the specific requirements, various target platforms on which the program code 116 is to be executed can be used. The program code 116 can be generated for a communicating overall system (e.g., the robot controller and the PLC controller). The program code 116 can optionally have predefined parts to which the program code 116 can be adapted by a developer.

[0106] The formation of the program code 116 does not necessarily have to be carried out, as will be described in more detail later. The control information of the control model 116m can, for example, be implemented directly by the control device 702 of the machine 114.

[0107] The formation of the control model 116m and / or the process model 104m does not necessarily have to be carried out in an analogous manner. For example, processed training data can also be supplied to the control device 702 of the machine 114 as control information, which is then interpreted by the control device 702 of the machine 114 and converted into control signals for controlling the kinematic chain of the machine.

[0108] Code generation 107 can clearly demonstrate finding a transformation from the dependent model into the control language of a concrete manufacturer-specific machine control system. In one example, code generation 107 takes place in the form of templates that exist for a target language. These templates have instances of the platform-dependent model 116m as input and describe, at the metamodel level, how text fragments are generated from them. In addition to pure text output, these templates also have control structures (e.g., branches). A template engine, in turn, has a template and an instance of the platform-independent model as input and generates one or more text files from them, which can be added to the program code 116. It is understood that any other form of code generation 107 can also be used, e.g., without using templates and / or only based on mathematical mappings.

[0109] By means of the code generation 107, a control program 116 can be formed which can be executed by the corresponding machine 114.

[0110] Code generation 107 can be performed, for example, for a machine controller 702 and / or a PLC controller 702. Code generation 107 can, for example, generate human-readable code segments (i.e., source code) and / or machine-readable code segments (i.e., machine code). The source code can be generated for various target languages, e.g., depending on which target language is suitable for the respective machine. Optionally, the source code can be subsequently adapted and edited, e.g., by a developer.

[0111] The generation 305 of the control model 116m can thus be carried out on the basis of the training data and a model 114m of the machine 114.

[0112] Fig.4 illustrates the method 300 according to various embodiments in a schematic side view 400.

[0113] The method 300 may further comprise 401: calibrating 403 the system 200. The calibration may comprise performing a calibration sequence when the handheld device 100 is attached to the machine 114, e.g., to its manipulator. For this purpose, the system 200 may comprise a fastening device 402 by means of which the handheld device 100 can be detachably coupled to the machine 114. The fastening device 402 may, for example, be configured for magnetic fastening, for form-fitting and / or force-fitting fastening, e.g., by means of a clip, by means of a hook-and-loop fastener, or by means of another form-fitting element (e.g., screws).

[0114] The calibration sequence may include: moving the end effector 114w of the machine 114, e.g., by controlling one or more than one (e.g., each) actuator of the kinematic chain of the machine 114; and acquiring the spatial information of the handheld device 100 (e.g., analogous to 301). For example, the position and / or movement of the handheld device 100 may be calibrated with respect to the coordinate system of the machine and / or a global coordinate system. Based on the information thereby obtained, the model 114m of the machine 114 (e.g., the control model, e.g., a machine-type-specific model or the delta formation) may be updated 401.

[0115] Optionally, the program code 116 can be generated subsequently. The program code 116 can then be generated 107 based on the updated model 114m of the machine 114. However, the program code 116 does not necessarily have to be generated. For example, the learned data can be executed directly on the control device of the data processing system 502 (see Fig.5 ) to control the machine, as described in more detail below.

[0116] The calibration sequence can clearly enable calibration of the robot 114 in the global coordinate system.

[0117] The method 300 may alternatively or additionally to the calibration 403 of the system 200 comprise the following: executing the control information to be transmitted to the machine 114 (e.g., in the form of program code, in the form of the process model 104m, and / or in the form of training data) using the model 114m of the machine 114. The model 114m of the machine 114 may be configured to emulate the operation of the machine 114. The model 114m of the machine 114 may, for example, comprise a virtual image of the machine 114.

[0118] Illustratively, a test instance can be provided by means of the model 114m of the machine 114, on which the control information can be tested for the degree of its integrity, its task fulfillment and / or its freedom from conflict.

[0119] Optionally, the control information can be adjusted, e.g., manually and / or automatically, based on the result of executing the control information using the model 114m of the machine 114. This makes it possible to increase the degree of integrity, the degree of task fulfillment, and / or the degree of conflict-freeness.

[0120] Fig.5 illustrates the system 200 according to various embodiments in a schematic perspective view 500, which further comprises a handset-external device. The handset-external device can comprise a data processing system 502 configured to generate 107 the program code 116 or, more generally, to generate 107 control information, as will be described in more detail later. For this purpose, the training data can be provided to the data processing system 502, e.g., by transmitting it from the locating device 112 and / or from the handset 100 to the data processing system 502. The transmission can occur, for example, according to a wireless communication protocol.

[0121] The data processing system 502 can be separate from the machine 114 or can also be part of it. The data processing system 502 of the machine 114 can, for example, have its control device or be formed therefrom, e.g., having a programmable logic system. Clearly, the training data can be processed on a data processing system 502 separate from the machine and later transferred to the robot controller. Alternatively or additionally, the training data can be processed (e.g., partially or completely) on the robot controller itself. Likewise, e.g., if the generation of the program code 116 is omitted, the machine can be controlled directly by means of the data processing system 502 separate from the machine. In the latter case, the data processing system 502 can determine one or more control commands based on the control information and control the machine by means of the control command, e.g.,via a programming interface and / or by means of a programming communication protocol (API). Then, steps 305, 307, and 316, described in more detail below, can be omitted.

[0122] In the following, reference is made to a data processing system 502 that is separate from the machine. The description can also apply analogously to a data processing system 502 that is integrated into the machine 114 and / or provides the machine with the control commands directly.

[0123] The data processing system 502 can optionally be communicatively coupled 502k to the machine 114, e.g., to the machine controller 702, e.g., if the latter is separate from the machine 114, or can be a component of the machine 114. The communicative coupling 502k can be provided, for example, by means of a cable 502k or, alternatively, wirelessly. The machine 114 can be controlled by means of the coupling 502k, e.g., according to the calibration sequence. Alternatively or additionally, the generated program code 116 can be transmitted to the machine 114 by means of the coupling 502k. Communication with the machine 114 can occur according to a communication protocol of the machine, e.g., according to a programming communication protocol, a network communication protocol, and / or a fieldbus communication protocol.The code generation 107 and / or the communication with the machine 114 can thus be provided for one or more than one machine 114, optionally of different types, optionally taking into account possible different PLC control systems.

[0124] When the data processing system 502 receives the first activation information, it can begin recording the training data (also referred to as activating the training). Of course, the data processing system 502 can also begin recording data beforehand. The training data then refers to the portion of the data that is also used for training. For this purpose, the recorded data can be filtered, for example, manually or on a tablet.

[0125] When data processing system 502 receives a second activation information, it can stop recording the training data (also referred to as deactivating the training). The activation and deactivation can be repeated, and the training data thus recorded can be consolidated. Program code 116 can then be generated 107 based on the training data.

[0126] The data processing system 502 can, for example, enable a software-supported method 300 for training an industrial robot 114, which is also accessible to a technical layperson 106. For example, a non-programmer 106 can be enabled to train an industrial robot 114 in a fully integrated manner.

[0127] Using the method 300, at least one task expert 106 (e.g., a mechanic or a welder) can demonstrate one or more activities of the process sequence by way of example using the training device 302. Based on this, the necessary control software 116 of the robot 114, including all required software components, can be generated fully automatically.

[0128] The method 300 can comprise in 301: capturing the training data by means of one or more than one sensor of the locating device 112 and / or the training device 302 (e.g., its handheld device 100). For example, the handheld device 100 can provide its position and / or acceleration as part of the training data. Alternatively or additionally, at least one sensor of the locating device 112 (also referred to as an external sensor) can provide the position and / or acceleration of the training tool 214 and / or the handheld device 100 as part of the training data. Other measured variables can also be captured, which represent an actual state of the training tool 214 and / or the handheld device 100, e.g., their trajectory 111 (e.g., position and / or movement). Optionally, the input unit 108 can have at least one sensor that captures a handling according to the activity as part of the training data. The training data can be transmitted to the data processing system 502 (e.g.,a PC, a laptop, etc.) which is communicatively connected to the locating device 112 and / or to the handset 100 (e.g. its output unit 110) by means of its communication unit (e.g. by radio).

[0129] The locating device 122 may optionally emit a locating signal using a locating signal source 1308 (e.g., an infrared laser), as will be described in more detail later.

[0130] Optionally, the data processing system 502 can implement a system management system by means of which the components of the system can be managed and / or registered. For example, a component that registers with the system can be or will be registered as a component of the system, e.g., the handset 100 and / or the interchangeable attachment 210. Thus, multiple interchangeable attachments 210 and / or multiple handsets 100 can also be managed per system.

[0131] Reference is made herein to the fact that the input unit 108 is a component of the handset 100 (also referred to as the handset-internal input unit 108). What has been described can apply analogously to a handset-external input unit. The handset-external input unit can, for example, be provided alternatively or in addition to the handset-internal input unit 108. For example, the handset-external input unit can be provided by means of a handset-external device (also referred to as a handset-external activation device). The handset-external activation device can, for example, comprise or be formed from a smartphone, a radio switch, a removable attachment (as will be described in more detail later), or the or an additional computing system 502.The handset-external input unit can clearly provide the same function as the handset-internal input unit 108, so that the input of activation information for activating the training of the machine is performed via the handset-external input unit. The handset-external input unit does not necessarily have to be a physical input unit, but can also be an emulated or virtual input unit.

[0132] The handset-external activation device can, for example, be registered in the system and optionally exchange data, e.g., wirelessly, with the system component (also referred to as the processing component) that records and / or processes the training information. The processing component of the system can, for example, be the control device of the handset 100 if it has the data processing system 502, or the control device of the machine if it has the data processing system 502. Alternatively, the handset-external activation device can record and / or process the training information itself.

[0133] In general, the system may comprise one or more than one handset-external device, of which at least one handset-external device processes the training data (i.e., provides the processing component) and at least one or the handset-external device optionally provides the input unit.

[0134] The handset-external activation device can, for example, enable two-handed training of the machine.

[0135] Analogously, the handset-external activation device may comprise a handset-external output unit, which may be provided alternatively or in addition to the output unit 110 of the handset 100 (also referred to as a handset-internal output unit). The handset-external output unit may be configured to output the activation information for activating the training of the machine 114.

[0136] The (handset-internal and / or external) input unit can optionally implement voice control or gesture control for entering activation information. Alternatively or additionally, the handset 100 and / or the handset-external activation device can implement voice control or gesture control for entering training data. For example, it can be enabled for the user to input one or more parameters of the activity to be trained (e.g., activity-specific and / or tool-specific process parameters) using voice or gesture. For example, the user can specify a flow rate using voice or gesture.

[0137] Alternatively or in addition to gesture control, muscle tension control can be implemented, for example. This can enable the detection of gestures or parameters based on the user's muscle tension. Gesture control can be implemented, for example, using muscle tension control and / or a video camera that captures the user's behavior.

[0138] More generally, the activation information and / or one or more parameters of the activity to be trained can be determined based on (e.g., contactless) recorded user behavior (e.g., speech, facial expressions, gestures, movement, etc.). For example, user behavior can be used to determine whether training should be started, paused, and / or ended.

[0139] Fig.6 illustrates a machine 114 according to various embodiments in a schematic layout diagram 600.

[0140] The machine 114 herein may be a machine programmable by means of a control program 116. Once programmed, the machine 114 may be configured to autonomously perform one or more process activities, and optionally to vary the process activity (i.e., the execution of the task) within limits depending on sensor information. The control device 702 may, for example, comprise a programmable logic controller (PLC).

[0141] The machine 114 may include a control device 702 configured to control at least one actuator 704 of the machine 114 according to the control program 116. The control device 702 may, for example, include one or more processors and / or a storage medium. The manipulator of the machine 114 may include a kinematic chain 706 along which an effect of the at least one actuator 704 is transmitted, e.g., along the coupling of the links of the kinematic chain 706 to one another.

[0142] The kinematic chain 706 can have a positioning device 114p and an end effector 114w that can be positioned by means of the positioning device 114p. The end effector 114w can be understood as the last link of the kinematic chain 706 of the machine 114, which is configured to act directly on a workpiece, e.g., to machine it (i.e., to process it). The sum of all operations, such as the act on the workpiece, for example a preparation step therefor or a post-processing step therefor, can be part of the process activity. The process activity can, for example, be primary forming, inspecting, joining (e.g., welding, coating, screwing, inserting, contacting, gluing or otherwise assembling or assembling), separating (e.g., grinding, milling, sawing or otherwise machining, punching or disassembling), forming, heating, relocating (e.g.,Gripping, loading, rotating or moving), or similar. The process activity can be path-based, i.e., can be mapped by moving the end effector 114w along a trajectory 113.

[0143] The positioning device 114p can have at least one actuator 704, which is configured to displace the end effector 114w to a position (also referred to as positioning). The end effector 114w can have at least one actuator 704, which is configured to carry out the process activity, e.g., by means of a tool 124w of the end effector 114w. The tool 124w can generally provide a function corresponding to the process activity, by means of which the workpiece is acted upon. The tool can, for example, have a primary forming tool, a joining tool (e.g., screwdriver, glue gun, or welding device), a displacement tool (e.g., gripper), a separating tool, or the like. The joining tool can, for example, have a coating tool (e.g., a paint spray gun, a powder coating gun), or be formed therefrom.

[0144] Optionally, the machine 114 can have at least one internal sensor 114i configured to detect an operating point of the kinematic chain 706, e.g., to implement a closed-loop control. The internal sensor 114i can, for example, be a component of a stepper motor that detects its current operating point (e.g., its position). Alternatively or in addition to the at least one internal sensor 114i, the machine 114 can also have a sensor 114i external to its frame and / or its end effector, e.g., a camera that visually detects the machine 114.

[0145] If the process activity is simulated by the programmable machine 114, the machine 114 as a whole can be brought to an operating point that is as close as possible to the process activity based on the spatial information. The operating point can, for example, define the position to which the end effector 114w should be brought (by moving it) and what effect it should provide there. The operating point can, for example, describe the sum of the states of the individual actuators 704 of the machine 114.

[0146] The storage medium may be part of the control device 702 and / or provided separately therefrom. The storage medium may, for example, comprise an electronic semiconductor storage medium, e.g., a read-only memory (ROM) or a random access memory (RAM), a memory card, a flash memory, a universal serial bus stick (USB stick), a solid-state drive (SSD), a hard disk drive (HDD), a memory disk (MD), a holographic storage medium, an optical storage medium, a compact disc, a digital versatile disc (DCV), and / or a magneto-optical disk.

[0147] Training was described above with reference to a machine 114. Analogously, what has been described can apply to a plurality of separate, e.g., communicating, machines 114 (e.g., a process line), as well as to a machine having multiple positioning devices and / or end effectors.

[0148] Fig.7 illustrates a system 700 according to various embodiments in a schematic perspective view, e.g., configured like system 200, wherein system 700 comprises handset 100 and a locating device 112, wherein locating device 112 comprises multiple locating units 112a, 112b. Each of the locating units 112a, 112b can be configured to determine the position of handset 100, e.g., by means of tracking. Alternatively or additionally, handset 100 itself can be configured to determine the position. For example, locating units 112a, 112b can be configured to project an optical pattern into space (e.g., by means of an infrared laser), which can be detected by one or more optoelectronic sensors of handset 100.

[0149] Fig.8 illustrates a system 800, e.g., configured like system 200 or 700, according to various embodiments in a schematic perspective view. Along the longitudinal direction 711a, the handheld device 104 can be delimited by two end faces 101a, 101b, of which a second end face 101b is opposite the coupling structure 102 and has a sensor section 802 (more generally also referred to as locating section 802). The longitudinal direction 711a of the handheld device 100 can be directed from the second end face to the first end face 101a. The sensor section 802 can have one or more than one (e.g., optoelectronic) sensor, as described above, which can be used to determine the training trajectory 111, e.g., at least 3 (4, 5, or at least 10) sensors.

[0150] Alternatively or in addition to the sensors, one or more emitters can be arranged in the locating section 802 (also referred to as emitter section 802), which communicates with the locating device 112. The locating device 112 can then feed the signals received from the emitters to capture the spatial information of the handset 100. Reference is made below to the sensors of section 802. What has been described for the sensors of section 802 can also apply analogously to the emitters of the emitter section 802, in which case, for example, the signal direction would then be reversed.

[0151] The more sensors the sensor section 802 has, the greater the accuracy of the position determination. Optionally, at least two (or more pairs) of the sensors of the sensor section 802 can differ from each other in their orientation. This facilitates all-round detection.

[0152] The handle 104 can have a smaller circumference (e.g., along a self-contained path extending transversely to the longitudinal direction 711a) than the sensor section 802. In other words, the sensor section 802 can be widened. Alternatively or additionally, at least two sensors of the sensor section 802 can have a distance (also referred to as sensor distance) from one another that is greater than an extent of the handle 104 (also referred to as transverse extent). The transverse extent and / or the sensor distance can be transverse to the longitudinal direction 711a and / or parallel to one another. This increases the accuracy of the position determination. Clearly, the accuracy can increase the greater the distances between the sensors and / or the larger the area spanned by the sensors (also referred to as sensor area).

[0153] Optionally, one or more than one sensor of the handset used to determine the training trajectory 111 may be arranged within the handle 104, e.g., a rotation sensor 812 and / or a position sensor 814. This increases the accuracy of determining the training trajectory 111, e.g., due to the distance to the sensor section 802.

[0154] The arrangement of the sensor section (e.g. with optoelectronic sensors) on the second end face 101b and / or away from the handle minimizes concealment of the sensors of the sensor section and thus facilitates the determination of the spatial information.

[0155] The handset 100 may have one or more than one feedback unit (also referred to as a signal generator), e.g., an optical feedback unit 822 (e.g., having a light source), e.g., a haptic feedback unit 824 (e.g., having a vibration source, e.g., having an unbalance motor), and / or an acoustic feedback unit (e.g., having a loudspeaker).

[0156] The interchangeable attachment 210 can optionally have a circuit 852 configured to implement a function. The function of the interchangeable attachment 210 can be configured according to the activity to be trained and / or controlled by manipulating the handset 100 (e.g., its input unit 108). Alternatively or additionally, the interchangeable attachment can have an input unit, by means of which, for example, a manipulation according to the activity can be recorded or the function of the interchangeable attachment can be operated.

[0157] The function of the interchangeable attachment 210 can be controlled, for example, using the input unit of the interchangeable attachment 210 and / or the handheld device 100 (more generally, the training device 302). For example, a gripping movement of a gripper can be triggered and / or controlled using the input unit.

[0158] Examples of the function of the interchangeable attachment 210 may include: detecting a physical quantity acting on the interchangeable attachment 210 (for this purpose, the circuit may have at least one sensor), emitting radiation (for this purpose, the circuit may have at least one radiation source, e.g., a light source), exchanging data (for this purpose, the circuit may have at least one additional interface), moving one or more than one component of the interchangeable attachment 210. The function may, for example, be configured according to the process activity.

[0159] The handheld device 100 can be configured to supply the circuit 852 with energy and / or exchange data with it. For example, the sensor of the circuit 852 can be read, and the read-out data can be transmitted to the data processing system via the output unit 110. For example, the radiation source can be adjusted (e.g., its radiation intensity, its aperture angle, its radiation wavelength, etc.). For example, the radiation source can comprise a light source, an ultraviolet radiation source (e.g., for curing adhesive), or a thermal radiation source (e.g., a radiant heater). By means of the light source (e.g., emitting visible light), an illuminated area can be projected onto the workpiece, for example, for inspecting the workpiece or for marking a location on the workpiece that is to be machined.

[0160] The supply of energy to the circuit 852 and / or the exchange of data with the circuit 852 can, for example, be carried out via a cable connection (by means of a contact-interchangeable attachment interface). For this purpose, the handset 100 can, for example, have a power line whose output 842a provides a first contact on the first end face 101a of the handset. Correspondingly, the circuit 852 can have a power line whose input 842e provides a second contact. The first contact and the second contact can, when the interchangeable attachment 210 is coupled to the handset 100, be electrically and / or physically connected to one another (also referred to as a power supply connection). In an analogous manner, a data exchange connection can be provided by means of contacts 843. The interchangeable attachment 210 can alternatively or additionally have an electrical energy source (e.g.a battery) which is arranged to supply the circuit 852 with energy.

[0161] More generally, the interchangeable attachment 210 and the handset 100 (e.g., their batteries) can exchange electrical energy for power supply, e.g., from the interchangeable attachment 210 to the handset 100 or vice versa. For example, the interchangeable attachment 210 can also be used to charge the battery of the handset 100 or operate independently of the battery of the handset 100.

[0162] In the same way, the interchangeable attachment 210 and the handset 100 can exchange data, e.g., from the interchangeable attachment 210 to the handset 100 or vice versa. For example, the function of the input unit 108 of the handset 100 can be transferred to an input unit of the interchangeable attachment 210, so that the input of activation information for activating the training of the machine can be performed (e.g., selectively or only) on the interchangeable attachment 210. The input unit of the interchangeable attachment 210 can, for example, exchange data with the control device of the handset 100 via contacts 843.

[0163] For example, the interchangeable attachment 210 may also have a sleeve into which the handle and / or the input unit 108 of the handset 100 can be at least partially (ie partially or completely) inserted and / or which at least partially covers them (cf., for example, interchangeable attachment 210f in Fig.9 ). For example, the sleeve can have a recess into which the handle and / or the input unit 108 of the handheld device 100 can be inserted. This allows for more individual adaptation of the training device 302 to the activity to be trained. Optionally, such an interchangeable attachment 210 can be configured to assume the function of the input unit 108 when coupled to the handheld device 100, so that the activation information for activating the training of the machine can be entered on the interchangeable attachment 210.

[0164] The data exchange connection and / or power supply connection can also be wireless (using a wireless interchangeable interface).

[0165] For example, the energy can be coupled wirelessly into the circuit 852 by means of induction. For example, the data can be exchanged wirelessly, e.g., via Bluetooth and / or RFID. Alternatively or additionally, the data (e.g., in the form of a data signal) can be modulated onto a signal (e.g., the current and / or voltage) of the power supply connection (also referred to as carrier frequency technology). The data signal can, for example, be modulated onto one or more carrier signals of the power supply connection.

[0166] The coupling structure 102 and the mating coupling structure 212 may, for example, provide a plug-in coupling.

[0167] Fig.9 illustrates a system 900, e.g., configured as one of the systems 200, 700, or 800, according to various embodiments in a schematic perspective view, wherein the system 900 comprises the handset 100 and a plurality of interchangeable attachments 210a to 210g. Of the plurality of interchangeable attachments 210a to 210g, one interchangeable attachment can be selectively (e.g., always precisely) coupled to the coupling structure 102.

[0168] The plurality of interchangeable attachments 210a to 210g can, for example, comprise an interchangeable attachment 210b with a relocation tool (also referred to as a pick-and-place tool), e.g., a gripper. The relocation tool can, for example, comprise interchangeable gripper jaws, two or more gripper jaws, and / or a driven gripping function. For this purpose, its circuit 852 can, for example, comprise an actuator that drives the gripping function (e.g., a gripper setting).

[0169] The plurality of interchangeable attachments 210a to 210g can, for example, comprise an interchangeable attachment 210c with a coating tool. This can, for example, comprise a sensor that detects a flow regulation to be trained.

[0170] The plurality of interchangeable attachments 210a to 210g can, for example, have an interchangeable attachment 210d with an inspection tool. The inspection tool can, for example, represent an optical quality assurance activity. For example, the area to be inspected can be projected onto the workpiece using the inspection tool.

[0171] The plurality of interchangeable attachments 210a to 210g can, for example, comprise an interchangeable attachment 210e with a cutting tool, e.g., a deburring tool. Its circuit 852 can, for example, comprise a multi-axis force sensor, a variable stiffness, and / or an interchangeable deburring tip. The multi-axis force sensor can, for example, detect a force acting on the deburring tip. The stiffness can, for example, represent a force that counteracts a deflection of the deburring tip relative to the multi-axis force sensor. The deburring tool can, for example, represent deburring as an activity.

[0172] The plurality of interchangeable attachments 210a to 210g can, for example, comprise an interchangeable attachment 210f with a screwing tool. Its circuit 852 can, for example, comprise a sensor (e.g., a single-axis or multi-axis force sensor) that can detect a force acting on the screwing tip. The screwing tool can, for example, comprise an interchangeable screwing tip (the screwing end effector, e.g., a screw bit).

[0173] The plurality of interchangeable attachments 210a to 210g can, for example, comprise an interchangeable attachment 210g with an adhesive tool. The adhesive tool can, for example, comprise an interchangeable adhesive tip. The circuit 852 of the interchangeable attachment 210g can, for example, comprise a sensor that can detect a working area swept by the adhesive tip and / or a sensor (e.g., a single-axis or multi-axis force sensor) that can detect a force acting on the adhesive tip.

[0174] The circuit 852 of the interchangeable attachment with coating tool and / or inspection tool can, for example, have a workspace display unit that displays the workspace 812, e.g., by illuminating it with a light source of the workspace display unit. The size of the displayed workspace can optionally be changed, whereby the actually set workspace or working point can be taken into account during training to form the control model 116m.

[0175] The circuit 852 of the interchangeable attachment with coating tool, screwing tool, displacement tool, and / or with inspection tool can, for example, have the additional interface.

[0176] The additional interface can, for example, have an additional input unit that is configured to input information about an operating point of the tool (also referred to as operating point information). The operating point information can, for example, represent a start, a duration, and / or a strength of the activity, e.g., a coating (e.g., the flow rate used for this purpose). The additional input unit can, for example, have one or more than one switch, slider, force sensor, or the like. The additional interface can, alternatively or additionally, have an additional feedback unit that is configured to output feedback. The additional feedback unit can, for example, be an optical feedback unit (e.g., having a light source), a haptic feedback unit (e.g., having a vibration source, e.g., having an unbalance motor), and / or an acoustic feedback unit (e.g.,The feedback can, for example, indicate the status of the operating point, acknowledge its change, acknowledge the detection of a user input, acknowledge the (physical, electrical, and / or communicative) coupling and / or uncoupling of the interchangeable attachment, or acknowledge the detection of a mechanical impact on the interchangeable attachment. The optical feedback unit can, for example, have a display or other indicator.

[0177] Fig.10 illustrates the handset 100 according to various embodiments in a schematic layout diagram 1000.

[0178] The handheld device 100 can be a mobile device programmable using a control program. Once programmed, the handheld device 100 can be configured to autonomously capture at least parts of a process activity demonstrated using the handheld device 100 and transmit them as training data via the output unit 110. The training data can include or be formed from the activation information and / or the spatial information. The spatial information can include, for example, a position, movement, and / or orientation of the handheld device 100.

[0179] Alternatively or additionally, the handset 100 may be configured to detect itself, e.g., its position, movement and / or orientation.

[0180] The handheld device 100 may include a control device 1702 configured to read at least the input unit 108 and / or control the output unit 110. For example, the control device 1702 may be configured to send the or each activation information acquired by the input unit 108 according to a communication protocol via the output unit 110 (e.g., to the data processing system and / or a network). Examples of the input unit 108 may include: one or more than one switch (e.g., a button), e.g., a contactless switch, a touch-sensitive surface (e.g., resistive and / or capacitive), a virtual switch (e.g., implemented by a display).

[0181] The output unit 110 can, for example, be a wireless output unit 110 and optionally also have a transceiver for receiving data. For example, the control device 1702 can exchange data with the data processing system and / or the network using the transceiver of the output unit 110. For example, the output unit 110 can have: a Bluetooth transceiver, a WLAN transceiver, or a mobile radio transceiver.

[0182] Optionally, the output unit 110 can also be configured to register the handset 100 in the system. Registration can occur, for example, as soon as the handset 100 is switched on. Thus, the system can detect whether or when the handset 100 is ready to train at least one movement and at least one activity of the machine. If the input unit is a component of a handset-external device, the handset-external device can be configured analogously to register itself in the system or to manage the registration of the handset 100 (e.g., if the handset-external device implements system management).

[0183] The control device 1702 may, for example, include one or more processors and / or storage media.

[0184] The control device 1702 can, for example, be configured to capture the spatial information using one or more than one sensor 802s (if present) of the handheld device 100. For example, sensors of the handheld device 100 can include: a GPS sensor, a position sensor (e.g., an orientation sensor and / or a position sensor), an acceleration sensor, a rotation sensor, a speed sensor, an air pressure sensor, an optoelectronic sensor, a radar sensor.

[0185] The control device 1702 can, for example, be configured to exchange data with an interchangeable attachment 210 coupled to the handset 100 by means of an interface 843 (if present) of the handset 100.

[0186] For example, software may be executed on the control device 1702 that provides one or more of the above functions, has a programming interface, and / or cyclically reads the status of the components of the handset 100.

[0187] The handset 100 may include a battery 1704 (e.g., an accumulator) configured to supply electrical energy to the electrical components 108, 110, 822, 824, 802s of the handset 100. Optionally, the handset 100 may include a charging port, via which external electrical energy can be supplied to the battery 1704 to charge the battery 1704.

[0188] The battery 1704 can, for example, be configured to exchange energy with an interchangeable attachment 210 coupled to the handset 100 via an interface 842a (if present) of the handset 100. The exchange of energy can, for example, be controlled by the control device 702.

[0189] Based on a user input detected by the input unit 108 (e.g., force, repetition, speed, location, etc.), it can be determined, for example, whether the user input fulfills a criterion. If the criterion is parameterized according to the type of input unit 108 (i.e., mapped to a property that can be detected by the input unit 108), the property detected by the input unit 108 can be compared with the parameterized criterion to determine whether the criterion is met. If the user input fulfills a first criterion, the first activation information can be detected. If the user input fulfills a second criterion, the second activation information can be detected. The functions provided by the input unit 108 can, however, also be provided in whole or in part by means of the other device external to the handset, e.g., by means of a radio switch and / or by means of a mobile computing device (e.g.,a tablet), on which, for example, an application is executed which emulates the functions of the input unit 108.

[0190] The handheld device 100 can optionally have one or more than one feedback unit 822, 824. The control device 1702 of the handheld device 100 can, for example, be configured to output (e.g., haptic and / or optical) feedback using the one or more than one feedback unit 822, 824. The feedback can, for example, represent a training status, acknowledge a change to the training status (e.g., activation or deactivation), acknowledge the detection of a user input, acknowledge the (physical, electrical, and / or communicative) coupling and / or uncoupling of an interchangeable attachment, represent a status of the handheld device 100 (e.g., its battery charge), and / or acknowledge the detection of a mechanical impact on the interchangeable attachment. Examples of the feedback unit 822, 824 include: a light source (e.g., an LED), a sound source (e.g., a loudspeaker), a vibration source (e.g., an unbalance motor).

[0191] Fig.11 illustrates the method 300 according to various embodiments in a schematic flow diagram 1100. The method 300 may include: in 1101, first communicatively connecting the plurality of components of the system to one another; and in 1103, second communicatively connecting the system to the machine. The first connection may, for example, be wireless. The second connection may, for example, be via a cable, e.g., a network cable, that connects the machine to the data processing system. Optionally, the data processing system may be supplied with power from the machine via the network cable.

[0192] The method 300 can optionally comprise: Calibration 1103 of the system. By means of the locating device 112 (e.g., its locating signal), a position and / or movement of the handheld device can be detected (e.g., its exact position in space can be determined). For this purpose, the handheld device can be attached to the machine using the fastening device. Depending on the type of machine, different types of fastening devices can be used, for example. The detection of the handheld device can be carried out, for example, using infrared light (e.g., a light pattern) emitted by the locating device 112. Calibration 1103 can, for example, be carried out fully automatically, for example when an input meets a predetermined criterion.

[0193] The method 300 can comprise, in 1101, training the machine using the system. The person can couple a suitable interchangeable attachment to the handheld device and perform the activity using the training device thus formed. Optionally, the person can execute an application using the data processing system, by means of which various settings and / or information relating to the activity to be performed can be set. The training can further comprise recording the training data. The training data can, for example, comprise spatial information with an accuracy of millimeters (or tenths of a millimeter). The training data can optionally indicate a detected mechanical interaction (e.g., a force) with the interchangeable attachment when performing the activity. For example, a pressure acting on the interchangeable attachment during the activity can be detected using a force sensor of the handheld device.For example, the pressure intensity can be considered as a parameter of the activity when training a grinding process. The training can also involve creating the control model based on the training data. The control model can contain multiple pieces of control information that are executed sequentially. One piece of control information can, for example, represent the operating point to which the machine should be brought.

[0194] Training may optionally involve post-processing the training data, the control model, and / or the process model. This post-processing can be performed, for example, by a user interacting with the data processing system. Alternatively or additionally, the post-processing can be performed using artificial intelligence. This facilitates the training of more complex processes.

[0195] The training may further include generating the program code. For this purpose, the program code may be generated, for example, in a programming language of the machine. The programming language in which the program code is written may be determined by the data processing system, for example, by determining the type of machine.

[0196] The multiple components of the system may include the handset, the data processing system, the locating device 112 and / or (e.g., if capable of communication) the interchangeable attachment 210.

[0197] Fig.12 illustrates a system 1200, e.g., configured as one of the systems 200, 200, 700, 800, or 900, in a method 300 according to various embodiments in a schematic communication diagram 1200.

[0198] The data processing system 502 can optionally have a mobile terminal 1212 (e.g., a mobile device, e.g., a tablet) on which a user interface 1202 (also referred to as a front end) of the application is executed. Alternatively or additionally, the data processing system 502 can have a stationary terminal 1214 (e.g., a data processing system, e.g., a server) on which a substructure 1204 (also referred to as a back end) of the application (e.g., as a processing component) is executed.

[0199] The stationary terminal 1214 can, for example, be wired 502k to the machine 114, e.g., via a network 302n. The stationary terminal 1214 can, for example, be connected wirelessly 502d (e.g., via WLAN 302w) and / or wired (e.g., via a long cable from the terminal 1214) to the mobile terminal 1212. The stationary terminal 1214 can, for example, be connected wirelessly (e.g., via Bluetooth 302b) to processing software 1206, which is executed, for example, on the handset 100 and / or the locating device 112. The processing software 1206 can be configured to transmit the training data (or a pre-processed version thereof) to the backend 1204. One or more than one (e.g., each) of the wireless connections 302w, 302b, 302n may alternatively or additionally be provided by means of a wired connection and vice versa.

[0200] Alternatively or in addition to the terminal 1214, non-local logic (e.g., a cloud) can also be used to provide the processing component. For example, a central server can be configured to provide a processing component to each of a plurality of systems. This saves resources. For example, in this case, the transmission of training data and / or control information to or from the processing component can occur via the Internet or network-internal resources (e.g., EdgeClouds).

[0201] The front end 1202 (e.g., its user interface) may include a configuration and calibration manager 1202a, a training data visualization 1202b, and a process activity planner 1202c.

[0202] The backend 1204 may include a processing component 1402a that collects the training data and / or performs the location, e.g., based on the training data, determines the trained trajectory 111 and / or the associated operating points along the trained trajectory 111. The backend 1204 may include a process model generation component 1402b that determines the process model 104m based on the trained trajectory 111 and / or the associated operating points along the trained trajectory 111. The backend 1204 may include an optional process model adjustment component 1402c that modifies the determined process model 104m based on instructions originating from the process activity planner 1202c. The back-end 1204 may include a control model generation component 1402d that determines the control model 116m based on the process model 104m.

[0203] The backend 1204 may include the code generator 412, which generates the program code 116 based on the control model 116m and stores it on a storage medium 702m (e.g., data memory) of the machine 114. A processor 702p of the control device 702 may read the storage medium 702m to execute the program code 116. As already described above, the code generator 412 may be omitted, e.g., if the control commands are transmitted directly to the machine without generating the program code.

[0204] Fig.13 illustrates a trajectory determination mechanism 1300 of the system 200 in a schematic communication diagram.

[0205] The trajectory determination mechanism 1300 may be configured to determine the training trajectory 111, e.g., at least partially based on the spatial information 1301 about the handheld device 100. The trajectory determination mechanism 1300 may include at least one measurement chain component 1304 and an evaluation component 1306. The measurement chain component 1304 may include a sensor arrangement 1302 and one or more transducers 1304a, 1304b.

[0206] The sensor arrangement 1302 (e.g., each of its sensors) may be configured to detect one or more measured variables that represent the spatial information about the handheld device 100 (e.g., from which the spatial information can be derived). The sensors of the sensor arrangement 1302 may, for example, comprise internal or external sensors that are registered in the system (e.g., via a communication connection).

[0207] A sensor (also referred to as a detector) can be understood as a transducer that is designed to qualitatively or quantitatively detect a property of its environment corresponding to the sensor type, e.g., a physical or chemical property and / or a material quality. The measurand is the physical quantity to which the measurement by the sensor applies.

[0208] The or each sensor 1304a, 1304b can be coupled to at least one sensor of the sensor arrangement 1302 and configured to provide a measured value as an image of the detected measured variable of the at least one sensor. The provided measured values ​​(illustratively, measured data) can be supplied to the evaluation component 1306 as part of the training data. The measured data can contain specific values ​​for the spatial information, e.g., about a movement (e.g., rotation and / or translation) in space, a position in space, and / or an orientation in space.

[0209] For example, various physical quantities can be recorded, e.g. a force caused by the movement, or the change in an optical signal caused by the movement. For example, a camera can be used that records a stream of image data of the room in which the handheld device is arranged, wherein the spatial information about the handheld device can be determined based on the image data. However, it is also possible, for example, to measure a distance between a reference point of the locating device 112 (e.g. its locating unit 112a) and the handheld device, e.g. using radar, lidar or sonar. However, it is also possible, for example, to detect an orientation of the handheld device using an inclination sensor of the handheld device. However, it is also possible, for example, to use a photosensor of the handheld device to detect a movement relative to an optical locating signal (e.g. an optical pattern).For example, an inertial sensor on the handset can detect the current movement state of the handset.

[0210] According to various embodiments, the components of the trajectory determination mechanism 1300 (e.g., sensors or transducers) can be distributed across various components of the system. For example, the training device 302 (e.g., the handheld device 100) can have one or more than one sensor 802s (also referred to as a handset-internal trajectory sensor) of the sensor arrangement 1302 and / or one or more than one transducer 1304a. Alternatively or additionally, the handset-external device (e.g., its locating device 112) can have one or more than one sensor 112s of the sensor arrangement 1302 (also referred to as a handset-external trajectory sensor) and / or one or more than one transducer 1304a. The evaluation component 1306 can, for example, be part of the data processing system 502, e.g., as a component of the application (e.g.,as processing component 1402a) are executed by the data processing system 502, or transmit the determined training trajectory 111 to the data processing system 502.

[0211] The evaluation component 1306 can be configured to determine the training trajectory 111 based on the supplied measurement data 1301. To this end, the evaluation component 1306 can be configured to correlate the measurement data, interpret them, consider their origin, consider their connection to the spatial information via the handheld device, etc. For example, results from measured values ​​from different sensors can be superimposed on one another, thus yielding a more precise training trajectory 111. With knowledge of the connection between the measured variable and the spatial information, the measured value can be mapped to a value of the spatial information, e.g., to a location coordinate and / or its change over time. The more sensors used, the more precise the trajectory 111 can be.

[0212] Optionally, the trajectory determination mechanism 1300 may comprise one or more than one handset-external location signal source 1308. The or each location signal source 1308 may be configured to emit a location signal 1308s, e.g., toward the handset and / or at least into the space in which the handset is arranged. Multiple location signals 1308s may, for example, be superimposed on one another (see, for example, Fig. 7 ).

[0213] The locating signal 1308s can, for example, comprise an infrared signal (comprising light in the infrared range), an ultrasonic signal, a radio signal, and / or a visible light signal (comprising light in the visible range). The ultrasonic signal can, for example, enable high accuracy (e.g., in a time-of-flight measurement). The infrared signal enables cost-effective implementation. Furthermore, with the infrared signal, no direct connection between transmitter and receiver is necessary, since the infrared signal can reach the receiver via reflections. This makes it possible to at least determine whether an infrared signal is present in a room. A transmitter identification can optionally be modulated onto an electromagnetic locating signal (e.g., light signal, radio signal, infrared signal), whereby different locating signal sources 1303 can be differentiated based on their locating signal.Unlike infrared and visible light signals, a radio signal can penetrate walls and other obstacles, making reception easier. The light signal can be detected even with a technically simple charge-coupled device sensor. The or each location signal source 1308 may comprise, for example, a laser scanner.

[0214] The spatial information can be determined using one or more than one positioning signal 1308s, e.g., based on one or more of the following measurement mechanisms: range mechanism, propagation time mechanism, propagation time difference mechanism, angle of incidence mechanism, and / or signal strength mechanism. The or each positioning signal source 1308 can serve as a reference point for the measurement mechanism, knowing its position in space. For example, a geometric relationship (e.g., distance, angle, etc.) with respect to the reference point can be determined, and the spatial information about the handset can be determined based on this. A position of the handset can then be determined based on a distance to one or more than one positioning signal source. Optionally, the orientation of the handset with respect to a positioning signal source can also be determined based on the respective distance between two sensors of the handset.

[0215] With the range mechanism, each locating signal source 1308 can provide a cell the size of the range of the locating signal 1308s. The detected cell can be assigned to a locating signal source and its position, for example, via the transmitter identification, wherein the size of the cell sets an upper limit for the distance of the sensor 802 from the locating signal source 1308. With the runtime mechanism, the time difference between the transmission and reception of the locating signal 1308s (also referred to as the runtime) can be measured. The time difference can be assigned to a locating signal source and its position, for example, via the transmitter identification, and converted into a distance to it. With the runtime difference mechanism, the runtime of two locating signals can be compared with each other, and the distance to the corresponding locating signal sources can be determined based on this.With the angle of incidence mechanism, the angle of incidence of the locating signal 1308s can be determined, which can then be converted into an orientation, for example. Based on multiple angles of incidence, a position can also be determined, for example. With the signal strength mechanism, the signal strength of the locating signal can be converted into a distance from the locating signal source.

[0216] Alternatively or additionally, a visual capture of the handheld device and / or its surroundings can be performed using the camera, and spatial information can be determined based on this. Data from various sensors (e.g., image data and position data, or image data and acceleration data) can also be overlaid to improve accuracy.

[0217] The or each location signal source 1308 may, for example, be provided as part of a location unit 112a, 112b. The or each location unit 112a, 112b may alternatively or additionally also comprise at least one trajectory sensor 112s of the handset-external device (also referred to as handset-external trajectory sensor 112s).

[0218] As a result, time-based training data, for example at high frequency, which describe the complete process activity are acquired and recorded by means of the trajectory sensors 802s, 112s of the training device 302 and / or the locating device 112.

[0219] Examples of a handheld device-external trajectory sensor 112s include: a camera 112, a distance sensor 112, a sonar sensor 112, and / or a radar sensor 112. Examples of a handheld device-internal trajectory sensor 802s include: a camera 112, a motion sensor (e.g., a rotation sensor, a speed sensor, and / or an acceleration sensor), an inertial sensor, an attitude sensor (e.g., an orientation sensor and / or a position sensor), an infrared sensor, and / or an air pressure sensor. The position sensor may, for example, also include a GPS sensor. The orientation sensor may, for example, include a gyro sensor, a gravity sensor, and / or a magnetic field sensor (e.g., for determining orientation in the Earth's magnetic field). The air pressure sensor may, for example, enable information about a vertical position of the handheld device 100 to be determined, which makes it possible to perform more accurate triangulation.

[0220] The training trajectory 111 can be determined, for example: by means of laser tracking (e.g., in the infrared range and / or by means of an infrared laser), by means of optical tracking (e.g., by means of a camera and / or pattern recognition), by means of radar (e.g., by means of a radar transceiver), by means of ultrasound (e.g., by means of an ultrasound transceiver), by means of a global positioning system (GPS), by means of an inertial measurement unit (IMU) of the handheld device. An IMU of the handheld device can, for example, have several different inertial sensors, such as one or more acceleration sensors and / or one or more yaw rate sensors.

[0221] The training trajectory 111 can, for example, represent a spatial distribution of the work location. The work location can designate the location in space 701, 703, 705 at which an effect of the process tool is to take place. For example, the work location can be stationary with respect to the training device 302 (e.g., the coordinate system 711). For example, the work location can be arranged at the tip of the training tool 210 and / or its position can be or become dependent at least on the training tool 210 of the training device 302.

[0222] The system can, for example, have a model of the training device 302, which represents the training tool 214 coupled to the handheld device 100 (e.g., its type), wherein the model of the training device 302 can be taken into account when determining the training trajectory 111. The model of the training device 302 can, for example, specify a position of the working point in the coordinate system 711 of the training device 302. Alternatively or additionally, the model of the training device 302 can, for example, specify a position of the longitudinal direction 711a of the handheld device in the coordinate system 711 of the training device 302.

[0223] In one embodiment, the system comprises a plurality of separate locating units 112a, 112b, each of which is configured to emit a pulsed infrared signal as a locating signal. The or each pulsed infrared signal is detected by a plurality of separate optoelectronic sensors of the sensor section. The measurement data determined from the detected infrared signal is transmitted to the stationary terminal as training data, together with data representing one or more inputs on the training device (e.g., the input unit of the handheld device and / or the interchangeable attachment). Based on the data and taking into account the interchangeable attachment of the training device, the stationary terminal determines the training trajectory and, optionally, an operating point for each point of the training trajectory. The operating point is determined, for example, based on the handling of the function of the interchangeable attachment.The recording of the measurement data is started, for example, in response to the operation of the input unit of the handset.

[0224] In one embodiment, the handheld device can determine which interchangeable attachment the training device has (also referred to as interchangeable attachment identification), i.e. the type of interchangeable attachment can be determined. The interchangeable attachment identification takes place, for example, using RFID, e.g. by reading a radio tag of the interchangeable attachment. The radio tag can, for example, have and / or transmit a stored interchangeable attachment identifier, e.g. a number. The interchangeable attachment identification takes place alternatively or additionally by detecting a resistive resistance of the power supply connection, in which resistance the multiple interchangeable attachments differ from one another. The interchangeable attachment identification takes place alternatively or additionally by detecting the function of the circuit of the interchangeable attachment. Which interchangeable attachment the training device has can also be specified by user input on the computer system.The result of the interchangeable attachment identification can be taken into account when determining the control model (e.g. the training trajectory).

Claims

1. Hand-held device (100) for training at least one movement and at least one activity of a machine (114), the hand-held device (100) comprising: • a handle (104); • an input unit (108) configured to input activation information for activating the training of the machine (114); • an output unit (110) configured to output the activation information for activating the training of the machine (114) to an apparatus external to the hand-held device; • a coupling structure (102) for releasably coupling an interchangeable attachment (210) configured according to the at least one activity; and characterized in that the hand-held device includes: • one or more than one infrared sensor (802s) which is configured to detect a location signal external to the hand-held device.

2. Hand-held device (100) according to claim 1, wherein the output unit (110) comprises a wireless communication apparatus for communicating with the apparatus external to the hand-held device; and / or wherein the input unit (108) is configured to detect a handling of the hand-held device (100) when performing the activity.

3. Hand-held device (100) according to claim 1 or 2, wherein the activity relates to a sum of processes, and wherein information about the activity provided and / or recorded by the hand-held device comprises: • a type of activity, • spatial information that describes the spatial sequence of processes, and • temporal information that is associated with the spatial information and that describes the duration of one of the processes.

4. Hand-held device (100) according to any of claims 1 to 3, further comprising: a feedback unit (822, 824) which is configured to output feedback to a user of the hand-held device (100).

5. Hand-held device (100) according to any of claims 1 to 4, further comprising: a mechanical sensor configured to detect a mechanical impact on the coupling structure or the interchangeable attachment, wherein the output unit is configured to output impact information about the detected mechanical impact to the apparatus external to the hand-held device.

6. Hand-held device (100) according to any of claims 1 to 5, further comprising: one or more than one additional first sensor (802s) which is configured to detect spatial information about: • a movement of the hand-held device (100) in space, • a position of the hand-held device (100) in space, and / or • an alignment of the hand-held device (100) in space. wherein preferably the one or more than one additional first sensor (802s) comprises a motion sensor and / or a position sensor.

7. Hand-held device (100) according to any of claims 1 to 6, wherein the coupling structure (102) is configured to couple the interchangeable attachment (210) in a form-fitting manner; and / or wherein the coupling structure (102) is arranged on the front side of the hand-held device (100).

8. System (200, 700, 800, 1200), comprising: • a hand-held device (100) according to any of claims 1 to 7, which further comprises an interface (842a, 843) which is configured to implement a function together with a circuit (852) of the interchangeable attachment (210); and • the apparatus external to the hand-held device and / or the interchangeable attachment (210), wherein the interchangeable attachment (210) comprises the circuit (852) which is configured to provide the function; wherein preferably the interchangeable attachment (210) comprises an end portion in the form of a tool (214) which represents the activity of the machine (114).

9. System (200, 700, 800, 1200) according to claim 8, further comprising: at least one additional interchangeable attachment (210a to 210g), wherein the additional interchangeable attachment is configured according to another activity which is different from the activity, and wherein the additional interchangeable attachment (210a to 210g) and the interchangeable attachment (210) are configured in such a way that they can be exchanged for one another.

10. System (200, 700, 800, 1200) according to either of claims 8 or 9, wherein the apparatus external to the hand-held device comprises a data processing means (502) which is configured to determine control information for the machine (114) on the basis of the activation information and the activity, wherein preferably the data processing means (502) comprises a communication interface which is configured according to a communication protocol of the machine (114); and wherein the data processing means is configured to communicate with the machine by means of the communication interface, wherein further preferably the data processing means (502) is configured to transmit the control information to the machine (114) by means of the communication interface.

11. System (200, 700, 800, 1200) according to claim 10, further comprising: a fastening apparatus which is configured to fasten the hand-held device to the machine, wherein the data processing means (502) is further configured to perform a calibration sequence by controlling the machine (114) by means of the communication interface when the hand-held device is fastened to the machine by means of the fastening apparatus.

12. Method comprising: • detecting spatial information of the hand-held device (100) according to any of claims 1 to 12 when the activation information has been input; • determining control information for the machine (114) on the basis of the spatial information and the activity.

13. Nonvolatile storage medium comprising code segments configured, when executed by a processor, to perform the method according to claim 12.

14. Hand-held device (100) for training at least one movement and at least one activity of a machine (114), the hand-held device (100) comprising: • a handle (104); • an input unit (108) configured to input activation information for activating the training of the machine (114); • an output unit (110) configured to output the activation information for activating the training of the machine (114) to an apparatus external to the hand-held device; • a tool that is configured according to the at least one activity; and characterized in that the hand-held device includes: • one or more than one first infrared sensor (802s) which is configured to detect a preferably electromagnetic location signal external to the hand-held device.

15. System (200, 700, 800, 1200) for training at least one movement and at least one activity of a machine, comprising: a hand-held device (100) (114) comprising: • a handle (104); • a tool or a coupling structure (102) for releasably coupling an interchangeable attachment (210), wherein the tool or the interchangeable attachment (210) is configured according to the at least one activity; • one or more than one first infrared sensor (802s) which is configured to detect a preferably electromagnetic location signal external to the hand-held device; an apparatus external to the hand-held device, comprising: • an input unit configured to input activation information for activating the training of the machine; and • an output unit (110) configured to output the activation information for activating the training of the machine (114).