Creating and executing robot applications

Abstract

The invention relates to a method for creating a robot application in which the following steps are repeated several times: selecting a routine from a predefined set of selectable routines (A-F) by means of data processing based at least in part on machine learning on the basis of a voice input (S); parameterising at least one parameter of the selected routine by means of data processing based at least in part on machine learning on the basis of the voice input and / or a context of the robot application to be created; and adding the selected and parameterised routine to the robot application. The invention also relates to a system and to a computer program (product).

Description

[0001] 120529P524PC 1 / 22 KUKA Deutschland GmbH

[0002] 2024P00005WO

[0003] Description

[0004] Creating and implementing robot applications

[0005] The present invention relates to a method for creating or carrying out a robot application, as well as a system and a computer program or computer program product for carrying out a method described herein.

[0006] The object of the present invention is to improve the creation, and in particular the execution, of robot applications.

[0007] This problem is solved by a method with the features of claim 1 and 10, respectively. Claims 11 and 12 protect a system, computer program, and computer program product for carrying out a method described herein. The dependent claims relate to advantageous embodiments.

[0008] According to one embodiment of the present invention, the following steps are taken to create a robot application:

[0009] - Selections (each) of a routine

[0010] - from a predefined set of selectable routines

[0011] - by means of data processing based at least partially on machine learning

[0012] - based on (each) one voice input;

[0013] - Parameterizing one or more parameters of the (respectively) selected routine

[0014] - by means of data processing based at least partially on machine learning

[0015] - based on the (respective) speech input and / or a context of the robot application to be created, in particular therefore

[0016] - Parameterizing one or more parameters of the (respectively) selected routine based on the (respective) speech input and / or 120529P524PC 2 / 22 KUKA Deutschland GmbH

[0017] 2024P00005WO

[0018] - Parameterizing one or more parameters of the (respectively) selected routine based on the context of the robot application to be created;

[0019] - in a further training course, parameterizing one or more parameters of the (respective) selected routine based on the (respective) speech input and the context of the robot application to be created; and

[0020] Adding the (respective) selected and parameterized routine to the robot application can be repeated several times. According to one embodiment of the present invention, a robot application created according to the invention or according to a method described herein is executed.

[0021] By creating routines using or with the aid of artificial intelligence

[0022] - based on voice input

[0023] - selected from a predefined set of selectable routines and

[0024] By parameterizing the selected routines, the robot application can be advantageously created faster, more securely, and / or by non-experts. In particular, more general chatbot interfaces that are not fine-tuned to specific routines can be used, and / or chatbot errors, especially so-called hallucinations or unexpected and incorrect routines, can be avoided or reduced.

[0025] In one embodiment, a robot application comprises a movement and / or a work process of one or more robots and / or a robot environment, in a further development, tools, conveying devices and the like that interact with the robot(s), and / or a program for controlling the robot(s) and / or the robot environment, wherein the robot(s) in one further development (each) has a stationary or mobile base and / or a robot arm with at least three, preferably at least six, in particular at least seven, joints or axes of movement, and / or the robot environment has one or more with 120529P524PC 3 / 22 KUKA Deutschland GmbH

[0026] 2024P00005WO includes tools, in particular machining and / or holding tools, and / or conveying equipment, that interact with or are set up or used for this purpose by the robot(s). The robot application may, in particular, comprise a real robot application or movement and / or work process of one or more real robots and / or a real robot environment and / or a (corresponding) program and / or a virtual robot application or movement and / or work process of one or more virtual robots and / or a virtual robot environment and / or a (corresponding) program. Accordingly, the execution of a robot application may, in particular, involve the actual execution of a (real) robot application.Movement and / or work process of one or more real robots and / or a real robot environment and / or a virtual, in particular simulated, execution of a (virtual) robot application or movement and / or work process of one or more virtual robots and / or a virtual robot environment, in particular, include.

[0027] One or more of the routines, in each execution, comprise actions, in particular movements and / or functions, of the (real and / or virtual) robot(s) and / or robot environment, and / or commands or instructions for the robot(s) (environment), and / or functions of a robot simulation and / or robot programming and / or simulation environment. In one execution, the procedure for creating a robot application comprises providing, or in a further development, specifying, the (predefined) set of selectable routines. In one execution, the selectable routines are specified in textual form; in a further development, they each comprise a command and / or a parameter and / or a description, in particular a functional description, and / or in textual form.Additionally or alternatively, the selectable routines, especially their descriptions, include (program) code and / or images in one execution, and image sequences in a further training.

[0028] In one implementation, selecting one or more of the routines (each) involves the following steps: 120529P524PC 4 / 22 KUKA Deutschland GmbH

[0029] 2024P00005WO

[0030] - Embedding the speech input into a vector called the input vector in a vector space, wherein this vector space contains vectors called function vectors into which the selectable routines are embedded, in an execution based on the description of the respective routine; and

[0031] - Selection of one of the selectable routines taking into account a relation, preferably proximity, between the input vector assigned to the speech input and the function vectors assigned to the selectable routines.

[0032] In the present context, embedding is understood in particular, in a manner customary in the field, as an assignment, preferably bijective, between the data to be embedded, here the speech input or the selectable routines, and the vectors, preferably such that the distance between vectors is smaller the more similar the data assigned to them (to be embedded) are to each other or the better they match or correspond to each other.

[0033] By transforming speech input and selectable routines into a common vector space, these can be handled advantageously, in particular suitable routines can be selected particularly efficiently and / or reliably, and / or speech input can be processed particularly efficiently and / or easily, thereby (further) improving the creation of the robot application.

[0034] In a further education course, selecting a routine, taking into account a relation between the (respective) input vector and the function vectors, involves (at least) the following steps:

[0035] - Determining the distances of the input vector to the function vectors; and

[0036] - Selecting a routine based on a determined distance between the input vector and the function vector assigned to that (selected or to be selected) routine. The distance between two vectors u, v can encompass a mathematical norm, in particular, a cosine metric or similarity u vl (||u|| • ||vj|) with the scalar product u ■ v and the magnitudes ||u||, || vj| has proven particularly suitable. (See 120529P524PC 5 / 22 KUKA Deutschland GmbH)

[0037] 2024P00005WO, which in particular results from this, means that a distance between vectors of the vector space is understood in particular as a similarity of these vectors, whereby other metrics for the distance are also suitable or possible.

[0038] By using distances in the (common) vector space, the most suitable routines can be selected particularly advantageously, especially in a particularly efficient and / or reliable manner, and thereby the creation of the robot application can be (further) improved.

[0039] Preferably, the routine that, according to a machine-learned model, shows the greatest similarity to the speech input is selected.

[0040] In a further training course, selecting a routine taking into account the determined distance of the respective input vector to the function vectors involves a nearest-neighbor classification ("nearest-neighbor algorithm") of the function vectors, whereby in one execution a predetermined number of nearest neighbors (among or from the function vectors) to the input vector are determined and / or the routine whose assigned function vector is closest to the input vector is selected.

[0041] Additionally or alternatively, in one implementation a ranking or ranking process, or in a further training, a reranking or adjustment of a ranking of the selectable routines is carried out, preferably based on a match or compatibility of the speech input with the respective routine determined by a machine-learned model, preferably based on the description of the respective routine, wherein in one further training the determined nearest neighbors, or in another further training the specified number of nearest neighbors, are ranked (re)ranked and the remaining routines are discarded or can be disregarded (in the (respective) selection).

[0042] Therefore, two particularly advantageous training courses for selecting a routine are suggested: 120529P524PC 6 / 22 KUKA Deutschland GmbH

[0043] 2024P00005WO

[0044] - The routine whose assigned function vector is closest to the input vector is selected; or

[0045] - Several routine candidates are identified based on the proximity of their associated function vectors to the input vector, and from these, based on a match or compatibility of the speech input with the respective routine determined by a machine-learned model, preferably its description, the best routine or the one that best matches the speech input according to the machine-learned model is selected, whereby the selection is of course not limited to these two approaches.

[0046] Through a nearest-neighbor algorithm and (optional) ranking, the most suitable routines can be selected particularly advantageously, especially efficiently and / or reliably, thereby (further) improving the creation of the robot application.

[0047] In addition to or as an alternative to a nearest-neighbor algorithm and / or reranking, in one implementation a speech input is classified as an anomaly if the determined distance of the input vector associated with this speech input to the function vectors fulfills a predefined condition; in a further development, a determined distance of the input vector associated with this speech input to the function vectors, in particular a minimum, maximum, or statistically averaged distance, exceeds a predefined limit, or the like. In one implementation, in another further development, this limit is specified based on statistical evaluations and / or statistically and / or probabilistically determined distributions, preferably such that statistically more frequent and / or more probable speech inputs or input vectors are not classified as anomalies, or statistically rarer and / or less probable speech inputs or input vectors are not classified as anomalies.Input vectors can be classified as anomalies. If an anomaly is classified, a corresponding message can be issued.

[0048] This allows unsuitable voice inputs to be filtered out particularly advantageously, especially efficiently and / or reliably, thereby (further) improving the creation of the robot application. 120529P524PC 7 / 22 KUKA Deutschland GmbH

[0049] 2024P00005WO

[0050] In one implementation, reranking is performed using a (machine-learned) Large Language Model (“LLM”). Additionally or alternatively, in another implementation, one or more parameters of one or more selected routines are parameterized using one or more, preferably different, (machine-learned) Large Language Models (LLMs). Additionally or alternatively, in another implementation, language input is embedded in an input vector and / or the routines are embedded in the function vectors using at least one (machine-learned) embedding model; in a further development, this is performed using one or the same or different (machine-learned) Large Language Models (LLMs). By using, in particular, these machine-learned models for selecting and / or embedding the routines in the function vectors, the system achieves a more efficient and efficient reranking process.Parameterizing a routine using data processing that is at least partially based on machine learning allows for particularly advantageous handling of speech inputs and routines, especially enabling the selection of suitable routines to be particularly efficient and / or reliable, and / or the processing of speech input to be particularly efficient and / or simple, thereby (further) improving the creation of the robot application.

[0051] In one implementation, the parameterization of one or more parameters of one or more selected routines is performed based on the voice input on which that routine was selected. For example, if a voice input is "The robot's TCP should perform a linear movement to the nearest welding point at a speed of 0.8 m / s," then, due to the small distance between the input vector associated with this voice input and the function vector associated with this command, the command "LinMotion(working point, speed)" can be selected. Then, its parameter "speed" can be set based on the phrase "at a speed of 0.8 m / s," and its parameter "working point" can be set based on the context of the robot application to be created.In this context, "context" is generally understood to mean information that characterizes the situation of an entity in interaction with other entities, in particular a relationship between one or more references of one or more robots and / or a robot environment (the robot application). 120529P524PC 8 / 22 KUKA Deutschland GmbH.

[0052] 2024P00005WO

[0053] Additionally or alternatively, in one embodiment the parameterization of several parameters of one or more selected routines is carried out step by step, preferably in such a way that one of the parameters of the (respective) routine is parameterized or its value is set successively.

[0054] This allows the routines to be parameterized in a particularly advantageous, especially efficient and / or reliable way, thereby (further) improving the creation of the robot application.

[0055] In one version, one or more of the voice inputs (each) are written voice inputs, or in a training course, voice inputs entered via keyboard, touchscreen, or the like. Additionally or alternatively, one or more of the voice inputs (each) are acoustic voice inputs, or in a training course, voice inputs entered via microphone or the like and / or converted into text form using speech recognition.

[0056] This allows the robot application to be created more quickly and / or safely and / or by non-experts.

[0057] As already mentioned, the selectable routines in one version include descriptions, preferably at least partially in textual form; in a further development, the descriptions may also include images, especially image sequences, or the like. The descriptions preferably characterize the respective routine; in particular, they may describe its functionality.

[0058] Preferably, the routines are selected based on these descriptions or their match or correspondence with the respective language input; in further training, the routines are embedded in the vector space based on their descriptions.

[0059] In this way, voice input can be implemented in routines in a particularly advantageous, especially efficient and / or reliable way.

[0060] In one implementation, these descriptions are adapted using data processing based at least partially on machine learning, preferably based on speech input. 120529P524PC 9 / 22 KUKA Deutschland GmbH

[0061] 2024P00005WO

[0062] This allows the creation of robot applications to adapt to the language inputs used to adjust the descriptions, and thus further language inputs can be implemented in routines in a particularly advantageous, especially more efficient and / or reliable way.

[0063] In one implementation, an LLM-based chatbot interface is provided, implemented, or used for creating a robot application, whereby the chatbot can advantageously select from a predefined set of routines and / or parameterize the selected routines using user or voice input and / or context and / or stepwise. This allows the robot application to be created advantageously faster and / or more securely and / or by non-experts.

[0064] According to one embodiment of the present invention, a system for creating a robot application, in a further development for carrying out the created robot application, is set up and / or has, in particular in terms of hardware and / or software, especially programming, for carrying out a method described herein:

[0065] - Means of selecting a routine from a given set of selectable routines by means of data processing based at least partially on machine learning and based on speech input;

[0066] - Means of parameterizing at least one parameter of the selected routine by means of data processing based at least partially on machine learning, based on speech input and / or a context of the robot application to be created; and

[0067] - Means of adding the selected and parameterized routine to the robot application.

[0068] In one version, the system or its means exhibit:

[0069] - Means for embedding speech input in an input vector in a vector space, wherein the vector space has function vectors in which the selectable routines are embedded; and / or 120529P524PC 10 / 22 KUKA Deutschland GmbH

[0070] 2024P00005WO

[0071] - Means of selecting one of the selectable routines, taking into account a relation between the input vector associated with the speech input and the function vectors associated with the selectable routines; and / or

[0072] - Means of determining distances between the input vector and the function vectors; and / or

[0073] - Means of selecting a routine taking into account a determined distance of the input vector to the function vector assigned to that routine; and / or

[0074] - Means of performing a nearest-neighbor classification of the function vectors and / or a ranking of the selectable routines for selecting a routine, taking into account a determined distance of an input vector to function vectors; and / or

[0075] - Means of classifying a speech input as an anomaly if the determined distance of the input vector associated with this speech input to the function vectors satisfies a predefined condition; and / or

[0076] - at least one Large Language Model to perform the ranking; and / or

[0077] - at least one Large Language Model for parameterizing one or more parameters of at least one selected routine based on speech input and / or a context of the robot application to be created, preferably stepwise; and / or

[0078] - Means of adapting descriptions of routines based on speech input using data processing that is at least partially based on machine learning; and / or

[0079] - the robot(s) and / or robot environment and / or a robot simulation and / or robot programming environment and / or robot simulation environment and / or the data processing(s) that are at least partially based on machine learning.

[0080] A means according to the present invention can be designed using hardware and / or software, in particular at least one, preferably data- or signal-connected, in particular digital, processing unit, in particular a microprocessor unit (CPU), 120529P524PC 11 / 22 KUKA Deutschland GmbH

[0081] 2024P00005WO

[0082] A computer program product may include a graphics processing unit (GPU) or similar device, and / or one or more programs or program modules. The processing unit may be configured to execute instructions implemented as a program stored in a memory system, to acquire input signals from a data bus, and / or to output signals to a data bus. A memory system may include one or more, in particular different, storage media, especially optical, magnetic, solid-state, and / or other non-volatile media. The program may be designed to embody or execute the procedures described herein, enabling the processing unit to perform the steps of such procedures and thus, in particular, to create or execute the robot application. A computer program product may, in one version, include a storage medium, in particular a computer-readable and / or non-volatile medium, for storing a program or program module.of instructions or with a program or instructions stored thereon, in particular, being. In an execution, the execution of this program or these instructions by a system or a controller, in particular a computer or an arrangement of several computers, causes the system or the controller, in particular the computer(s), to execute a procedure described herein or one or more of its steps, or the program or instructions are configured for this purpose.

[0083] In one embodiment, one or more, in particular all, steps of the procedure are fully or partially computer-implemented, or one or more, in particular all, steps of the procedure are fully or partially automated, in particular by the system or its means.

[0084] Further advantages and features will become apparent from the dependent claims and the exemplary embodiments. These are shown, in part schematically:

[0085] Fig. 1: A system for creating and executing a robot application according to an embodiment of the present invention; and 120529P524PC 12 / 22 KUKA Deutschland GmbH

[0086] 2024P00005WO

[0087] Fig. 2: a part of the system and a method for creating and executing the robot application according to an embodiment of the present invention

[0088] Fig. 1 shows a system for creating and executing a robot application according to an embodiment of the present invention, and Fig. 2 shows a part of this system as well as a method for creating and executing the robot application according to an embodiment of the present invention. The system comprises (see Fig. 1) a robot 1000 and a computer system 2000, which may in particular include a robot programming and / or simulation environment and / or robot control. The computer system 2000 may in particular include several computers, and the robot 1000 may be a virtual or simulated robot or a real robot.

[0089] In step S10, a set of selectable routines is specified, which are schematically indicated by squares and capital letters in Fig. 2. For example, A can be a function or command "lnsertObject(object_name: str)" for inserting an object parameterized by the string "object_name", B a function or command "DeleteObject(object_name: str)" for deleting an object parameterized by the string "object_name", C a function or command "LinMotion(point_name: str, velocity: float = 0.1 , blending: bool = False)" for linear movement of a TCP of robot 1000 to the target point parameterized by the string "point_name" with the floating-point number

[0090] a speed parameterized by "velocity" and preset with the default value "0.1" and the blending selectable by the boolean parameter "blending" and preset with the default value "False", D an analogous function or command "PTPotion(point_name: str, velocity: float = 0.1 , blending: bool = False)" for a PTP movement of the TCP, , E a function or command "OpenWeldingDialogue(welding_object: str, power_source: str, welding_gun: str)" for configuring a welding of the environment object or workpiece parameterized by the string "welding_object" with the string

[0091] The energy source parameterized by "power_source" and the welding tool parameterized by the string "welding_gun", and F a function or command "TransformObject(transform_object: str,...)" to transform a by the 120529P524PC 13 / 22 KUKA Deutschland GmbH

[0092] 2024P00005WO

[0093] The string "transform_object" illustrates the parameterized object according to the further parameters, which are not listed here.

[0094] The routines (here exemplified as A - F) each have a functional description, preferably at least partially in textual form, and in further training also including images or the like. Using routine C ("LinMotion(point_name: str, velocity: float = 0.1 , blending: bool = False)") as an example, such a description might read as follows:

[0095] "Generates a linear motion to a target point named point_name. The motion is linear in Cartesian space and is performed at the specified velocity [m / s]. The motion can be overridden at the target point or not."

[0096] Parameters: point_name: String - the name of the target point of the linear movement. velocity: Floating-point number - the speed of the movement. blending: Boolean variable - whether the movements are blended or not.

[0097] In step S15, the routines are embedded in a vector space, preferably based on their descriptions in function vectors, using a L(arge )L(anguage )M(odels)s LLMi, which are schematically indicated in Fig. 2 by circles and corresponding lowercase letters, i.e., routine A is embedded in the function vector a associated with it, routine B in the function vector b, routine C in the function vector c, routine D in the function vector d, routine E in the function vector e, and routine F in the function vector f.

[0098] In step S20, a speech input S, also indicated by a square, is provided.

[0099] This speech input S is processed in step S25 using the (same)

[0100] L(arge )L(anguage )M(odels)s LLMi embedded in an input vector s in the vector space. 120529P524PC 14 / 22 KUKA Deutschland GmbH

[0101] 2024P00005WO

[0102] The embedding model “text-embedding-ada” from OpenAI serves as an example.

[0103] In step S30, an anomaly detection method (ADD) classifies a speech input as an anomaly and discards it accordingly if the determined distance of the input vector associated with this speech input to the function vectors meets a predefined condition. This condition is schematically indicated in Fig. 2 by exceeding a distance threshold for a hatched input vector s, into which, for example, the speech input "Let's play football" would be embedded. This distance threshold can advantageously be predefined, and in particular adjusted, based on static evaluations of previously processed speech inputs or the like. An error message can be output in step S30 if an anomaly is classified.

[0104] In step S35, a Nearest-Neighbour-Search (NNS) is used to determine, for example, the three nearest neighbors from the function vectors a - f to the input vector s that have the smallest distance, again, in an advantageous embodiment, the smallest cosine metrics.

[0105] These are replaced in step S40 by another

[0106] L(arge )L(anguage )M(odels)s LLM2 ge-rerankt, which is configured, for example, by an instruction

[0107] "You are a computer assistant who selects suitable functions from lists. You only answer in natural numbers (the indices of the list)" and some examples, by way of illustration.

[0108] {'User': "Move the robot 5 cm in the z-direction and rotate it around the y-axis by 15 degrees.

[0109] List of functions: [

[0110] 'create_point_at_tcp(robot_name: str)', 120529P524PC 15 / 22 KUKA Deutschland GmbH

[0111] 2024P00005WO

[0112] 'transform_object(object_name: str, x: float = 0, y: float = 0, z: float = 0, a: float = 0, b: float = 0, c: float = 0)',

[0113] 'create_linear_motion(point_name: str, velocity: float = 0.1 , blending: bool = False)' ], 'Assistant': '1'}.

[0114] The list for this model is the output of NNS. If the LLM2 model outputs an index j != 0, the element of j is moved to the top of the list. This step can also be used to detect unusual speech inputs and, in this case, output the message: "No routine in the list is suitable for this speech input".

[0115] In the example, the user has voice input

[0116] "Please create a linear motion with a speed of 0.8 m / s" was entered.

[0117] Accordingly, routine C (LinMotion(point_name: str, velocity: float = 0.1, blending: bool = False)) is selected (Fig. 2: step S40) and parameterized in step S50 by another LLM3 (Language Model). The simulation context is included and taken into account. This context can be determined from the simulation state and transformed into text form for LLM3, for example, in the following form:

[0118] "The user has selected point 'P5'" or "The only welding source in the scene is 'ABCD1234'".

[0119] The LLM3 receives as input a description of the selected routine, the user's voice input, and context (information) of the simulation (environment) in text form and can be configured as follows, for example:

[0120] 'You are a KL Assistant that identifies parameters for routines. The parameters are taken either directly from the user's voice input or from the additional context description. Ensure that the parameter names exactly match the routine description. For example, if the routine description contains 'point_name', the parameter should be 120529P524PC 16 / 22 KUKA Deutschland GmbH'.

[0121] 2024P00005WO

[0122] Use 'point_name' and not "point". Prioritize voice input over context.

[0123] Always reply in JSON format.'

[0124] Examples: [

[0125] {'User': 'Create a linear movement to P5. Routine description: [... docstring of the routine...] Context: empty, 'Assistant': '{"Point": "P5"}'},

[0126] {'User': 'Create a linear motion. Routine description: [... routine docstring...] Context: P12 selected', 'Assistant': '{" point": "P12"}'},

[0127] {'User': 'Generate welding source "SFR10_as". Routine description: [...

[0128] Docstring of the routine...] Context: empty', 'Assistant': '{"object_name": "SFR10_as"}'},

[0129] {'User': Remove object "AB12". Routine description: [... docstring of the routine...] Context: Object "CD34" selected, 'Assistant': '{"object_name": "AB12"}'}].

[0130] Further speech inputs may be processed. Otherwise, the robot application created in this way by adding the corresponding routines is completed and executed if necessary (Fig. 2: Step S60). The descriptions of the routines can also be adapted to better match the speech inputs.

[0131] In the present disclosure, "has an X" does not generally imply an exhaustive list, but is a shorthand for "has at least one X" and also includes "has two or more X" as well as "has Y in addition to X". Although exemplary examples were explained in the preceding description, it should be noted that a multitude of variations are possible.

[0132] For example, the reranking step by LLM2 can be omitted, and the routine whose associated function vector is closest to the input vector can be selected. 120529P524PC 17 / 22 KUKA Deutschland GmbH

[0133] 2024P00005WG

[0134] Furthermore, it should be noted that the exemplary embodiments are merely examples and are not intended to restrict the scope of protection, applications, or structure in any way. Rather, the preceding description provides the skilled person with a guideline for implementing at least one exemplary embodiment, whereby various modifications, particularly with regard to the function and arrangement of the described components, can be made without departing from the scope of protection as defined by the claims and these equivalent combinations of features.

[0135] 120529P524PC 18 / 22 KUKA Deutschland GmbH

[0136] 2024P00005WO

[0137] List of reference signs

[0138] 1000 robots

[0139] 2000 Computer AF routine af function vector

[0140] S speech input s input vector

[0141] LLM 1 / 2 / 3 Large Language Model NNS Nearest Neighbor Search

[0142] ADD Anomaly Detection Based on Distance

[0143] APP Application

Claims

120529P524PC 19 / 22 KUKA Deutschland GmbH 2024P00005WO Patent claims 1. Method for creating a robot application, wherein the steps are: - Selecting a routine from a given set of selectable routines (AF) using data processing based at least partially on machine learning and language input (S); - Parameterizing at least one parameter of the selected routine using data processing based at least partially on machine learning, using speech input and / or a context of the robot application to be created; and - Adding the selected and parameterized routine to the robot application can be repeated several times.

2. The method according to claim 1, characterized in that the selection of a routine comprises the steps: - Embedding the speech input in an input vector (s) in a vector space, wherein the vector space has function vectors (af) in which the selectable routines are embedded; and - Selecting one of the selectable routines, taking into account a relation between the input vector assigned to the speech input and the function vectors assigned to the selectable routines.

3. Method according to claim 2, characterized in that the selection of a routine taking into account a relation between input vector and function vectors comprises the steps: - Determining the distances of the input vector to the function vectors; and - Selecting a routine taking into account a determined distance between the input vector and the function vector assigned to that routine.

4. Method according to claim 3, characterized in that - selecting a routine taking into account a determined distance of an input vector to function vectors - a nearest-neighbor classification of the function vectors and / or - a ranking of the selectable routines 120529P524PC 20 / 22 KUKA Deutschland GmbH 2024P00005WO; and / or that - a speech input is classified as an anomaly if the determined distance of the input vector assigned to this speech input to the function vectors fulfills a predefined condition.

5. Method according to claim 4, characterized in that the ranking is carried out using a Large Language Model (LLM2).

6. Method according to one of the preceding claims, characterized in that the parameterization of one or more parameters of at least one selected routine is carried out stepwise and / or by means of at least one Large Language Model (LLM3) based on a language input and / or a context of the robot application to be created.

7. Method according to one of the preceding claims, characterized in that at least one voice input comprises a written or acoustic voice input.

8. Method according to one of the preceding claims, characterized in that the selectable routines have descriptions.

9. Method according to claim 8, characterized in that these descriptions are adapted by means of data processing based at least partially on machine learning, in particular on the basis of speech inputs.

10. Method for performing a robot application, comprising the following steps: - Creating a robot application according to a method according to one of the preceding claims; and - Execute (S60) the created robot application.

11. System for creating a robot application, which is configured and / or comprises a method according to one of the preceding claims: 120529P524PC 21 / 22 KUKA Deutschland GmbH 2024P00005WO - Means of selecting a routine from a given set of selectable routines (AF) using data processing based at least partially on machine learning and based on speech input (S); - Means of parameterizing at least one parameter of the selected routine Routine using data processing based at least partially on machine learning, based on speech input and / or a context of the robot application to be created; and - Means of adding the selected and parameterized routine to the robot application.

12. Computer program or computer program product, wherein the computer program or computer program product, in particular stored on a computer-readable and / or non-volatile storage medium, contains instructions which, when executed by one or more computers or a system according to claim 11, cause the computer(s) or system to perform a method according to any one of claims 1 to 10.

Images