Controlling robots using language model-generated programs

Abstract

System for controlling a robot comprising: one or more computers; and one or more storage devices communicatively coupled to the one or more computers, the one or more storage devices storing instructions that, when executed by the one or more computers, cause the one or more computers to perform the following operations: Obtaining natural language instructions to interact with the robot; Generate an input sequence that includes: (i) the natural language instruction and (ii) a context sequence specifying one or more application programming interfaces (APIs) that can be called to obtain outputs generated by one or more sensors of the robot, to control the robot, or both; processing the input sequence using a neural code generation network to generate as output a computer code sequence in a programming language defining a computer program, the computer code sequence comprising computer code representing a call to one of the one or more APIs and arguments for the call; generating the computer program from the computer code sequence; and Running the computer program to interact with the robot.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims priority to U.S. Application No. 63 / 407,607, filed September 16, 2022, the disclosure of which is hereby incorporated by reference. GENERAL STATE OF THE ART

[0002] This specification concerns controlling robots using language and using machine learning models.

[0003] Machine learning models receive an input and generate an output, such as a predicted output, based on the received input. Some machine learning models are parametric models and generate output based on the received input and the values ​​of the model's parameters.

[0004] Some machine learning models are deep models that use multiple model layers to generate an output for a received input. For example, a deep neural network is a deep machine learning model that includes an output layer and one or more hidden layers, each of which applies a nonlinear transformation to a received input to generate an output. SUMMARY

[0005] This description generally describes a system implemented by computer programs on one or more computers at one or more locations that controls a robot that interacts with an environment using language model generated programs (LMP).

[0006] Certain embodiments of the subject matter described in this specification may be implemented to achieve one or more of the following advantages.

[0007] Providing users with the ability to control robots using natural language commands may be desirable for many types of robotics tasks.

[0008] However, it is difficult to implement a policy that successfully compels a robot to follow any given user instruction. Especially for robots that use language, there is a need to ground (or localize) it to establish a reference to the physical world and bridge connections between words, perceptions, and actions.

[0009] Some conventional methods analyze language using lexical analysis to extract semantic representations that provide information for policies, but these techniques have difficulty dealing with unknown instructions.

[0010] Other methods learn grounding from beginning to end (from speech to action), but these techniques require extensive data that can be costly to acquire on real robots.

[0011] This description, on the other hand, describes techniques for using a pre-trained neural code generation network, e.g., a language model pre-trained on computer code data, to generate robot policy code ("language model-generated programs") from natural language commands. Specifically, the generated code can express functions or feedback loops that process perceptual outputs (e.g., outputs from open-vocabulary object detectors) and parameterize primitive control APIs for controlling the robot. Specifically, the LMPs can accept new natural language commands and autonomously reassemble the available API calls to generate new policy code that executes the commands.Thus, the described system can be generalized to new instructions and work with any set of available APIs, allowing the system's outputs to move the robot in the environment and generate natural language text answers to questions posed in the natural language instructions.

[0012] The details of one or more embodiments of the subject matter are set forth in the accompanying figures and the following specification. Further features, aspects, and advantages will become apparent from the description, drawings, and claims. BRIEF DESCRIPTION OF THE DRAWINGS Fig. 1 shows an example robot policy system. Fig. Figure 2 is a flowchart of an example process for controlling the robot. Fig. 3 is a flowchart of an example process for generating a computer program from an output sequence. Fig. 4 shows an example of an LMP.

[0013] Like reference numerals in the different drawings indicate like elements. DETAILED DESCRIPTION

[0014] Fig. 1 shows an example robot policy system 100. The robot policy system 100 is an example of a system implemented as computer programs on one or more computers at one or more locations in which the systems, components, and techniques described below are implemented.

[0015] The robot policy system 100 controls a robot 104 that interacts with an environment 106 to perform tasks.

[0016] The robot 104 may be any suitable type of robot, e.g., a robot arm, a humanoid robot, a quadruped robot, a robot in a vehicle, e.g., an autonomous vehicle, and so on.

[0017] As a generalized example, the task may involve one or more of, for example, navigating to a particular location in the environment, identifying a particular object in the environment, manipulating the particular object in a particular way, and so on.

[0018] In some cases, and as described below, the task is specified by a natural language instruction 122 received by the system 100.

[0019] The robot 104 generally includes one or more sensors 105 that sense the environment, such as one or more of the following: camera sensors, laser sensors, radar sensors, temperature sensors, microphone sensors, proprioceptive sensors, such as optical encoders, gyroscopes, accelerometers, inertial measurement sensors, and so on.

[0020] Robot 104 also generally includes software that processes measurements from these sensors to generate outputs, such as machine learning models or other software that detects, locates, or otherwise characterizes objects in the environment. For example, robot 104 may include machine learning models that perform object localization, open vocabulary object recognition, and so on.

[0021] The robot 104 is controlled by specifying parameters 108 for one or more primitive controllers. A primitive controller is a function that maps a set of parameters to a sequence of one or more control inputs for the robot. For example, a primitive controller may be a function that maps parameters specifying a target angular velocity for the robot to a sequence of control inputs that cause the robot to have the target angular velocity. As another example, another primitive controller may be a function that maps a set of parameters specifying a starting location of a robot object and an end location of the object to a sequence of control inputs that cause the robot to move the object from the starting location to the end location.As another example, another primitive controller may be a function that maps a set of parameters specifying a starting location of a gripper of a robot object and an end location of the gripper to a sequence of control inputs that cause the robot to move the gripper from the starting location to the end location. As another example, another primitive controller may be a function that maps a set of parameters specifying a starting location of a gripper of a robot object and an end location of the gripper to a sequence of control inputs that cause the robot to move the gripper from the starting location to the end location.

[0022] A control system 107 for the robot 104 then maps these primitive control parameters 108 into control inputs for one or more controllable elements of the robot, e.g., position, velocity, or force / torque / acceleration data for one or more joints or other control elements of the robot. That is, the control system 107 executes the function specified by the primitive controller 108 to map the set of parameters to the corresponding sequence of control inputs.

[0023] In other words, system 100 has access to a set of primitive controls for robot 104, each of which maps a set of parameters to a set of control inputs for robot 104. By providing the primitive control parameters 108 for a given primitive control to control system 107, system 100 can cause control system 107 to map the parameters 108 to control inputs for the robot, thereby controlling robot 104.

[0024] More specifically, the system 100 controls the robot through language model generated programs (LMP) 122.

[0025] An LMP 122 is a computer program generated from a computer code sequence produced as output by a neural code generation network 120.

[0026] A neural code generation network 120 is a neural network that receives an input sequence of tokens in a vocabulary for a programming language, e.g., Python, C++, C, or another programming language, and processes the input sequence to generate an output sequence of tokens in the vocabulary that specifies a computer program in the programming language.

[0027] For example, the neural code generation network 120 may have been trained on a language modeling objective using a corpus of training computer code sequences from training computer programs. That is, the neural code generation network 120 may have been trained on a next token prediction task that requires the neural network 120 to predict the next token in a computer code sequence, given the preceding tokens in the computed code sequence.

[0028] As a particular example, the neural code generation network 120 may have the architecture of an autoregressive neural language model network.

[0029] The language model neural network is called an autoregressive neural network because the neural network autoregressively generates an output sequence of tokens by generating each individual token in the output sequence under the condition of a current input sequence that includes all tokens preceding the respective text token in the output sequence, that is, the tokens already generated for all previous positions in the output sequence preceding the specific position of the specific token, and a context input that provides the context for the output sequence. For example, when generating a token at a given position in the output sequence, the current input sequence may include the context sequence and the tokens at all previous positions preceding the given position in the output sequence.As a specific example, the current input sequence may include the context sequence followed by the tokens at all preceding positions preceding the given position in the output sequence. Optionally, the context and the current output sequence may be separated by one or more predetermined tokens within the current input sequence.

[0030] More specifically, to generate a specific token at a specific position within a candidate output sequence, the neural network 120 may process the current input sequence to generate a score distribution, e.g., a probability distribution, that assigns a corresponding score, e.g., a corresponding probability, to each token in the token vocabulary. The neural network 120 may then select a token from the vocabulary as the specific token based on the score distribution. For example, the neural network 120 may greedily select the token with the highest score or, for example, sample a token from the distribution using kernel sampling or another sampling technique.

[0031] As a specific example, the speech neural network model 120 may be an autoregressive, transformer-based neural network that includes (i) a plurality of attention blocks, each applying a self-supervision operation, and (ii) an output subnetwork that processes an output of the last attention block to generate the score distribution.

[0032] The neural network 120 may represent an acceptable Transformer-based neural network architecture. Examples of such architectures are described in J. Hoffmann, S. Borgeaud, A. Mensch, E. Buchatskaya, T. Cai, E. Rutherford, D. d. L. Casas, LA Hendricks, J. Welbl, A. Clark, et al. Training compute-optimal large language models, arXiv preprint arXiv:2203.15556, 2022; JW Rae, S. Borgeaud, T. Cai, K. Millican, J. Hoffmann, HF Song, J. Aslanides, S. Henderson, R. Ring, S. Young, E. Rutherford, T. Hennigan, J. Menick, A. Cassirer, R. Powell, G. van den Driessche, LA Hendricks, M. Rauh, P. Glag, J. P. Huang, J. Huang, R. Ring, S. Young, E. Welbl, S. Dathathri, S. Huang, J. Uesato, J. Mellor, I. Higgins, A. Creswell, N. McAleese, A. Wu, E. Elsen, SM Jayakumar, E. Buchatskaya, D. Budden, E. Sutherland, K. Simonyan, M. Paganini, L. Sifre, L. Marten, A. Li, A. Kunco, A. Kunco. Nematzadeh, E. Gribovskaya, D. Donato, A. Lazaridou, A. Mensch, J. Lespiau, M. Tsimpoukelli, N. Grigorev, D. Fritz,T.Sottiaux, M. Pajarskas, T. Pohlen, Z. Gong, D. Toyama, C. de Masson d'Autume, Y. Li, T. Terzi, V. Mikulik, I. Babuschkin, A. Clark, D. de Las Casas, A. Guy, C. Jones, J. Bradbury, M. Johnson, B. A. Hechtman, L. Weidinger, I. Gabriel, W. S. Isaac, E. Lockhart, S. Osindero, L. Rimell, C. Dyer, O. Vinyals, K. Ayoub, J. Stanway, L. Bennett, D. Hassabis, K. Kavukcuoglu und G. Irving. Scaling language models: Methods, analysis & insights from training gopher. CoRR, abs / 2112.11446, 2021; Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li und Peter J Liu beschriebenen. Exploring the limits of transfer learning with a unified text-to-text transformer. arXiv preprint arXiv:1910.10683, 2019; Daniel Adiwardana, Minh-Thang Luong, David R. So, Jamie Hall, Noah Fiedel, Romal Thoppilan, Zi Yang, Apoorv Kulshreshtha, Gaurav Nemade, Yifeng Lu und Quoc V. Le. Towards a human-like open-domain chatbot. CoRR, abs / 2001.09977, 2020; und Tom B Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. arXiv-Vorabdruck arXiv:2005.14165, 2020.

[0033] In general, however, the transformer-based neural network includes a sequence of attention blocks, and while processing a given input sequence, each attention block in the sequence receives a respective hidden output state for each input token in the given input sequence. The attention block then updates at least the hidden state for the last token in the given input sequence, at least in part by applying self-attention, to generate a corresponding hidden output state for the last token. The hidden input states for the first attention block are embeddings of the input tokens in the input sequence, and the hidden input states for each subsequent attention block are the hidden output states generated by the preceding attention block.

[0034] In this example, the output subnetwork processes the hidden output state generated by the last attention block in the sequence for the last input token in the input sequence to generate the point distribution.

[0035] Because the neural network 120 is autoregressive, the system 100 can generally use the same neural network 110 to generate multiple different candidate output sequences in response to the same query, e.g., using beam search decoding from point distributions generated by the neural network 110, using a sample-and-rank decoding strategy, using different random seeds for the pseudorandom number generator used for sampling for different passes through the neural network 120, or using another decoding strategy that takes advantage of the autoregressive nature of the neural network.

[0036] To control the robot 104, the system 100 obtains a natural language instruction 112 to interact with the robot 104.

[0037] For example, the system 100 may receive the instruction 112 as input, e.g., as text or audio input, from a user who wishes to cause the robot 104 to navigate through the environment or to obtain information generated as a result of the robot navigating the environment.

[0038] For example, instruction 112 may be an instruction for robot 104 to move an object to a specified location in the environment, navigate to a specified location in the environment, locate a specified item in the environment, or respond to a query about the environment.

[0039] The system 100 generates an input sequence 114 that includes (i) the natural language instruction 112, represented as tokens from the vocabulary, and (ii) a context sequence that specifies one or more application programming interfaces (APIs) that can be called to obtain outputs generated by processing measurements from the robot's one or more sensors for controlling the robot, or both. For example, the natural language instruction can be formatted as a comment in an input sequence of code.

[0040] That is, the API may include one or more APIs to query certain types of information obtained from the sensor and one or more APIs to parameterize the primitive controls used to control the robot.

[0041] The system processes the input sequence using the neural code generation network 120 to produce as output a computer code sequence in the programming language that defines a computer program for interacting with the robot.

[0042] Generally, the computer code sequence includes computer code that represents a call to one or more of the APIs, as well as arguments for the call. That is, by including the context sequence in the input sequence, the neural code generation network 120 is capable of generating an output sequence that uses the API to obtain information from the environment, control the robot, or both.

[0043] The system 100 generates the computer program, i.e., the LMP 122, from the computer code sequence and then executes the computer program to interact with the robot. As described in more detail below, generating the computer program from the computer code sequence may include, for example, one or more of adapting the code sequence to a syntax for an execution function or performing hierarchical function generation by parsing the abstract syntax tree of a code block and checking for functions that do not exist in the given scope.

[0044] Interacting with the robot may include controlling the robot. Depending on the API call(s) included in the computer code sequence, controlling the robot includes causing the robot to move within an environment and / or causes information derived from sensor readings to be retrieved. That is, if the computer code sequence includes an API call to an API that parameterizes the primitive controls, executing the computer program causes the robot to move within the environment. If the computer code sequence includes an API call to an API that provides information from sensors, executing the computer program causes the information to be retrieved and processed according to the rest of the sequence.

[0045] In some cases, such as when instruction 112 includes a question about the environment, LMP 122 may return a natural language response. System 100 may then provide a response to the user who submitted instruction 112, such as natural language text or speech using a text-to-speech engine.

[0046] As will be appreciated by one skilled in the art, executing the computer program for interacting with the robot may include causing the generation of one or more control inputs for one or more controllable elements of the robot (e.g., by the control system 107). The control inputs generated by the control system 107 for controlling the robot may include, for example, torques for the robot's joints or higher-level control commands. In other words, the control inputs may include, for example, position, velocity, or force / torque / acceleration data for one or more joints of a robot or parts of another mechanical agent. Control inputs may additionally or alternately include electronic control data, such as motor control data, or more generally, data for controlling one or more electronic devices within the environment whose control affects the observed state of the environment.

[0047] Fig. Figure 2 is a flowchart of an example process 200 for controlling the robot. For simplicity, the process 200 is described as being performed by a system of one or more computers located at one or more locations. For example, a robot policy system, such as the robot policy system 100 of Fig. 1, which is programmed accordingly according to this specification, perform process 200.

[0048] The system obtains a natural language instruction to interact with the robot (step 202).

[0049] The system generates an input sequence that includes (i) the natural language instruction and (ii) a context sequence that specifies one or more application programming interfaces (APIs) that can be called to obtain outputs generated by the one or more sensors of the robot for controlling the robot, or both (step 204).

[0050] In some implementations, the context sequence includes a k-shot prompt, where no integer is greater than or equal to one. In other words, the input sequence further includes k, i.e., one or more example sequences, each including (i) a natural language example statement and (ii) an example computer code sequence in the programming language generated in response to the natural language example statement. Generally, the example computer code sequences include calls to the API for controlling the robot and calls to the API for obtaining outputs from one or more of the robot's sensors. By including the k-shot prompt in the input sequence, the system provides the code generation neural network with information about which APIs are available to invoke, as well as examples of how the APIs can be used to perform the natural language input.As described above, the natural language instruction in each example can be encoded as a comment. A simplified example of a one-time prompt might look like this: # when you see an orange, move backward. if detect- object("orange"): robot.set_velocity(x=-0,1, y=0, z=0)where "# if you see an orange, move backward" is an example of a natural language statement formatted as commented code, "detect_object("orange")" is a call to an API that performs open-vocabulary object detection in images captured by the robot's sensors, and "robot.set_velocity(x=-0,1, y=0, z=0)" is a call to a primitive control API that sets the robot's angular velocity to the parameters specified in the call, i.e., with parameters x-velocity -0.1, y-velocity 0, and z-velocity 0.

[0051] In the simplified example above, the natural language instruction obtained by system 100 could include, for example, "Move right until you see the apple." The API call specified in the context sequence can then be reassembled by pre-trained neural code generation network 120 to generate a computer code sequence as follows: while not detect_object("apple"): robot. set velocity(x=0, y=0,1, z=0)

[0052] In some implementations, instead of or in addition to the k-shot prompt, the context sequence may include text that identifies each API and provides a natural language description of the function of each API, for example, also formatted as a code comment.

[0053] In some implementations, instead of the above, the context sequence may include code formatted as import statements that inform the neural network what APIs are available, as well as type hints on how to use those APIs.

[0054] In some implementations, the natural language instruction may be a natural language follow-up instruction that follows a previous instruction (and corresponding movement / control of the robot). In these implementations, the input sequence may also include the previous natural language instruction and the output sequence generated for the previous natural language instruction. This allows the code generation neural network to generate programs that utilize the context from previously generated programs and allows users to provide feedback that results in a change in the robot's control.

[0055] The system processes the input sequence using a neural code generation network to generate as output a computer code sequence in a programming language that defines a computer program (step 206). Generally, the computer code sequence includes computer code that represents a call to one or more APIs, as well as arguments for the call.

[0056] In some cases, the computer code sequence may also include code that represents a call to a library for the programming language that is not specified in the context sequence because the neural code generation network has been pre-trained on a code prediction task as described above, e.g., by using NumPy or other similar libraries to elicit spatial reasoning with coordinates.

[0057] The system generates the computer program from the computer code sequence of the computer (step 208) and then executes the computer program to interact with the robot (step 210). This allows the system to control the robot according to the computer program.

[0058] To execute the computer program, the system may invoke an execution function for the programming language with an input string derived from the computer code sequence, e.g., with an input string that formats the computer code sequence according to a syntax for the execution function.

[0059] More specifically, the system can call the execution function with the input string and with (i) a global directory specifying one or more APIs and (ii) a local directory that can be filled with variables defined during execution.

[0060] The local directory may be initialized as empty when the execution function is called and may be populated with values ​​when the computer program executes. For some natural language statements, the system may provide one or more values ​​from the local directory in response to the natural language statement after the computer program executes.

[0061] For example, the natural language instruction may include a request for the system to provide the user with information about the environment, such as who the natural language instruction was received from. As part of executing the program, the system populates the local directory with the values ​​specified in the request for information, and after execution, the system may provide one or more values ​​in response to the request. For example, the generated program may include a call to an API that outputs text and parameterizes the call with one or more values ​​stored in the local directory.

[0062] In some implementations, the system may first check whether the computer program can be executed safely before executing it. For example, the system may verify that there are no import statements, calls to certain functions that have been identified as risky, such as the "eval" or "exec" functions in Python, or special variables, such as variables that begin with "-" in a Python program.

[0063] Additionally, in some implementations, the system generates computer programs that are hierarchical.

[0064] This will be explained below with reference to Fig. 3 described in more detail.

[0065] Fig. Figure 3 is a flowchart of an example process 300 for generating a computer program from an output sequence. For simplicity, the process 300 is described as being executed by a system of one or more computers located at one or more locations. For example, a robot policy system, such as the robot policy system 100 of Fig. 1, which is programmed accordingly according to this specification, perform process 300.

[0066] The system may continue process 300 until the computer code sequence no longer includes any undefined functions. Once the computer code sequence no longer includes any undefined functions, the system may execute the computer program defined by the code sequences and the additional function definitions as described above.

[0067] The system determines whether the computer code sequence includes computer code that calls an undefined function (step 302).

[0068] That is, the system identifies all function calls in the computer code sequence and then determines whether any of the function calls call a function that has not been defined, for example, in a library for the programming language, within the computer code sequence, or within the local directory.

[0069] For example, in Python, the system can analyze the abstract syntax tree of a code block to search for functions that are not present in the given scope.

[0070] If the system identifies a function call that invokes an undefined function, the system generates a new input sequence that includes a natural language statement to define the function (step 304).

[0071] For example, the new input sequence may include a natural language statement "# define function: get_obj_bbox_area(obj_name)", where "get_obj_bbox_area(obj_name)" is the name of the function that was called in the computer code sequence and was determined to be undefined.

[0072] In some implementations, the new input sequence also includes one or more of the natural language statement, the context sequence, or the output sequence generated by the code generation neural network. This can provide the context for the code generation neural network to generate the definition of the undefined function.

[0073] The system then processes the new input sequence using the neural code generation network to generate a new output sequence that includes new computer code that defines the function (step 306).

[0074] The system then adds the new computer code that defines the function, depending on the scope of the program. For example, the system can add data specifying the new computer code that defines the function to the global directory, the local directory, or both. As another example, the system can generate a modified computer code sequence that replaces the original computer code with the new computer code.

[0075] The system can then repeat the process 300 times with the code that defines the function, e.g., to ensure that the function definition itself does not call an undefined function, and if not, that the neural code generation network is used to define the function. The system can continue iterating process 300 until all functions in the output sequence and all functions in the new output sequences have definitions.

[0076] In this way, the system can implement hierarchical program generation, which allows the neural code generation network to hierarchically generate complex programs that, when successfully executed, cause the robot to respond to natural language instructions. For example, the programs can sequence primitive controls or create action trajectories.

[0077] Fig. 4 shows an example 400 of an LMP 410 generated by the neural code generation network in response to a natural language instruction 402.

[0078] In the example in Fig. 4, instruction 402 reads: “Stack the blocks on the empty pot.”

[0079] As in Fig. 4, the system processes an input sequence including instruction 402 using neural code generation network 120 to generate LMP 410 written in Python.

[0080] As can be seen from the example, the LMP 410 calls multiple APIs. For example, the LMP 410 includes two API calls to an API that provides perception output ("detect objects") and one API call to an API that parameterizes primitive controls for the robot ("pick_place").

[0081] In addition, the neural code generation network 120 in the example of Fig. 4, the LMP 410 is generated hierarchically. In particular, an initial sequence 412 generated by the neural network 120 includes calls to two undefined functions: "is_empty" and "stack_objects." In response to determining that these functions are undefined, the system has used the neural network 120 to generate new sequences 414 and 416 that define each of the functions. As can be seen from Fig.As can be seen in Figure 4, the new sequence 416, which defines the "stack_objects" function, includes a call to the API that parameterizes primitive controls. Thus, the system has hierarchically generated a complex string of control inputs for the robot, which causes the robot to stack several blocks onto the empty pot.

[0082] In this specification, the term "configured" is used in the context of systems and computer program components. A system comprising one or more computers may be configured to perform particular operations or actions by having software, firmware, hardware, or a combination thereof installed in the system that, when used, causes the system to perform the operations or actions. That one or more computer programs may be configured to perform particular operations or actions means that the one or more programs include instructions which, when executed by a data processing device, cause the device to perform the operations or actions.

[0083] Embodiments of the subject matter and the functional operations described in this specification may be implemented in digital electronic circuits or in physically embodied computer software, firmware, or hardware, including the structures disclosed in this specification and their structural equivalents, or in combinations of one or more thereof. Embodiments of the subject matter described in this specification may be implemented as one or more computer programs, i.e., one or more modules of computer program instructions encoded on a physical non-transitory storage medium for execution by, or for controlling the operation of, a computing device.The computer storage medium may be a machine-readable storage device, a machine-readable storage substrate, a random or serial access storage device, or a combination of one or more of these. Alternatively or additionally, the program instructions may be encoded on a synthetically generated propagated signal, for example, a machine-generated electrical, optical, or electromagnetic signal generated to encode information for transmission to a suitable receiving device for execution by a data processing device.

[0084] The term "data processing device" refers to data processing hardware and encompasses all types of devices, apparatus, and machines for processing data, including, for example, a programmable processor, one or more processors or computers. The device may further consist of or include special-purpose logic circuits, for example, an FPGA (field-programmable gate array) or an ASIC (application-specific integrated circuit). In addition to hardware, the device may optionally include code that generates an execution environment for the respective computer program, for example, code representing processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of these.

[0085] A computer program, which may also be called or described as a program, software, software application, app, module, software module, script, or code, may be written in any programming language, including compiled or interpreted languages, or declarative or procedural languages, and it may be delivered in any form, including as a stand-alone program or module, component, subroutine, or other unit suitable for use in a computing environment. A program may, but does not have to, correspond to a file in a file system. A program may be stored in a part of a file that contains other programs or data, such as one or more scripts stored in a markup language document, in a single file dedicated to the program in question, or in several coordinated files, such asFiles that store one or more modules, subroutines, or sections of code. A computer program can be deployed to run on one computer or on multiple computers located at one site or distributed across multiple sites and interconnected by a data communications network.

[0086] In this specification, the term "engine" is used broadly to refer to a software-based system, subsystem, or process programmed to perform one or more specific functions. Generally, an engine is implemented as one or more software modules or components and installed on one or more computers at one or more locations. In some cases, one or more computers are dedicated to a specific engine; in other cases, multiple engines may be installed and running on the same computer or computers.

[0087] The processes and logic sequences described in this specification can be performed by one or more programmable computers executing one or more computer programs to perform functions by processing input data and generating output. The processes and logic sequences can also be performed by special-purpose logic circuits, such as an FPGA or ASIC, or by a combination of special-purpose logic circuits and one or more programmed computers.

[0088] Computers suitable for executing a computer program may be based on microprocessors of a general or special type, or both, or on any other type of central processing unit. Generally, a central processing unit receives instructions and data from a read-only memory or a random access memory, or both. The essential elements of a computer are a central processing unit for performing or executing instructions, and one or more memory devices for storing instructions and data. The central processing unit and memory may be supplemented by or integrated with special logic circuitry. Generally, a computer also includes one or more mass storage devices for storing data, e.g.magnetic, magneto-optical, or optical data storage devices, or is operatively coupled to one or more of these devices to receive data from them, transmit data to them, or both. However, a computer need not include such devices. Furthermore, a computer may be embedded in another device, such as a mobile phone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a Global Positioning System (GPS) receiver, or a portable storage device such as a Universal Serial Bus (USB) memory stick, to name a few.

[0089] Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and storage devices, including, for example, semiconductor memory devices, such as EPROM, EEPROM, and flash memory devices; magnetic disks, such as internal hard disks or removable disks; magneto-optical disks, and CD-ROM and DVD-ROM disks.

[0090] To provide interaction with a user, the subject matter embodiments described in this specification may be implemented on a computer having a display device, e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor for displaying information to the user, and a keyboard and pointing device (e.g., a mouse or trackball) with which the user can provide input to the computer. In addition, other types of devices for interacting with a user may be used; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including auditory, voice, or tactile input.Furthermore, a computer can interact with a user by sending and receiving documents to and from a device used by the user. For example, it can send web pages to a web browser on a user's device in response to requests received from the web browser. A computer can also interact with a user by sending text messages or other messages to a personal device, such as a smartphone running a messaging application, and receiving reply messages from the user in return.

[0091] For example, computing devices for implementing machine learning models may also include dedicated hardware accelerator units for processing general and computationally intensive parts of machine learning training or production, i.e., inference workloads.

[0092] Machine learning models can be implemented and deployed using a machine learning framework, such as a TensorFlow framework or a Jax framework.

[0093] Embodiments of the subject matter described in this specification may be implemented in a computing system that includes a back-end component, e.g., as a data server; or that includes a middleware component, e.g., an application server; or that includes a front-end component, e.g., a client computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described herein; or any combination of such back-end, middleware, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication, e.g., a communications network. Examples of communications networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0094] The computing system may include clients and servers. A client and a server are generally remote from each other and typically interact over a communications network. The client-server relationship arises because the respective computers run computer programs that are in a client-server relationship with each other. In some embodiments, a server transmits data, e.g., an HTML page, to a user device, e.g., to display data to and receive user input from a user interacting with the device acting as a client. Data generated on the user device, e.g., a result of the user interaction, may be received from the device at the server.

[0095] Although this specification contains many specific implementation details, these should not be construed as limitations on the scope of any invention or the scope of what may be claimed, but rather as descriptions of features that may be specific to particular embodiments of particular inventions. Certain features described in this specification in connection with separate embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in connection with a single embodiment may also be implemented in multiple embodiments separately or in any suitable combination.Furthermore, although features are described above as operating in certain combinations and are initially claimed as such, in some cases one or more features of a combination may be extracted from that combination, and the claimed combination may relate to a subordinate combination or a modification of a subordinate combination.

[0096] Although the operations are depicted in a particular order in the drawings and claims, this should not be understood to mean that it is required to achieve the desired results by performing these operations in the particular order shown or in the order stated, or that all of the illustrated operations be performed. In certain circumstances, multitasking and parallel processing may be advantageous. Furthermore, the separation of the various system modules and components in the embodiments described above should not be understood to mean that such separation is required in all embodiments. It should also be understood that the described program components and systems may generally be integrated together into a single software product or bundled into multiple software products.

[0097] Certain embodiments of the subject matter have been described. Other embodiments are within the scope of the following claims. For example, the actions recited in the claims may be performed in a different order and still achieve the desired results. By way of example, the processes depicted in the accompanying figures do not necessarily require the particular order or sequential order shown to achieve the desired results. In some cases, multitasking and parallel processing may be advantageous. QUOTES CONTAINED IN THE DESCRIPTION

[0000] This list of documents submitted by the applicant was generated automatically and is included solely for the convenience of the reader. This list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions. Cited patent literature

[0000] US 63 / 407,607

[0001] Cited non-patent literature

[0000] J. Hoffmann, S. Borgeaud, A. Mensch, E. Buchatskaya, T. Cai, E. Rutherford, D. d. Casas L, Hendricks LA, Welbl J, Clark A, et al. Training compute-optimal large language models, arXiv preprint arXiv:2203.15556, 2022

[0032] JW Rae, S. Borgeaud, T. Cai, K. Millican, J. Hoffmann, HF Song, J. Aslanides, S. Henderson, R. Ring, S. Young, E. Rutherford, T. Hennigan, J. Menick, A. Cassirer, R. Powell, G. van den Driessche, LA Hendricks, M. Rauh, P. Glag, J. P. Huang, J. Huang, R. Ring, S. Young, E. Welbl, S. Dathathri, S. Huang, J. Uesato, J. Mellor, I. Higgins, A. Creswell, N. McAleese, A. Wu, E. Elsen, SM Jayakumar, E. Buchatskaya, D. Budden, E. Sutherland, K. Simonyan, M. Paganini, L. Sifre, L. Marten, A. Li, A. Kunco, A. Kunco. Nematzadeh, E. Gribovskaya, D. Donato, A. Lazaridou, A. Mensch, J. Lespiau, M. Tsimpoukelli, N. Grigorev, D. Fritz, T. Sottiaux, M. Pajarskas, T. Pohlen, Z. Gong, D. Toyama, C. de Masson d'Autu, Y. Milik, T. V. Mili, V. Y. Babuschkin, A. Clark, D. de Las Casas, A. Guy, C. Jones, J. Bradbury, M. Johnson, BA Hechtman, L. Weidinger, I. Gabriel, WS Isaac, E. Lockhart, S. Osindero, L. Rimell, J. Dyer, O. Vinyals, K. Ayoub, J. L. Stanway, K. Bennett, D. K. Bennett, Hassac. Kavukcuoglu and G. Irving.Scaling language models: Methods, analysis & insights from training gopher. CoRR, abs / 2112.11446, 2021

[0032] . Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li und Peter J Liu beschriebenen. Exploring the limits of transfer learning with a unified text-to-text transformer. arXiv preprint arXiv:1910.10683, 2019

[0032] Daniel Adiwardana, Minh-Thang Luong, David R. So, Jamie Hall, Noah Fiedel, Romal Thoppilan, Zi Yang, Apoorv Kulshreshtha, Gaurav Nemade, Yifeng Lu und Quoc V. Le. Towards a human-like open-domain chatbot. CoRR, abs / 2001.09977, 2020

[0032] Tom B Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. arXiv-Vorabdruck arXiv:2005.14165, 2020

[0032]

Claims

[1] System for controlling a robot comprising: one or more computers; and one or more storage devices communicatively coupled to the one or more computers, the one or more storage devices storing instructions that, when executed by the one or more computers, cause the one or more computers to perform the following operations: Obtaining natural language instructions to interact with the robot; Generate an input sequence that includes: (i) the natural language instruction and (ii) a context sequence specifying one or more application programming interfaces (APIs) that can be called to obtain outputs generated by one or more sensors of the robot, to control the robot, or both; processing the input sequence using a neural code generation network to generate as output a computer code sequence in a programming language defining a computer program, the computer code sequence comprising computer code representing a call to one of the one or more APIs and arguments for the call; generating the computer program from the computer code sequence; and Running the computer program to interact with the robot. [2] The system of claim 1, wherein the input sequence further comprises one or more example sequences each comprising: (i) a natural language example instruction and (ii) an example computer code sequence in the programming language generated in response to the natural language example statement. [3] A system according to any preceding claim, wherein generating the computer program from the computer code sequence comprises: Determining that the computer code sequence includes a first computer code that calls an undefined function; and in response to this: Generating a new input sequence comprising a natural language statement for defining the function; and Processing the new input sequence using the neural code generation network to produce a new output sequence comprising new computer code defining the function. [4] The system of claim 3, wherein generating the computer program from the computer code sequence comprises: Generating a modified computer code sequence that replaces the first computer code with the new computer code. [5] A system according to any one of the preceding claims, wherein executing the computer program for interacting with the robot comprises: Calling an execution function for the programming language with an input string derived from the computer code sequence. [6] The system of claim 5, wherein calling the execution function further comprises: Call the execution function with: (i) a global directory that specifies at least one or more APIs, and (ii) a local directory that can be filled with at least variables defined during execution. [7] The system of claim 6, wherein the method further comprises: after executing the computer program, providing one or more values ​​from the local directory in response to the natural language instruction. [8] A system according to claim 6 or claim 7 when dependent on claim 3, wherein generating the computer program comprises adding data specifying the new computer code defining the function to the global directory, the local directory, or both. [9] A system according to any preceding claim, wherein the computer code sequence comprises computer code representing a call to a library for the programming language not specified in the context sequence. [10] A system according to any preceding claim, wherein the neural code generation network is an autoregressive neural network that autoregressively generates each computed code token in the computer code sequence under the constraint of the input sequence. [11] The system of claim 10, wherein the neural code generation network is a transformer neural network. [12] A system according to any preceding claim, wherein the neural code generation network has been trained on a language modeling objective on a corpus of training computer code sequences of training computer programs. [13] One or more non-transitory computer storage media storing instructions which, when executed by one or more computers, cause the one or more computers to perform the operations of any one of claims 1-12.

Images