Integrating software tools into the neural network response of a language model through tool embedding.

Abstract

Methods and systems for one or more computers, one method comprising maintaining software tool usage data for each software tool in a set of software tools, the software tool usage data including a software tool selection embedding and a corresponding software tool embedding. The method includes receiving a query input; generating a software tool selection input sequence; processing the software tool selection input sequence to generate a software tool selection output identifying a particular software tool; and generating a software tool invocation input sequence including the corresponding software tool embedding for that particular software tool and an embedded representation of the query input.

Description

[0001] Cross-reference of related applications

[0002] This application claims priority to U.S. Provisional Application No. 63 / 616,230, filed December 29, 2023, entitled “Integrating Software Tools in Language Model Neural Network Responses Through Tool Embedding,” which is incorporated herein by reference in its entirety. Background Technology

[0003] This specification relates to the integration of software tools with neural networks.

[0004] A neural network is a machine learning model that uses one or more non-linear units to predict the output in response to a received input. In addition to the output layer, some neural networks also include one or more hidden layers. The output of each hidden layer is used as the input to the next layer in the network (i.e., the next hidden layer or output layer). Each layer of the network generates an output from the received input based on the current values ​​of its corresponding set of parameters. Summary of the Invention

[0005] This specification describes a system implemented as a computer program on one or more computers in one or more locations, which integrates the capabilities of a set of software tools with a language model neural network.

[0006] Some implementations of this system are particularly useful in parallel or distributed computing environments because they are specifically adapted to separate the process of learning to select software tools from the process of using those tools, typically accomplished by a language model or visual language model neural network. Specifically, some implementations of this system allow such a language model or visual language model neural network implemented on a first computer system to maintain a fixed set of neural network parameters (particularly trainable parameters, such as weights), while allowing a second computer system to learn the tools used. This offers several advantages, including the possibility of having a shared language model or visual language model neural network on a first computer system and having multiple second computer systems, each of which, as described below, can learn to use different sets of corresponding software tools based on one or more different software tool selection embeddings and / or one or more different software tool embeddings.

[0007] Language model neural networks are pre-trained neural networks trained on training data from various sources and domains. In some cases, the training data does not include specific information, such as task-specific information, recent events, etc. To overcome this deficiency in training data, the system described in this specification relates to providing data and services related to a set of software tools to systems implementing language model neural networks.

[0008] A software toolset can include software programs, each operable to perform an operation to execute a corresponding function. For example, a software toolset could include: scheduling software for scheduling meetings based on a user's calendar, booking software for booking airline accommodations, transcription software for providing meeting transcripts, and so on. Integrating the functionality of these software tools into the general functionality of language model neural networks can expand the number of real-world problems and use cases that systems implementing language model neural networks (e.g., large language models) can solve.

[0009] Here are some other application examples of the system described in this article: Software tools may include one or more image processing tools, such as tools specifically designed for processing a particular type or different types of images. Some examples may be images of natural environments or specific types of natural environments, urban landscapes or urban landscape-type images, astronomical images, medical images or specific types of medical images, and so on. The query input may define the image processing task to be performed on pixels of a still or moving image; and the response to the query input may define the result of the image processing task. The image processing task may be, for example, an image classification task, where pixels of a still or moving image are processed to generate a response that classifies the image content; or it may be an object detection task, where an image is processed to generate a response that locates the coordinates (e.g., bounding box coordinates) of one or more objects or object types in the image; or it may be an instance detection task, where an image is processed to generate a response that identifies the shape of one or more objects or object types in the image.

[0010] Software tools can include one or more materials design tools for designing physical materials. Some examples include protein design tools (e.g., tools related to AlphaFold), retrosynthetic tools (such as those described in WO2021 / 263238), or materials design tools (such as those described in Yang et al., “Generative Hierarchical Materials Search”, arXiv:2409.06762, 2024). The query input can define the materials design task being performed, such as specifying one or more desired properties of the material; the response to the query input can define the outcome of the materials design task. The material can then be physically manufactured (synthesized), for example, for testing.

[0011] The software tool may include one or more robot control tools for controlling a mechanical robot to perform physical tasks. The query input may define the physical task to be performed by the robot; the response to the query input may define the outcome of the physical task, for example, it may include one or more observations of the task outcome, i.e., observations captured from the real world that have been returned as tool output and characterize the outcome of the physical task (in the case of the previous example, the outcome might be a synthetic material).

[0012] This specification generally describes a prompt adjustment method in which the system learns and adjusts the parameters of soft prompts corresponding to a specific software tool. Soft prompts are learned embedded representations of task-specific data, presented as continuous embedding matrices. After prompt adjustment, during inference, the system appends the learned soft prompts to hard prompts, and the language model neural network processes the combined prompts. Hard prompts are embedded representations that consist of embeddings generated by processing word sequences (including query inputs to the language model neural network, such as prompts, questions, etc.) using the embedding layers of the language model neural network. By processing the combined prompts (soft prompts appended to or prepended to hard prompts), the language model neural network processes task-specific data using learned embedded representations while keeping the network parameters unchanged.

[0013] Specific embodiments of the subject matter described in this specification may be implemented in order to achieve one or more of the following advantages.

[0014] The range of methods for providing task-specific data to language model neural networks to integrate the use of software tools into the use of language model neural networks can range from supervised fine-tuning methods (high cost but high performance) to few-shot learning methods (low cost but low performance).

[0015] Supervised fine-tuning methods require relearning the model's parameter space. In many cases, language model neural networks have a large number of parameters to relearn, which is a computationally expensive task. Furthermore, using task-specific training data for supervised fine-tuning risks reducing performance on other tasks.

[0016] Few-shot learning methods involve processing a few examples from a specific software tool and using them as part of the input to each language model neural network. While this approach does not require fine-tuning of the language model neural network, few-shot learning methods typically produce suboptimal performance compared to supervised fine-tuning.

[0017] Furthermore, few-shot learning methods are computationally more expensive during inference compared to the methods described in this specification. With few-shot learning methods, several examples are included in the prompt at each inference, causing the neural network to process more lexical units, thus increasing the required computational resources. Specifically, the computational requirements of transformer-based models are quadratically increased based on the number of lexical units processed by the transformer model. Therefore, since adding examples to each prompt adds a large number of lexical units to each input to the model, employing few-shot learning methods significantly increases the computational cost per inference. In contrast to few-shot learning, the methods described in this specification involve storing embedded representations of prompts and / or examples (e.g., API documentation) instead of the actual prompts and / or examples, resulting in a more compact storage method (e.g., storing compressed data).

[0018] Furthermore, compared to few-shot learning methods, the methods described herein send fewer lexical units between computational resources during inference. Additionally, the methods of this invention require less data storage compared to few-shot learning methods. For example, few-shot learning methods require storing data including long cues and / or examples (e.g., API documentation examples). In contrast, the methods of this disclosure require storing embedded representations (e.g., software tool selection embeddings and software tool embeddings), which are more compact than the associated raw data.

[0019] The method described in this specification provides a high-performance solution (e.g., with accuracy comparable to supervised fine-tuning) for integrating software tool functionality into the functionality of a language model neural network, which is less computationally and economically costly than supervised fine-tuning methods.

[0020] Compared to methods that require explicit inclusion of API keys in the language model output, this method reduces the risk of API key leakage by leveraging software tools to select embeddings and corresponding software tool embeddings. By encoding API interaction details into embeddings, rather than storing them in their raw format, the system reduces the risk of accidentally leaking API keys or other sensitive information in the language model output. This improves the security of systems interacting with software tools.

[0021] Specifically, the described technique involves a training process that includes learning parameters for soft cues (e.g., embedding vectors associated with software tool data, or embedding vectors associated with a set of software tools in a software tool suite). In some examples, this training process can include a learned parameter space several orders of magnitude smaller than the learned parameter space of a language model neural network. By reducing the number of learned parameters, the training system requires less computation, energy consumption, data transfer, and data storage resources. Furthermore, the described method demonstrates that the accuracy of software tool-related outputs (e.g., software tool identifiers) is comparable to that of the outputs of supervised fine-tuning models.

[0022] Furthermore, the described techniques provide a flexible method for adding and removing task-specific features to a system implementing a language model neural network. For example, in an example system implementing supervised fine-tuning, the system must relearn the parameter space of the language model neural network to add training data relevant to new software tools. In contrast, the techniques described in this specification require a small, task-specific learning task to learn software tool-specific embeddings to provide soft cues relevant to new software tools. By creating a flexible system that can add and remove software tools without relearning the parameters of the language model neural network, the system requires less computational, energy consumption, data transfer, and storage resources to add and / or remove specific software tool features. Similarly, in response to changes in the features of a specific software tool, the system can update the specific learned embeddings (soft cues) associated with that specific software tool by relearning the associated embeddings. Relearning the parameters of the language model neural network is not required when the relevant training data changes.

[0023] In addition to the embodiments described in this specification, the following embodiments are also innovative: In a first aspect, a method is performed by one or more computers, the method comprising maintaining software tool usage data for each software tool in a set of software tools, the software tool usage data including a software tool selection embedding and a corresponding software tool embedding. The method includes: receiving a query input; generating a software tool selection input sequence, the software tool selection input sequence including the software tool selection embedding and a first set of embeddings representing the query input. The method includes: processing the software tool selection input sequence using a language model neural network to generate a software tool selection output, the software tool selection output identifying a specific software tool in the set of software tools; generating a software tool invocation input sequence, the software tool invocation input sequence including the corresponding software tool embedding for that specific software tool and a second set of embeddings representing the query input.

[0024] In some implementations, the method further includes: using a language model neural network to process the software tool call input sequence to generate a software tool call output specifying the input for that particular software tool; providing the input to the particular software tool; obtaining an output from the particular software tool; and generating a response to the query input based on the output obtained from the particular software tool.

[0025] In some implementations, the method includes receiving data specifying a new software tool to be added to the set of software tools; and in response to receiving the data specifying the new software tool to be added to the set of software tools: generating a corresponding software tool embedding for the new software tool; and updating the software tool usage data to include the corresponding software tool embedding for the new software tool.

[0026] In some implementations, the method includes generating a corresponding software tool embedding for the new software tool, comprising: initializing the corresponding software tool embedding for the new software tool; receiving first training data, the first training data including a plurality of training software tool call input sequences for the new software tool, and for each training software tool call input sequence, the first training data including a corresponding baseline real software tool call output for the new software tool; and using the first training data to perform cue adjustments on the corresponding software tool embedding for the new software tool to update the corresponding software tool embedding for the new software tool.

[0027] In some implementations, in response to receiving data specifying a new software tool to be added to the set of software tools, the method includes: updating the software tool selection embedding to take into account the addition of the new software tool to the set of software tools; and updating the software tool usage data to include the updated software tool selection embedding.

[0028] In some implementations, the method includes updating the software tool selection embedding to account for the addition of the new software tool to the set of software tools, comprising: receiving second training data comprising a plurality of training software tool selection input sequences, and for each training software tool selection input sequence, the second training data comprising data identifying a corresponding benchmark real software tool to be selected in response to the training software tool selection input sequence, wherein for at least a subset of the training software tool selection input sequences, the corresponding benchmark real software tool is the new software tool. The method further comprises: performing cueing adjustments on the software tool selection embedding using the training data to update the corresponding software tool embedding for the new software tool.

[0029] In some implementations, the software tool has been trained to select embeddings by prompting adjustments, while keeping the language model neural network unchanged.

[0030] In some implementations, the corresponding software tool embedding for the software tool in the set of software tools has been learned through cueing adjustments, while keeping the language model neural network unchanged.

[0031] In some implementations, the software tool selection output is a sequence of outputs that specifies an identifier for that particular software tool.

[0032] In some implementations, the software tool selection output is a corresponding likelihood score for each of one or more software tools, and the method further includes selecting the particular software tool based on the corresponding likelihood score.

[0033] In some implementations, each software tool receives input including a corresponding value for each of one or more input parameters for that software tool, and the software tool invokes an output sequence that specifies a corresponding particular value for each of the one or more input parameters for that particular software tool.

[0034] In some implementations, generating a response to the query input based on the output obtained from the particular software tool includes: generating a software tool output-input sequence based at least on the output obtained from the particular software tool; and processing the software tool output-input sequence using a language model neural network to generate an output sequence that defines the response to the query input.

[0035] In some implementations, the method is implemented in a distributed computing system comprising a first computer system implementing the language model neural network and a second computer system implementing one or both of a tool selection neural network and a tool embedding neural network. The method further comprises maintaining software tool usage data on the second computer system. The first computer system performs the following steps: receiving the query input; processing the software tool selection input sequence; processing the software tool invocation input sequence; providing the input to the specific software tool; obtaining the output from the specific software tool; and generating the response to the query input. The second computer system performs the following steps: maintaining the software tool usage data; generating the software tool selection input sequence; and generating the software tool invocation input sequence.

[0036] In some implementations, the distributed computing system includes more than one second computer system, each of which is configured to support different groups of software tools for different corresponding user groups, and each of the second computer systems maintains different response software tool usage data for generating different corresponding software tool selection input sequences and software tool invocation input sequences.

[0037] In some implementations, the method includes: learning the data used by the software tool without modifying the learned parameters of the language model neural network.

[0038] In some implementations, the software tool includes at least one image processing tool, wherein the query input defines an image processing task, and wherein the response to the query input defines the result of the image processing task.

[0039] In some implementations, the software tool includes at least one material design tool for designing physical materials, wherein the query input defines a material design task, and wherein the response to the query input defines the result of the material design task.

[0040] In some implementations, the software tool includes at least one robot control tool for controlling the mechanical robot to perform a physical task, wherein the query input defines the physical task, and wherein the response to the query input defines the result of the physical task.

[0041] In another aspect, a method, performed by one or more computers, for generating a corresponding software tool embedding for a new software tool, includes: initializing the corresponding software tool embedding for the new software tool; receiving first training data, the first training data including a plurality of training software tool invocation input sequences for the new software tool, and for each training software tool invocation input sequence, the first training data including a corresponding baseline real software tool invocation output for the new software tool. The method further includes: performing cue adjustments on the corresponding software tool embedding for the new software tool using the first training data to update the corresponding software tool embedding for the new software tool.

[0042] In some implementations, in response to receiving data specifying a new software tool to be added to the set of software tools, the method includes: updating the software tool selection embedding to take into account the addition of the new software tool to the set of software tools; and updating the software tool usage data to include the updated software tool selection embedding.

[0043] In another aspect, one or more non-transitory computer-readable media store embeddings generated by performing operations, the operations including: maintaining software tool usage data for each software tool in a set comprising a plurality of software tools, the software tool usage data including software tool selection embeddings and corresponding software tool embeddings. The operations include: receiving query input; and generating a software tool selection input sequence, the software tool selection input sequence including the software tool selection embedding and a first plurality of embeddings representing the query input. The operations include: processing the software tool selection input sequence using a language model neural network to generate a software tool selection output, the software tool selection output identifying a specific software tool in the set of software tools; and generating a software tool invocation input sequence, the software tool invocation input sequence including the corresponding software tool embedding for that specific software tool and a second plurality of embeddings representing the query input.

[0044] While this specification contains numerous details of specific implementations, these details should not be construed as limiting the scope of any invention or the scope that may be claimed, but rather as descriptions of features that may be characteristic of particular embodiments of a particular invention. Certain features described in this specification within the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments. Furthermore, although features may be described above as operating in certain combinations and even initially claimed in this way, in some cases one or more features from the claimed combination may be removed from the combination, and the claimed combination may involve sub-combinations or variations thereof.

[0045] Details of one or more embodiments of the subject matter of this specification are set forth in the accompanying drawings and the following description. Other features, aspects, and advantages of this subject matter will become apparent from the description, drawings, and claims. Attached Figure Description

[0046] Figure 1 It is a diagram of an example system that includes a language model neural network system that interacts with a software tool system.

[0047] Figure 2 It is a diagram of an example system that includes a language model neural network and software tool system.

[0048] Figure 3 This is a flowchart of an example process for generating output in response to a query input from a language model neural network system with integrated software tools.

[0049] Figure 4 This is a flowchart of an example process for updating data used by software tools.

[0050] Figure 5 This is an example graph used to compare training methods for language model neural networks configured to select software tools.

[0051] Figure 6 An example graph is shown to compare training methods for language model neural networks configured to select software tools.

[0052] Figure 7 This is an example diagram used to compare training methods for language model neural networks configured to generate arguments to be processed by software tools.

[0053] In the various figures, the same reference numerals and names indicate the same elements. Detailed Implementation

[0054] This specification describes a system implemented as a computer program on one or more computers at one or more locations, which integrates the capabilities of a set of software tools with a system implementing a language model neural network. The set of software tools can act as a collection of plug-ins for the language model neural network, such as additional capability layers. The software tools provide the system with additional functionality not present in the traditional output of the language model neural network. In some examples, the software tools may include software applications that can communicate via an application programming interface (API), which allows other software applications (e.g., the system implementing the language model neural network) to provide input to and receive output from the software application associated with the software tools. For example, calendar applications, web browsers, travel planning applications, and IoT devices can all communicate via APIs on the web, on a local network, or via local computing resources. Generally, software tools capable of communicating through a “request” and “response” scheme with predictable and structured input-output formats can provide additional functionality to the language model neural network.

[0055] In some cases, the details of a specific software tool are not represented in the training data of a language model neural network. Many pre-trained language model neural networks are limited to performing general tasks with limited functionality when asked to generate information related to recent events, task-specific goals (such as mathematical operations), operations requiring interaction with external resources, and taking actions. Generally, language model neural networks are trained on general information and can perform tasks and reasoning exercises closely related to the training data. Retraining and fine-tuning language model neural networks (e.g., changing the parameters (weights) of the neural network) to suit specific functional and informational requirements can be computationally costly and time-consuming.

[0056] The following description describes a cue tuning method that integrates the functionality of a software tool into a system implementing the operation of a language model neural network (e.g., a general-purpose, pre-trained large language model). The cue tuning method involves learning embedded representations of software tool-specific data, and the language model neural network processing the learned embeddings along with the embedded representation of the input query.

[0057] Figure 1 This is an illustration of an example system 100 including a language model neural network system 102 (hereinafter referred to as system 102) that interacts with software tool system 104. In general, system 102 can interact with multiple software tool systems.

[0058] System 102 and software tool system 104 are examples of systems implemented as computer programs on one or more computers in one or more locations, in which the systems, components and technologies described below can be implemented.

[0059] System 102 includes a language model neural network 112 and performs communication operations (e.g., database requests, API calls, local software commands, etc.).

[0060] For example, system 102 receives query input 110 from client device 106. Query input 110 can be a question, prompt, instruction, or any other message to be processed by system 102. System 102 uses a lexer to lexer the query input 110 (e.g., a sequence of words) into a sequence of lexers, where each lexer in the sequence of lexers is mapped by the lexer to a numeric lexer identifier.

[0061] System 102 uses neural network layers of language model neural network 112 to process sequences of lexical identifiers, which includes embedding layers (e.g., neural network layers that convert sequences of lexical identifiers into one or more embedding vectors). The layers of language model neural network 112 may also include one or more transformer layers (e.g., self-attention networks and feedforward networks) that process the embedding vectors sequentially.

[0062] In addition to using the language model neural network 112 to process query input 110, system 102 also accesses software tool usage data 114 via database queries, API calls, local software commands, etc. System 102 accesses and processes software tool usage data 114 to interact with the software tool in response to query input 110.

[0063] The software tools use data 114, which includes embeddings (e.g., one or more embedding vectors). The embeddings include (i) software tool selection embeddings and (ii) the corresponding software tool embeddings for each software tool in a set of multiple software tools.

[0064] System 102 processes a software tool selection embedding for selecting relevant software tools from a set of software tools in response to query input 110. The software tool selection embedding is an embedded representation of training data, wherein the training data includes example query inputs and corresponding software tools related to the hints and / or questions associated with the query input. For example, the example query input for a training data item is "book a flight for tomorrow from New York to Los Angeles," and the corresponding software tool for the example training data item is "airlinebooking software."

[0065] For each software tool in a set of multiple software tools, the corresponding software tool embedding is an embedded representation of the instructions for interacting with the corresponding software tool (e.g., required parameters, expected input data formatting, and API endpoints). Each embedding vector in the software tool selection embedding and each embedding vector in the software tool embedding have the same dimension as the embedding vector generated by the embedding layer of the language model neural network 112 of system 102.

[0066] To utilize the embeddings in the software tool usage data 114, system 102 appends or prepends the embeddings from the software tool usage data 114 (e.g., software tool selection embeddings or software tool embeddings) to the embeddings generated by the embedding layer of the language model neural network 112. For example, the embedding layer of the language model neural network 112 generates an embedded representation of the query input 110. System 102 appends or prepends the generated embedded representation to the embeddings from the software tool usage data 114. The remaining layers of the language model neural network 112 (e.g., transformer layers) process the combined embedding structure.

[0067] System 102 sends input message 120 to software tool system 104 to initiate the execution of an operation associated with the corresponding software tool. For example, software tool system 104 may perform an operation associated with an airline booking software system. Input message 120 may include one or more input parameters for the software tool. In response, software tool system 104 returns response message 116 to system 102. Response message 116 may be a sequence of outputs specifying particular values ​​for each of the one or more input parameters for a particular software tool.

[0068] In some implementations, the transmission of input message 120 and the reception of response message 116 are steps in a series of data processing steps performed by system 102 to generate response 108 in response to query input 110 and send it to client device 106. For example, as a step in a multi-step process, system 102 may receive response message 116, which is lexicalized and then processed by system 102's language model neural network 112.

[0069] In some implementations, system 102 processes response message 116 from software tool system 104 to generate response 108 to be received by client device 106. In some implementations, language model neural network 112 processes response message 116 and query input 110 (e.g., response message 116 and query input 110 are both included in a single prompt) to generate response 108 to be received by client device 106.

[0070] Figure 2This is a diagram of an example system 200 including a language model neural network 112 that interacts with a software tool system 216. Example system 200 is an example of a system implemented as a computer program on one or more computers at one or more locations, where the systems, components, and techniques described below can be implemented.

[0071] System 200 receives query input 202, which initiates a sequence of data processing steps. In some implementations, system 200 receives query input 202 via an API connection to a computer, which executes operations related to a user interface (e.g., ...). Figure 1 The system 200 receives query input 202 from a system implemented on a common device (e.g., the common device implements the operation of system 200 and the operation associated with the user interface for receiving and sending query input 202). In some other implementations, system 200 receives query input 202 as part of an automatic data processing pipeline, wherein the output of system 200 can be processed by a data processing step in a subsequent step of the automatic pipeline.

[0072] System 200 includes a software tool selection sequence generator 204. The software tool selection sequence generator 204 processes query input 202 and software tool selection embedding 206.

[0073] System 200 can maintain software tool usage data (e.g., Figure 1 The software tool usage data (114) includes, for each software tool in the set of available software tools, a software tool selection embedding 206 and a corresponding software tool embedding 210. In this example, the software tool selection embedding 206 is a learned vector representation of training data that includes example input queries and corresponding indicators of one or more software tools associated with each corresponding example input query.

[0074] In response to query input 202, software tool selection sequence generator 204 can generate a tool selection input sequence 222, which includes a software tool selection embedding 206 and an embedding sequence representing query input 202. In some implementations, software tool selection sequence generator 204 appends the software tool selection embedding 206 to the query input 202 embedding. In some other implementations, software tool selection sequence generator 204 prepends the software tool selection embedding 206 to the query input 202 embedding.

[0075] System 200 can generate a sequence of embeddings representing query input 202 in any suitable manner, such as by lexicalizing query input 202 and generating a corresponding embedding for each lexical, or, when query input 202 is a multimodal query input, generating one or more embeddings for each modality in query input 202.

[0076] In some implementations, query input 202 includes one or more modalities, including text, images, audio, video, or a combination of one or more modalities. For example, language model neural network 112 may be a visual language model that can process visual (image and video) data as well as text data as query input 202. As another example, language model neural network 112 may include an audio neural network, where query input 202 includes audio data. Generally, depending on the specific architecture of language model neural network 112, query input 202 may include multiple modalities.

[0077] System 200 can use language model neural network 112 to process tool selection input sequence 222 to generate software tool selection output 224, which identifies a specific software tool in a set of software tools. For example, the output of language model neural network 112 in response to processing tool selection input sequence 222 is a sequence of lexical identifiers. After decoding the lexical identifier sequence, the output can be represented as text including identifiers (e.g., indexes) corresponding to a specific software tool, the name of the specific software tool, or a set of likelihood scores corresponding to a set of software tools. Each likelihood score represents the probability that a specific software tool is relevant to query input 202. Based on the output of language model neural network 112, system 200 determines that a specific software tool is relevant to query input 202. In some implementations, system 200 selects the software tool with the highest likelihood score and continues to interact with the selected software tool (e.g., sends a web request). In some other implementations, system 200 can provide a user with a list of software tools with associated likelihood scores via a user interface (e.g., the user interface of client device 106) so that the user can select a software tool from a set of multiple software tools based on the likelihood scores. In some other implementations, the user selects more than one software tool based on the likelihood score of each software tool (e.g., software tools associated with likelihood scores above a threshold).

[0078] The invocation input sequence generator 212 can generate a tool invocation input sequence 226 (obtained from the software tool selection output 224), which includes the corresponding software tool embedding of a specific software tool retrieved from software tool embeddings 210 maintained as part of the software tool usage data, and an embedding sequence characterizing the query input 202. Similar to the software tool selection sequence generator 204, the invocation input sequence generator 212 can append or prepend the software tool embedding to the embedded representation of the query input 202. The tool invocation input sequence 226 includes information about the query input 202 (via the embedded representation of the query input 202) and information about the specific software tool's requirements for the software tool (e.g., parameters and protocols). For example, a sample software tool for booking airline flights might require the tool invocation input sequence 226 to include embedded representations of date ranges, departure points, and destinations. Furthermore, a specific software tool might expect data to be formatted according to a specific protocol (e.g., JSON or XML). Additionally, a specific software tool might expect the tool invocation to be formatted based on a specific communication protocol (e.g., REST or SOAP).

[0079] The language model neural network 112 processes the software tool call input sequence 226 to generate a tool call output 228, which specifies the input to a particular software tool. The tool call output 228 is a sequence of terms that, after being decoded and converted into text, can describe the API call to the specific software tool. In some implementations, additional text processing steps are applied to the decoded tool call output 228 before the request is sent to the specific software tool (e.g., formatting to JSON, removing unnecessary text, etc.).

[0080] System 200 can provide tool invocation output 228 as input to and from software tool system 216 associated with a specific software tool via API, local software commands, or any other method of communicating with software services, and obtain output 230 from software tool system 216. System 200 includes a response integrator 218 for generating a response 220 to query input 202 based on the output 230 obtained from the specific software tool. In some implementations, response integrator 218 uses language model neural network 112 to generate a response to query input 202 by processing an input sequence generated based on responses from the specific software tool, and in some implementations, based on query input 202 itself.

[0081] Integrating software tools with language model neural networks can extend the functionality of those networks. For example, software tools can provide users with the ability to perform actions, such as booking flights, purchasing goods, requesting transportation, initiating communications, or arranging meetings.

[0082] Turning to a specific example method for enhancing the functionality of the language model neural network 112 through software tool integration, system 200 can implement a cueing adjustment method for generating and processing software tool selection embeddings 206 and software tool embeddings 210 to be processed by the language model neural network 112. That is, cueing adjustment can refer to learning embeddings, such as software tool selection embedding 206 or software tool embedding 210.

[0083] Hint adjustment is a method that includes using a language model neural network 112 to process hints, wherein the hints include embedded inputs associated with query input 202 and general instructions (i.e., “hard hints”), as well as learned embedded inputs (e.g., software tool selection embedding 206 or an embedding from software tool embedding 210). The learned embeddings are referred to as “soft hints”, which provide the language model neural network 112 with context about a specific task (e.g., selecting a software tool or generating a call output associated with a specific software tool). In some implementations, soft hints are associated with data reflecting the properties and parameters (e.g., API endpoints, API arguments, and / or expected return values) of a specific software tool. In this disclosure, the soft hint corresponding to the software tool selection task is the software tool selection embedding 206, and the soft hint corresponding to one or more specific software tools is the software tool embedding 210.

[0084] In some implementations, the soft prompt (e.g., the software tool selection embedding 206, or the software tool embedding in software tool embedding 210) is a tensor of dimension [length of the soft prompt] × [length of the language model neural network embedding]. System 200 can define the embedding dimension for the language model neural network 112. For example, a smaller language model neural network can have a dimension of 256 or 512. A larger language model neural network can have a larger dimension of 1024 or 2048. The "length of the soft prompt" relates to the number of lexical units embedded in the soft prompt. For example, the number of lexical units that make up the soft prompt can be set to 50, 100, 150, or any number. System 200 can use the following regarding... Figure 3 The description of the cue adjustment techniques described in this disclosure relates to specific tasks and is used to learn specific soft cues for each task (e.g., selecting software tools, generating tool call output, etc.).

[0085] In some implementations, system 200 maintains and accesses data used by software tools (e.g., Figure 1The software tools use a collection of embedded sequences from one or more databases associated with data 114. In some implementations, system 200 generates a hard hint associated with the embedded representation of query input 202 in response to receiving query input 202.

[0086] Figure 3 This is a flowchart of an example process 300 for generating output in response to a query input from a language model neural network system with integrated software tools. For convenience, process 300 will be described as being executed by a system of one or more computers located in one or more locations. For example, a system appropriately programmed according to this specification... Figure 2 The system 200 described herein can execute process 300.

[0087] The system maintains (302) software tool usage data for each software tool in a set comprising multiple software tools. This software tool usage data includes software tool selection embeddings and corresponding software tool embeddings. An embedding may include one or more embedding vectors.

[0088] The system receives (304) query input. The query input may be associated with one of the software tools. For example, the query input may indicate the need to perform a specific task that can be executed by one of the software tools. As another example, the query input may include one or more parameters to be processed by one of the software tools. As yet another example, the query input may include instructions for a software execution environment to execute code, such as code used to interact with and / or control another software system via API calls.

[0089] Query input can include questions, instructions, wishes, or any communication that may indicate the system needs to perform a task. Some examples of query input include, but are not limited to, the following:

[0090] In some implementations, the system determines whether responding to a query input might require a software tool by using rule-based systems, pattern matching, machine learning models, or other methods to determine whether communication with a software tool is needed to generate a response to the query input.

[0091] The system generates (306) a software tool selection input sequence, which includes a software tool selection embedding and a first embedding sequence representing the query input. The software tool selection embedding is a sequence of one or more embedding vectors having the same dimension as the embedding vectors of the first embedding sequence representing the query input. The first embedding sequence can be generated from the embedding layer of a language model neural network. The first embedding sequence can be appended to or prepended to the software tool selection embedding to generate the software tool selection input sequence. In some implementations, the software tool selection input sequence includes other information besides the query input. For example, the software tool selection input sequence may include information related to the natural language request to select the software tool most relevant to the query input.

[0092] In some implementations, language model neural networks (e.g., Figure 1 The language model neural network 112) processes the software tools to select the input sequence and generate a word output sequence. When decoded into text, the word output sequence indicates that none of the software tools in the set of one or more software tools are related to the received query input. In this case, the system can avoid communicating with the software tools and can instead use only the language model neural network to generate the response to the query input.

[0093] In some implementations, the system generates software tool selection embeddings by processing training data using a language model neural network and adjusting the values ​​of the software tool selection embeddings based on an evaluated error between the baseline true values ​​of the training data and the output of the language model neural network. The training data includes query input and baseline true labels, which define which particular software tool from a set of one or more available software tools is most relevant to solving the problem or answering the query input. The training data may include the following entries:

[0094] Each training data entry includes a query input and a benchmark real-world software tool output determined to be associated with that specific query input. In some other implementations, the system can configure the training data in any way, including the query input and appropriate labels.

[0095] If new software tools are introduced, the system can generate updated software tool selection embeddings by re-executing the cueing adjustment technique using a new training dataset that includes examples of the new software tools and related queries, as shown in the example. Figure 4 As described in the description.

[0096] The system uses a language model neural network to process (308) software tool selection input sequences to generate a software tool selection output that identifies a specific software tool in a set of software tools. In some implementations, the software tool selection output is a sequence of lexical identifiers that can be decoded and converted into a character sequence.

[0097] In some implementations, the language model neural network outputs a unique identifier that represents a specific software tool that can be used to process the corresponding query input. For example, the language model neural network can output the software tool name, unique identifier, or set of likelihood scores for one or more software tools from a set of available software tools. Therefore, the system can select the appropriate tool based on the distribution of the likelihood scores. For example, for the query input "UseTicket Vendor 1, no Ticket Vendor 2 to buy concert tickets.", the query input mentions two product names, such as Ticket Vendor 1 and Ticket Vendor 2. However, the target software tool of the query input seems to have switched from Ticket Vendor 1 to Ticket Vendor 2. In this case, the system can configure the output of the language model neural network to include "Ticket Vendor 1", "TixVendr1", "Ticket Vendor 1 0.9, Ticket Vendor 2 0.5, Online Ticket Marketplace 0.2", or any other identification parameter or set of likelihood scores referring to one or more available software tools.

[0098] The system generates (310) a software tool call input sequence, which includes the corresponding software tool embedding of a specific software tool and a second embedding sequence representing the query input.

[0099] Software tool embeddings contain information related to a specific software tool, which the language model neural network needs to generate the input the system will send to that tool. The system appends or prepends the software tool embedding to the query input embedding sequence. For example, to access information from the Ticket Vendor 1 tool, the system needs to understand how to send web requests to the computational resources that perform the operations associated with the Ticket Vendor 1 tool, how to format the corresponding web requests (if applicable), and what parameters and information are needed to retrieve information from the resources available to generate a response to the query input.

[0100] In some implementations, the system generates software tool embeddings for each of one or more software tools in a manner similar to the methods described above for selecting embeddings for generating software tools. The system may receive a training dataset that includes samples of input queries and corresponding invocation instructions specific to the software tool and the task implied by the query input. For example, the system can learn the software tool embeddings for the Flights tool using the following training data:

[0101] The left column represents the query input, and the right column represents the baseline real response of the required parameters that should be extracted from the query input and sent to a software tool (such as the Flights API). In some implementations, the system performs a separate execution of a language model neural network to determine the values ​​associated with each requirement (e.g., required parameters, required format, expected return parameters, API endpoint, etc.) of a particular software tool call.

[0102] For example, if the system identifies the Flights tool as the most suitable software tool using a software tool selection process that employs a software tool selection embedding, the system may include a software tool invocation embedding trained on Flights training data to generate appropriate parameters and arguments to communicate with Flights resources or APIs. Each software tool in the set of available software tools is configured to receive input, which includes a corresponding value for each of one or more input parameters, as shown in the table above.

[0103] In some implementations, the system uses a language model neural network to process (312) the software tool call input sequence to generate a software tool call output that specifies the input to a particular software tool. In some implementations, the system uses a separate computing environment (e.g., a separate server) to process (312) the software tool call input sequence. For example, the software tool call input sequence may be generated on a user device, for example, by a small language model, and then sent to a large neural network model hosted on a remote computing resource that has access to one or more software tools.

[0104] In some implementations, the output of a software tool invocation from a language model neural network is a sequence of output values ​​for each of one or more input parameters specifying a particular software tool. In other words, the system can prompt the language model neural network to generate output that mimics the right column of the table above and the corresponding values ​​extracted from the query input shown in the left column of the table above. For example, if the query input is “Can you find me a flight from Denver to Chicago for 1 adult and 1 child in Economy Class Seating for one way on 10 September 2023 with Economy Air”, the output of the language model neural network can include a JSON object containing parameters and corresponding values ​​from the query input: {“origin”: “Denver”,“destination”: “Chicago”, “earliest_departure_date”: “2023-09-10”, “airline_codes”; “Economy”, “one_way”: true, “num_adult_passengers”: 1, “num_child_passengers”: 1, “num_infant_in_seat_passengers”: 0}. By configuring the training data to include expected parameters and corresponding values ​​extracted from the query input, the system can prompt the language model neural network to format the software tool call output in a format understood by Flights resources.

[0105] In some implementations, the system provides input (314) to a specific software tool. In some implementations, the system provides a set of corresponding input arguments for the specific software tool, generated by a language model neural network. In some implementations, this may include an API endpoint, a web request, or any other means of communicating with the specific software tool.

[0106] In some implementations, the system obtains (316) output from a specific software tool. For example, in the case of a Flights query input “Can you find me a flight from Denver to Chicago for 1 adult and 1 child in Economy Class Seating for one way on 10 September 2023 with Economy Air?”, where the language model neural network processes the software tool call input sequence (a combination of the software tool embedding for Flights and the query input embedding) to generate the software tool call {“origin”: “Denver”, “destination”: “Chicago”, “earliest_departure_date”: “2023-09-10”, “airline_codes”;“Economy”, “one_way”: true, “num_adult_passengers”: 1, “num_child_passengers”: 1, “num_infant_in_seat_passengers”: 0}, the system can obtain a list of flights conforming to the requested criteria from the Flights API, provided by a JSON object.

[0107] In some implementations, the system generates (318) a response to the query input based on the output obtained from a specific software tool. The system can use a language model neural network to process the input sequence of the software tool's output to generate an output sequence that defines the response to the query input. In other words, the system can use a language model neural network to process the output from a specific software tool to generate a natural language response to the query input. For example, the system can format a list of flights matching the requested criteria from the example above into a list that can be displayed in a chat interface, a spreadsheet, or any other suitable mode that displays or otherwise provides data from the corresponding software tool.

[0108] Figure 4 This is a flowchart of an example process 400 for updating software tool usage data. For convenience, process 400 will be described as being executed by a system of one or more computers located in one or more locations. For example, a system appropriately programmed according to this specification... Figure 2 The system 200 described herein can execute process 400.

[0109] Such as about Figures 2 to 3As described, for each software tool in a set of multiple software tools represented by software tool usage data, the software tool usage data may include software tool selection embedding and software tool embedding.

[0110] The system receives (402) data specifying a new software tool to be added to the software tool set. The new software tool data may include sample queries related to the new software tool (e.g., sample airline itinerary query, sample calendar event query, etc.). Additionally, the new software tool data may include data related to retrieving data from the software tool (e.g., API format requirements, parameter requirements, API endpoints, etc.).

[0111] In response to receiving new software tool data, the system generates (404) a corresponding software tool embedding for the new software tool.

[0112] In some implementations, in order to generate a software tool embedding associated with the new software tool data, the system initializes the corresponding software tool embedding for the new software tool using a random embedding, an existing embedding from a similar software tool, or any other initialization method.

[0113] The system can receive a training data set specific to the new software tool (e.g., training data included in the new software tool data), which includes multiple samples of the new software tool invocation input sequences and corresponding benchmark real software tool invocation outputs. The system can perform cue adjustments on the corresponding software tool embeddings for the new software tool to update the corresponding software tool embeddings. The system can compute a loss function for each training step, such as the next lexical prediction loss, such as the negative log-likelihood loss, etc., and pass it through a language model neural network (e.g., ...). Figure 1 The language model neural network 112) backpropagates the error and only updates the parameters associated with the new software tool embedding, while keeping the parameters of the underlying language model neural network unchanged (especially the trainable parameters, such as weights).

[0114] The system updates (406) the software tool usage data to include the corresponding software tool embedding for the new software tool. In some implementations, the system updates the database or data store that stores the software tool usage data to include the software tool embedding associated with the new software tool.

[0115] In addition to updating the software tool usage data using embeddings associated with the new software tool, the system also updates (408) the software tool selection embeddings to account for the addition of the new software tool to the software tool set. The system includes training data associated with the new software tool and uses a cue-based adjustment method to determine the updated software tool selection embeddings. Furthermore, the system updates (410) the software tool usage data to include the updated software tool selection embeddings.

[0116] In some implementations, the system updates the training dataset used to learn the software tool selection embeddings to include corresponding training data for the new software tool. For example, if the new software tool is an Online Shopping API, the system can append training data to include entries such as "Put video game console on the Online Shopping wishlist." The system can use the new training dataset, taking previous software tool selection embeddings as initialization points, to perform cue adjustments on the software tool selection embeddings. The system can compute a loss function for each training step and backpropagate the error through a language model neural network, updating only the parameters associated with the software tool selection embeddings while keeping the parameters of the underlying language model neural network unchanged. After the system has learned the software tool selection embeddings generated from the training data including data related to the new software tool, it can generate the most likely software tool to solve a problem, answer a question, or perform an action, where the most suitable software tool may be the new software tool. For example, before adding a new software tool that provides book reviews to the software tool selection embeddings, an input query such as "What is the top-rated spy novel of this year?" might not return suggested software tools. After adding a new software tool that provides book reviews to the software tool selection embedding, the hint is likely to fall within the set of examples provided for the new software tool, and the language model neural network will return the new software tool as a possible source of information in response to the input query.

[0117] The above-described technology can be implemented in a distributed computing system comprising a first computer system implementing a language model neural network 112 and a second computer system implementing one or both of a tool selection neural network and a tool embedding neural network. The tool selection neural network and the tool embedding neural network can be implemented in the same or different computing systems; the latter approach facilitates updating only a portion of the system, such as adding or removing tools. Software tool usage data can be maintained on the second computer system.

[0118] In this implementation, the first computer system may perform one or more of the following steps: receiving query input; processing a software tool selection input sequence; processing a software tool invocation input sequence; providing input to a specific software tool; obtaining output from the specific software tool; and generating a response to the query input. The second computer system may perform one or more of the following steps: maintaining software tool usage data; generating a software tool selection input sequence; and generating a software tool invocation input sequence. The specific software tool, or each specific software tool, may be implemented on one or more third computer systems.

[0119] There can be multiple second computer systems. For example, each of the second computer systems can be configured to support different groups of software tools, for example, for different corresponding user groups. Then, each of the second computer systems can maintain different corresponding software tool usage data (i.e., different software tool selection embeddings and / or different software tool embeddings) to generate different corresponding software tool selection input sequences and software tool invocation input sequences using each of the second computer systems.

[0120] This type of implementation is convenient because the software tool uses data, i.e., the software tool selects embeddings and the software tool embeddings can, for example, use hints to adjust the learning without modifying the learned parameters of the language model neural network, such as weights.

[0121] Figure 5 This is an example graph 500 used to compare training methods for language model neural networks configured to select software tools based on query input. Graph 500 includes a horizontal axis 502 representing multiple training steps used to train the neural network and a vertical axis 504 representing the prediction accuracy of the trained neural network. Graph 500 compares two methods for training neural networks. A first set of data values ​​506 represents a supervised fine-tuning training method, and a second set of data values ​​508 represents a cue-based fine-tuning training method.

[0122] Chart 500 illustrates accuracy data related to a software tool selection task. For example, the evaluated task in Chart 500 relates to an action associated with selecting a software tool from a set of multiple software tools. The vertical axis 504 represents the prediction accuracy of the corresponding neural network used to predict (i.e., select) the correct software tool based on processing a specific query input and a software tool selection embedding.

[0123] The first set of data values, 506, represents the supervised fine-tuning training method for the language model neural network. In other words, training data related to the selection of software tools is included in the training data of the language model neural network, and all parameters of the network are relearned based on the updated training data.

[0124] The second set of data values ​​508 represents a cue-adjusted training method, wherein the training process represented by value 508 includes learning the software tool selection embedding as a soft cue and processing the soft cue appended to the same hard cue used to generate value 506 to predict the relevant software tool based on a set of multiple software tools. Compared to data value 506, the method associated with data value 508 only needs to learn the parameters of the software tool selection embedding (rather than the parameters of the full language model neural network).

[0125] As shown in the comparison of the first data set 506 and the second data set 508, the cue-based training method exhibits a significant accuracy improvement with fewer training steps compared to the supervised fine-tuning training method. However, the cue-based training method requires more training steps to achieve the same asymptotic accuracy level as the supervised fine-tuning method. Furthermore, example graph 500 compares the two training methods, where the supervised fine-tuning method requires the system to learn approximately 350 million parameters, while the cue-based method requires the system to learn approximately 75,000 parameters, demonstrating a reduction of several orders of magnitude in the number of parameters to be learned.

[0126] The system trained the two models compared in Chart 500 using the same training data and provided each model with the same prompts (e.g., prompts indicating which software tool to select). For example, the shared prompts provided between the two models used to generate the comparison in Chart 500 could be: "The task is to call an API tool from the following tools: 'Tool A', 'Tool B', 'Tool C', …, 'Tool N'". The corresponding training data could include entries such as: "Add cat food to my list called Tool A – Tool A", "Open Tool A, not Tool B", "Use Tool C and show me the status of my order from the last week", and "Use Tool C, not Tool N to buy concerttickets". Each training data entry includes a cue (the part of the example cue before the dash) and a label (the part of the example cue after the dash). Other formats for displaying the training data input and corresponding labels are also possible.

[0127] The system evaluates the accuracy of training neural networks trained using supervised fine-tuning and cueing by determining the exact match between the labels of training data elements (e.g., “Tool A” or “Tool B”) and the outputs of the corresponding trained networks.

[0128] Figure 6 Example graphs 600 and 650 are shown for comparing training methods of language model neural networks configured to select software tools. The first example graph 600 shows a comparison of the predicted set of software tools 602 with the corresponding benchmark set of real software tools 604 as represented in the training data.

[0129] Example chart 600 illustrates the accuracy of a language model neural network trained using a cue-based tuning method. The first gradient scale 608 indicates the degree of correlation between the predicted software tool set 602 and the corresponding benchmark real software tool set 604.

[0130] Example chart 650 includes similar components and represents the accuracy of a language model neural network trained using a supervised fine-tuning method. Example charts 600 and 650 show light-colored squares as data points with higher relevance compared to dark-colored squares. The first diagonal correlation line 606 indicates a high correlation between the predicted tool 602 and the baseline true tool 604 represented in the training data. A trained model demonstrating perfect predictive ability would generate a chart similar to example chart 600, where all diagonal pixels are at their maximum values ​​and all off-diagonal pixels have zero values.

[0131] Example graphs 600 and 650 depict the correlations with the training phase after 14,000 training steps, as shown by example graph 500, where the output accuracy associated with the neural network trained through supervised fine-tuning and cueing converges.

[0132] As shown in example graph 600, a network trained using cue-tuned methods depicts a high correlation between two different tools (e.g., tool C in the predicted set of software tools and tool J in the benchmark set of real software tools). This correlation is depicted as bright off-diagonal pixels. This correlation can be due to various influences, including different software tools sharing multiple common features. For example, tools C and J may solve the same problem, use similar techniques, share a common user base, etc., causing the neural network processing the cue-tuned input embeddings to confuse mentions of tools C and J in the training data. In contrast, example graph 650, representing the accuracy of a supervised fine-tuned neural network, depicts fewer off-diagonal correlated elements but a greater number of “class-unclassed” predictions. Indicators of the trained neural network are tools not represented in the training data or indicators of “other” (represented in the predicted set of software tools 602 as “…”). The output of ") shows a "no category" prediction.

[0133] Figure 7 This is an example graph 700 used to compare training methods for language model neural networks configured to generate actual parameters to be processed by software tools. The training methods compared include supervised fine-tuning and cue-based tuning, represented by example data values ​​706 and 708, respectively.

[0134] Chart 700 includes a horizontal axis 702 representing the number of training steps used to train the neural network and a vertical axis 704 representing the accuracy of the trained neural network's predictions about the software tool parameters.

[0135] Chart 700 includes accuracy data related to the software tool's parameter generation task. For example, the evaluated task in Chart 700 is compared with... Figure 1The output of the language model neural network 112 is related to the accuracy of the prediction. The vertical axis 704, representing the prediction accuracy, is related to the accuracy of the corresponding neural network used to predict the correct parameters to be provided to a specific selected software tool based on processing a specific query input.

[0136] The first set of data values, 706, represents the supervised fine-tuning training method for the language model neural network. In other words, training data related to the generation of actual parameters is incorporated into the training data of the language model neural network, and all parameters of the network are relearned based on the updated training data.

[0137] The second set of data values ​​708 represents a cue-based training method, where the training process, represented by value 708, involves learning updated software tool embeddings as soft cues and processing the soft cues using the same hard cues used to generate value 706 to predict the set of actual parameters to be provided to the selected software tool, as indicated by the hard cues. The method associated with data value 708 does not achieve the same high asymptotic accuracy as data value 706. However, the method associated with data value 708 only needs to learn the embedded representations of the specific software tool without needing to relearn all the parameters of the language model neural network.

[0138] In some implementations, the system formats the training data for both training methods into a format including the target prediction value "{tool_name:intent}", where each training data element includes a pair of tools and a corresponding intent. Other applications may require generating multiple arguments from a list of possible arguments as output from the language model's neural network. The prediction accuracy depicted in example graph 700 can be evaluated based on precision, recall, F1 score, or any other metric sensitive to the correctness of the predicted output in relation to the provided training data.

[0139] This specification describes a system implemented as a computer program, wherein at least one computer program implements a language model neural network (e.g., language model neural network 112). The language model neural network is a generative neural network that has parameters and can be configured through training to process an input sequence consisting of lexical units from a vocabulary according to the parameters, thereby generating an output sequence of a generative task consisting of lexical units from the vocabulary based on the input sequence. For example, the input sequence may include cues that provide context for the output sequence.

[0140] After training, the training system or another inference system can deploy a generative neural network on one or more computing devices to perform inference on one or more generative tasks, i.e., to generate new output sequences for generative tasks based on new input sequences.

[0141] A lexicon may include any of a variety of lexicons that represent text symbols or other symbols. For example, a lexicon may include one or more of characters, subwords, words, punctuation marks, numbers or other symbols that appear in a corpus of natural language text and / or computer code.

[0142] Alternatively, the vocabulary of lexical units may include lexical units that can represent data other than text. For example, the vocabulary of lexical units may include image lexical units representing a discrete set of image patch embeddings of an image, which may be generated by an image encoder neural network based on image patches processed from an image. As another example, the vocabulary of lexical units may include audio lexical units representing code vectors in the codebook of a quantizer—e.g., a residual vector quantizer.

[0143] In some implementations, a generative neural network can be configured as an autoregressive language model neural network. A language model neural network is called an autoregressive neural network when it generates an output sequence of regressively generated lexical units from an output sequence conditioned on the current input sequence, which includes any (e.g., all) lexical units in the output sequence preceding the specific lexical unit (i.e., any previously generated lexical units in the output sequence preceding the specific lexical unit at a particular position in the output sequence) and the input sequence.

[0144] For example, when generating a lexical at any given position in the output sequence, the current input sequence can include any lexical at a preceding position in both the input and output sequences. As a specific example, the current input sequence can include the input sequence followed by any (e.g., all) lexical at preceding positions in the output sequence. Optionally, the input sequence and the current output sequence can be separated by one or more predetermined lexicals within the current input sequence.

[0145] More specifically, to generate a specific word at a specific position within the output sequence, a generative neural network can process the current input sequence to generate a score distribution, such as a probability distribution, that assigns a corresponding score, such as a corresponding probability, to each word in the vocabulary. The generative neural network can then use the score distribution to select words from the vocabulary as specific words. For example, the generative neural network can greedily select the word with the highest score, or it can sample words from the distribution, for example, using kernel sampling or another sampling technique.

[0146] As a specific example, a generative neural network may be or include an autoregressive Transformer-based neural network, which includes (i) a sequence of multiple attention blocks, each attention block having a self-attention operation applied; and (ii) an output subnetwork that processes the output of the last attention block to generate a score distribution.

[0147] Generative neural networks can be any of a variety of Transformer-based language model neural network architectures. Examples of such neural network architectures include those described in the following: Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu, Exploring the limits of transfer learning with a unified text-to-text transformer, arXiv preprint arXiv:1910.10683, 2019; Daniel Adiwardana, Minh-Thang Luong, David R. So, Jamie Hall, Noah Fiedel, Romal Thoppilan, Zi Yang, Apoorv Kulshreshtha, Gaurav Nemade, Yifeng Lu, and Quoc V. Le, Towards a human-like open-domain chatbot, CoRR, abs / 2001.09977, 2020; Tom B Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al., Language models are few-shot learners, arXiv preprint arXiv:2005.14165, 2020; Aakanksha Chowdhery, et al., PaLM: Scaling Language Modeling with Pathways, arXiv preprint arXiv:2204.02311; Rohan Anil, et al., Palm 2 technical report, arXiv preprint arXiv:2305.10403, 2023; and Gemini Team et al., Gemini: a family of highly capable multimodal models, arXiv preprints arXiv:2312.11805 (2023), arXiv:2408.00118, 2024, arXiv:2412.03555.

[0148] However, generally, Transformer-based language model neural networks consist of a sequence of attention blocks, and during processing of a given input sequence, each attention block in the sequence receives the corresponding input hidden state for each input word in the given input sequence. The attention block then updates the hidden state of at least the last word in the given input sequence by applying self-attention, at least in part, to generate the corresponding output hidden state for the last word. The input hidden state for the first attention block is the embedding of the input words in the input sequence, while the input hidden state for each subsequent attention block is the output hidden state generated by the previous attention block.

[0149] In this example, the output subnetwork processes the output hidden state generated by the last attention block in the sequence for the last input term in the input sequence to generate a score distribution.

[0150] Additional methods for implementing neural networks for language models include the RecurrentGemma family of language models, which combine linear recursion with local attention, as described in the arXiv preprint arXiv:2404.07839 (2024). Similarly, the Griffin neural network model, as described in the arXiv preprint arXiv:2402.19427, is a hybrid model that combines gated linear recursion with local attention. Text diffusion models, as described in the arXiv preprint arXiv:2211.15089, represent discrete language variables in a continuous space.

[0151] For example, a generative neural network can generate text sequences, where each output sequence generated by the generative neural network is a sequence of text lexical units from a vocabulary of text lexical units, which includes one or more of, for example, characters, subwords, words, punctuation marks, numbers, or other symbols appearing in natural language text. For example, an inference system can use a generative neural network to generate text sequences and provide said text sequences to a user.

[0152] As another example, a generative neural network can generate images or videos with multiple frames (each frame being an image) by generating images (e.g., as a sequence of pixels or through an iterative denoising process). For example, the output sequence generated by a generative neural network includes multiple color values ​​of pixels in an image arranged in a specified order. As another example, the output sequence generated by a generative neural network includes multiple terms representing image patch embeddings, which can then be processed by a decoder neural network to generate an image. For example, an inference system can use a generative neural network to generate images or videos conditioned on an input sequence that includes a textual description of the content of the image or video.

[0153] As another example, the input sequence is a text sequence, and the output sequence is another text sequence, such as completion of the input text sequence, interpretation of the input text sequence, response to a question posed in the input sequence, or a text sequence about a topic specified by the input text sequence. As another example, the input sequence can be input other than text, such as multiple pixels included in an image, and the output sequence can be a text sequence describing the input.

[0154] As another example, the input sequence represents the data to be compressed, such as image data, text data, audio data, or any other type of data; and the output sequence is a compressed version of the data. The terms included in the output sequence can include any representation of the compressed data, such as symbols or embeddings to be decoded by a corresponding neural network.

[0155] As a specific example, the inference system can be part of a dialogue system, and the input sequence can include audio or text from the most recent conversation round submitted by the user during the dialogue, while the output sequence is the next round in the conversation, such as text or audio as a response to the most recent conversation round. Optionally, the input sequence can also include one or more historical conversation rounds that occurred earlier in the conversation.

[0156] As another specific example, the inference system can be part of a machine translation system, and the input sequence can include source language text, while the output sequence can include target language text, which is the source text translated into the target language.

[0157] As another specific example, the reasoning system can be part of a natural language processing system. For instance, if the input sequence is a sequence of words in the original language, such as sentences or phrases, then the output sequence can be a summary of the input sequence in the original language—that is, a sequence with fewer words than the input sequence but retaining the basic meaning of the input sequence. As another example, if the input sequence is a sequence of words that form a question, then the output sequence can be a sequence of words that form the answer to the question.

[0158] As another specific example, the reasoning system can be part of a computer-aided medical diagnostic system. For instance, the input sequence could be a data sequence from an electronic medical record, while the output sequence could each be a predicted treatment sequence.

[0159] As another specific example, the inference system can be part of a computer code generation system, and the input sequence can include a textual description of the desired code segment or an excerpt of computer code in a programming language, and the output sequence can include computer code, such as an excerpt of code described by the input sequence or an excerpt of code in a computer program that follows the input sequence.

[0160] As another specific example, an inference system can be part of a multimodal system that processes multimodal input sequences, such as both text and image input sequences, or both text and audio input sequences, and generates output sequences in a single data modality or multiple data modalities, such as a combination of two or more text, image, and audio output sequences. Examples of such multimodal systems include image description systems, text-based image search systems, image-based question answering systems, and so on.

[0161] As another specific example, the inference system can be part of a digital agent, where the agent autonomously performs specific tasks or assists a user by utilizing the output from the inference system. Furthermore, the inference system can process real-time streaming data between the digital agent and the user.

[0162] As another specific example, the inference system may be part of or associated with a robot control system, i.e., a system for controlling one or more mechanical agents. The input sequence may include a natural language description of one or more tasks for the one or more mechanical agents, and the output sequence may include a sequence of instructions (e.g., joint angles, torques, speeds, etc.) for the one or more mechanical agents, which instruct the one or more mechanical agents to perform one or more tasks described in the input sequence.

[0163] In a similar example, the inference system may be part of or associated with a control system in a manufacturing environment used to manufacture a product, i.e., a system for controlling a manufacturing unit or operating a machine to manufacture a product. In another similar example, the inference system may be part of or associated with a control system in a service facility that includes multiple electronic devices.

[0164] As another specific example, the inference system can be part of or associated with a search system that facilitates searching for resources on the Internet. Resources can be any data available via the Internet. Resources can be identified by the resource address associated with them. Resources include web pages, word processing documents, portable document format (PDF) documents, images, videos, and news feeds, to name just a few.

[0165] In this specific example, the search system may receive a search query submitted by a client device and, in response, identify resources related to the search query in the form of search results, returning the search results to the user device in a search results page. The search results page may include search results data generated by the search system that identifies the resources in response to the search query and includes links to those resources. The search results page may additionally include results in the form of an output sequence generated by an inference system based on an input sequence derived from the search query.

[0166] Generative neural networks are typically trained using a multi-stage approach: a pre-training stage followed by a fine-tuning stage, where at least the fine-tuning stage takes place at the training system. For example, the training system may receive data from another system specifying a pre-trained generative neural network and then fine-tune the pre-trained network.

[0167] During the pre-training phase, the generative neural network is pre-trained on one or more large datasets by a training system or another system based on optimizing one or more unsupervised or self-supervised objective functions (e.g., maximum likelihood objective functions). Then, in some cases, it is fine-tuned for a generative task through supervised learning, reinforcement learning with human feedback (RLHF), reinforcement learning with AI feedback (RLAIF), cue tuning, instruction tuning, etc. The generative task may include any combination of one or more generative tasks mentioned below and possible other tasks. This fine-tuning may use different training objectives, different datasets, or both.

[0168] One or more large datasets used during the pre-training phase may include: a large dataset of text in one or more natural languages ​​(e.g., text publicly available from the Internet or another text corpus), a large dataset of computer code in one or more programming languages ​​(e.g., computer code publicly available from the Internet or another code repository), a large dataset of audio samples (e.g., audio recordings or waveforms representing audio recordings), a large dataset of images (where each image includes an array of pixels), a large dataset of videos (where each video includes a sequence of time frames), or a large multimodal dataset that includes a combination of two or more of these datasets.

[0169] In this specification, the term "configured" is used in connection with computing systems and environments, as well as computer program components. A computing system or environment is considered "configured" to perform specific operations or actions when it possesses the necessary software, firmware, hardware, or combinations thereof that enable it to perform specific operations or actions during operation. For example, configuring a system may involve installing a software library with specific algorithms, updating firmware with new instructions for data manipulation, or adding hardware components to gain enhanced processing power. Similarly, one or more computer programs are "configured" to perform those intended operations or actions when they contain instructions that, when executed by a computing device or hardware, cause the device to perform specific operations or actions.

[0170] The embodiments and functional operations described in this specification can be implemented in various forms, including digital electronic circuit systems, software, firmware, computer hardware (covering the disclosed structures and their equivalents) or any combination thereof. The subject matter can be implemented as one or more computer programs, which are essentially modules of computer program instructions encoded on a tangible, non-transitory storage medium for execution by a computing device or hardware or for controlling the operation of a computing device or hardware. The storage medium can be a storage device such as a hard disk drive or solid-state drive (SSD), a storage medium, a random or serial access memory device, or a combination thereof. Alternatively or additionally, program instructions can be encoded on transmitted signals such as machine-generated electrical, optical, or electromagnetic signals, designed to carry information for transmission to a receiving device or system for execution by a computing device or hardware. Furthermore, implementations can leverage emerging technologies such as quantum computing or neuromorphic computing for specific applications and can be deployed in distributed or cloud-based environments where components reside on different machines or within a cloud architecture.

[0171] The term "computing device or hardware" refers to the physical components involved in data processing and encompasses all types of devices and machines used for this purpose. Examples include processors or processing units, computers, multiple processors or computers working together, graphics processing units (GPUs), tensor processing units (TPUs), and specialized processing hardware such as field-programmable gate arrays (FPGAs) or application-specific integrated circuits (ASICs). In addition to hardware, computing devices or hardware may also include code that creates the execution environment for computer programs. This code can take the form of processor firmware, protocol stacks, database management systems, operating systems, or combinations of these elements. In the context of general-purpose computing graphics processing units (GPGPUs), embodiments can particularly benefit from leveraging the parallel processing capabilities of GPUs, where code specifically designed for GPU execution is employed, often referred to as kernels or shaders. Similarly, TPUs excel at running optimized tensor operations crucial to many machine learning algorithms. By utilizing these accelerators and their specialized programming models, the system can achieve significant speedups and efficiency gains for tasks involving artificial intelligence and machine learning, particularly in fields such as computer vision, natural language processing, and robotics.

[0172] Computer programs (also known as software, applications, modules, scripts, code, or simply programs) can be written in any programming language, including compiled or interpreted languages, as well as declarative or procedural languages. They can be deployed in various forms, such as standalone programs, modules, components, subroutines, or any other unit suitable for use in a computing environment. A program may or may not correspond to a single file in a file system and can be stored in various ways. This includes being embedded within a file containing other programs or data (e.g., a script within a markup language document), residing in a dedicated file, or distributed across multiple coordination files (e.g., files storing modules, subroutines, or code segments). Computer programs can execute on a single computer or across multiple computers, whether located at a single site or distributed across multiple sites and interconnected via data communication networks. The specific implementation of a computer program can involve a combination of traditional programming languages ​​and specialized languages ​​or libraries designed for GPGPU programming or TPU utilization, depending on the chosen hardware platform and desired performance characteristics.

[0173] In this specification, the term "engine" broadly refers to a software-based system, subsystem, or process designed to perform one or more specific functions. An engine is generally implemented as one or more software modules or components installed on one or more computers, which may be located at a single site or distributed across multiple locations. In some cases, one or more dedicated computers may be used for a particular engine, while in others, multiple engines may operate concurrently on the same one or more computers. Examples of engine functions in the context of AI and machine learning include data preprocessing and cleaning, feature engineering and extraction, model training and optimization, inference and prediction generation, and post-processing of results. The specific design and implementation of an engine will depend on the overall architecture and the distribution of computational tasks across various hardware components, including CPUs, GPUs, TPUs, and other specialized processors.

[0174] The processes and logic flows described in this specification can be executed by one or more programmable computers, which run one or more computer programs to perform functions by manipulating input data and generating output. Additionally, graphics processing units (GPUs) and tensor processing units (TPUs) can be used to implement aspects of these processes and logic flows concurrently, significantly accelerating execution. This approach offers significant advantages for computationally intensive tasks common in AI and machine learning applications, such as matrix multiplication, convolution, and other operations exhibiting high parallelism. By leveraging the parallel processing capabilities of GPUs and TPUs, significant speedup and efficiency gains can be achieved compared to relying solely on CPUs. Alternatively, or in combination with programmable computers and dedicated processors, these processes and logic flows can also be implemented using dedicated processing hardware, such as field-programmable gate arrays (FPGAs) or application-specific integrated circuits (ASICs), to achieve even higher performance or energy efficiency in specific use cases.

[0175] Computers capable of executing computer programs can be based on general-purpose microprocessors, special-purpose microprocessors, or a combination of both. They can also utilize any other type of central processing unit (CPU). Additionally, graphics processing units (GPUs), tensor processing units (TPUs), and other machine learning accelerators can be employed to enhance performance, particularly for tasks involving artificial intelligence and machine learning. These accelerators typically work in conjunction with the CPU to handle specialized computations, while the CPU manages overall system operation and other tasks. Typically, the CPU receives instructions and data from read-only memory (ROM), random access memory (RAM), or both. The components of a computer include a CPU for executing instructions and one or more memory devices for storing instructions and data. The specific configuration of the processing unit and memory will depend on factors such as the complexity of the AI ​​model, the amount of data being processed, and the desired performance and latency requirements. Implementations can be carried out on a wide range of computing platforms, from small embedded devices with limited resources to large-scale data center systems with high-performance computing capabilities. Systems may include storage devices such as hard disks, SSDs, or flash memory for persistent data storage.

[0176] Computer-readable media suitable for storing computer program instructions and data encompass all forms of non-volatile memory, media, and storage devices. Examples include semiconductor memory devices such as read-only memory (ROM), solid-state drives (SSDs), and flash memory devices; hard disk drives (HDDs); optical media; and optical discs such as CDs, DVDs, and Blu-ray discs. The specific type of computer-readable media used will depend on factors such as the size of the data, access speed requirements, cost considerations, and the desired level of portability or durability.

[0177] To facilitate user interaction, embodiments of the subject matter described herein can be implemented on computing devices equipped with display devices (such as liquid crystal displays (LCDs) or organic light-emitting diode (OLED) displays) for presenting information to users. Input can be provided by the user through various means, including keyboards, touchscreens, voice commands, gesture recognition, or other input modalities, depending on the specific device and application. Additional input methods may include sound, voice, or tactile input, while feedback to the user may take the form of visual, auditory, or tactile feedback. Furthermore, the computer can interact with the user by exchanging documents with the user's device or application. This may involve sending web content or data in response to a request, or sending and receiving text messages or other forms of messages via mobile devices or messaging platforms. The choice of input and output modalities will depend on the specific application and the desired form of user interaction.

[0178] Machine learning models can be implemented and deployed using machine learning frameworks such as TensorFlow or JAX. These frameworks provide comprehensive tools and libraries that facilitate the development, training, and deployment of machine learning models.

[0179] Embodiments of the subject matter described in this specification can be implemented within a computing system comprising one or more components, depending on the specific application and requirements. These components may include: backend components, such as backend servers or cloud-based infrastructure; optional middleware components, such as middleware servers or application programming interfaces (APIs), for facilitating communication and data exchange; and frontend components, such as client devices having a user interface, web browser, or app through which users can interact with the implemented subject matter. For example, the described functionality may be implemented only on the client device (e.g., for on-device machine learning) or deployed as a combination of frontend and backend components for more complex applications. Where present, these components may be interconnected using any form or medium of digital data communication, such as communication networks like local area networks (LANs) or wide area networks (WANs), including the Internet. The specific system architecture and component selection will depend on factors such as the scale of the application, the need for real-time processing, data security requirements, and the desired user experience.

[0180] Computing systems can include geographically separated clients and servers that interact via communication networks. The specific type of network (such as a local area network (LAN), wide area network (WAN), or the Internet) will depend on the accessibility and scale of the application. The client-server relationship is established through computer programs running on the respective computers and designed to communicate with each other using appropriate protocols. Depending on the nature of the exchanged data and the system's security requirements, these protocols may include HTTP, TCP / IP, or other specialized protocols. In some embodiments, the server sends data or instructions to a user device acting as a client, such as a computer, smartphone, or tablet. The client device can then process the received information, display the results to the user, and potentially send data or feedback back to the server for further processing or storage. This allows for dynamic interaction between the user and the system, enabling a wide range of applications and functionalities.

[0181] While this specification contains numerous details of specific implementations, these details should not be construed as limiting the scope of any invention or the scope that may be claimed, but rather as descriptions of features that may be characteristic of particular embodiments of a particular invention. Certain features described in this specification within the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments. Furthermore, although features may be described above as operating in certain combinations and even initially claimed in this way, in some cases one or more features from the claimed combination may be removed from the combination, and the claimed combination may involve sub-combinations or variations thereof.

[0182] Similarly, although operations are depicted in the accompanying drawings and described in a specific order in the claims, this should not be construed as requiring such operations to be performed in the specific order shown or in a sequential order, or requiring all shown operations to be performed to achieve the desired result. In some contexts, multitasking and parallel processing may be advantageous. Furthermore, the separation of various system modules and components in the above embodiments should not be construed as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0183] Specific embodiments of this subject matter have been described. Other embodiments are within the scope of the appended claims. For example, the actions recited in the claims can be performed in a different order and still achieve the desired result. As an example, the processes depicted in the figures do not necessarily require the specific order or sequential order shown to achieve the desired result. In some cases, multitasking and parallel processing can be advantageous.

Claims

1. A method executed by one or more computers, the method comprising: Maintain software tool usage data, for each software tool in a set comprising multiple software tools, the software tool usage data including software tool selection embedding and corresponding software tool embedding; Receive query input; A software tool selects an input sequence, the software tool selecting an input sequence including the software tool selection embedding and a first plurality of embeddings characterizing the query input; A language model neural network is used to process the software tool selection input sequence to generate a software tool selection output that identifies a specific software tool in the set of software tools. as well as A software tool invocation input sequence is generated, the software tool invocation input sequence including a corresponding software tool embedding for the specific software tool and a second plurality of embeddings characterizing the query input.

2. The method of claim 1, further comprising: The language model neural network is used to process the software tool call input sequence to generate a software tool call output, which specifies the input to the particular software tool. Provide the input to the specific software tool; The output is obtained from the specific software tool. as well as A response to the query input is generated based on the output obtained from the specific software tool.

3. The method according to any of the preceding claims, further comprising: Receive data specifying the new software tool to be added to the set of software tools; as well as In response to receiving the data specifying the new software tool to be added to the set of software tools: Generate a corresponding software tool embedding for the new software tool; and Update the software tool usage data to include the corresponding software tool embedding for the new software tool.

4. The method of claim 3, wherein generating a corresponding software tool embedding for the new software tool comprises: Initialize the corresponding software tool embedding for the new software tool; Receive first training data, the first training data including a plurality of training software tool call input sequences for the new software tool, and for each training software tool call input sequence, the first training data including a corresponding benchmark real software tool call output for the new software tool; as well as The first training data is used to perform cue adjustments on the corresponding software tool embedding for the new software tool to update the corresponding software tool embedding for the new software tool.

5. The method of claim 3 or claim 4, further comprising: In response to receiving the data specifying the new software tool to be added to the set of software tools: Update the software tool selection embedding to take into account the addition of the new software tool to the set of software tools; and Update the software tool usage data to include the updated software tool selection embedding.

6. The method of claim 5, wherein updating the software tool selection embedding to take into account the addition of the new software tool to the set of software tools comprises: Receive second training data, the second training data including a plurality of training software tool selection input sequences, and for each training software tool selection input sequence, the second training data including data identifying a corresponding benchmark real software tool selected in response to the training software tool selection input sequence, wherein for at least a subset of the training software tool selection input sequences, the corresponding benchmark real software tool is the new software tool; as well as The training data is used to perform cue adjustments on the software tool selection embedding to update the corresponding software tool embedding for the new software tool.

7. The method of any of the preceding claims, wherein the software tool has been learned to select embeddings by prompting adjustments while keeping the language model neural network unchanged.

8. The method of any of the preceding claims, wherein the respective software tool embeddings for the software tools in the set of software tools have been learned through cueing adjustments, while keeping the language model neural network unchanged.

9. The method of any of the preceding claims, wherein the software tool selection output is an output sequence specifying an identifier for the particular software tool.

10. The method of any one of claims 1 to 8, wherein the software tool selection output is a corresponding likelihood score for each of one or more of the software tools, and wherein the method further comprises: The specific software tool is selected based on the corresponding likelihood score.

11. The method of any preceding claim, wherein each software tool receives input, the input including a corresponding value for each of one or more input parameters for the software tool, and wherein the software tool invocation output is an output sequence specifying a corresponding particular value for each of the one or more input parameters for the particular software tool.

12. The method of any preceding claim, wherein generating a response to the query input based on the output obtained from the particular software tool comprises: The software tool output-input sequence is generated based at least on the output obtained from the specific software tool. as well as The language model neural network is used to process the input sequence of the software tool to generate an output sequence that defines the response to the query input.

13. The method of any one of claims 1 to 12, wherein the method is implemented in a distributed computing system, the distributed computing system comprising a first computer system implementing the language model neural network and a second computer system implementing one or both of a tool selection neural network and a tool embedding neural network, the method further comprising: Maintain the software tool usage data on the second computer system; and The first computer system performs the following steps: Receive the query input; The software tool is used to select the input sequence. Process the input sequence of the software tool call; Provide the input to the specific software tool; The output is obtained from the specific software tool; and Generate the response to the query input; and The second computer system performs the following steps: Maintain the usage data of the software tools; The software tool generates the input sequence to be selected; and Generate the input sequence for calling the software tool.

14. The method of claim 13, comprising a plurality of second computer systems, wherein each of the second computer systems is configured to support different groups of software tools for different corresponding user groups, and wherein each of the second computer systems maintains different corresponding software tool usage data for generating different corresponding software tool selection input sequences and software tool invocation input sequences.

15. The method of claim 13 or 14, further comprising: The software tool learns to use data without modifying the learned parameters of the language model neural network.

16. The method of any one of claims 1 to 15, wherein the software tool comprises at least one image processing tool, wherein the query input defines an image processing task, and wherein the response to the query input defines a result of the image processing task.

17. The method of any one of claims 1 to 16, wherein the software tool includes at least one material design tool for designing physical materials, wherein the query input defines a material design task, and wherein the response to the query input defines a result of the material design task.

18. The method of any one of claims 1 to 17, wherein the software tool includes at least one robot control tool for controlling a mechanical robot to perform a physical task, wherein the query input defines the physical task, and wherein the response to the query input defines the result of the physical task.

19. A method, executed by one or more computers, for generating a corresponding software tool embedding for a new software tool to be added to software tool usage data, for each software tool in a set comprising a plurality of software tools, the software tool usage data including a software tool selection embedding and a corresponding software tool embedding, the method comprising: Initialize the corresponding software tool embedding for the new software tool; Receive first training data, the first training data including a plurality of training software tool call input sequences for the new software tool, and for each training software tool call input sequence, the first training data including a corresponding benchmark real software tool call output for the new software tool; as well as The first training data is used to perform cue adjustments on the corresponding software tool embedding for the new software tool to update the corresponding software tool embedding for the new software tool.

20. The method of claim 19, further comprising: In response to receiving data specifying the new software tool to be added to the set of software tools: Update the software tool selection embedding to take into account the addition of the new software tool to the set of software tools; and Update the software tool usage data to include the updated software tool selection embedding.

21. One or more non-transitory computer-readable media, the non-transitory computer-readable media storing an embedding generated by performing operations, the operations including: Maintain software tool usage data, for each software tool in a set comprising multiple software tools, the software tool usage data including software tool selection embedding and corresponding software tool embedding; Receive query input; A software tool selects an input sequence, the software tool selecting an input sequence including the software tool selection embedding and a first plurality of embeddings characterizing the query input; A language model neural network is used to process the software tool selection input sequence to generate a software tool selection output that identifies a specific software tool in the set of software tools. as well as A software tool invocation input sequence is generated, the software tool invocation input sequence including the corresponding software tool embedding for the specific software tool and a second plurality of embeddings characterizing the query input.

22. A system comprising: One or more computers; And one or more storage devices for storing instructions, which, when executed by the one or more computers, cause the one or more computers to perform the operation of the corresponding method as claimed in any one of claims 1 to 20.

23. One or more non-transitory computer-readable media storing instructions that, when executed by one or more computers, cause the one or more computers to perform the operations of the corresponding methods as claimed in any one of claims 1 to 20.

Images