Selection inference neural network system

Abstract

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generating a response to query input using a selective inference neural network.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims priority to U.S. Provisional Application No. 63 / 402,382, filed on August 30, 2022. The disclosure of the prior application is considered part of and incorporated by reference into the disclosure of the present application. Background Art

[0002] This specification relates to generating responses to query inputs using neural networks.

[0003] A neural network is a machine learning model that uses one or more layers of nonlinear units to predict an output for a received input. Some neural networks include one or more hidden layers in addition to the output layer. The output of each hidden layer is used as the input to the next layer in the network, that is, the next hidden layer or output layer. Each layer of the network generates an output from the received input based on the current values ​​of the corresponding set of parameters. Summary of the invention

[0004] This specification describes a system implemented as a computer program on one or more computers in one or more locations that generates responses to queries related to an environment.

[0005] Specifically, the system receives a context input including context information related to an environment and a query input including a query related to the environment.

[0006] The system then uses the contextual input to generate a response to the query, such as a natural language response.

[0007] The system generates a response by repeatedly alternating between performing selection steps and inference steps. As a result, the system is also able to generate trace data that provides natural language, interpretable information characterizing how the system arrived at the response.

[0008] In one aspect, a method includes obtaining a context input including context information, the context information including one or more natural language sentences each representing a fact or rule related to the environment; receiving a query input including a query related to the environment; generating a response to the query input by performing update iterations until a termination criterion is satisfied, the generating including at each update iteration: processing a selection input including the context input and the query input using a selection neural network to generate a selection output for the update iteration, the selection output including one or more natural language sentences from the natural language sentences of the context input; processing an inference input including the selection output of the update iteration using an inference neural network to generate an inference output including a natural language sentence representing a new fact for the update iteration; generating a first stop input based on the inference output of the update iteration and the query input; processing the first stop input using a first stop neural network to generate a first stop output indicating whether the inference output contains information necessary to respond to the query input; and determining whether the termination criterion is satisfied based on the first stop output.

[0009] In some implementations, generating further includes, in response to determining that the termination criterion is not satisfied, updating the context input to include the natural language sentence in the inference output of the update iteration.

[0010] In some implementations, the method further includes determining that a termination criterion is satisfied when the stop output indicates that the inference output contains information necessary to respond to the query input.

[0011] In some implementations, the method further includes determining that a termination criterion is satisfied when the stop output indicates that the inference output does not contain information necessary to respond to the query input but a maximum number of update iterations have been performed.

[0012] In some implementations, generating further includes, in response to determining that the inferred output does not contain information necessary to respond to the query input but a maximum number of update iterations have been performed, providing as a response a natural language output indicating that the query has not been answered.

[0013] In some embodiments, the method further includes: in response to determining that the stop output indicates that the inference output does contain information necessary to respond to the query input, providing (i) a natural language output derived from a natural language statement in the inference output of the last update iteration as a response.

[0014] In some implementations, the response further includes (ii) an inference trace comprising natural language sentences in the inference output at each update iteration except the last update iteration.

[0015] In some implementations, the inference trace further includes a selection output of the update iteration.

[0016] In some embodiments, the method further includes: generating a second stalling input from at least the inference output at the update iteration; and processing the second stalling input using a second stalling neural network to generate a second stalling output, the second stalling output identifying a natural language output derived from the natural language sentence in the inference output of the last update iteration.

[0017] In some embodiments, the second stopping neural network is the same neural network as the first stopping neural network.

[0018] In some implementations, the first stop neural network and the second stop neural network are language model neural networks, wherein the first stop input comprises a first stop text prompt, and wherein the second stop input comprises a second, different stop text prompt.

[0019] In some implementations, the query input further includes a plurality of natural language options, wherein the second stop input further includes a plurality of natural language options, and wherein the second stop output identifies one of the plurality of natural language options as a natural language statement.

[0020] In some implementations, the inferred input does not include the query input or any natural language statements from the context input that are not included in the selected output of the update iteration.

[0021] On the other hand, a method includes obtaining a context input including context information, the context information including one or more natural language statements each representing a fact or rule related to the environment; receiving a query input including a query related to the environment; and generating a response to the query input conditioned on the context input, including, at each bundle step in a sequence of one or more bundle steps: obtaining a plurality of candidate reasoning trajectories generated by a selection inference system as part of generating a response to the query input conditioned on the context input, each candidate reasoning trajectory including a corresponding selection output of the update iteration and a corresponding inference output of the update iteration for each of one or more update iterations; and maintaining up to a predetermined number of candidate reasoning trajectories in the bundle for further consideration by the selection inference system, including: processing each reasoning trajectory using a value neural network to generate a corresponding score for each reasoning trajectory, the corresponding score estimating the likelihood that the candidate reasoning trajectory is a valid reasoning trajectory, the valid reasoning trajectory being included in the true value reasoning trajectories that result in a true value response to the query input; ranking the candidate reasoning trajectories based on the corresponding scores; and removing all candidate reasoning trajectories except a predetermined number of the highest ranked candidate reasoning trajectories from the bundle according to the ranking.

[0022] In some implementations, generating a response further includes: at a final bundle step in the sequence, determining that each candidate inference trajectory not removed from the bundle has been finalized, and in response, generating a response based on the highest ranked candidate inference trajectory.

[0023] In some implementations, the value neural network is configured to generate a value output that specifies a probability distribution over a vocabulary of word-grams, and wherein the score is based on a probability assigned by the probability distribution to a predetermined word-gram from the vocabulary.

[0024] In some implementations, the value neural network is a language model neural network.

[0025] In some implementations, a value neural network has been trained on training examples, each training example including (i) a corresponding partial reasoning trace and (ii) a label indicating whether the partial reasoning trace is a valid reasoning trace included in the corresponding full reasoning trace.

[0026] In some embodiments, the method further includes: at each bundle step except the last bundle step in the sequence, determining that at least one of the candidate reasoning trajectories that have not been removed from the bundle has not been finalized, and in response, causing the selection inference system to generate, for each candidate reasoning trajectory that has not been removed from the bundle and has not been finalized, a corresponding selection output of a new update iteration and a corresponding inference output of the new update iteration after the last update iteration in the candidate reasoning trajectory.

[0027] In some embodiments, a selection inference system is a system configured to perform the following operations: obtain context input; receive query input; receive a current candidate reasoning trajectory, the current candidate reasoning trajectory including a corresponding selection output of the update iteration and a corresponding inference output of the update iteration for each update iteration in one or more update iterations; and perform a new update iteration to update the current candidate reasoning trajectory to add a new selection output and a new inference output to the candidate reasoning trajectory, including: using a selection neural network to process the selection input to generate a new selection output of a new update iteration, the selection input including (i) context input, (ii) one or more natural language sentences derived from the corresponding inference output in the candidate reasoning trajectory, and (iii) query input, the new selection output including one or more natural language sentences from the natural language sentences of the context input, one or more natural language sentences derived from the corresponding inference output, or both; and using an inference neural network to process the inference input including the selection output of the new update iteration to generate a new inference output, wherein the new inference output includes a natural language sentence representing a new fact of the new update iteration.

[0028] Particular embodiments of the subject matter described in this specification can be implemented so as to realize one or more of the following advantages.

[0029] One problem with using neural network-based approaches to make control decisions for controlling, for example, an intelligent agent or a manufacturing plant is that it is often difficult to infer why a particular decision has been made—the neural network is simply a "black box" that is trusted to perform the task. Similarly, when diagnosing faults in mechanical systems, it can be useful to know the reasons behind the response.

[0030] Embodiments of the described system can solve this and other problems by providing a reasoning trace that can be displayed or otherwise provided to a user or stored for later review. This can be particularly useful when control decisions or fault diagnosis involve safe operation of a mechanical agent or manufacturing plant, or other real-world scenarios where explainable responses are critical. Embodiments of the described system can provide a reasoning trace in human-interpretable natural language, for example, as a sequence of logical steps, presented as natural language statements leading from a query to a response in a causal chain.

[0031] More specifically, by generating a response by repeatedly alternating between: 1) selection, which involves picking a subset of relevant information sufficient to make a single inference step; and 2) inference, which sees only the limited information provided to it by the selection output and uses it to infer new intermediate evidence about the process of producing the final answer, the described techniques ensure that the intermediate inference outputs (and optionally, the selection outputs) provide an interpretable reasoning trail to justify the final answer. Furthermore, the reasoning produced by the described techniques is causal, in that each step follows and depends on the previous steps, and each inference is made independently based solely on the limited information provided by the selection output, without direct access to the query input or previous reasoning steps. This results in high-quality responses while simultaneously providing a reasoning trail, i.e., an interpretable natural language "trace" of the reasoning performed by the system in generating the response.

[0032] In addition, the system can dynamically determine when to stop performing update iterations for any given query and provide a response based on the most recently generated inference output. In doing so, the system can both improve the quality of the generated responses and reduce the amount of computing resources consumed by the selection inference process. In addition, the system can use this "halting" mechanism to determine when to generate a response that indicates that the system cannot accurately answer the query input. This can greatly improve the utility and safety of the system when deployed in a real-world environment, for real-world tasks, or both, where accuracy is prioritized because generating inaccurate responses can have significant negative consequences.

[0033] Additionally, the specification describes techniques for using a value neural network to maintain a bundle of multiple candidate reasoning trajectories and determine which trajectories to update and which candidate to use to generate a final response. By utilizing a value neural network, the system is able to efficiently search in a large space of candidate reasoning trajectories to generate an accurate final response by ensuring that the candidate reasoning trajectories maintained through the search process are both valid and likely to be included in the true value reasoning trajectory while cutting back on candidate reasoning trajectories that are unlikely to be valid, unlikely to be included in the true value, or both.

[0034] The details of one or more embodiments of the subject matter of this specification are set forth in the accompanying drawings and the description below. Other features, aspects, and advantages of the subject matter will become apparent from the description, drawings, and claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1A An example selection inference system is shown.

[0036] Figure 1B A choice inference system for controlling an agent interacting with an environment is shown.

[0037] Figure 1C A simplified example of selection and inference steps performed when the system is used to control a vehicle is shown.

[0038] Figure 1D An overview of the operation of the system at a given update iteration is shown.

[0039] Figure 2 is a flow chart of an example process for generating a response to a query input.

[0040] Figure 3 An example of utilizing a stopping neural network when a query input includes multiple options is shown.

[0041] Figure 4 is a flow diagram of an example process for updating a bundle of candidate responses to a query input.

[0042] Like reference numbers and designations in the various drawings indicate like elements. DETAILED DESCRIPTION

[0043] This specification describes a system implemented as a computer program on one or more computers in one or more locations that uses contextual information about an environment to generate responses to queries related to the environment.

[0044] Figure 1AAn example selection inference system 100 is shown. Selection inference system 100 is an example of a system implemented as a computer program on one or more computers in one or more locations in which the systems, components, and techniques described below can be implemented.

[0045] The selection inference system 100 is a system that receives a query input 110 and a context input 120 and generates a response 150 to the query input 110 using the context input 120 by performing update iterations.

[0046] The query input 110 includes queries related to the environment. For example, each query can be a text fragment in natural language.

[0047] The context input 120 includes context information that provides context about the environment. More specifically, the context information in the context input 120 includes one or more natural language sentences that each represent facts or rules related to the environment.

[0048] In some implementations, the environment is a real-world environment, and the system 100 facilitates reasoning in the real-world environment, such as controlling real-world systems in the environment in a principled and logical manner.

[0049] As described later, in some embodiments, the system 100 can also provide trajectory data that includes a causal explanation of why a particular response 150 has been generated, which can promote trust in the system, particularly in environments where safety is important.

[0050] For example, in some embodiments, the environment is a real-world environment, and the response 150 is used to control a mechanical agent acting in the real-world environment, such as a robot or an autonomous or semi-autonomous vehicle, to perform a task.

[0051] Figure 1B An example of the operation of selection inference system 100 is shown when the environment is a real-world environment navigated by mechanical agent 102 .

[0052] exist Figure 1B In the example of FIG. 1 , the agent 102 is shown as a vehicle. However, more generally, the mechanical agent 102 can be any suitable agent that is controlled by a control system as the agent navigates in a real-world environment.

[0053] The agent 102 includes one or more sensors 104 that capture observations of the environment, for example at specified time intervals, as the agent 102 navigates the environment.

[0054] For example, observations may include, for example, one or more of the following items used to capture observations when the agent interacts with the environment: images, object positioning data, and sensor data, such as sensor data from images, distance or positioning sensors, or from actuators. For example, in the case of a robot, observations may include data characterizing the current state of the robot, such as one or more of the following: joint positioning, joint velocity, joint force, torque or acceleration, such as torque feedback for gravity compensation, and the global or relative posture of an item held by the robot. In the case of a robot or other mechanical agent or vehicle, observations may similarly include one or more of the positioning, linear or angular velocity, force, torque or acceleration, and global or relative posture of one or more components of the agent. Observations may be defined in 1, 2, or 3 dimensions and may be absolute and / or relative observations. Observations may also include, for example, sensed electronic signals, such as motor current or temperature signals; and / or, for example, image or video data from a camera or LIDAR sensor, such as data from a sensor of the agent or data from a sensor located separately from the agent in the environment.

[0055] The agent 102 is also associated with a control system 106, which uses the observations generated by the sensors 104 to generate control signals for controlling the agent 102. Specifically, the control system 106 generates control signals that cause the agent 102 to follow a planned trajectory in the environment by first determining an appropriate action for the agent 102 to perform, e.g., as part of performing a specified task, such as navigating to a particular location, identifying a particular object, moving a particular object to a given location, manipulating a particular object in some manner, etc., and then generating control signals that cause the agent 102 to perform the action.

[0056] The control system 106 can be deployed on-board the intelligent agent 102 or can be deployed remotely from the intelligent agent 102 and can transmit control signals to the intelligent agent 102 via a data communication network.

[0057] The control signal can be a control input for controlling the agent. For example, when the agent is a robot, the control signal can be, for example, a torque of a joint of the robot or a higher level control command. As another example, when the agent is an autonomous or semi-autonomous land, air, or marine vehicle, the control signal can include actions to control navigation, such as steering and movement of the vehicle, such as braking and / or acceleration of the vehicle. For example, the control signal can be, for example, a torque to a control surface or other control element, such as a steering control element of a vehicle, or a higher level control command.

[0058] In other words, the control signal can include, for example, position, velocity, or force / torque / acceleration data of one or more joints of a robot or component of another mechanical agent.

[0059] In these examples, similar to control system 106 , system 100 can be deployed onboard agent 102 , or can be deployed remotely from agent 102 .

[0060] In these embodiments, the system 100 is used to provide an additional layer of control on top of the control system 106 , and the system 100 or another component can determine the query input 110 based on information received by the control system 106 .

[0061] That is, the query input 110 may be determined by receiving a control signal from the agent control system 106 and then determining, based on the control signal, one or more natural language queries relevant to, for example, the agents 102 in the environment.

[0062] In some implementations, the agent control system 106 is an autonomous or semi-autonomous control system, for example, which autonomously or semi-autonomously controls the navigation of the agent 102 .

[0063] In some other implementations, the agent control system 106 has an interface for receiving control commands, for example, from a human operator.

[0064] In both applications, the described system 100 can be used to provide an additional layer of control, such as for safety purposes. For example, the described system 100 can be used to inhibit control of the mechanical agent 102 in a manner that may be dangerous or contrary to one or more rules.

[0065] One or more rules related to the control of the agent may be explicitly entered, such as as part of the context input, or may be implicit, such as included as natural language statements when training neural networks used by the system 100, particularly when these neural networks each include a language model as described later. As an example, such rules may include rules related to permitted movement of a vehicle, such as traffic regulations. As another example, such rules may include rules related to decisions made to ensure safe behavior of the mechanical agent, such as to inhibit damage to the mechanical agent or a person.

[0066] Thus, each query input 110 may relate to an action to be performed by the agent, such as the action of the control system 106 under consideration.

[0067] For example, query input 110 can define an action to be performed by mechanical agent 102, e.g., in the form of a question such as “Do I turn left?” or “Is it safe for the agent to turn left?”.

[0068] As another example, the query input 110 can ask what action is to be performed by the mechanical agent, such as, "Which way should the agent turn?".

[0069] In the case of a robot, the query input 110 can relate to a subtask within a series of subtasks to be performed to perform a task, e.g., “What do I do next?” or “Do I pick up object X?” The subtasks themselves can include a series of primitive actions for moving parts of the robot to, for example, open a gripper.

[0070] Typically, query input 110 includes one or more natural language queries. More specifically, query input 110 can include a natural language description that defines the information to be provided in a response from system 100. That is, query input 110 explicitly or implicitly determines what is desired in a response.

[0071] Response 150 to query input 110 is used to control mechanical agent 102 in the real world environment. More specifically, response 150 is used to control the actions to be performed by mechanical agent 102.

[0072] As one example, response 150 may inhibit an action that would otherwise be performed, ie, the response may determine whether to perform an action defined by the query input.

[0073] As another example, response 150 may define an action to be performed, eg, where the query input implicitly or explicitly requests an action to be determined.

[0074] In these embodiments, obtaining context input 120 can include obtaining one or more observations of a real-world environment, and because the environment includes an agent, context input 120 potentially includes one or more observations of the agent. The observations can be obtained from one or more sensors, which can but need not be the agent's sensors 104. As described above, the observations can include still or moving images, as used herein, including LIDAR point clouds and / or other sensor data from one or more sensors that sense the state of the environment or the agent.

[0075] The one or more observations are processed, for example, by a first machine learning model to generate a natural language representation of the one or more observations, i.e., to generate natural language text describing the observations included in the context input 120.

[0076] There are many different types of machine learning models that can be used to achieve this. For example, so-called visual language models are typically configured to describe an image or video using natural language, for example, to perform an image or video captioning task. More generally, such models are able to perform many different types of image processing tasks by formulating the task as a text generation problem, for example, to detect or classify objects in an image or video. Correspondingly, other machine learning models can be trained to generate natural language text that describes data from other types of sensors, for example, to represent physical positioning or forces as natural language statements that describe, for example, an agent or environment or a portion thereof. Natural language representations of one or more observations are used to provide one or more natural language statements in the context input 120.

[0077] Additionally, the context input 120 can include one or more rules related to the current position of the agent 102 in the environment that are not generated directly from observations generated by the sensors 104 .

[0078] For example, the context input 120 can also have the following rules: common sense populated from the weights of the trained machine learning model, a manual of driving rules designed manually, or obtained from a knowledge graph or the Internet. Examples of such rules include "if a car's electrical systems get wet, they will short circuit", "if a car drives in the direction of something, it will end up there", "if the car's electrical systems are broken, the car is not safe to drive", etc. Other examples of such rules include "the speed limit in the current location is 30 mph", "right turns are allowed after stop at this red light", "turning across double yellow lines is prohibited", etc.

[0079] Thus, in these examples, system 100 can be used to evaluate the consequences of potential actions being considered by control system 106 before being transmitted as control signals to agent 102.

[0080] In some other embodiments, the environment is a real-world environment that includes a manufacturing plant, such as a manufacturing plant for manufacturing products such as chemical, biological or mechanical products or food. As used herein, "manufacturing" a product also includes refining starting materials to produce a product, or processing starting materials, such as removing contaminants, to produce a clean or recycled product. A manufacturing plant may include multiple manufacturing units, such as containers for chemical or biological substances, or machines for processing solids or other materials. The manufacturing units are configured so that intermediate versions or components of the product can be moved between manufacturing units during the manufacture of the product, such as via pipelines or mechanical transport. In an embodiment, the system is used to control one or more manufacturing units in a manufacturing unit or to control the movement of intermediate versions or components of a product between manufacturing units.

[0081] Thus, in these embodiments, obtaining context input 120 may then include obtaining one or more observations of a manufacturing unit or movement from one or more sensors. The sensors may include any type of sensor that monitors a manufacturing unit or movement, for example, a sensor configured to sense mechanical movement or force, pressure, temperature; an electrical condition such as current, voltage, frequency, impedance; an amount, level, flow / movement rate, or flow / movement path of one or more materials; a physical or chemical condition, such as a physical state, shape, or configuration or a chemical state such as pH; a configuration of a unit, such as a mechanical configuration of a unit or a valve configuration; an image or video sensor for capturing an image or video observation of a manufacturing unit or movement; or any other suitable type of sensor. In embodiments, one or more observations are processed to generate a natural language representation of the one or more observations, for example as previously described. The natural language representation of the one or more observations is used in one or more natural language statements of a natural language statement that provides context information.

[0082] The query input may involve an action to control the operation of one or more manufacturing units in the manufacturing unit or to control movement. The response to the query input is used to control the operation of one or more manufacturing units in the manufacturing unit or to control movement. For example, the response to the query input may be used to control - e.g., minimize - energy or other resource usage, or to control manufacturing to obtain a desired quality or characteristic of a product. For example, the action may include an action to control an item of equipment of a factory or an action to change a setting that affects the movement of a manufacturing unit or product or an intermediate or component thereof, e.g., to adjust or turn on / off an item of equipment or a manufacturing process.

[0083] In some embodiments, the manufacturing plant has a plant control system to control the manufacturing unit or control movement. The query input can be generated by receiving a control signal from the plant control system and generating one or more natural language queries for the query input from the control signal. In a manner similar to that previously described, the plant control system can be autonomous, semi-autonomous, or human-controlled.

[0084] In a manner similar to that previously described, the system can implement rules, for example, to control or limit energy or other resource allocation, or to ensure target quality or characteristics of a product, or to constrain the operation of a plant - such as a manufacturing unit - within safe boundaries.

[0085] In some embodiments, the environment is a real-world environment, and the method is used to diagnose faults in a mechanical system operating in the real-world environment. Obtaining contextual input may then include obtaining one or more observations of the mechanical system (which herein include observations of the operation of the mechanical system) from, for example, one or more sensors as described above. These are processed, for example as described above, to generate a natural language representation of one or more observations of one or more natural language sentences used to provide contextual information. In these embodiments, the query input relates to the operation of the mechanical system, and the response to the query input is used to identify a fault in the mechanical system. For example, the query input may include a general query, such as "Is the system working correctly?" or "What is wrong with the system?" or a specific query, such as "Is there a fault with component X?".

[0086] In some embodiments, the environment is a real-world environment of a service facility including multiple items of electronic equipment, such as a server farm or a data center, such as a telecommunications data center, or a computer data center for storing or processing data, or any service facility. The service facility may also include auxiliary control equipment that controls the operating environment of the items of equipment, such as environmental control equipment, such as temperature control, such as cooling equipment, or airflow control or air conditioning equipment. Then, obtaining contextual input can include obtaining observations of the state of the environment, which may include any electronic signals representing the operation of the facility or the equipment in the facility. For example, the representation of the state of the environment can be derived from observations made by any sensor sensing the state of the physical environment of the facility or by any sensor sensing the state of one or more items of equipment or one or more items of auxiliary control equipment. These sensors include sensors configured to sense the following: electrical conditions, such as current, voltage, power or energy; the temperature of the facility; fluid flow, temperature or pressure within the facility or within the cooling system of the facility; or physical facility configuration, such as whether the vent is open. These are processed—for example, as described above—to generate a natural language representation of one or more observations of one or more natural language sentences in the natural language sentences used to provide contextual information. The query input can relate to the operation of the facility, such as to adjust the operation of one or more electrical components to control (e.g., minimize) the use of a resource, such as a task to control the use of electricity or water. For example, the query input can ask which components to turn on to reduce the use of a resource, or whether it is safe to turn a given component on or off. The system can then determine how to operate the electrical component based on the generated response, such as by turning one or more components on or off as indicated by the response.

[0087] In some embodiments, the environment is a real-world environment of a power generation facility—e.g., a renewable power generation facility such as a solar farm or a wind farm—and the query input can relate to how to control power generated by the facility, for example, controlling the delivery of power to a distribution grid, for example, to meet demand or reduce the risk of mismatch between elements of the grid, or to maximize power generated by the facility.

[0088] Typically, observations of environmental conditions may include any electronic signals representing electrical or mechanical operation of power generation equipment in a power generation facility. For example, representations of environmental conditions may be derived from observations made by any sensor that senses the physical or electrical state of equipment in a power generation facility that is generating electricity, or the physical environment of such equipment, or the condition of auxiliary equipment that supports the power generation equipment. Such sensors may include sensors configured to sense: electrical conditions of the equipment, such as current, voltage, power, or energy; temperature or cooling of the physical environment; fluid flow; or the physical configuration of the equipment; and observations of electrical conditions of the power grid, such as from local or remote sensors. Observations of environmental conditions may also include one or more predictions about future operating conditions of the power generation equipment, such as predictions of future wind levels or solar irradiance or predictions of future electrical conditions of the power grid.

[0089] As another example, the environment can be an educational environment, for example, the system can be deployed as part of an educational software program that assists a user in learning or practicing one or more corresponding skills. In these examples, the context input 120 can include natural language statements that describe or reference a scene or scenario in a real-world or imagined environment, and the query input 110 can be a question about a scene or scenario that requires logical reasoning. As described below, the trajectory data generated by the system 100 as part of generating the response 150 can be used to provide the user with insight into the logical reasoning required to generate the response 150. That is, the trajectory data generated by the system 100 can be used to generate interpretations of questions and their corresponding responses to assist the user in learning one or more skills.

[0090] As a specific example, an educational software program can generate questions and accompanying contextual information (or receive this data from an external source). An example of such a question could be, for example, "Why does an astronaut need an oxygen backpack?" The system 100 (given a context of basic facts) can generate the following reasoning trajectory to assist the user in understanding the concept: "(1) Humans need oxygen to survive. There is no oxygen in space. Therefore, humans need a supply of oxygen to survive in space. (2) Oxygen can be supplied to astronauts through an oxygen backpack and humans need a supply of oxygen to survive in space. Therefore, humans can use an oxygen backpack to help astronauts survive in space."

[0091] As another example, the environment can be an information retrieval environment, e.g., the system can be deployed as part of a search engine or other software that allows a user to search for information in a document corpus, such as the Internet or another electronic document corpus. In these examples, the query input 110 can be any appropriate natural language query, and the context input 120 can include relevant sentences from the document corpus, i.e., as identified by searching the corpus using conventional information retrieval techniques. The system 100 can then use the context input 120 to generate a response 150 to the query input 110 in a manner that allows the user to view the trajectory of the logical reasoning required to generate the response 150, i.e., to increase the user's confidence in the authenticity of the response 150 generated by the system 100.

[0092] As another example, the environment can be a software testing or evaluation environment, e.g., the system can be deployed as part of a system that tests software before deployment or evaluates already deployed software to identify bugs. In these examples, when the system tests software before deployment, the query input 110 can be a natural language query about whether the software will execute as expected, and the context input 120 can include code snippets from the software code and optionally natural language statements describing the computer system on which the software will execute. The system 100 can then use the context input 120 to generate a response 150 to the query input 110 that indicates whether the code will execute as expected in a manner that allows the user to view the trace of logical reasoning required to generate the response 150, i.e., to increase the user's confidence in the authenticity of the response 150 generated by the system 100, e.g., to identify which snippets the system focused on when determining whether execution will proceed as expected. When the system monitors the execution of the code after deployment, the query input 110 can be a natural language query about whether the software program or a part of the software program has been executed as expected, and the context input 120 can include one or more of the following: code snippets from the software code, system logs, program logs, or other artifacts that should be left on the computer by running the program, or verification rules that express requirements for the execution of the software program, or natural language statements describing the computer system on which the software is executed. The system 100 can then use the context input 120 to generate a response 150 to the query input 110, which indicates whether the code has been executed as expected, optionally including a reasoning trace that allows the user to view the trace of logical reasoning required to generate the response 150, that is, to increase the user's confidence in the authenticity of the response 150 generated by the system 100, such as to identify which snippets the system pays attention to when determining whether the execution will proceed as expected. As a specific example, the software program can be part of the computer boot up, and the system can generate a response each time the computer is started to verify whether the computer will run correctly after startup.

[0093] As another example, the environment can be a computer security monitoring environment, for example, the system can be deployed as part of a system that monitors the security of one or more computers. For example, the environment can be a computer network security monitoring environment, and the system can be deployed as part of a system that monitors the security of one or more computers on a computer network such as a wireless network, a cellular network, a local area network, and / or the Internet. As another example, the environment can alternatively or additionally be a computer system security monitoring environment, and the system can be deployed as part of a system that monitors the system for the presence of unresolved software vulnerabilities such as computer viruses and / or zero-day exploits. Software vulnerabilities can be resolved by updating the software (e.g., patching) and / or removing (e.g., uninstalling) the software from the computer system. In these examples, the query input 110 can be a natural language query that queries whether a computer security incident has been resolved (e.g., "has the incident been resolved?"), and the context input 120 can include relevant statements from the system log, i.e., statements that are potentially related to the event being queried. Computer security events can be, for example, data leaks, unauthorized logins or other accesses to protected systems, detection of computer viruses, or detection of software vulnerabilities. An event can be "resolved" when the underlying event is no longer a threat to the security of the computer system, e.g., a computer virus has been removed, access to a protected system has been removed, a data breach has been mitigated, or software with a vulnerability has been updated or removed. The system 100 can then use the context input 120 to generate a response 150 to the query input 110 in a manner that allows the user to view the trace of logical reasoning required to generate the response 150, i.e., to increase the user's confidence in the authenticity of the response 150 generated by the system 100. In particular, the response 150 can be a natural language statement indicating whether the event has been resolved, optionally including the trace of reasoning.

[0094] Back to Figure 1A As described above, to generate response 150, system 100 uses selection neural network 130 and inference neural network 140.

[0095] More specifically, the system 100 generates a response by performing multiple update iterations.

[0096] At each update iteration, the system 100 performs a selection step by using the selection neural network 130 to select a proper subset (ie, less than all) of the natural language sentences in the context input 120 .

[0097] The system 100 then performs an inference step using the selected proper subset and the inference neural network 140 to generate a new natural language sentence representing a new fact, i.e., an "inferred" fact. The fact is called "inferred" because the fact does not exist in the context input 120, but is determined by the system 100 from the context input 120.

[0098] At each update iteration, the system 100 also determines whether that update iteration should be the last update iteration by determining whether a termination criterion is satisfied.

[0099] Specifically, the system 100 generates a first stop input based on the inference output of the update iteration and the query input, and processes the first stop input using the first stop neural network 170 to generate a first stop output indicating whether the inference output contains information necessary to respond to the query input.

[0100] The system 100 determines whether a termination criterion is satisfied based on the first stop output.

[0101] For example, when the stop output indicates that the inference output contains information necessary to respond to the query input, the system 100 can determine that the termination criteria are satisfied.

[0102] As another example, the system 100 can determine that the termination criteria is satisfied when the stop output indicates that the inference output does not contain information necessary to respond to the query input but a maximum number of update iterations have been performed.

[0103] At each update iteration except the last, the system 100 updates the context input 120 with the new fact, ie, by adding a natural language sentence representing the new fact to the context input 120 .

[0104] At the last update iteration, the system 100 uses the new facts to generate a response 150. For example, the system 100 can provide as the response 150 a natural language output derived from the new natural language sentence generated at the last update iteration.

[0105] As a specific example, the system 100 generates a second stop input from at least the inference output at the update iteration, and processes the second stop input using the second stop neural network 170 to generate a second stop output that identifies a natural language output derived from a natural language sentence in the inference output of the last update iteration.

[0106] In some implementations, the first stopping neural network 170 and the second stopping neural network 170 are identical, i.e., the system 100 uses the same stopping neural network 170 to determine whether to terminate and generates a final output by querying the neural network 170 with different inputs.

[0107] Thus, the system 100 iteratively adds new facts to the context input 120 , and then generates a response 150 to the query input 110 using the new facts generated at the last update iteration.

[0108] More specifically, the selection neural network 130 is a neural network configured to process a selection input including the context input 120 (as of the current update iteration) and the query input 110 to generate a selection output including one or more natural language sentences from the context input 120. Because the context input 120 is updated at each iteration, the selection output can identify different natural language sentences at different update iterations.

[0109] Inference neural network 140 is a neural network configured to process inference input including selection output to generate inference output including natural language sentence representing new facts of update iteration. Because the selection output can be different for different update iterations, the inference output can also be different across update iterations.

[0110] In some implementations, the inference input does not include the context input 120 or the query input 110. That is, the inference neural network 140 has access only to a proper subset of the context input 120 included in the selection output, and does not have access to the remaining natural language sentences in the context input 120 or the query input 110.

[0111] Thus, at each update iteration, the system 100 processes the selection input including the context input 120 (as of that update iteration) and the query input 110 using the selection neural network 130 to generate a selection output for that update iteration, which includes one or more natural language sentences from the natural language sentences of the context input 120. These natural language sentences can include sentences included in the original context input obtained by the system 100, sentences added to the context input at a previous update iteration, or both.

[0112] In some embodiments, the selection neural network 130 is configured to generate, i.e., regress, a text sequence including one or more natural language sentences from the natural language sentences in the context input 120. In some of these embodiments, the system 100 can employ constrained sampling to ensure that the tokens generated by the neural network 130 are (i) valid continuations of a natural language sentence in the context input 120, (ii) the beginning of another natural language sentence in the context input 130, or, alternatively, (iii) one or more predetermined tokens that represent, for example, separators between natural language sentences or the end of the regressed text sequence.

[0113] In some other implementations, neural network 130 can be configured to generate a text sequence that includes placeholder references to natural language sentences in context input 120, e.g., generating sentences such as “sent 1. We know that sent 4,” where “sent 1” is a reference to the first sentence in context input 120, and “sent 4” is a reference to the fourth sentence in context input 120. System 100 can then substitute the actual sentences to generate a selection output.

[0114] In some other embodiments, each selection step includes multiple internal steps. At each internal step, the selection neural network 130 is used to add the new sentence to the selection output of the selection step.

[0115] Then, the system 100 processes the inference input including the selected output of the update iteration using the inference neural network 140 to generate an inference output including a natural language sentence representing the new fact of the update iteration. As described above, if the update iteration is not the last update iteration, the system 100 updates the context input 120 to include the natural language sentence in the inference output of the update iteration.

[0116] In some implementations, during the generation of a response, the system 100 can maintain a "bundle" of multiple candidate reasoning traces. The system 100 can use the value neural network 180 to determine which candidates should be kept in the bundle and which candidates should be discarded after each (unfinalized) candidate in the bundle has been updated to generate one or more updated candidates. The system 100 can also use the value neural network 180 to determine which candidate(s) to use to generate a final response to the query input after all candidates in the bundle have been finalized or after different termination criteria have been met.

[0117] Because of the way the system 100 generates the response 150, the system 100 is also able to provide as output trajectory data 160 that provides an interpretable natural language summary of the causal reasoning performed by the system 100 to generate the response 150. In particular, because the system 100 alternates between the following two steps: 1) selection, which involves picking a subset of relevant information sufficient to perform a single inference step; and 2) inference, which sees only the limited information provided to it by the selection output and uses it to infer new intermediate evidence in the process of producing the final answer, the system ensures that the intermediate inference output (and optionally, the selection output) provides an interpretable reasoning trajectory to prove that the final answer is reasonable. In addition, the reasoning produced by the system 100 is causal because each step follows and depends on the previous step, and each inference is performed independently based only on the limited information provided by the selection output without direct access to the query input or the previous steps of the reasoning. That is, the reasoning trajectory includes the selection output and the inference output from each update iteration, and the trajectory data 160 can include the inference output and optionally the selection output from each update iteration.

[0118] In some embodiments, the system 100 provides the trajectory data 160 to the user, for example, along with the response 150. When multiple candidate trajectories are retained in the bundle, the system can designate the trajectory data of the trajectory used to generate the final response as the trajectory data 160. In some other embodiments, the system 100 stores the trajectory data 160 in association with the data identifying the response 150.

[0119] For example, trajectory data 160 can be accessed later by a user requesting to view the "reasoning" performed by system 100 that resulted in a particular control signal being transmitted to the mechanical agent.

[0120] As a simplified example, when context input 120 indicates that a pedestrian is near a road on which the agent is traveling and query input 110 is "what action to take" - e.g., "continue driving" or "stop driving" - the trajectory data can include select outputs such as "There is a person crossing the road" and "we know that if someone is crossing the road it is not safe to drive", and the inference can be "Therefore, it is not safe to drive".

[0121] As another simplified example, when context input 120 indicates that there is a lake near the vehicle, and query input 110 asks whether the vehicle should drive toward the lake, trajectory data 160 can include intermediate inferences such as: “therefore the car will end up in the lake” --> “lake means wet” --> “wet means short circuiting” --> “short circuiting means not safe”, and response 150 can be “we should not drive forward”.

[0122] The selection neural network 130 can generally have any suitable architecture that allows a neural network to be used to map the selection input to a proper subset of the context input 120. Similarly, the inference neural network 140 can generally have any suitable architecture that allows a neural network to be used to map the inference input to a new natural language sentence.

[0123] As a specific example, both the selection neural network 130 and the inference neural network 140 can be corresponding language model neural networks. In general, a language model neural network is a neural network that has been trained so that given a text prompt that includes a sequence of tokens of a natural language, the neural network can generate the next token in the sequence. This process can be repeated to expand the text prompt one token at a time to generate a natural language output, that is, to generate a natural language output autoregressively token by token. At each "time step", the language model neural network processes the current sequence to generate a probability distribution over a vocabulary of tokens. The probability distribution can then be used to select the next token, for example by sampling from the distribution using nucleus sampling or another sampling technique or by selecting the highest probability token. Tokens in the vocabulary can include any of a variety of tokens, such as words, subwords, characters, punctuation and other symbols, and some combination of numbers. In general, a language model neural network is trained on a text corpus consisting of tokens from the vocabulary (and optionally, other tokens that can be mapped to specified out-of-vocabulary tokens) to predict the next token in the token sequence from the training data.

[0124] Surprisingly, but widely accepted, large language model neural networks are able to perform tasks that they were not explicitly trained to perform. For example, they are able to perform translation tasks (assuming the training corpus includes words in different languages), arithmetic, and many other tasks.

[0125] A language model neural network can be caused to perform a specific task by providing a natural language description of a desired response as input or "prompt." In some cases, the prompt can be a few-shot prompt, where a few - e.g., 1 to 10 - examples of the query and sample outputs are provided in text prior to the actual query.

[0126] Alternatively or additionally, a language model neural network can be "fine-tuned" to perform a specific task by obtaining a pre-trained language model neural network trained on a large corpus of examples as previously described and then further training a portion of all of the language model neural networks on a relatively small number of examples specific to the type of task to be performed.

[0127] Thus, a trained language model neural network is able to perform control and diagnostic tasks of the type described. Where the system is to follow rules for generating responses, these may be included in the context information, e.g. in the prompts, and or in sentences included in a corpus as training data or in the data used to fine-tune the language model neural network.

[0128] In other words, in some embodiments, the selection neural network 130 and the inference neural network 140 are the same pre-trained language model neural network. In these embodiments, each selection input includes a type of few-trial prompt that causes the language model to generate a selection output, and each inference input includes a different type of few-trial prompt that causes the language model to generate an inference output.

[0129] For example, the few-try prompts for the inference neural network 140 can include one or more example inference input-inference output pairs arranged according to a predetermined syntax (followed by a subset of context inputs identified by corresponding selection outputs), while the few-try prompts for the selection neural network 130 can include one or more example selection input-selection output pairs arranged according to a predetermined syntax (followed by context inputs and query inputs arranged according to the predetermined syntax).

[0130] For example, the few-try prompt in each choice input can have the following form: # n-shot prompt # First example. <context 1> <query 1> # Example selection <fact>. We know that <fact>[ and <fact>]*. Therefore,( <fact>We know <fact>[and <fact>]*.therefore,) … # Problem to solve. <context> <query> In this example, the statements after # are not included in the prompt (or are optional) and are included in the example only for explanation. <context>Represents the natural language text in the context input, <query>Represents a query input, "..." indicates that the first example is followed by one or more additional examples of the same format, each <fact>is a natural language sentence copied from the corresponding context, and [ and <fact>]* implies that the system is able to select more than one fact per inference step, where the total number of facts in a given inference step is a hyperparameter.

[0131] As another example, the few-trial hint in each inference input can be of the following form: #n-shot inference prompt # First example. <fact>. We know that <fact>[ and <fact>]*. Therefore, <new fact>.( <fact>We know <fact>[and <fact>]*. Hence, <new fact>.) … # Problem to solve. <output of the selection step>Therefore, (< output of the Selectionstep>. Therefore,) In this example, each example in the prompt<new fact> From the sample <fact>the generated ("inferred") inferred facts, and <output of the selection step>is one or more facts in the selection output of the selection step, formatted in the same way as the examples in the few-try prompt.

[0132] The language model neural network can be a large language model neural network, e.g., one having greater than 1 billion, 10 billion, or 100 billion trained parameters. The language model neural network may have been trained on greater than 10 billion, 100 billion, or 1 trillion words or word-grams representing words or other textual tokens, e.g., subwords (also referred to as "word fragments").

[0133] In some other embodiments, both the selection neural network 130 and the inference neural network 140 are language model neural networks having the same architecture and pre-trained on the same large corpus, but the selection neural network 130 is fine-tuned on a first dataset of example selection input-selection output pairs and the inference neural network 140 is fine-tuned on a second dataset of example inference input-selection input pairs, or both. For example, fine-tuning the selection neural network 130 can allow the selection neural network 130 to more efficiently generate outputs that include placeholder references to facts from context inputs, as described above.

[0134] In some of these embodiments, each selection input and each inference output each includes a corresponding few-trial prompt, while in other of these embodiments, no few-trial prompt is included.

[0135] In some embodiments, the language model neural network is an autoregressive transformer neural network, wherein the transformer neural network is characterized by having a series of self-attention neural network layers. The self-attention neural network layer has an attention layer input for each element of the input and is configured to apply an attention mechanism on the attention layer input to generate an attention layer output for each element of the input; there are many different attention mechanisms that can be used. In some embodiments, the language model neural network can be an expert mixture model.

[0136] The stop neural network 170 and the value neural network 180 can also be language model neural networks, for example, having any of the forms described above.

[0137] That is, one or more stop neural networks 170 and value neural networks 180 can be configured to generate outputs of desired types by including corresponding prompts, such as few-trial prompts, in the inputs of the neural networks or by fine-tuning on a fine-tuning dataset, or both. Some example prompts are given later.

[0138] Figure 1C A simplified example of selection and inference steps performed when the system 100 is used to control a vehicle, such as an autonomous or semi-autonomous vehicle, is shown.

[0139] exist Figure 1C In the example of FIG. 1 , the query input 176 is “should the vehicle turn left?” and (as of FIG. 1 ) Figure 1C The example selection step shown in FIG. 1 is a context input 174 of “there is a lake to the left, lakes are full of water, water can damage vehicles.” To perform the selection step, the system 100 generates a selection input that includes a k-trial prompt 172 (in Figure 1C 17 shows a portion of a sample context for one of the examples in the k-test prompt 172, a sample query input for the example, and a sample selection output for the example), context input 174, and query input 176. The system 100 processes the selection input using the selection neural network 130 to generate a selection output 181 that states "lakes are full of water, and water can damage vehicles, therefore."

[0140] The system 100 then generates a k-trial prompt (in Figure 1C 182) and an inference input of a selection output 181. The system 100 processes the inference input using the inference neural network 140 to generate an inference output 186, which is a new inference fact, i.e., a new inference fact stating "going in the lake can damage the vehicle." The system 100 then adds 188 the new inference fact to the context input 174 for the next selection step.

[0141] Figure 1D An overview of the operation of the system at a given update iteration is shown.

[0142] like Figure 1D As shown, the system 100 receives a question and a context. The system also receives a set of natural language options, and the question indicates that the response should be to select one of the natural language options.

[0143] At each update iteration, the system processes the problem and context up to that update iteration using the selection neural network 130 to generate a selection output.

[0144] The system then processes the selection output using the inference neural network 140 to generate an inference output.

[0145] The system then processes the stopping input generated from the inference output using the stopping neural network 170 to determine whether the termination criteria are met.

[0146] If the termination criteria are not met, the system adds new facts from the inference output to the context.

[0147] If the termination criteria are met, the system processes the options and infers the output using the same stopping neural network 170 or a different stopping neural network 170 to generate an answer to the question.

[0148] This is referenced below Figure 2 Describe in more detail.

[0149] Figure 2 is a flow chart of an example process 200 for generating a response to a query input. For convenience, process 200 will be described as being performed by a system of one or more computers located in one or more locations. For example, a data generation system appropriately programmed according to the present specification, such as Figure 1A The selection inference system 100 depicted in , is capable of performing the process 200 .

[0150] The system obtains context input including context information (step 202). Typically, the context information includes one or more natural language sentences each representing a fact or rule related to the environment.

[0151] The system receives a query input including a query related to an environment (step 204).

[0152] The system then generates a response to the query input by performing update iterations until a termination criterion is met.

[0153] Specifically, at each update iteration, the system performs steps 206 to 214 .

[0154] The system processes the selection input including the context input and the query input using the selection neural network to generate the selection output of the update iteration (step 206). As described above, the selection output includes one or more natural language sentences from the natural language sentences of the context input.

[0155] The system processes the inference input including the selected output of the update iteration using the inference neural network to generate the inference output of the update iteration (step 208). As described above, the inference output includes a natural language sentence representing the new facts of the update iteration.

[0156] The system generates a first stop input based on the inference output of the update iteration and the query input (step 210). The first stop input will be described in more detail below.

[0157] The system processes the first stalling input using a first stalling neural network to generate a first stalling output indicating whether the inference output contains information necessary to respond to the query input (step 212).

[0158] The system determines whether termination criteria are met based on the first stop output (step 214 ).

[0159] For example, the system can determine that the termination criteria is satisfied when the stop output indicates that the inference output contains information necessary to respond to the query input.

[0160] As another example, even if the stop output indicates that the inferred output does not contain necessary information, the system can determine that the termination criterion is satisfied when a maximum number of update iterations have been performed. That is, when the stop output indicates that the inferred output does not contain information necessary to respond to the query input but a maximum number of update iterations have been performed, the system determines that the termination criterion is satisfied.

[0161] If the termination criteria are met, the system sets the current update iteration as the last update iteration, ie, does not perform any more update iterations, and provides a response to the query input.

[0162] For example, if the termination criteria are met because the inferred output does not contain the information necessary to respond to the query input but a maximum number of update iterations have been performed, the system can generate a natural language output indicating that the query has not been answered, such as "no answer available" or "response not found" or "unknown".

[0163] As another example, in response to determining that the stop output indicates that the inference output does contain information necessary to respond to the query input, the system provides, in response, a natural language output derived from a natural language statement in the inference output of the last update iteration.

[0164] Optionally, the system can also provide as part of the response a reasoning trace that includes the corresponding natural language statements in the inference output at each update iteration except the last update iteration. That is, the reasoning trace can include the corresponding natural language statements in the inference output at each update iteration except the last update iteration. The reasoning trace can also optionally include the selection output of the update iteration.

[0165] In some implementations, the system generates a natural language output derived from a natural language sentence in an inference output of a last update iteration by applying a predetermined rule to the natural language sentence in the inference output.

[0166] In some other implementations, the system generates a second stalling input from at least the inference output at the update iteration, and processes the second stalling input using a second stalling neural network to generate a second stalling output that identifies a natural language output derived from a natural language sentence in the inference output of the last update iteration.

[0167] For example, the first stop neural network and the second stop neural network can be the same neural network, such as the same language model neural network as described above, and the system can cause the neural network to generate the second stop output by including a specified prompt as part of the second stop input.

[0168] That is, when both the first stop neural network and the second stop neural network are the same language model neural network, the first stop input used to determine whether the inferred output contains information necessary to respond to the query includes a first stop text prompt, and the second stop input used to generate a natural language output includes a different second stop text prompt.

[0169] For example, the first stop text prompt can have the form: "Question: {question} Given {inference}. Do you know the answer?". In this example, "question" is the text of the query input, and {inference} is a natural language sentence in the inference output. The stop neural network can then output a first natural language output, such as "yes" or "definitely" or "I do", indicating that the answer is known (and therefore, the inference output does contain the information necessary to respond to the query input), or output a second natural language output, such as "no" or "not sure" or "I don't", indicating that the answer is unknown (and therefore, the inference output does not contain the information necessary to respond to the query input). As a specific example, when the score assigned to the first natural language output by the first stop neural network, such as the log likelihood, is higher than the score assigned to the second natural language output by the stop neural network, the system can determine that the inference output does contain the information necessary to respond to the query input. As another specific example, when a score assigned to a first natural language output, such as a log-likelihood, exceeds a predetermined threshold, the system can determine that the inferred output does contain the information necessary to respond to the query input.

[0170] Optionally, the stop prompt can also include one or more examples, ie, can be a k-trial prompt as described above.

[0171] The system can use any of a plurality of different forms of the second stop prompt.For example, the form that the second stop prompt uses can depend on the query received as input.

[0172] As a specific example, the query input can include a plurality of natural language options as possible responses to the query as well as a context-dependent query.

[0173] In this example, the second stop input can include a plurality of natural language options, and the second stop output can identify one of the plurality of natural language options as a natural language sentence.

[0174] As an example, the second stop input can have the form: "Given {inference}. Which of the following most closely matches: {choices}? Answer:" where {choices} identifies a plurality of options in the query input. The system can then select the natural language option to which the stop neural network assigns the highest score as the natural language statement to be included in the response.

[0175] As another specific example, the query can ask whether a particular statement is true or false. In this example, the second stop input can be of the form: "Given {inference}. Is {statement} true or false? Answer:" where {statement} is the statement from the query. The system can then select between "true" and "false" to stop the option to which the neural network assigns the highest score as the natural language statement to be included in the response.

[0176] Optionally, the stop prompt can also include one or more examples, ie, can be a k-trial prompt as described above.

[0177] If the termination criteria are not met, for example, if the stop output indicates that the inference output does not contain necessary information but a maximum number of update iterations have not been performed, the system updates the context input to include the natural language sentence in the inference output of the update iteration.

[0178] Figure 3 An example 300 of utilizing a halter neural network when a query input includes multiple options is shown.

[0179] like Figure 3 As shown, after the select inference step has been performed to generate an inference output, the system provides a first stop input that includes a first stop prompt, a question, and a natural language sentence from the inference output as input to a stopper neural network.

[0180] The system then selects "yes" or "no" as the output of the stopper neural network.

[0181] If the system selects "no" and the maximum number of update iterations has been performed, the system output response to the question is "Unknown".

[0182] If the system selects "yes", the system generates a second stop input to the stop neural network, the second stop input including a second stop prompt, the options, and the inferred output, and provides the output of the stop neural network as an answer to the question.

[0183] In some implementations, as described above, during generation of a response, the system can maintain a "bundle" of multiple candidate reasoning trajectories.

[0184] In these embodiments, the system can use a value neural network to determine which candidates should be kept in the bundle and which candidates should be discarded after each (unfinalized) candidate in the bundle has been updated to generate one or more updated candidates. Figure 2 Describing the stopping neural network, if the system has determined that the termination criteria for a candidate in the bundle is satisfied, then the candidate is considered to be finalized.

[0185] The system can also use a value neural network to determine which candidate to use to generate a final response to a query input after all candidates in a bundle have been finalized or after different termination criteria have been met.

[0186] Figure 4 4 is a flow chart of an example process 400 for updating a bundle of candidate responses to a query input. For convenience, process 400 will be described as being performed by a system of one or more computers located in one or more locations. For example, a data generation system appropriately programmed according to the present specification, such as Figure 1A The selection inference system 100 depicted in , is capable of performing the process 400 .

[0187] The system obtains context input (step 402). As described above, the context input includes context information, which in turn includes one or more natural language sentences, each of which represents a fact or rule related to the environment.

[0188] The system receives a query input including a query related to an environment (step 404).

[0189] The system generates a response to the query input conditioned on the context input by executing a sequence of one or more bundle steps (step 406).

[0190] In general, at any given one of the bundle steps, the system obtains a plurality of candidate inference trajectories generated by the selection inference system as part of generating a response to a query input conditioned on a context input.

[0191] The selection inference system can be any suitable system that generates an inference trajectory in response to a query and context input. For example, the selection inference system can be as described in reference Figures 1A to 3 Operate as described.

[0192] As described above, for each update iteration of the one or more update iterations, each candidate inference trace includes a corresponding selection output of the update iteration and a corresponding inference output of the update iteration.

[0193] That is, each of the candidate inference trajectories is generated by the selection inference system from a corresponding one of the candidate inference trajectories that was in the bundle at the end of the previous bundle step, i.e., by performing an update iteration on the corresponding candidate inference trajectory, or for the first bundle step, by performing an initial update iteration starting from the context input and the query input.

[0194] The system then maintains up to a predetermined number of candidate inference trajectories in the bundle for further consideration by the selection inference system.

[0195] Specifically, to determine which candidates to maintain, the system processes each candidate reasoning trajectory using a value neural network to generate a corresponding score for the candidate reasoning trajectory.

[0196] The score of a candidate inference trajectory estimates the likelihood that the candidate inference trajectory is a valid inference trajectory included in the true-valued inference trajectory that results in a true-valued response to the query input.

[0197] An inference trace is "valid" when it is logically valid, ie, each inference output in the inference trace can be derived from the corresponding selection output.

[0198] A ground-truth response to a query input is a response that correctly answers the query posed by the query input.

[0199] Therefore, a true-valued inference trajectory is a valid inference trajectory, where the final inference output contains the information necessary to respond to the query input, i.e., generate a true-valued response to the query input. In some cases, the system requires that a true-valued inference trajectory is an inference trajectory that includes a minimum number of inference steps from the set of all valid inference trajectories, where the final inference output contains the information necessary to respond to the query input.

[0200] As described above, the value neural network can be a language model neural network. In these examples, the system can determine a score for a given candidate reasoning trajectory by determining a score assigned to a predetermined natural language output, such as a log likelihood, that indicates that the candidate reasoning trajectory is valid by processing the candidate reasoning trajectory by the value neural network. For example, the predetermined natural language output can be "correct" or "valid" or another such natural language sentence.

[0201] In other words, as described above, the value neural network is configured to process the candidate inference trajectory to generate a value output of a probability distribution over a specified word-gram vocabulary, and the score of the candidate inference trajectory is based on the probability assigned to the predetermined word-gram from the vocabulary by the probability distribution. That is, when the predetermined natural language output is represented by a single word-gram, the score can be based only on the probability of the single word-gram. When the predetermined natural language output is represented by multiple word-grams, the score can be based on the probability of the first word-gram of the multiple word-grams, for example, so that the score is the logarithm of the probability of the first word-gram, or based on the probability of the first word-gram and the subsequent probability of the subsequent word-gram in the subsequent probability distribution generated by the value neural network, for example, the logarithm of the product of the first probability and the subsequent probability.

[0202] As a specific example, an input to a value neural network can include a k-trial prompt that causes the value neural network to generate one of the predetermined natural language outputs by processing the inference trace.

[0203] As another specific example, the value neural network can be a pre-trained language model that has been fine-tuned so that the value neural network accurately determines whether a candidate inference trajectory is "correct".

[0204] For example, the value neural network can have been trained on training examples, each training example including (i) a corresponding partial reasoning trace and (ii) a label indicating whether the partial reasoning trace is a valid reasoning trace included in the corresponding complete reasoning trace. In these examples, the positive label indicating that the partial reasoning trace is valid can be a predetermined natural language output.

[0205] The system can then rank the candidate reasoning trajectories based on the corresponding scores, and remove all but a predetermined number of the highest ranked candidate reasoning trajectories from the bundle according to the ranking.

[0206] Typically, the system continues to execute beam steps until it reaches the final beam step.

[0207] The last bundle step is the bundle step in which every candidate inference trajectory that was not removed from the bundle has been finalized, ie, every candidate inference trajectory that remains in the bundle after ranking and removal has been performed is a finalized candidate inference trajectory.

[0208] When a given beam step is not the last beam step in the sequence, that is, when the system determines that at least one of the candidate reasoning trajectories that have not been removed from the beam has not been finalized, the system causes the selection inference system to generate a corresponding selection output for a new update iteration after the last update iteration in the candidate reasoning trajectory and a corresponding inference output for the new update iteration for each candidate reasoning trajectory that has not been removed from the beam and has not been finalized, for example, as described above. The candidate reasoning trajectories can then be updated using the new selection and inference outputs to generate a candidate reasoning trajectory for the next beam step. In some cases, the system can cause the selection inference system to generate multiple updated candidate reasoning trajectories for each candidate reasoning trajectory that has not been removed from the beam and has not been finalized, for example by sampling from a probability distribution generated by a selection neural network and an inference neural network.

[0209] The system can also enable the selection inference system to perform the above reference Figure 2 Steps 210 to 214 are described to determine, for each candidate reasoning trajectory, whether the candidate reasoning trajectory has become finalized.

[0210] That is, more generally, the system can cause the selection inference system to perform one or more iterations of process 200 for each unfinalized candidate inference trajectory in the bundle to generate one or more updated candidate inference trajectories from each of the unfinalized candidates.

[0211] At the last beam step, the system generates a response based on one or more of the candidate reasoning trajectories in the beam (step 408). For example, the system can generate a response based on the highest ranked candidate reasoning trajectory according to the ranking generated at the last beam step. As a specific example, the system can use a second stopping neural network to generate a response from the last inference output of the highest ranked candidate reasoning trajectory, as described above with reference to Figure 2 and 3 described.

[0212] This specification uses the term "configured" in connection with system and computer program components. For a system of one or more computers to be configured to perform a particular operation or action, it means that the system has installed thereon software, firmware, hardware, or a combination thereof, which, in operation, causes the system to perform the operation or action. For one or more computer programs to be configured to perform a particular operation or action, it means that the one or more programs include instructions that, when executed by a data processing device, cause the device to perform the operation or action.

[0213] The embodiments of the subject matter and functional operations described in this specification can be implemented in digital electronic circuit systems, in tangibly embodied computer software or firmware, in computer hardware, including the structures disclosed in this specification and their structural equivalents, or in a combination of one or more of them. The embodiments of the subject matter described in this specification can be implemented as one or more computer programs, that is, one or more modules of computer program instructions encoded on a tangible non-transitory storage medium, for execution by a data processing device or for controlling the operation of a data processing device. The computer storage medium can be a machine-readable storage device, a machine-readable storage substrate, a random or serial access memory device, or a combination of one or more of them. Alternatively or additionally, the program instructions can be encoded on an artificially generated propagation signal, such as a machine-generated electrical signal, optical signal, or electromagnetic signal, which is generated to encode information for transmission to a suitable receiver device for execution by a data processing device.

[0214] The term "data processing apparatus" refers to data processing hardware and encompasses all kinds of apparatus, devices and machines for processing data, including, for example, a programmable processor, a computer or a plurality of processors or computers. The apparatus can also be or further include a dedicated logic circuit system, such as an FPGA (field programmable gate array) or an ASIC (application-specific integrated circuit). In addition to hardware, the apparatus can optionally include code that creates an execution environment for a computer program, for example, code constituting processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them.

[0215] A computer program, which may also be referred to or described as a program, software, software application, application, module, software module, script or code, can be written in any form of programming language, including compiled or interpreted languages, or declarative or procedural languages; and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine or other unit suitable for use in a computing environment. A program may, but need not, correspond to a file in a file system. A program can be stored in a portion of a file that holds other programs or data, such as one or more scripts stored in a markup language document, in a single file dedicated to the program in question, or in multiple coordinated files, such as in files that store one or more modules, subroutines or code portions. A computer program can be deployed to execute on one or more computers located at one site or distributed across multiple sites and interconnected by a data communications network.

[0216] In this specification, the term "database" is used broadly to refer to any collection of data: the data need not be structured in any particular way or at all, and it can be stored on a storage device in one or more locations. Thus, for example, an index database can include multiple data collections, each of which can be organized and accessed differently.

[0217] Similarly, in this specification, the term "engine" is used broadly to refer to a software-based system, subsystem, or process that is programmed to perform one or more specific functions. Typically, an engine will be implemented as one or more software modules or components installed on one or more computers in one or more locations. In some cases, one or more computers will be dedicated to a specific engine; in other cases, multiple engines can be installed and run on the same computer or multiple computers.

[0218] The processes and logic flows described in this specification can be performed by one or more programmable computers executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows can also be performed by a dedicated logic circuit system such as an FPGA or ASIC, or by a combination of a dedicated logic circuit system and one or more programmed computers.

[0219] Computers suitable for executing computer programs can be based on general or special microprocessors or both, or any other kind of central processing unit. Typically, the central processing unit will receive instructions and data from a read-only memory or a random access memory or both. The basic elements of a computer are a central processing unit for implementing or executing instructions and one or more memory devices for storing instructions and data. The central processing unit and the memory can be supplemented by a special logic circuit system or incorporated into the special logic circuit system. Typically, the computer will also include or be operably coupled to receive data from one or more large-capacity storage devices for storing data such as a disk, a magneto-optical disk or an optical disk or transfer data or both. However, the computer does not need to have such a device. In addition, the computer can be embedded in another device, such as a mobile phone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a global positioning system (GPS) receiver or a portable storage device, such as a universal serial bus (USB) flash drive, to name a few.

[0220] Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, including, for example, semiconductor memory devices, such as EPROM, EEPROM, and flash memory devices; magnetic disks, such as internal hard disks or removable disks; magneto-optical disks; and CD ROM and DVD-ROM disks.

[0221] To provide interaction with a user, embodiments of the subject matter described in this specification can be implemented on a computer having a display device for displaying information to the user, such as a CRT (cathode ray tube) or LCD (liquid crystal display) monitor, and a keyboard and pointing device, such as a mouse or trackball, through which the user can provide input to the computer. Other kinds of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback, such as visual feedback, auditory feedback, or tactile feedback; and input from the user can be received in any form, including acoustic, voice, or tactile input. In addition, the computer can interact with the user by sending documents to and receiving documents from a device used by the user; for example, by sending a web page to a web browser on a user's device in response to a request received from the web browser. In addition, the computer can interact with the user by sending a text message or other form of message to a personal device, such as a smart phone running a messaging application, and receiving a response message from the user in return.

[0222] The data processing apparatus for implementing the machine learning model can also include, for example, dedicated hardware accelerator units for processing common and computationally intensive portions of machine learning training or production (i.e., inference) workloads.

[0223] Ability to implement and deploy machine learning models using machine learning frameworks such as the TensorFlow framework.

[0224] Embodiments of the subject matter described in this specification can be implemented in a computing system that includes a back-end component, such as a data server, or includes a middleware component, such as an application server, or includes a front-end component, such as a client computer with a graphical user interface, a web browser, or an application through which a user can interact with an implementation of the subject matter described in this specification, or any combination of one or more such back-end, middleware, or front-end components. The components of the system can be interconnected by digital data communication of any form or medium, such as a communication network. Examples of communication networks include local area networks (LANs) and wide area networks (WANs), such as the Internet.

[0225] The computing system can include a client and a server. The client and the server are usually remote from each other and typically interact through a communication network. The relationship between the client and the server is generated by computer programs running on respective computers and having a client-server relationship with each other. In some embodiments, the server transmits data such as HTML pages to a user device, for example, for displaying data to a user interacting with the device acting as a client and receiving user input therefrom. Data generated at the user device, such as the result of a user interaction, can be received from the device at the server.

[0226] Although this specification contains many specific implementation details, these should not be interpreted as limitations on the scope of any invention or on the scope that can be claimed, but as descriptions of features that can be specific to a particular embodiment of a particular invention. Certain features described in the context of separate embodiments in this specification can also be implemented in combination in a single embodiment. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable sub-combination. In addition, although features can be described as working in certain combinations as above and even initially claimed as such, in some cases, one or more features from the claimed combination can be removed from the combination, and the claimed combination can be directed to a sub-combination or a variation of the sub-combination.

[0227] Similarly, although operations are depicted in a particular order in the drawings and recited in the claims, this should not be understood as requiring that such operations be performed in the particular order shown or in a sequential order, or that all of the operations shown be performed to achieve the desired results. In some cases, multitasking and parallel processing may be advantageous. In addition, the separation of various system modules and components in the above-described embodiments should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0228] Specific embodiments of the subject matter have been described. Other embodiments are within the scope of the following claims. For example, the actions recited in the claims can be performed in a different order and still achieve the desired results. As an example, the processes depicted in the accompanying drawings do not necessarily require the particular order shown or sequential order to achieve the desired results. In some cases, multitasking and parallel processing may be advantageous.< / output> < / fact> < / output> < / fact> < / fact> < / fact> < / fact> < / fact> < / fact> < / fact> < / fact> < / query> < / context> < / query> < / context> < / fact> < / fact> < / fact> < / fact> < / fact> < / fact>

Claims

1. A method executed by one or more computers, the method comprising: Obtaining context input including context information, the context information including one or more natural language sentences each representing a fact or rule related to an environment; receiving a query input comprising a query related to the environment; A response to the query input is generated by performing update iterations until a termination criterion is satisfied, the generating comprising at each update iteration: Processing a selection input including the context input and the query input using a selection neural network to generate a selection output for the update iteration, the selection output including one or more natural language sentences from natural language sentences of the context input; Processing an inference input including the selected output of the update iteration using an inference neural network to generate an inference output including a natural language sentence representing a new fact of the update iteration; generating a first stop input based on the inference output of the update iteration and the query input; processing the first stall input using a first stall neural network to generate a first stall output, the first stall output indicating whether the inference output contains information necessary to respond to the query input; as well as A determination is made whether the termination criteria is satisfied based on the first stop output.

2. The method of claim 1 , the generating further comprising, in response to determining that the termination criteria are not satisfied: The context input is updated to include the natural language sentence in the inference output of the update iteration.

3. The method according to claim 1 or claim 2, further comprising: The termination criterion is determined to be satisfied when the stop output indicates that the inference output contains the information necessary to respond to the query input.

4. The method according to any one of claims 1 to 3, further comprising: The termination criterion is determined to be satisfied when the stop output indicates that the inference output does not contain the information necessary to respond to the query input but a maximum number of update iterations have been performed.

5. The method according to claim 4, wherein the generating further comprises: In response to determining that the inference output does not contain the information necessary to respond to the query input but a maximum number of update iterations have been performed, providing as the response a natural language output indicating that the query has not been answered.

6. The method according to any preceding claim, further comprising: In response to determining that the stop output indicates that the inference output does contain the information necessary to respond to the query input, providing (i) a natural language output derived from the natural language sentence in the inference output of the last update iteration as the response.

7. The method according to claim 6, wherein: The response further includes (ii) an inference trace including natural language sentences in the inference output at each update iteration except the last update iteration.

8. The method according to claim 7, wherein: The inference trace further includes the selected output of the update iteration.

9. The method according to any one of claims 6 to 8, further comprising: generating a second stop input from at least the inferred output at the update iteration; as well as The second stalling input is processed using a second stalling neural network to generate a second stalling output, the second stalling output identifying the natural language output derived from the natural language sentence in the inference output of the last update iteration.

10. The method according to claim 9, wherein: The second stopping neural network is the same neural network as the first stopping neural network.

11. The method according to claim 10, wherein: The first stop neural network and the second stop neural network are language model neural networks, wherein the first stop input comprises a first stop text prompt, and wherein the second stop input comprises a different second stop text prompt.

12. The method according to any one of claims 9 to 11, wherein: The query input further includes a plurality of natural language options, wherein the second stop input further includes the plurality of natural language options, and wherein the second stop output identifies one of the plurality of natural language options as the natural language sentence.

13. A method according to any preceding claim, wherein: The inferred input does not include the query input or any natural language statements from the context input that are not included in the selected output of the update iteration.

14. A method performed by one or more computers, the method comprising: Obtaining context input including context information, the context information including one or more natural language sentences each representing a fact or rule related to an environment; receiving a query input comprising a query related to the environment; and Generating a response to the query input conditioned on the context input includes, at each bundle step in a sequence of one or more bundle steps: obtaining a plurality of candidate inference trajectories generated by a selection inference system as part of generating a response to the query input conditioned on the context input, each candidate inference trajectory comprising, for each update iteration of one or more update iterations, a corresponding selection output of the update iteration and a corresponding inference output of the update iteration; as well as Maintaining up to a predetermined number of candidate inference trajectories in a bundle for further consideration by the selection inference system includes: processing each inference trajectory using a value neural network to generate a corresponding score for each inference trajectory, the corresponding score estimating a likelihood that the candidate inference trajectory is a valid inference trajectory included in the true-valued inference trajectory that results in a true-valued response to the query input; ranking the candidate inference trajectories based on the corresponding scores; and Based on the ranking, all but the predetermined number of highest ranked candidate inference trajectories are removed from the bundle.

15. The method according to claim 14, wherein: Generating the response further comprises: At a final bundle step in the sequence, each candidate inference trajectory not removed from the bundle is determined to have been finalized, and in response, the response is generated based on the highest ranked candidate inference trajectory.

16. The method according to claim 14 or 15, wherein: The value neural network is configured to generate a value output that specifies a probability distribution over a vocabulary of word-grams, and wherein the score is based on a probability assigned by the probability distribution to a predetermined word-gram from the vocabulary.

17. The method according to claim 16, wherein: The value neural network is a language model neural network.

18. The method according to any one of claims 14 to 17, wherein: The value neural network has been trained on training examples, each training example including (i) a corresponding partial inference trajectory and (ii) a label indicating whether the partial inference trajectory is a valid inference trajectory included in the corresponding complete inference trajectory.

19. The method according to any one of claims 14 to 18, further comprising: At each bundle step other than the last bundle step in the sequence, it is determined that at least one of the candidate reasoning trajectories that have not been removed from the bundle has not been finalized, and in response, the selection inference system generates, for each candidate reasoning trajectory that has not been removed from the bundle and has not been finalized, a corresponding selection output of a new update iteration after the last update iteration in the candidate reasoning trajectory and a corresponding inference output of the new update iteration.

20. The method according to any one of claims 14 to 19, wherein: The selection inference system is a system configured to: Obtaining the context input; receiving the query input; receiving a current candidate reasoning trajectory, the current candidate reasoning trajectory comprising, for each update iteration of one or more update iterations, a corresponding selected output of the update iteration and a corresponding inferred output of the update iteration; as well as Performing a new update iteration to update the current candidate reasoning trajectory to add the new selection output and the new inference output to the candidate reasoning trajectory includes: Processing a selection input using a selection neural network to generate the new selection output of the new update iteration, the selection input comprising (i) the context input, (ii) one or more natural language sentences derived from the corresponding inference outputs in the candidate reasoning trace, and (iii) the query input, the new selection output comprising one or more natural language sentences from the context input, one or more natural language sentences derived from the corresponding inference outputs, or both; and An inference input including a selected output of the new update iteration is processed using an inference neural network to generate the new inference output, wherein the new inference output includes a natural language sentence representing a new fact of the new update iteration.

21. A method according to any preceding claim, wherein: The environment is a real-world environment, and wherein the method is used to control a mechanical agent acting in the real-world environment to perform a task; wherein Obtaining the context input comprises obtaining one or more observations of the real-world environment from one or more sensors, and processing the one or more observations to generate a natural language representation of the one or more observations; and wherein The query input relates to an action to be performed by the agent; The method further comprises: one or more of the natural language sentences using the natural language representations of the one or more observations to provide the context information; and The response to the query input is used to control the mechanical agent in the real-world environment.

22. The method according to claim 21, wherein: The mechanical agent has an agent control system to control the actions of the mechanical agent, wherein the query input includes one or more natural language queries, and wherein receiving the query input includes: receiving a control signal from the agent control system; and The one or more natural language queries are generated from the control signal.

23. The method according to claim 21 or 22, wherein: The mechanical agent comprises an autonomous or semi-autonomous vehicle that navigates the real-world environment, and wherein the actions comprise actions that control movement of the vehicle in the real-world environment.

24. A method according to any one of claims 21 to 23, wherein: The environment is a real-world environment, The context input is derived from at least observations characterizing a current state of the real-world environment, the observations being generated from measurements from one or more sensors configured to sense the real-world environment, The query input includes data characterizing a planned navigation of the agent, and The response to the query input represents an action to be performed by the agent in response to the observation.

25. The method according to claim 24, wherein: The agent is a robot or an autonomous vehicle.

26. The method according to claim 24 or 25, further comprising: Navigation of the agent is controlled based on the response to the query input.

27. The method according to any one of claims 1 to 20, wherein: The environment is a manufacturing plant for manufacturing a product, the manufacturing plant comprising a plurality of manufacturing cells, the plurality of manufacturing cells being configured to enable intermediate versions or components of the product to be moved between the manufacturing cells during the manufacture of the product, and wherein the method is used to control one or more of the manufacturing cells or to control the movement of the intermediate versions or components of the product between the manufacturing cells; wherein Obtaining the context input comprises obtaining one or more observations of the manufacturing unit or the movement from one or more sensors, and processing the one or more observations to generate a natural language representation of the one or more observations; and wherein The query input relates to controlling the operation of one or more of the manufacturing units or controlling the movement; The method further comprises: one or more of the natural language sentences using the natural language representations of the one or more observations to provide the context information; and The response to the query input is used to control operation of one or more of the manufacturing units or to control the movement.

28. The method according to claim 27, wherein: The manufacturing plant has a plant control system to control the manufacturing unit or to control the movement, wherein the query input comprises one or more natural language queries, and wherein receiving the query input comprises: receiving a control signal from the plant control system; and The one or more natural language queries are generated from the control signal.

29. The method according to any one of claims 1 to 20, wherein: The environment is a real-world environment, and wherein the method is used to diagnose faults in a mechanical system operating in the real-world environment; wherein Obtaining the context input comprises obtaining one or more observations of the mechanical system from one or more sensors, and processing the one or more observations to generate a natural language representation of the one or more observations; and wherein said query input relates to said operation of said mechanical system; The method further comprises: one or more of the natural language sentences using the natural language representations of the one or more observations to provide the context information; and The response to the query input is used to identify a fault in the mechanical system.

30. The method according to any one of claims 1 to 20, wherein: The environment is a computer security system, and wherein the method is used to determine whether a computer security incident has been resolved on a computer network, and wherein the context input includes data derived from a system log characterizing the computer security incident, data characterizing the computer network, or both.

31. The method according to any one of claims 1 to 20, wherein: The environment is a computer software evaluation system, wherein the method is used to determine whether a piece of software code will execute or has executed on a computer system as expected, and wherein the context input includes data characterizing one or more of the following: the piece of software code, the execution of the piece of software code, the computer system on which the code will execute or has executed, a product of the execution of the software code, or one or more verification rules for the execution of the piece of software code.

32. The method according to any one of claims 1 to 20, wherein: The environment is a real-world environment, and wherein the method is used to control one or more electrical components in a facility comprising a plurality of electrical components; wherein Obtaining the context input comprises obtaining one or more observations of the facility from one or more sensors, and processing the one or more observations to generate a natural language representation of the one or more observations; and wherein The query input relates to the operation of the facility; The method further comprises: one or more of the natural language sentences using the natural language representations of the one or more observations to provide the context information; and The response to the query input is used to determine how to control the one or more electrical components in the facility.

33. The method of claim 32, wherein: The one or more electrical components control heating and / or cooling of the facility.

34. A method according to any preceding claim, wherein: The query input includes a natural language description defining the information to be provided by the response.

35. One or more computer-readable storage media storing instructions that, when executed by one or more computers, cause the one or more computers to perform the operations of a corresponding method according to any preceding claim.

36. A system comprising one or more computers and one or more storage devices storing instructions which, when executed by the one or more computers, cause the one or more computers to perform the operations of the respective methods of any one of claims 1 to 34.

Images