Updating constraints for computerized assistant operations

Abstract

A method of adapting a computerized assistant program to satisfy an update constraint. The method includes maintaining a dialog history including a first utterance indicative of an initial constraint. The method also includes receiving a second utterance indicative of a new constraint that conflicts with the initial constraint. The method also includes identifying a revised function statement parameterized by a reference to an initial computerized assistant program configured to satisfy the initial constraint and a reference to the new constraint. The method also includes executing instructions derived from the revised function statement to return a revised computerized assistant program configured to satisfy the new constraint.

Description

Background Technology

[0001] Computerized assistants can be programmed to respond to user utterances with appropriate actions. For example, in response to a user telling the assistant to schedule a meeting, the assistant can add the meeting to the user's calendar. State-of-the-art computerized assistants employ one or more machine learning models to provide this assistance. Training these machine learning models can require large amounts of annotated training data. Annotating training data can be very time-consuming and technically challenging for human annotators. Summary of the Invention

[0002] This summary is provided to introduce a selection of concepts in a simplified form, which will be further described in the detailed description below. This summary is not intended to identify key or essential features of the claimed subject matter, nor is it intended to limit the scope of the claimed subject matter. Furthermore, the claimed subject matter is not limited to embodiments that address any or all of the shortcomings pointed out in any part of this disclosure.

[0003] A method for adjusting a computerized assistant program to satisfy updated constraints includes maintaining a dialogue history that includes a first utterance indicating the initial constraints. The method also includes receiving a second utterance indicating a new constraint that conflicts with the initial constraints. The method further includes identifying a revision function statement parameterized by references to an initial computerized assistant program configured to satisfy the initial constraints and references to the new constraints. The method also includes executing instructions derived from the revision function statement to return a revised computerized assistant program configured to satisfy the new constraints.

[0004] Therefore, methods, computer programs, computer systems, and computerized assistant systems as detailed in the following claims are provided. Attached Figure Description

[0005] Figure 1 The computational architecture of the computerized assistant is shown.

[0006] Figure 2 A method for updating computerized assistant actions based on new constraints is shown.

[0007] Figures 3A-3F Exemplary dialogue histories are shown, each including an initial computerized assistant and a revised computerized assistant based on the new constraints.

[0008] Figure 4 An exemplary computing system is shown. Detailed Implementation

[0009] Humans can interact with computer assistant systems via natural language. When interacting via natural language, humans can express an initial request that includes one or more constraints to be satisfied (e.g., the expected and / or required behavior of the computerized assistant). Humans can express modifications to the initial request by changing previous constraints and / or adding new constraints. For example, a user can make subsequent requests such as “change the meeting to the afternoon,” “show only one-on-one meetings,” or “schedule at least 2 hours between the two flights.” This disclosure relates to techniques for training computerized assistants to efficiently handle additional constraints by predicting appropriate revision function statements that specify the additional constraints and utilizing a specialized revision model to process the revision function statements.

[0010] Figure 1An exemplary computing architecture 120 of a computerized assistant 122 is illustrated. The computerized assistant 122 is configured to receive one or more user utterances 126 from a user 124 (e.g., via a microphone, keyboard, touchscreen, and / or any other suitable input device). The computerized assistant 122 is configured to generate and / or execute instructions to perform actions based on the user utterances 126 to assist the user 124. For example, the computerized assistant 122 may include a code generation machine 128 configured to translate the user utterances 126 into an initial computerized assistant program, including instructions in one or more computer-readable programming languages ​​executable by the computerized assistant. The initial computerized assistant program may be configured to satisfy initial constraints indicated by the user utterances 126. In some examples, the code generation machine 128 may include a previously trained machine learning model configured to generate the initial computerized assistant program. For example, the previously trained machine learning model may be trained to generate the program based on one or more labeled data tuples. As an example, the labeled data tuples may include exemplary utterances received from a human annotator and an exemplary computerized assistant program. The computerized assistant 122 may also include a dialogue history machine 134 configured to maintain the dialogue history 300. In this example, the dialogue history 300 may include multiple user utterances and / or computerized assistant programs configured to respond to user utterances. As a non-limiting example, the dialogue history 300 includes a first user utterance 302 and an initial computerized assistant program 304 configured to respond to the first user utterance 302. For example, the first user utterance 302 may indicate a constraint, and the initial computerized assistant program 304 may be configured to satisfy that constraint. Continuing with this non-limiting example, the dialogue history 300 may also include a second user utterance 306 indicating a new constraint. The dialogue history 300 may also include a revision function statement 308 indicating a revision to the initial computerized assistant program 304. For example, the revision function statement 308 may indicate how to revise the initial computerized assistant program 304 to generate a revised computerized assistant program 310 that will satisfy the new constraint indicated in the second user utterance.

[0011] The use of revision function statements can reduce the costs associated with training machine learning systems. For example, training a code generation machine 128 to generate suitable programs might require a large amount of training data in the form of exemplary dialogues that indicate exemplary user utterances and exemplary computerized assistant programs for responding to those user utterances. For example, the code generation machine 128 could be trained on dozens, hundreds, or thousands of exemplary dialogues. Typically, a human annotator might need to review multiple exemplary user utterances and, for each exemplary utterance, create a suitable computerized assistant program.

[0012] According to this technology, a human annotator can provide simple revision function statements as exemplary computerized assistant programs for responding to user utterances, rather than explicitly specifying the implementation details of the computerized assistant program. Thus, a code generation machine 128 can be trained to generate appropriate revision function statements when responding to user utterances. For example, the human annotator can be able to use revision function statements that only reference the appropriate program from an earlier part of the exemplary dialogue, along with new constraints from the user utterance. By utilizing a simple and invariant revision function statement format, the human annotator can be able to respond to multiple different exemplary user utterances using the same simple revision function statements parameterized with the initial program to be revised and the revised new constraints. The human annotator can quickly / easily provide annotations in response to new constraints in user utterances without having to provide the full details of the computer program to respond to the new constraints. Instead, revision function statements can be provided by the human annotator in a simple, invariant format. Therefore, the use of revision function statements can significantly reduce the costs associated with training machine learning systems (e.g., by reducing the costs associated with teaching human annotators how to create training data and / or compensating human annotators for providing annotations).

[0013] Figure 2 A method 200 is shown for adjusting a computerized assistant program to meet updated constraints. In 202, method 200 includes maintaining a dialogue history of interactions between the computerized assistant and one or more users, for example... Figure 1 The dialogue history 300. The dialogue history can be instantiated as any suitable computer-readable data structure to store data (e.g., utterances, actions, and / or contextual information) related to any interaction between the computerized assistant and one or more users. As used herein, utterances include any communication between the user and the computerized assistant, such as the user's audible speech, the text entered by the user, the user's keyboard commands, the computerized assistant's audible speech, and / or the computerized assistant's visual representation of human-readable text. The utterances stored in the dialogue history can specify one or more constraints. Constraints can include any contextual information instructing the computerized assistant on expected / required behavior, such as constraints on what action the computerized assistant should take and / or when it should take an action.

[0014] Speech can include any communication between a user and a computerized assistant, for example, via any suitable communication mode. In some examples, speech is spoken language communication. In some examples, speech can include other communication modes, such as nonverbal communication and / or input provided by the user via a computer device. For example, as used herein, speech can refer to sign language, nonverbal gestures (e.g., waving, nodding, changes in posture), button presses, keyboard input (e.g., speech entered via a text chat interface), and / or touchscreen input on a mobile device. For example, a computerized assistant can be configured to recognize one or more gestures associated with a specific user request (e.g., a user gesture of clapping to turn on a multimedia device). Alternatively or additionally, a computerized assistant can be configured to generally recognize user context indicated by gestures or computer input. For example, if a user nods or provides keyboard input, and the computerized assistant asks a confirmation question, such as “Should I buy a plane ticket for tomorrow?”, the computerized assistant can be configured to recognize user gestures / input indicating an affirmative answer to the question. As another example, if a user shrugs while coordinating details of a planned meeting, the computerized assistant can be configured to recognize the user's indecisiveness and automatically select the details without further user intervention. Thus, the user can interact with the computerized assistant through any combination of communication modes (e.g., specifying and / or modifying constraints). Although the examples here are described in terms of the user's verbal utterances, the techniques disclosed herein are applicable to handling new user constraints provided through any appropriate interaction between the user and the computerized assistant.

[0015] As an example, Figure 3A The diagram shows a dialogue history 300A including a first user utterance 302A, in which the user requests the computerized assistant to "create a meeting tomorrow from 10:00 AM to 10:30 AM". The user utterance relates to creating the meeting, and the constraints include a start time of 10:00 AM, an end time of 10:30 AM, and the date of tomorrow.

[0016] return Figure 2 In step 204, method 200 further includes identifying an initial computerized assistant program configured to satisfy initial constraints. For example, such as... Figure 3A As shown, based on the received first user utterance 302A, the computerized assistant can automatically generate an initial computerized assistant program 304A, which is configured to perform an action in response to the first user utterance (e.g., the action is configured to satisfy constraints in the first user utterance).

[0017] As a non-limiting example, the initial computerized assistant program 304A is shown in an exemplary programming language specific to computerized assistants. For example, Figure 3AAn initial computerized assistant program 304A in a dataflow programming language based on dataflow functions is shown (e.g., illustrated using bracketed function call syntax in the example). Alternatively or additionally, the methods described herein can be applied to any other suitable programming language (e.g., source code, bytecode, assembly code, functional, imperative, object-oriented, and / or any other programming language).

[0018] The initial computerized assistant program 304A is configured to store a variable "x0" (shown in square brackets) indicating tomorrow's date, and another variable "x1" indicating an executable function configured to create a new calendar event with a defined set of constraints. Reference Figure 1 The code generation machine 128 of the computerized assistant 122 is configured to generate an initial computerized assistant program. For example, the code generation machine 128 may include a machine learning model trained via supervised training on labeled data. As a non-limiting example, the labeled data may include exemplary utterances and exemplary programs configured to respond to the utterances. Thus, based on training, the code generation machine 128 can be configured to respond to a given utterance using an appropriate program configured to respond to the utterance. For example, the utterance may indicate constraints, and the code generation machine 128 may be configured to generate a program configured to satisfy the constraints. Although... Figure 3A Not shown, executing the computerized assistant program 304A can cause the computerized assistant to emit output to communicate with the user (e.g., text and / or voice output). For example, the computerized assistant can be configured to output a description of any running program and / or the results of the running program based on function calls to the program. As a non-limiting example, the computerized assistant can be configured to describe the initial computerized assistant program 304A based on a call to the "createEvent" function by outputting a description saying, "I am creating a new calendar event for 10:00 AM to 10:30 AM". The computerized assistant can generate descriptive text and / or voice based on the computerized assistant program in any suitable manner, such as by matching the program to a natural language template and recursively populating the fields of the template with data from the program and / or data generated by recursively calling the natural language template. Reference Figure 1 The computerized assistant 122 may include a description machine 130 configured to generate descriptions of any output from the computerized assistant program. For example, the description machine 130 may be configured to cause the computerized assistant 122 to output one or more utterances 136, for example, in the form of spoken audio and / or human-readable text. In some examples, the description machine 130 may be configured to save the generated descriptions to the dialogue history (e.g., the descriptions may be provided to a dialogue history machine 134 to be saved to the dialogue history 300).

[0019] Alternatively, or in addition to describing the procedure, a computerized assistant can be configured to perform computational actions by interacting with other computer devices, programs, and / or application programming interfaces (APIs) (e.g., the "createEvent" function can be configured to interact with a calendar application via an API to save new calendar events for the user). The computerized assistant can thus assist the user with various computational and / or real-world tasks (e.g., sending emails, making phone calls, and / or making purchases). In some examples, the conversation history further includes references to the initial computerized assistant. For example, the conversation history can track all computerized assistants that were generated and / or executed (e.g., to track previously executed actions and / or actions generated but not yet executed).

[0020] Computerized assistants can be configured to engage in multi-turn interactions with a user by responding to a first utterance and / or receiving and responding to subsequent utterances. In some examples, the user may utter a second utterance to modify and / or build upon the computerized assistant's response to the first utterance. Although the examples described here pertain to a first utterance and a second subsequent utterance that occurs later in a multi-turn dialogue, the second utterance can occur at any appropriate time. For example, the second utterance may occur in different dialogues involving the same and / or different users. As another example, the second utterance may actually occur before the first utterance; for example, when parsing the first utterance, the computerized assistant may be configured to consider the previously received second utterance to ensure that the first utterance is processed according to such an earlier second utterance. It should be noted that the same utterance can be used as both a "first" and a "second" utterance in the same or different interactions between the user and the computerized assistant. For example, a utterance may specify two different constraints that conflict within the same utterance, and these constraints can be resolved according to the methods of this disclosure. As another example, a utterance may specify a constraint for updating a previous action, resulting in an updated action. However, a subsequent utterance may specify a new constraint to update the (already updated) action. For example, if a user asks, "What are my schedules this afternoon?", the computerized assistant can respond by listing schedules that occur after 12 PM. If the user then asks, "What about after 10 AM?", the computerized assistant can update its previous action (e.g., updating the action of listing schedules in the afternoon) based on new constraints from the user's utterance (e.g., between 10 AM and 12 PM). If the user then further asks, "What about after 9 AM?", the computerized assistant can update its previous action (e.g., updating the action of listing schedules starting at 10 AM to instead list schedules between 9 AM and 10 AM). In other words, the computerized assistant can respond to new constraints in the user's utterance by updating any previous actions, including actions that have already been updated regarding constraints provided in the previous round.

[0021] Regardless of when the second utterance occurs, it can indicate a new constraint that may conflict with the initial constraint. For example, constraints may conflict when they have different breadths (e.g., one constraint is wider or narrower than another), incompatible details (e.g., constraints specify contradictory facts), or other differences that would lead to an expected revision of the initial computerized assistant program.

[0022] return Figure 2 In 206, method 200 includes receiving a second discourse indicating a new constraint that conflicts with the initial constraint. Figure 3AAs shown, in the second user utterance 306A, the user can request that the meeting be extended to 60 minutes (instead of the previously indicated 30-minute duration). In some examples, method 200 optionally further includes adding the second utterance to the dialogue history. Based on the utterances collected in the dialogue history and / or responses to said utterances, the computerized assistant can conduct multi-turn dialogues, where the computerized assistant responds to two or more user utterances over a period of time. Reference Figure 1 The computerized assistant 122 can be configured to respond to the user 124 in multiple rounds, for example, by manipulating the code generation machine 128 to generate new computerized assistant programs to respond to each incoming user utterance 126. In some examples, the dialogue history machine 134 can be configured to store second utterances indicating new constraints that conflict with the initial constraints in the dialogue history 300. Figure 3A The second user utterance 306A in the dialogue history 300A shown is the same.

[0023] return Figure 2 In method 200, at 208, method 200 further includes identifying revision function statements parameterized by references to the initial computerized assistant program and references to new constraints. Revision function statements are simple, human-readable statements that specify the initial computerized assistant program and new constraints (e.g., new constraints that will be used to revise the program). Therefore, revision function statements are concise and relatively easy to read / create by human annotators and / or relatively easy to generate by code generation machines compared to reading / creating / generating different extended code specific to each type of revision program.

[0024] Figure 3AA non-limiting example of revision function statement 308A is shown. In the example shown, revision function statement 308A is written in the example programming language of the computerized assistant, which uses the syntax "revise(InitialProgram, Constraint 1, ... Constraint N)," where "Initial Program" indicates a reference to the initial program to be revised. The revision function statement is further configured to parameterize using one or more constraints. The revision function statement specifies new constraints to be applied to the initial computerized assistant program, and can specify new constraints in general such that any computerized assistant program can be revised against the new constraints, where "Constraint 1" to "Constraint N" indicate any suitable number (e.g., up to N) of different constraints. The syntax shown is not limiting, as any other suitable syntax can be used. For example, revision function statements can include any suitable programming language syntax that allows references to any computerized assistant program (regardless of the specific characteristics of a particular computerized assistant program) and / or references to constraints (regardless of the specific characteristics of the constraints).

[0025] Revise function statements are configured to be parameterized using multiple different types of computerized assistants. For example, the same revision function statement, such as revise(Initial Program, Constraint 1, ... Constraint N), can be parameterized with different types of computerized assistants (e.g., by different choices of “Initial Program”), such as programs used to schedule meetings, buy airline tickets, order food, and / or any other computerized assistant.

[0026] The revised function statement 308A includes the variable "'xl", which indicates a reference to the executable portion of the initial computerized assistant program 304A. In the revised function statement 308A, the variable "xl" specifically indicates a call to the "createEvent" function starting on the line following the variable assignment "[xl] = ...". Although Figure 3AAn explicit reference variable named "xl" is shown. The reference to the initial computerized assistant program can be any suitable reference, such as a filename / line of code, a function name, a reference to a specific previously generated program in the computerized assistant's dialogue history, and / or a search history function configured to select a previously generated program in the computerized dialogue history (e.g., based on salient features such as "meeting" or "schedule"). As another example, different variables referencing different parts of the initial program and / or different initial programs can be used. In this example, the revised function statement 308A is further parameterized by a new constraint indicating that the event duration of the meeting should be 60 minutes.

[0027] return Figure 2 In step 210, method 200 further includes executing instructions derived from the revision function statement to return a revised computerized assistant program configured to satisfy the new constraints. In other words, for a given computerized assistant program, the instructions are configured to return a corresponding revised computerized assistant program based on the new constraints.

[0028] In some examples, revision model machine 132 may be configured to store and / or generate instructions executable by a computerized assistant. Revision model machine 132 is configured to process revision function statements concerning any new constraints specified as parameters in any initial computerized assistant program and / or as revision function statements. Therefore, instructions from revision model machine 132 are configured to generate a revised computerized assistant program based on the revision function statements and the initial computerized assistant program upon execution.

[0029] In some examples, the revision model machine 132 may utilize one or more revision models to generate a revised computerized assistant program. For example, the revision model may include executable code, parameters, and / or data configured to return the revised program. As a non-limiting example, the revision model may include a previously trained machine learning model (e.g., having executable code for evaluating and / or training the model, parameters generated during model training, and / or data that can be used as examples for training and / or evaluation). As another non-limiting example, the model may include a rule-based model (e.g., having executable code for evaluating one or more predefined rules based on parameters / data that define such rules).

[0030] In some examples, the revised model is a previously trained machine learning model trained on human-annotated training data, including revised function statements parameterized with an exemplary initial computerized assistant program and constraints labeled with a corresponding exemplary revised computerized assistant program. For example, each training data example may include utilizing a corresponding exemplary revised computerized assistant program received from a human annotator (e.g., Figure 3AThe exemplary revised function statements marked with the revised Computerized Assistant Program 310A (e.g., as a non-limiting example) Figure 3A (Revision function statement 308A). In some examples, the revision model can be configured to return, for a given revision function statement, an exact corresponding revised computerized assistant received from the human annotator along with that revision function statement. Furthermore, the revision model can be configured to evaluate any revision function statement and return a suitable revised computerized assistant (e.g., even if neither the revision function statement nor the revised computerized assistant is specifically included in the training data).

[0031] In some examples, the revision model includes multiple predefined rules configured to transform an initial computerized assistant program based on new constraints. In some examples, the multiple predefined rules may include rule-based models. For example, the rule-based model may be a domain-specific rule-based model, including multiple predefined rules for that domain (e.g., predefined rules for revising scheduling procedures). In some examples, the revision model includes a combination of one or more predefined rule-based models and / or one or more previously trained machine learning models. For example, revision model machine 132 may provide one or more domain-specific revision models, each configured to revise a computerized assistant program for that domain (e.g., separate domain-specific models for each of meetings, airlines, and / or food). Therefore, revision model machine 132 can be extended to new domains by adding new domain-specific revision models.

[0032] In some examples, the revision model machine 132 is configured to determine the appropriate revised computerized assistant based on the revision function statement 308A and optionally further based on any suitable contextual information (e.g., based on user preferences, user schedules, and / or previous interactions with the user, such as the user's previous utterances and / or previous computerized assistants and / or responses stored in the dialogue history 300). As an example, such contextual information may be provided to the revision model of the revision model machine 132 (e.g., a machine learning model previously trained on an example annotation indicating the contextual information, and / or a rule-based model with one or more rules for revising the procedure based on the contextual information).

[0033] The revision model machine 132 is configured to process a given revision function statement based on the statement's parameters, such as based on specific details of the initial computerized assistant program and / or based on specific details of the new constraint. For example, the revision model machine 132 may be configured to identify and utilize appropriate models for a given type of initial program and / or be configured to apply revisions to other logical structures (e.g., executable code) of that type of initial program. For example, the revision model machine 132 may be configured to determine which model(s) to use to process the revision function statement based on the initial computerized assistant program as a domain-specific program (e.g., processing a scheduling program using a domain-specific model of scheduling). In some examples, the revision model machine 132 may provide a domain-independent revision model configured to revise a computerized assistant program relating to any domain (e.g., meetings, airlines, and / or food). For example, a domain-independent revision model may be configured to handle a specific type of constraint (e.g., a domain-independent revision model for scheduling that can be applied to any scheduling-related scenario, such as meetings, airline tickets, and / or restaurant reservations). In some examples, the domain-independent revision model may be extended to handle new types of programs. In some examples, revision model machine 132 can provide a domain-specific revision model configured to revise a computerized assistant program concerning a specific domain (e.g., a meeting). Revision model machine 132 can include a collection of any suitable domain-specific and / or domain-independent models. Therefore, revision model machine 132 can automatically determine which domain-specific and / or domain-independent model(s) to use to process a particular revision function statement (e.g., based on the domain associated with the program and / or constraints).

[0034] The revision model machine 132 can be extended to handle different computerized assistant programs. The revision model machine 132 can be configured to provide revisions for any suitable initial type of computerized assistant program (e.g., a program for scheduling meetings, purchasing airline tickets, and / or ordering food), and / or extended to provide revisions for new types of computerized assistant programs. As an example, when the revision model machine is configured to utilize a machine learning model, the machine learning model can be extended by retraining it with training data that includes the new type of program. As another example, when the revision model machine 132 uses a rule-based model, the rule-based model can be extended by adding additional rules.

[0035] Even when the revision model machine 132 is extended to handle new programs, the format of the revision function statements remains unchanged. This unchanged format simplifies the process of collecting annotated training data from human annotators, thereby reducing the cost of training the machine learning system. For example, due to the consistency of the revision function statement format, human annotators can more easily create high-quality annotations. Furthermore, the revision function statements can remain simple (e.g., specifying the program and constraints without further details) while still being usable for revising new programs when the revision model machine 132 is extended. Moreover, the simplicity and invariance of the revision function statements can reduce the amount of training data required to train a sufficiently powerful machine learning system (e.g., training the revision model machine 132 to generate revisions and / or training the code generation machine 128 to generate revision function statements). Therefore, the invariant and simple revision function statements not only further reduce the cost of collecting annotations but also reduce the computational cost, memory storage cost, and latency associated with training the machine learning system. Furthermore, the relative simplicity of the revision function statements can reduce the computational cost associated with operating the trained machine learning system (e.g., reducing the latency and / or power cost of operating the revision model machine 132 and / or the code generation machine 128).

[0036] In other words, revision function statements can always follow a simple format, and revision model machine 132 can be programmed to handle new types of computerized assistant programs, thereby shielding human annotators from the underlying complexity. Human annotators and / or machine learning-trained components (e.g., code generation machine 128) can thus learn how to use revision function statements more easily, and revision function statements can be processed by revision model machine 132 to produce programs with effective revisions.

[0037] return Figure 2 In step 210, method 200 further includes executing instructions derived from the revision function statement to return a revised computerized assistant program configured to satisfy the new constraints. In other words, the revised computerized assistant program is generated based on the revision function statement and configured to satisfy the new constraints. The revision function statement can be parameterized with multiple different computerized assistant programs and configured to return a corresponding revised computerized assistant program based on the new constraints for a given computerized assistant program.

[0038] For example, such as Figure 3AAs shown, the revised computerized assistant program 310A is similar to the initial computerized assistant program 304A, but is modified to indicate a meeting that starts at 10:00 AM and lasts for 60 minutes (e.g., the new meeting will end at 11:00 AM instead of 10:30 AM). The revised computerized assistant program 310A is a non-limiting example of a revised program configured to satisfy new constraints based on the revision function statement 308A. For example, the revision function statement 308A indicates that the meeting will take one hour, but does not specify that the meeting must start at 10:00 AM. In another non-limiting example, the revision model machine 132 is configured to output a revised computerized assistant program that schedules a meeting from 9:30 AM to 10:30 AM. For example, if a user has another appointment scheduled for 10:30 AM, it can be completed before the other appointment by starting the one-hour meeting at 9:30 AM instead of 10:00 AM. In other words, more than one suitable revised computerized assistant program can exist based on the revision function statement. Therefore, the revision function statement 308A can be processed by the revision model machine 132 to generate a revised computerized assistant program 310A. For example, the revision model machine 132 can be configured to use one or more revision models to determine a suitable revised computerized assistant program. The revision model machine 132 can be configured to evaluate the revision model with respect to the initial computerized assistant program, new constraints, and / or based on any suitable contextual information (e.g., based on user preferences, user schedules, and / or previous interactions with the user). The revision model is configured to derive suitable instructions from the revision function statement, which are configured to generate the revised computerized assistant program based on the revision function statement and the initial computerized assistant program.

[0039] For example, refer to Figure 3A The revised function statement 308A can be modified by the revised model machine 132 (e.g., Figure 1 (As shown) the revised computerized assistant program 310A is translated using any suitable revision model. In the example, the revision model machine 132 can be configured to determine the revised computerized assistant program using any suitable programming technique. In some examples, the revision model includes multiple predefined rules configured to translate the computerized assistant program based on new constraints. Alternatively or additionally, in some examples, the revision model is a previously trained machine learning model trained based on exemplary revision function statements and exemplary revised computerized assistant programs received from human annotators. Therefore, the revision model can be any suitable combination of a previously trained machine learning model configured to generate the computerized assistant program, one or more predefined rules for revising the computerized assistant program, and / or a mixture of machine learning models and predefined rules.

[0040] return Figure 2 The revised computerized assistant can be used in any suitable manner, for example, to continue a multi-turn dialogue by responding to a second utterance from the user indicating new constraints. As a non-limiting example, in 212, method 200 optionally further includes saving a reference to the revised computerized assistant in the dialogue history. Therefore, the revised computerized assistant can be executed at any suitable time. As another non-limiting example, in 214, method 200 optionally further includes executing the revised computerized assistant (e.g., causing the computerized assistant to perform an action to assist the user). When the revised computerized assistant 310A is executed, the computerized assistant can be configured to issue appropriate descriptions, such as indicating that a previous action is being modified according to new constraints and / or indicating any new action being performed. For example, although not in Figure 3A As shown, however, the computerized assistant can be configured to issue text and / or voice descriptions, such as, “Okay, I will reschedule your meeting for 10 a.m. to 11 a.m.”

[0041] In some examples, executing a revised computerized assistant includes undoing one or more actions performed by the initial computerized assistant. For example, the computerized assistant can be configured to undo the initial computerized assistant before starting execution of the revised computerized assistant. As another example, the computerized assistant can be configured to identify when a new action in the new computerized assistant might conflict with a previous action performed in the initial computerized assistant, so as to undo such a previous action before executing the new action. In some examples, each action of the initial computerized assistant can be configured to allow the "undoing" of that action. In some examples, the "undoing" of an action may require specific steps to first consider the impact of the completed action. For example, if the action results in scheduling a new calendar event, the undoing action could include deleting the calendar event. As another example, if the action involves making an appointment / reservation with another entity (e.g., a restaurant reservation or meeting invitation), the undoing action could include appropriate steps to cancel the original appointment / reservation (e.g., sending a cancellation message). Typically, the computerized assistant can be configured to appropriately prompt the user before performing and / or undoing any action with real-world effects, such as prompting the user before making or canceling an appointment.

[0042] The revised model machine 132 is typically configured to solve the new constraints and initial constraints to find solutions that satisfy both the new and initial constraints. In some examples, the revised model machine 132 is configured to transform the new constraints, along with the initial constraints of the initial computerized assistant, into a constraint satisfaction problem (CSP) so that a new computerized assistant can be derived based on the solution to the CSP. In some examples, the CSP may include further constraints independent of the new constraints and / or user utterances (e.g., domain-independent and / or domain-specific constraints). For example, a CSP may include a domain-independent constraint indicating that "an event must begin before it ends," or a domain-specific constraint indicating that "air travel between connecting flights less than 20 minutes is not feasible."

[0043] For example, the revised model machine 132 can be configured to evaluate the cost function of the CSP solution in order to find the minimum-cost solution of the CSP. As a non-restrictive example, the CSP can be encoded as a constraint graph, which includes multiple nodes indicating constraint variables and multiple edges indicating implicit relationships between constraint variables. For example, the graph structure indicated by edges connecting nodes can be used to propagate relationships between constraints based on logically implied associativity and / or transitivity. As an example, solving a CSP can include finding a set of constraints (e.g., a subgraph of the constraint graph) that includes new constraints such that the constraints in that set are mutually satisfyable. In some examples, the individual nodes and / or edges of the constraint graph can be referred to as sub-constraints; for example, the subgraph can indicate sub-constraints of the initial constraints or sub-constraints of the new constraints.

[0044] In some examples, solving the constraint satisfaction problem may include finding multiple candidate constraint solutions for the constraint satisfaction problem, and selecting candidate constraint solutions based on a cost function. For example, the cost function may be based on a previously trained machine learning model and / or on multiple predefined rules. For example, each rule may indicate how to evaluate the cost of a sub-constraint and / or indicate a mathematical function used to aggregate the costs of multiple sub-constraints. In some examples, each candidate constraint solution for the constraint satisfaction problem is determined by a revised model machine (e.g., Figure 1 One of the models (revision model machine 132) is generated. For example, revision model machine 132 can be configured to operate multiple domain-independent and / or domain-specific models (e.g., rule-based models and / or machine learning models) such that each model returns a candidate constraint solution, such that multiple models generate multiple candidate constraint solutions for the constraint satisfaction problem.

[0045] In some examples, the solution to the CSP is a relaxation of the constraints of the initial computerized assistant program (e.g., a minimum-cost relaxation). Therefore, solving the CSP may include finding a logical relaxation of the initial constraints. For example, the new constraints may conflict with the initial constraints (e.g., in the sense that a CSP containing both the new and initial constraints does not produce a solution). Alternatively or additionally, the new constraints may have a different range than the initial constraints (e.g., because the new constraints are wider than the initial constraints, and / or because the new constraints are incompatible with certain sub-constraints of the initial constraints). To handle such differences in constraint range, the initial constraint problem can be relaxed until the new constraints are no longer trivial (e.g., the new constraints feasiblely shrink the solution space of the CSP), and / or until the new constraints no longer conflict.

[0046] For example, constraint relaxation may include finding a set of relaxed constraints that have fewer constraints compared to the initial constraints. This set of relaxed constraints may have an estimated cost based on any suitable cost function (e.g., a machine learning function and / or hard-coded cost rules). In some examples, relaxed constraints are configured to satisfy one or more solution conditions associated with the constraints. For example, a solution condition may be that user constraints are not implied by relaxed constraints (e.g., thus ensuring that new user constraints introduce new information that will lead to an appropriately updated computerized assistant program). As another example, a solution condition may be that the combination of new user constraints and relaxed constraints is satisfiable (e.g., thus ensuring that the updated computerized assistant program will satisfy the new user constraints while also satisfying as many of the initial constraints as possible). The revised model machine 132 may be further configured to translate the set of relaxed constraints into a new program. For example, the revised model machine 132 may be configured to generate a corresponding computerized assistant program instruction for each constraint in the relaxed constraint set, which ensures that a specific constraint is satisfied when the new computerized assistant program is executed.

[0047] In some examples, the logical relaxation of the initial constraints is determined based on identifying a conflict between the second utterance and the initial constraints. As an example, see [reference needed]. Figure 3AThe revised model can be configured to extract constraints from the initial program, such as "start = 10 AM" and "end = 10:30 AM". The revision function statement 308A specifies: "RevisionModel([OldProgram],Constraint{duration:?=(60.toMinutes())})", for example, indicating a new constraint "duration = 60 minutes". Therefore, the revised model can be further configured to find a relaxation of the initial constraint in conjunction with the new constraint ("duration = 60 minutes"), for example, by dropping the existing constraint "end = 10:30 AM" based on a conflict between the 60-minute meeting (specified by a second user utterance) and the initial constraint (the meeting starts at 10:00 AM and ends at 10:30 AM, lasting only 30 minutes). This relaxation ensures that the "duration" no longer comes from the "start" and "end" constraints (e.g., because the duration is now explicitly specified by the user in the new constraint). Furthermore, the relaxation ensures that the new combination of constraints ("start = 10AM" and "duration = 60 minutes") is satisfyable. Therefore, the revised model is configured to generate a revised computerized assistant program 310A, which can be executed to create new calendar events based on the new constraints.

[0048] The above description and Figure 3AThe results shown are non-limiting examples of suitable results that can be obtained by using a revision model for revising function statement 308A. Alternative revised computerized assistant programs with different functionalities can be obtained based on different training and / or hard-coded rules. For example, a revised computerized assistant program 310A can be derived based on rules and / or a cost function that indicate users generally prefer to retain the initial meeting start time. However, instead of a revision program for scheduling a meeting that starts at 10:00 AM and lasts for 60 minutes, using different rules that indicate users generally prefer to retain the meeting end time (e.g., based on scheduling follow-up activities after the meeting), the revision model can be configured to produce an alternative revised program for scheduling a meeting that starts at 9:30 AM and lasts for 60 minutes (e.g., to retain the original meeting end time while allowing the meeting start time to be revised). In some examples, the computerized assistant can be configured to operate the revision model to generate multiple alternative solutions, thereby selecting one of the alternative solutions based on user selection (e.g., by prompting the user to select a start time of 9:30 a.m. or 10:00 a.m.) and / or based on a separate rating function (e.g., a rating function that checks the user's calendar to assess conflicts that may already be scheduled between 9:30 a.m. and 10:00 a.m. with meetings scheduled between 9:30 a.m. and 10:30 a.m. and meetings that may already be scheduled between 10:30 a.m. and 11:00 a.m. with meetings scheduled between 10:00 a.m. and 11:00 a.m.).

[0049] Constraints can be evaluated for relaxation costs based on any suitable cost function (e.g., machine learning functions and / or hard-coded rules). Therefore, a revision model can be configured to revise arbitrarily complex computerized assistants based on appropriate constraint semantics and / or relaxation costs. As an example, relaxation costs could include rule-based functions such as "relaxation end time is cheaper than relaxation start time," so the model would prioritize relaxing "end" over relaxing "start." Alternatively or additionally, costs could be based on a machine learning model trained to identify the costs of relaxing different constraints in different contexts. For example, a machine learning model can be trained using supervised training on labeled data tuples. For example, the tuples could indicate the initial computerized assistant, new constraints from the user, and an appropriate revised computerized assistant configured to satisfy the new constraints. Thus, in some examples, the revision model is a previously trained revision model, and the methods of this disclosure include retraining a previously trained revision model program based on labeled data tuples including exemplary revision function statements and exemplary revised computerized assistants received from human annotators.

[0050] Figure 3AAn example is shown where the initial constraints are extended / abandoned to address new constraints (e.g., the meeting end time is relaxed to allow for longer meetings). Besides... Figure 3A In addition to the meeting arrangement example shown, the methods disclosed herein are also applicable to handling revised constraints arising from any other interactions between a user and a computerized assistant.

[0051] Figure 3B-3F The illustration shows various other scenarios in which the revision model machine 132 uses one or more revision models (e.g., machine learning-based and / or rule-based) to update the initial computerized assistant program based on new constraints. As a non-limiting example, the revision model can be configured to allow adding new constraints, narrowing / specifying existing constraints from the initial constraint set, expanding existing constraints, deleting existing constraints, and / or changing existing constraints into different constraints. By changing, adding, and / or deleting constraints, the revision model can determine a revised computerized assistant program that appropriately incorporates any new constraints specified by the user. Figure 3B-3F All unrestricted examples use the same revision function statement syntax, demonstrating how a revision model machine can be configured to revise any given program using the same revision function statement syntax. As mentioned above, this allows human annotators to provide annotations based on a simple, invariant revision function statement syntax to train the code generator to generate appropriate revision function statements for any unrestricted example and / or in any other suitable scenario. Furthermore, the trained code generator can be better equipped to generate appropriate revision function statements based on a simple / invariant syntax. Moreover, revision function statements only require specifying new constraints to revise the program accordingly, and these new constraints are automatically resolved along with any previously defined constraints (e.g., from earlier user statements, and / or from the initial computerized assistant program used to respond to those user statements). Because revision function statements only require listing new constraints, human annotators can more easily learn to create and / or approve appropriate training data.

[0052] Figure 3B Another example of dialogue history 300B is shown, where resolving new constraints from the second user utterance 306B leads to a tightening of the initial constraints. The initial constraints inferred from the first user utterance 302B are all events during the second day. This is based on output generated by the computerized assistant (e.g., speech audio and / or human-readable utterances, not in...) Figure 3BAs shown in the example, a user can indicate in a second user statement 306B that a narrower time window should be used for the event list. Therefore, the revised function statement 308B can include a narrower constraint indicating a time window for the afternoon. For example, the code generator can be configured to generate the constraint shown for a specific user between 12 PM and 5 PM based on user preferences, schedules, etc. As another example, "afternoon" can be defined in a hard-coded rule as a time between 12 PM and 5 PM (or any other suitable time). Alternatively or additionally, the code generation machine can be configured to output constraints related to any suitable afternoon time in a context-dependent manner; for example, for a user who ends the workday at 5 PM, output events between 12 PM and 5 PM, or for a user who ends the workday at 6 PM, output events between 12 PM and 6 PM. Although this example shows a constraint that explicitly specifies the start and end times of "afternoon," alternatively or additionally, constraints can be specified based on a function statement that can be evaluated to determine a suitable afternoon time. For example, instead of generating the constraint "Constraint[Event]{start:>(DateTime{date:[xO],time:Time(12:00)})},end:<=(DateTime{date:[xO],time:Time(17:00)})", the code generator can be configured to generate a constraint such as "Constraint[Event]{getSalient('afternoon')}" where "getSalient('afternoon')" is a function statement indicating that an appropriate afternoon time range should be evaluated based on any suitable machine learning function, hard-coded rules, user preferences and / or configuration settings, conversation history, etc.

[0053] Based on the new constraint indicating the afternoon time window, the initial "DateTime" constraint from the initial computerized assistant program 304B is tightened by the revised model to obtain new start and end constraints in the revised computerized assistant program 310B.

[0054] Figure 3CAnother example of dialogue history 300C is shown, where the initial constraints from the first user utterance 302C are broadened based on resolving new constraints from the second user utterance 306C. For example, the initial constraints derived from the first user utterance 302C and reflected in the initial computerized assistant 304C can be logically relaxed, where the logical relaxation of the initial constraints is based on identifying sub-constraints in the second utterance that logically imply the initial constraints. As shown, the initial constraint indicates an event after 12 PM. If the user does not hear what they expect, they may want to hear an event that started earlier. Therefore, the user indicates in the second user utterance 306C that they want to hear an event after 10 AM. However, "after 10 AM" logically implies the sub-constraint "after 12 PM". Therefore, the logical connection between "after 10 AM" and "after 12 PM" is "after 12 PM", because all times after 10 AM and after 12 PM are after 12 PM. Therefore, resolving new and initial constraints can include relaxing the initial constraint, thereby broadening it so that the new constraint introduces new information and leads to different results compared to the initial constraint. For example, given a new constraint in the revised function statement 308C that starts from times after 10:00 AM, the revised model is configured to generate a revised computerized assistant program 310C, which is configured to look for all events after 10:00 AM, ignoring the original constraint of "after 12:00 PM". Although Figure 3C Not shown, but based on appropriate rules and / or training, the revised model can alternatively or additionally generate a revised computerized assistant configured to find all events before 10 a.m. and before 12 p.m. (e.g., because the user did not find the events of interest among the events after 12 p.m., the events of interest may be between 10 a.m. and 12 p.m.).

[0055] Figure 3D Another example of dialogue history 300D is shown, where resolving new constraints leads to a revision of the initial constraints. As illustrated, in the first user utterance 302D, the user creates a meeting and invites "David A," which is handled by the initial computerized assistant 304C. However, the user later in the second user utterance 306D indicates that "David B" should actually be invited instead of "David A." Therefore, the revised function statement 308D indicates that "David B" should be invited instead of "David A," and the revised computerized assistant 310D thus indicates that both "David B" and "Adam C" should be invited, while "David A" should not be invited.

[0056] Figure 3EAnother example is shown where a user interacts with a computerized assistant to search for flights, as illustrated in dialogue history 300E. Based on the initial constraints indicated in the first user utterance 302E, the initial computerized assistant 304E is configured to search for suitable flights. However, as indicated by assistant utterance 305E (e.g., via voice audio and / or human-readable text), the computerized assistant cannot find any matching flights. Therefore, as indicated by the second user utterance 306E, the user requests an earlier flight for the previous day, Tuesday. The revised function statement 308E indicates a new constraint for finding flights on Tuesday. The resulting revised computerized assistant 310E is configured to search for flights on Tuesday, abandoning the original "Wednesday" constraint. Compared to the initial computerized assistant 304E, the revised computerized assistant 310E does not impose a time limit of 5 PM. For example, since the flight arrives the previous day, it may no longer be necessary to arrive before 5 PM (Tuesday) to ensure the user can complete their desired journey in Seattle (starting at 5 PM on Wednesday). The result obtained by the revised model reflects that the revised model was trained to achieve context-specific predictions based on annotated data.

[0057] Figure 3FAnother example of a dialogue history 300F is shown, illustrating a user's interaction with a computerized assistant to control a graphics editor (e.g., within a demo program). In a first user statement 302E, the user requests that the image title size be increased to font size 16. The computerized assistant is configured to execute an initial computerized assistant procedure 304F and confirm in assistant statement 305F whether the user is satisfied with the resulting change. As shown, the initial computerized assistant procedure 304F uses the "getSalient" search history function to search for salient images based on the interaction context between the user and the computerized assistant, such as the image the user is currently working on. After finding a salient image, the initial computerized assistant procedure 304F changes the font size of the relevant text box. However, as shown in a second user statement 306F, the user does not want the text box to be wider than the image. Therefore, the revision function statement 308F instructs that the text box width should be smaller than the image width. However, this reduced width of the text box may require a reduced font size. Therefore, the revised computerized assistant program 310F uses a font size of at least 14, which is a smaller minimum font size compared to the initial constraint of a font size of at least 16 (e.g., a wider and / or more relaxed constraint). As an example, a font size of 14 could be the minimum relaxation of the initial constraint (e.g., increasing the font size as close as possible to size 16 based on the user's intent in the first user utterance 302F), which allows new constraints to be resolved (e.g., slightly reducing the font size from the initial request 16 to allow the text box to be at most as wide as the image). As an example, a font size of 14 could represent the solution with the lowest cost / highest score among multiple candidate solutions (e.g., candidate solutions with different font sizes).

[0058] The methods and processes described herein can be attached to a computing system of one or more computing devices. In particular, such methods and processes can be implemented as an executable computer application, a network-accessible computing service, an application programming interface (API), a library, or a combination of the above and / or other computing resources.

[0059] Figure 4 A simplified representation of a computing system 400 is schematically illustrated, which is configured to provide any to all computing functions described herein. The computing system 400 may take the form of one or more personal computers, network-accessible server computers, tablet computers, home entertainment computers, gaming devices, mobile computing devices, mobile communication devices (e.g., smartphones), virtual / augmented / mixed reality computing devices, wearable computing devices, Internet of Things (IoT) devices, embedded computing devices, and / or other computing devices. For example, the computing system 400 may be configured to implement method 200. As another example, the computing system 400 may be a computerized assistant 122.

[0060] The computing system 400 includes a logic subsystem 402 and a storage subsystem 404. The computing system 400 may optionally include a display subsystem 408, an input subsystem 406, a communication subsystem 410, and / or... Figure 4 Other subsystems not shown.

[0061] Logical subsystem 402 includes one or more physical devices configured to execute instructions. For example, the logical subsystem may be configured to execute instructions as part of one or more applications, services, or other logical structures. The logical subsystem may include one or more hardware processors configured to execute software instructions. Additionally or alternatively, the logical subsystem may include one or more hardware or firmware devices configured to execute hardware or firmware instructions. The processor of the logical subsystem may be single-core or multi-core, and the instructions executed on it may be configured for sequential, parallel, and / or distributed processing. The various components of the logical subsystem may optionally be distributed across two or more separate devices that can be remotely located and / or configured for coordinated processing. Aspects of the logical subsystem may be virtualized and executed by remotely accessible networked computing devices configured in a cloud computing configuration.

[0062] Storage subsystem 404 includes one or more physical devices configured to temporarily and / or permanently store computer information, such as data and instructions executable by the logical subsystem. When the storage subsystem includes two or more devices, these devices may be co-located and / or remotely positioned. Storage subsystem 404 may include volatile, non-volatile, dynamic, static, read / write, read-only, random access, sequential access, location-addressable, file-addressable, and / or content-addressable devices. Storage subsystem 404 may include removable and / or built-in devices. The state of storage subsystem 404 can be changed—for example, to store different data—when the logical subsystem executes instructions.

[0063] The aspects of logic subsystem 402 and storage subsystem 404 can be integrated together into one or more hardware logic components. Such hardware logic components may include, for example, application-specific integrated circuits (PASIC / ASIC), application-specific standard products (PSSP / ASSP), system-on-a-chip (SOC), and complex programmable logic devices (CPLD).

[0064] When included, the display subsystem 408 can be used to present a visual representation of the data stored by the storage subsystem 404. This visual representation may take the form of a graphical user interface (GUI). The display subsystem 408 may include one or more display devices using virtually any type of technology. In some embodiments, the display subsystem may include one or more virtual reality, augmented reality, or mixed reality displays.

[0065] When included, the input subsystem 406 may include or interface with one or more input devices. Input devices may include sensor devices or user input devices. Examples of user input devices include a keyboard, mouse, touchscreen, or game controller. In some embodiments, the input subsystem may include or interface with selected Natural User Input (NUI) components. Such components may be integrated or peripheral, and the translation and / or processing of input actions may be handled on-board or off-board. Example NUI components may include a microphone for speech and / or voice recognition; an infrared, color, stereo, and / or depth camera for machine vision and / or gesture recognition; and a head tracker, eye tracker, accelerometer, and / or gyroscope for motion detection and / or intent recognition.

[0066] When included, the communication subsystem 410 can be configured to communicatively couple the computing system 400 to one or more other computing devices. The communication subsystem 410 may include wired and / or wireless communication devices compatible with one or more different communication protocols. The communication subsystem can be configured to communicate over personal, local area networks (LANs), and / or wide area networks (WANs).

[0067] The logical subsystem and storage subsystem can collaborate to instantiate one or more logical machines. As used herein, the term "machine" is used collectively to refer to a combination of hardware, firmware, software, instructions, and / or any other components that collaborate to provide computer functionality. In other words, "machine" is never an abstract concept; it always has a tangible form. A machine can be instantiated by a single computing device, or a machine can include two or more sub-components instantiated by two or more different computing devices. In some implementations, a machine includes local components (e.g., software applications executed by a computer processor) that collaborate with remote components (e.g., cloud computing services provided by a network of server computers). The software and / or other instructions that give a particular machine its functionality can optionally be stored as one or more unexecuted modules on one or more suitable storage devices. As an example, see [reference]. Figure 1 The computerized assistant 122 can be implemented as one or more machines, such as code generation machine 128, description machine 130, revision model machine 132 and / or dialogue history machine 134.

[0068] The machine can achieve this using any appropriate combination of existing and / or future machine learning (ML), artificial intelligence (AI), and / or natural language processing (NLP) technologies. Non-limiting examples of techniques that can be incorporated into implementations on one or more machines include support vector machines, multilayer neural networks, convolutional neural networks (e.g., spatial convolutional networks for processing images and / or videos, temporal convolutional neural networks for processing audio signals and / or natural language sentences, and / or any other suitable convolutional neural network configured to convolve and pool features in one or more temporal and / or spatial dimensions), recurrent neural networks (e.g., long short-term memory networks), associative memories (e.g., lookup tables, hash tables, Bloom filters, neural Turing machines, and / or neural random access memories), word embedding models (e.g., GloVe or Word2Vec), unsupervised spatial and / or clustering methods (e.g., nearest neighbor algorithms, topological data analysis, and / or k-means clustering), graphical models (e.g., (hidden) Markov models, Markov random fields, (hidden) conditional random fields, and / or AI knowledge bases), and / or natural language processing techniques (e.g., tokenization, stemming, region selection and / or dependency parsing, and / or intent recognition, segmentation models, and / or super-segmentation models (e.g., implicit dynamic models)).

[0069] In some examples, the machine and / or model can be tuned through training, thereby configuring the machine / model to perform the desired function. For example, Figure 1 The computerized assistant 122, including a code generation machine 128, a description machine 130, a revision model machine 132, and / or a dialogue history machine 134, can be trained based on annotation data indicating exemplary user utterances, the computerized assistant program, revision function statements, and / or any other suitable training data. As an example, instructions stored in the storage subsystem 404 are executable to operate and / or train the code generation machine 128, the description machine 130, the revision model machine 132, and / or the dialogue history machine 134 based on annotation data. For example, instructions are executable to operate a previously trained revision model to derive a revised computerized assistant program, and to retrain a previously trained revision model based on exemplary revision function statements and an exemplary revised computerized assistant program received from a human annotator.

[0070] In some examples, the methods and processes described herein can be implemented using one or more differentiable functions, wherein the gradient of the differentiable function can be computed and / or estimated with respect to the input and / or output of the differentiable function (e.g., with respect to training data, and / or with respect to a target function). Such methods and processes can be determined at least in part by a set of trainable parameters. Therefore, the trainable parameters of a particular method or process can be tuned by any suitable training procedure to continuously improve the functionality of the method or process.

[0071] Non-limiting examples of training procedures for tuning trainable parameters include supervised training (e.g., using gradient descent or any other suitable optimization method), zero-shot, few-shot, unsupervised learning methods (e.g., category-based classification derived from unsupervised clustering methods), reinforcement learning (e.g., feedback-based deep Q-learning) and / or generative adversarial neural network training methods, belief propagation, RANSAC (random sample consensus), contextual slot methods, maximum likelihood methods, and / or expectation maximization. In some examples, multiple methods, processes, and / or components of the system described herein can be trained simultaneously with respect to an objective function that measures the performance of the collective function of multiple components (e.g., with respect to reinforcement feedback and / or with respect to labeled training data). Simultaneous training of multiple methods, processes, and / or components can improve this collective function. In some examples, one or more methods, processes, and / or components can be trained independently of other components (e.g., offline training on historical data).

[0072] In some examples, a computerized assistant may incorporate one or more language models, for example, for processing user utterances. The language models may leverage lexical features to guide word sampling / searching for speech recognition. For example, a language model may be defined at least in part by the statistical distribution of words or other lexical features. For instance, a language model may be defined by the statistical distribution of n-grams, defining the transition probabilities between candidate words based on lexical statistics. The language model may further be based on any other suitable statistical features, and / or the results of processing the statistical features with one or more machine learning and / or statistical algorithms (e.g., confidence values ​​generated by such processing). In some examples, the statistical model may constrain which words can be recognized for an audio signal, for example, based on the assumption that the words in the audio signal come from a specific vocabulary.

[0073] Alternatively or additionally, the language model may be based on one or more neural networks previously trained to represent audio input and words in a shared latent space, for example, a vector space learned by one or more audio and / or word models (e.g., wav2letter and / or word2vec). Therefore, finding candidate words may include searching the shared latent space for audio input based on vectors encoded by the audio model, in order to find candidate word vectors for decoding using the word model. For one or more candidate words, the shared latent space can be used to evaluate the confidence level of having candidate words in the speech audio.

[0074] Language models can be used in conjunction with acoustic models configured to evaluate the confidence of candidate words in speech audio within the audio signal based on word-based acoustic features (e.g., Mel-frequency cepstral coefficients, formants, etc.). Optionally, in some examples, language models can be combined with acoustic models (e.g., the evaluation and / or training of the language model can be based on the acoustic model). The acoustic model defines the mapping between acoustic signals and basic sound units such as phonemes, e.g., tagged speech audio. Acoustic models can be based on any suitable combination of existing or future ML and / or AI models, such as: deep neural networks (e.g., Long Short-Term Memory, Temporal Convolutional Neural Networks, Restricted Boltzmann Machines, Deep Belief Networks), Hidden Markov Models (HMMs), Conditional Random Fields (CRFs) and / or Markov Random Fields, Gaussian Mixture Models, and / or other graphical models (e.g., Deep Bayesian Networks). The audio signal to be processed with the acoustic model can be preprocessed in any suitable manner, such as encoding at any suitable sampling rate, Fourier transform, bandpass filtering. Acoustic models can be trained to identify the mapping between acoustic signals and sound units based on training using labeled audio data. For example, an acoustic model can be trained on labeled audio data that includes speech audio and corrected text to learn the mapping between speech audio signals and sound units represented by the corrected text. Therefore, acoustic models can be continuously improved to enhance their utility in correctly recognizing speech audio.

[0075] In some examples, in addition to statistical models, neural networks, and / or acoustic models, language models can be combined with any suitable graphical model, such as HMM or CRF. Given the speech audio and / or other words identified so far, the graphical model can utilize statistical features (e.g., transition probabilities) and / or confidence values ​​to determine the probability of recognizing the word. Therefore, the graphical model can utilize statistical features, previously trained machine learning models, and / or acoustic models to define the transition probabilities between states represented in the graphical model.

[0076] This disclosure is presented by way of example and with reference to the accompanying drawings. Components, process steps, and other elements that may be substantially the same in one or more drawings are identified in a coordinated manner and described with minimal repetition. However, it should be noted that elements identified by coordinates may also differ to some extent. It should also be noted that some drawings may be schematic rather than drawn to scale. Various drawing scales, aspect ratios, and numbers of components shown in the drawings may be intentionally distorted to make certain features or relationships easier to see.

[0077] In one example, a method for adjusting a computerized assistant to satisfy updated constraints includes: maintaining a dialogue history including a first utterance indicating the initial constraints; receiving a second utterance indicating a new constraint that conflicts with the initial constraints; identifying an initial computerized assistant configured to satisfy the initial constraints; identifying a revision function statement parameterized by references to the initial computerized assistant and references to the new constraints; and executing instructions derived from the revision function statement to return a revised computerized assistant configured to satisfy the new constraints. In this example or any other example, the revision function statement is configured to be parameterized using multiple different computerized assistants, and the instructions derived from the revision function statement are configured to return a corresponding revised computerized assistant for a given computerized assistant based on the new constraints. In this example or any other example, the method further includes manipulating a revision model to derive instructions from the revision function statement, wherein the instructions are configured to generate a revised computerized assistant based on the revision function statement and the initial computerized assistant. In this example or any other example, the revision model is a previously trained machine learning model trained based on an exemplary revision function statement and an exemplary revised computerized assistant received from a human annotator. In this example or any other example, the revised model includes multiple predefined rules configured to transform the initial computerized assistant program based on the new constraints. In this example or any other example, the method also includes transforming the new constraints and the initial computerized assistant program into a constraint satisfaction problem. In this example or any other example, the constraint satisfaction problem is encoded as a constraint graph, including nodes indicating constraint variables and edges indicating implicit relationships between constraint variables. In this example or any other example, the constraint satisfaction problem includes a logical relaxation of the initial constraints. In this example or any other example, the logical relaxation of the initial constraints is based on identifying a conflict between the second utterance and the initial constraints. In this example or any other example, the logical relaxation of the initial constraints is based on recognizing that the second utterance logically implies a sub-constraint of the initial constraints. In this example or any other example, the method also includes finding multiple candidate constraint solutions to the constraint satisfaction problem and selecting candidate constraint solutions based on a cost function. In this example or any other example, the cost function is a previously trained machine learning function. In this example or any other example, the method also includes undoing one or more actions performed by the initial computerized assistant program. In this example or any other example, the dialogue history further includes references to the initial computerized assistant program. In this example or any other example, the method also includes adding the second utterance to the dialogue history. In this example or any other example, the method also includes executing the revised computerized assistant. In this example or any other example, the method also includes saving a reference to the revised computerized assistant in the conversation history.

[0078] In this example, the computer system includes: a logic subsystem; and a storage subsystem that holds instructions executable by the logic subsystem to implement the methods of any of the examples described herein. In this or any other example, the computer program is configured to perform the methods of any of the examples described herein when executed on the computer system.

[0079] In the example, the computerized assistant system includes: a dialogue history machine configured to maintain a dialogue history including a first utterance indicating initial constraints and a second utterance indicating new constraints that conflict with the initial constraints; a code generation machine configured to generate an initial computerized assistant program configured to satisfy the initial constraints and to generate revision function statements parameterized by references to the initial computerized assistant program and to references to the new constraints; and a revision model machine configured to execute instructions derived from the revision function statements to return a revised computerized assistant program configured to satisfy the new constraints.

[0080] It should be understood that the configurations and / or methods described herein are exemplary in nature, and these particular embodiments or examples should not be considered limiting, as many variations are possible. The specific routines or methods described herein may represent one or more of any number of processing strategies. Thus, the various actions shown and / or described may be performed in the order shown and / or described, in a different order, in parallel, or omitted. Similarly, the order of the above processes may be changed.

[0081] The subject matter of this disclosure includes all novel and non-obvious combinations and sub-combinations of various processes, systems and configurations, as well as other features, functions, actions and / or characteristics disclosed herein, and any and all their equivalents.

Claims

1. A method for adjusting a computerized assistant program to meet updated constraints, the method comprising: Maintain the dialogue history, including the first utterance that indicates the initial constraints; Receive a second statement indicating a new constraint that conflicts with the initial constraint; Identify the initial computerized assistant program configured to satisfy the initial constraints; Identify the revised function statements parameterized by references to the initial computerized assistant program and references to the new constraints; The operation revision model is used to derive instructions from the revision function statement, wherein the instructions are configured to generate a revised computerized assistant program based on the revision function statement and the initial computerized assistant program; and Execute the instructions derived from the revised function statement to return a revised computerized assistant program configured to satisfy the new constraints. The revision model is a previously trained machine learning model based on exemplary revision function statements and exemplary revised computerized assistant programs received from human annotators.

2. The method as described in claim 1, wherein, The revision function statement is configured to be parameterized using multiple different computerized assistants, and wherein the instructions derived from the revision function statement are configured to return a corresponding revised computerized assistant for a given computerized assistant based on the new constraints.

3. The method as described in claim 1, wherein, The revised model includes multiple predefined rules configured to transform the initial computerized assistant program based on the new constraints.

4. The method of any of the preceding claims further comprises converting the new constraint and the initial computerized assistant program into a constraint satisfaction problem.

5. The method of claim 4, wherein, The constraint satisfaction problem is encoded as a constraint graph, which includes nodes indicating constraint variables and edges indicating implicit relationships between the constraint variables.

6. The method of claim 4, wherein, The constraint satisfaction problem includes the logical relaxation of the initial constraints.

7. The method of claim 6, wherein, The logical relaxation of the initial constraint is based on identifying the conflict between the second utterance and the initial constraint.

8. The method of claim 6 or 7, wherein, The logical relaxation of the initial constraint is based on identifying sub-constraints that the second utterance logically implies the initial constraint.

9. The method of any one of claims 4 to 8, further comprising finding a plurality of candidate constraint solutions for the constraint satisfaction problem, and selecting a candidate constraint solution based on a cost function.

10. The method of claim 9, wherein, The cost function is a previously trained machine learning function.

11. The method of any of the preceding claims further comprises reversing one or more operations performed by the initial computerized assistant.

12. The method as claimed in any of the preceding claims, wherein, The dialogue history also includes references to the initial computerized assistant program.

13. The method of any of the preceding claims further comprises adding the second utterance to the dialogue history.

14. The method of any of the preceding claims further comprises executing the revised computerized assistant program.

15. The method of any of the preceding claims further comprises storing a reference to the revised computerized assistant program in the dialogue history.

16. A computer system, comprising: Logical subsystem; as well as A storage subsystem that stores instructions that can be executed by the logic subsystem to implement the method described in any of the preceding claims.

17. A computer program, when executed on a computer system, is configured to perform the method as claimed in any one of claims 1 to 15.

Images