Machine-learned language models that generate intermediate text analysis in the service of contextual text generation

Abstract

A system for generating intermediate text analysis in the service of contextual text generation is provided. [Solution] A structural tool (15) performs a sequence including obtaining a contextual text string including one or more contextual text tokens (12); processing the contextual text string using a machine-learned language model to generate one or more intermediate text strings including one or more intermediate text tokens (16); and processing the one or more intermediate text strings using a machine-learned language model (14) to generate an output text string including one or more output text tokens (18), wherein the one or more intermediate text strings include text analysis of the contextual text string that supports the output text string.

Description

[Technical Field]

[0001] Related Applications This application claims priority to and the benefit of U.S. Provisional Patent Application No. 63 / 191,563, filed May 21, 2021. U.S. Provisional Patent Application No. 63 / 191,563 is incorporated herein by reference in its entirety.

[0002] The present disclosure relates generally to the use of machine learning for language modeling. More particularly, the present disclosure relates to machine-learned language models that generate intermediate text analysis (e.g., including the use of structural tools such as APIs) in the service of contextual text generation. [Background technology]

[0003] Natural language processing (NLP) has seen rapid development in recent years, with such advances primarily attributable to improvements in learning-based algorithms and other aspects of machine learning or "neural" learning. One particular task in the field of NLP is contextual text generation. In contextual text generation tasks, an agent (e.g., a machine learning model) is tasked with generating output text from a given context. For example, the given context may include one or more contextual text strings. Thus, in some exemplary approaches to the contextual text generation task, a text-to-text model reads the input contextual text and then directly creates the output text.

[0004] One example of a contextual text generation task is a question answering task, where the input context is a question and the desired output is a response to the question. Another example of a contextual text generation task is dialogue generation, where the input context is a conversation history and the desired output is a next utterance, where the next utterance is responsive to, or possibly reasonable within, the context of the conversation history.

[0005] The current state-of-the-art model for contextual text generation is a left-to-right language model such as GPT3 (Brown et al., Language Models are Few-Shot Learners, arXiv:2005.14165), where " <input> <output>These tend to be Transformer-based neural models, either considered as a sequence, or sequence-to-sequence models like the original Transformer (Vaswani et al., Attention is All You Need, arXiv:1706.03762).

[0006] However, neural language models such as GPT3 and Transformers suffer from several drawbacks. Specifically, while neural language models exhibit significant intelligence, their knowledge is constrained by the information contained in (and learned from) their training datasets and / or information introduced in contextual text input. Thus, their knowledge of factual information is extremely limited and generally frozen in time. As such, when required to produce outputs containing factual information, models typically either misinterpret incorrect facts or feed outdated information. Reliance on incorrect factual information can lead to inefficiencies, where incorrect actions (e.g., computerized actions) need to be taken and corrected or possibly remediated, resulting in redundant and unnecessary resource (e.g., computational) usage.

[0007] Another exemplary drawback of neural language models is that their output is difficult to interpret or understand. Specifically, because such models often generate output directly from input, it is difficult to understand exactly why such a model generated the output or what aspects of the input led to that output. A lack of interpretability in language model output can lead to a lack of trust or confidence in the model output, which can result in unnecessary overhead or other effort (e.g., computerized operations) to "double-check" the accuracy or usefulness of the model's output. [Prior art documents] [Non-patent literature]

[0008] [Non-Patent Document 1] Brown et al., Language Models are Few-Shot Learners, arXiv:2005.14165 [Non-patent document 2] Vaswani et al., Attention is All You Need, arXiv:1706.03762 Summary of the Invention [Means for solving the problem]

[0009] Aspects and advantages of embodiments of the present disclosure will be set forth in part in the description that follows, or may be learned from the description, or may be learned through practice of the embodiments.

[0010] One exemplary aspect of the present disclosure is directed to a computing system for generating contextual text with improved interpretability. The computing system includes one or more processors and one or more non-transitory computer-readable media, the one or more non-transitory computer-readable media collectively storing a machine-learned language model that performs text analysis in service of the contextual text generation and instructions that, when executed by the one or more processors, cause the computing system to perform operations. The operations include obtaining a contextual text string that includes one or more contextual text tokens. The operations include processing the contextual text string with the machine-learned language model to generate one or more intermediate text strings that include one or more intermediate text tokens. The operations include processing the one or more intermediate text strings with the machine-learned language model to generate an output text string that includes one or more output text tokens. The one or more intermediate text strings include text analysis of the contextual text string that supports the output text string.

[0011] Another exemplary aspect of the present disclosure is directed to a computer-implemented method for improved contextual text generation.

[0012] The method includes obtaining a plurality of training tuples, each training tuple including an example context text string including one or more context text tokens, one or more example intermediate text strings including one or more intermediate text tokens, and an example output text string including one or more output text tokens. For each training tuple, the method includes inputting at least a portion of the training tuple to a language model, receiving a predicted next token as an output of the language model, the predicted next token being generated by the language model by processing the portion of the training tuple, evaluating a loss function that compares the predicted next token generated by the language model to an actual next token included in the training tuple, and modifying one or more values ​​of one or more parameters of the language model based on the evaluation of the loss function.

[0013] Other aspects of the present disclosure are directed to various systems, apparatus, non-transitory computer-readable media, user interfaces, and electronic devices.

[0014] These and other features, aspects, and advantages of various embodiments of the present disclosure will become better understood with reference to the following description and appended claims. The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate exemplary embodiments of the present disclosure and, together with the description, serve to explain associated principles.

[0015] Detailed descriptions of embodiments directed to persons skilled in the art are set forth herein, which refer to the accompanying figures. [Brief explanation of the drawings]

[0016] [Figure 1] FIG. 1 is a block diagram of an example machine-learned language model for generating text analytics in a service of contextual text generation, according to an example embodiment of the present disclosure. [Figure 2] FIG. 1 is a block diagram of an example machine-learned language model for generating text analytics in a service of contextual text generation, according to an example embodiment of the present disclosure. [Figure 3] FIG. 1 is a block diagram of an example training process for a machine-learned language model that generates text analytics in the service of contextual text generation, according to an example embodiment of the present disclosure. [Figure 4] FIG. 1 is a block diagram of an example training process for a machine-learned language model that generates text analytics in the service of contextual text generation, according to an example embodiment of the present disclosure. [Figure 5A] FIG. 1 is a block diagram of an exemplary computing system according to an exemplary embodiment of the present disclosure. [Figure 5B] FIG. 2 is a block diagram of an exemplary computing device according to an exemplary embodiment of the present disclosure. [Figure 5C] FIG. 2 is a block diagram of an exemplary computing device according to an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0017] Reference numbers that are repeated among the figures identify the same features in various implementations.

[0018] overview In general, the present disclosure is directed to systems and models that include and / or utilize one or more machine-learned language models to generate intermediate text analysis (e.g., including the use of structural tools such as APIs) in the service of contextual text generation. For example, a computing system may obtain a contextual text string that includes one or more contextual text tokens. The computing system may process the contextual text string with the machine-learned language model to generate one or more intermediate text strings that include one or more intermediate text tokens. The computing system may process the one or more intermediate text strings with the machine-learned language model to generate an output text string that includes one or more output text tokens. The one or more intermediate text strings may include text analysis of the contextual text string that supports the output text string.

[0019] Thus, aspects of the present disclosure improve the knowledge, grounding, and interpretability of machine-learned language models by teaching the model to generate text analytics (e.g., during its service) before generating output text in response to contextual text input (e.g., generating a response to a question or previous dialogue). The generation of such intermediate text analytics can improve the interpretability of the model output. In particular, the intermediate text analytics can be reviewed or inspected to interpret or understand how the model generated the output in response to the contextual input. This can also facilitate the assessment of the reliability and / or suitability of the output in serving a particular task.

[0020] According to another aspect of the present disclosure, in some implementations, text analysis can include and / or leverage the use of structural tools that provide access to additional information. For example, one or more intermediate text tokens included in the intermediate text analysis can include at least one tool token that triggers the use of a structural tool to access additional information not included in the contextual text string and / or not included in the training data on which the model was trained. Thus, the language model can invoke and use such structural tools to have access to additional information that may be state-of-the-art, factual, domain-specific, client- or user-specific, etc. This improves the knowledge available to the language model when forming text output and further improves the flexibility of the system by enabling the introduction of various information sources for various use cases. Because machine learning processes can be applied to minimize the computational overhead in invoking such services, e.g., tool tokens, and the order in which they are generated can be adapted to minimize computational overhead, such as latency and / or network usage, the techniques of the present disclosure can achieve improved or optimized integration with external tools.

[0021] By way of example, a structural tool in a machine-learned language model may have access to include a database lookup to access additional information from a database, an API call to request and receive additional information via an application programming interface (API), a programming language interpreter that performs one or more sequences of operations on input text tokens, a query service that queries results from a search engine, a knowledge graph, or a digital assistant, a communication client that generates and sends a communication (e.g., an email, a short message service message, a multimedia messaging service message, an application-based chat message, etc.) to another device or user, and / or various other forms of structural tool that generate or potentially provide access to additional information. Thus, structural tools are not limited to looking up information but can also have side effects or cause an action (e.g., booking a meeting, purchasing something, submitting a ticket to a person, etc.).

[0022] The machine-learned language models described herein can be trained in several different ways. In one example, human volunteers or crowdworkers can generate example intermediate analysis texts for several (e.g., thousands) of examples. For example, a human worker can be provided with a contextual input text and output text pair, and the human worker can generate an intermediate analysis text that shows an analysis of the contextual input text that results in or possibly supports the output text. The human worker can be provided with access to structural tools and their use of such tools, and the corresponding information obtained can be included in the example intermediate analysis text.

[0023] The intermediate analysis (either during training of examples or as they are generated by the model) may, in some instances, include step-by-step logic in a human-readable form, such as a multi-step solution to an algebraic problem. It may also include the use of external text-to-text tools, such as databases, python interpreters, search engines, etc., as described elsewhere herein. In some implementations, tool use in the intermediate analysis section may be marked and / or triggered by special tags that specify which tools were used and describe the tool's inputs and outputs. The intermediate text may include multiple instances of tool use, as well as any amount of free-form text.

[0024] Thus, intermediate analysis can involve the use of a tool (e.g., an API) that takes in a structured list of input parameters and returns a structured output (e.g., do_thing(a: int, b: List[str]) --> response_object). However, any structured input / output can also be serialized to and parsed from free text using some serialization method, such as Google Protos or JSON text serialization. From that perspective, a text-to-text interface can be a superset of a structured interface.

[0025] To generate an exemplary training dataset, exemplary intermediate analysis texts generated by a human annotator may be combined with contextual input text and output text pairs to form training tuples, where the training tuples include an exemplary contextual text string including one or more contextual text tokens, one or more exemplary intermediate text strings including one or more intermediate text tokens, and an exemplary output text string including one or more output text tokens. Thus, in some implementations, a human annotator may be provided with the exemplary contextual text strings and the exemplary output text strings, and the human annotator may generate the exemplary intermediate text strings. In other implementations, a human annotator may be provided with only the exemplary contextual text strings, and the human annotator may generate both the exemplary intermediate text strings and the exemplary output text strings.

[0026] A training dataset as described herein may be used to train a language model. For example, the training dataset may be used to fine-tune a pre-trained model on tera-scale unsupervised data. As an example, the model may be trained by using the model to predict the next token included in a training tuple (e.g., the next intermediate text token or the next output text token). A loss function may be used to evaluate the ability of the model to predict the next token. Parameters of the model may be updated based on the loss function (e.g., via backpropagation-based techniques). In some implementations, training the model on each training tuple may include iteratively training on each token included in the intermediate text string, token-by-token, followed by inclusion of each token in the output text string.

[0027] In another example, a language model may be trained to generate intermediate text in the service of contextual language generation using reinforcement learning techniques. For example, aspects of the intermediate text and / or output text generated by the model may be evaluated by an objective function to determine a reward, which may then be used to update the model parameters.

[0028] During inference, the language model can then be used to generate intermediate analyses given the input. In some implementations, the intermediate analyses can be generated one token at a time. In some implementations, each time the model finishes generating input to an external tool, the tool itself is invoked with this input to generate tool output, which is appended to the intermediate text and the model continues generating from there.

[0029] Aspects of the present disclosure therefore propose extending the (input, output) training example and language generation paradigm to have intermediate analyses that are also textual, so that instead of simply producing an output given the input, a language model can be trained to produce an intermediate analysis given the input, and then to produce an output given the input and the intermediate analysis.

[0030] Thus, whereas a dialogue agent (or any other contextual text generation model) is typically trained on (context, response) pairs so that it can directly generate a response to a given context, in an exemplary implementation of the present disclosure, a language model can instead be trained on (context, intermediate analysis, response) triples, and the model learns to generate (intermediate analysis | context) and (response | context, intermediate analysis).

[0031] In some implementations, the output text generated as described herein can be further processed using a text-to-speech system to generate audio output. As another example, input text can be generated from audio input using speech to a text system. For example, a virtual assistant can interact with a user through audio input and output, and audio / speech-to-text conversion can be used to enable processing by the virtual assistant to occur within the text domain described herein.

[0032] The systems and methods of the present disclosure provide several technical effects and benefits. As an example, the proposed models exhibit improved interpretability. For example, intermediate text analysis generated by the model can be reviewed or inspected to interpret or understand how the model generated the output in response to contextual input. Improved interpretability can lead to more efficient use of computational resources, such as processor usage, memory usage, etc. For example, a lack of interpretability in language model output can result in a lack of trust or confidence in the model output, which can result in unnecessary overhead or other effort (e.g., computerized operations) attempting to "double-check" the accuracy or usefulness of the model's output. By improving interpretability, trust in computerized systems can be improved. In particular, the reliability of model output can be verified and / or assessed to establish the system's usefulness for a particular task.

[0033] As another exemplary technical effect and benefit, the proposed approach enables a language model to leverage structural tools to access additional information, such as additional factual information. Thus, the language model can invoke and use such structural tools to have access to additional information, which may be state-of-the-art, factual, domain-specific, client or user-specific, etc. This improves the knowledge available to the language model when forming text output, further improving the flexibility of the system by allowing the introduction of different information sources for different use cases.

[0034] In addition to improving the quality of the model's output, the proposed use of structural tools also leads to savings in computational resources, such as processor usage, memory usage, and network bandwidth. Specifically, the knowledge available to previous language models was constrained by the information contained in (and learned from) their training datasets and / or information introduced in contextual text input. Their knowledge of factual information was therefore extremely limited and generally frozen in time. As a result, when required to produce output containing factual information, the model typically either misinterprets incorrect facts or supplies outdated information. Therefore, the entire language model needs to be retrained to keep the language model up-to-date as real-world facts change, to port the language model to new domains or sets of user information, or possibly to deploy the model to new situations where new information was unresolved. Retraining a language model requires the use of computational resources, such as processor usage, memory usage, and network bandwidth.

[0035] However, the use of structural tools proposed by this disclosure obviates the need to retrain a language model to keep it up to date as real-world facts change, to port the language model to a new domain or set of user information, or possibly to deploy the model to new situations where new information was unresolved. Instead, the model can simply be given access to additional information (e.g., via structural tools), which may be state-of-the-art, factual, domain-specific, client- or user-specific, etc. Thus, the model can be easily ported to different domains, uses, users, etc., and / or can provide responses that leverage the latest factual information without the need to retrain the model, thereby significantly saving computational resources. By encoding context in the form of intermediate analyses that can interface with (potentially external) information sources, processing can contribute to overcoming technical constraints in the provision of information and / or functionality.

[0036] Similarly, another exemplary technical effect is derived from the ability of a model to leverage external sources to obtain information without having to store all of the information necessary to respond to various inputs (e.g., in the form of learned relationships). In particular, past approaches required storage and use (e.g., on user devices with constrained memory and / or battery availability) of large models having a size (e.g., number of parameters) sufficient to learn and store relationships between various inputs and outputs. In contrast, some exemplary implementations of the present disclosure may enable “thin” (smaller) models to reside on user devices or other mobile clients or browsers. Thin models can leverage various structural tools (e.g., cloud services) to conserve battery, computation, storage, updates, and the like. Thus, smaller models with access to structural tools can achieve similar or better performance than large, self-contained models, thereby conserving computational resources such as memory usage, network bandwidth, and energy consumption.

[0037] Exemplary embodiments of the present disclosure will now be described in further detail with reference to the figures.

[0038] Example Language Models for Generating Intermediate Text Analysis 1 illustrates a block diagram of an exemplary machine-learned language model that generates text analytics in a contextual text generation service, according to an exemplary embodiment of the present disclosure. Specifically, language model 14 can receive a contextual text string 12 that includes one or more contextual text tokens. Language model 14 can process the contextual text string 12 to generate one or more intermediate text strings 16 that include one or more intermediate text tokens. Language model 14 can process the one or more intermediate text strings 16 to generate an output text string that includes one or more output text tokens 18. The one or more intermediate text strings 16 can include text analytics of the contextual text string 12 that support, result in, prove, or possibly indicate a logical analysis of the contextual text string 12 to generate the output text string 18.

[0039] According to aspects of the present disclosure, in some implementations, one or more intermediate text tokens 16 may include at least one tool token that triggers the use of structure tool 15 to access additional information not included in contextual text string 12. In some implementations, structure tool 15 may include a database search to access the additional information from a database. In some implementations, structure tool 15 may include an application programming interface (API) call to request and receive the additional information via an API. In some implementations, structure tool 15 may include a programming language interpreter that performs a sequence of one or more operations on the input text token. In some implementations, structure tool 15 may include a query service that queries results from a search engine, a knowledge graph, or a digital assistant. In addition to the tool token, one or more intermediate text tokens 16 may further include at least one natural language text token.

[0040] In some implementations, when the machine-learned language model 14 generates a tool token, the computing system can suspend the machine-learned language model, execute a structural tool 15 to access additional information, append the additional information to a current version of one or more intermediate text strings 16, and resume text generation using the machine-learned language model 14 based on the current version of the one or more intermediate text strings 16 and the appended additional information.

[0041] In some implementations, the machine-learned language model 14 operates on a token-by-token basis, and in some such implementations, when generating one or more intermediate text strings 16, the language model 14 recursively receives as input each generated intermediate text token 16.

[0042] Thus, in some implementations, processing the contextual text string 12 with the machine-learned language model 14 to generate one or more intermediate text strings 16 including one or more intermediate text tokens may be performed over several iterations. In a first iteration, the computing system may process the contextual text string 12 with the machine-learned language model 14 to generate a first intermediate text string 16 including one or more intermediate text tokens. The computing system may then append the first intermediate text string 16 to the contextual text string 12 to generate an updated contextual text string. Then, for each of one or more additional iterations and until the machine-learned language model outputs a closure token, the computing system may process the updated contextual text string with the machine-learned language model 14 to generate additional intermediate text strings 16 including one or more intermediate text tokens. The computing system may append the additional intermediate text string to the updated contextual text string to generate an updated contextual text string for the next iteration.

[0043] The machine-learned language model 14 can be or include various types of models, including, by way of example, a recurrent neural network, a multi-head self-attention model, a sequence-to-sequence model, and / or other forms of language models. The language model can be a cloze model or a left-to-right model. The language model can optionally have an encoder-decoder architecture.

[0044] In some example implementations, the machine-learned language model 14 may be a question-answering model, and the contextual text string 12 may be or include a question. In some example implementations, the machine-learned language model 14 may be a dialogue model, and the contextual text string 12 may be or include a dialogue history.

[0045] In some implementations, at least a portion of the contextual text string 12 includes or corresponds to text entered by a user. In some implementations, the computing system can provide at least an output text string 18 for display to a user.

[0046] 2 illustrates a block diagram of an exemplary machine-learned language model 14 that generates text analytics in the service of contextual text generation, according to an exemplary embodiment of the present disclosure. Notably, FIG. 2 is similar to FIG. 1, except that in FIG. 2, the contextual text string 12 is additionally input to the base language model 202. The base language model 202, in some implementations, is configured to generate a base output 204 directly from the contextual text string 12 without generating an intermediate text string. As shown in FIG. 2, the base output 204 is combined (e.g., appended or concatenated to) the contextual text string 12, and the combined string is then input to the machine-learned language model 14.

[0047] Use of base language model 202 in this manner can enable the role of machine-learned language model 14 to change to an error-correcting or "fact-checking" role. Notably, in FIG. 1, model 14 is primarily responsible for generating output text 18. In contrast, in FIG. 2, the role of model 14 may be to supplement or correct facts contained in base output 204. In this manner, an existing base language model 202 can be extended or leveraged through the addition of model 14 with access to structural tools 15.

[0048] For example, this allows the application of models 14 and tools 15 to any number of different existing base models that have already been trained for different tasks, contexts, domains, users, applications, etc. Thus, any application that already has a custom language model can be combined with additional models 14 to provide improved use of factual or up-to-date information when generating contextual language output.

[0049] 3 illustrates a block diagram of an example training process for a machine-learned language model that generates text analytics in the service of contextual text generation, according to an example embodiment of the present disclosure. Specifically, FIG. 3 illustrates a supervised training approach.

[0050] As shown in FIG. 3 , several training text tokens 312 may be obtained. Some of the training text tokens 312 may be input to a language model 314. The model 314 may predict a next predicted text token 316 for the training text token 312. For example, the next predicted text token may be an example intermediate text token or an example output token. The next predicted text token 316 may be compared to the ground truth text tokens included in the training text tokens 312 using a loss function 318. Parameters of the model 314 may be updated based on the loss function 318 (e.g., a logarithmic loss function or similar). The process shown in FIG. 3 may be performed iteratively and successively for each text token included in the training text tokens 312. For example, one way to do this is to compare the token sequence [ <tokenized-context>, EOS, <tokenized-reasoning>, EOS, <tokenized-response>The goal is to train a left-to-right language model on [[, EOS]]. The process can be performed over several different training examples.

[0051] In some examples, training data may be collected through a volunteer / crowdworker interface. As an example, a human annotator may interact with the base language model. After the base language model issues a response, the human annotator may open a feedback interface that allows the user to enter an "interim analysis" and / or edit the output of the base language model's response. The "interim analysis" may begin with a call to the base language tool, whose output may include several example responses to the current context generated by the base language model. The human annotator may then add an interim analysis that may include additional tool usage. To facilitate tool usage, a form may be provided that allows the human annotator to query the tool, and the tool may have a button for adding tool input / output to the interim analysis. When the human annotator finishes adding the interim analysis, the human annotator optionally modifies the base model's response and clicks "save," which returns the human annotator to the conversation interface. The changes are reflected in the base model's response, and it is the human annotator's turn to say something. In some implementations, the base model may be considered simply another tool that is used.

[0052] 4 illustrates a block diagram of an exemplary training process for a machine-learned language model that generates text analytics in the service of contextual text generation, according to an exemplary embodiment of the present disclosure. Specifically, FIG. 4 illustrates an enhanced training approach.

[0053] As shown in FIG. 4, one or more context tokens 412 may be input to a machine-learned language model 414, such as those described in FIGS. 1 and 2. The model 414 may generate an intermediate text string (e.g., which may include accessing or utilizing structural tools 415). Finally, the model 414 may generate one output token 416. A reward function 418 may determine a reward based on the output token 416. The reward function 418 may determine how well the output token 416 met or resulted in some objective satisfaction (e.g., user satisfaction). Optionally, the output token 416 may be provided to a user or other dialogue agent to result in additional new context tokens, which may restart the described process.

[0054] Example text generation with intermediate analysis This section provides some examples of intermediate text analysis that illustrate various exemplary applications.

[0055] Call base agent, then fact check / clarification:

[0056] context:

[0057] Agent: Yes

[0058] User: Hi, can you tell me anything about Mexico?

[0059] Interim analysis:

[0060] <tool: base-pass-context> # automatically gets the context

[0061] <tool-output>

[0062] - Mexico, officially the United Mexican States, is a country in the southern part of North America.

[0063] - Mexico is bordered by the United States to the north, the Pacific Ocean to the south and west, Guatemala, Belize and the Caribbean Sea to the southeast, and the Gulf of Mexico to the east.

[0064] - The President of Mexico is the head of state and the head of the Mexican government.

[0065]

[0066] <tool: assistant>

[0067] Mexico's current president

[0068] <tool-output>

[0069] Andres Manuel Lopez Obrador

[0070]

[0071] response:

[0072] - Mexico, officially the United Mexican States, is a country in the southern part of North America. The current president of Mexico is Andres Manuel Lopez Obrador.

[0073] End-to-end customer support

[0074] Context 1:

[0075] Customer: Hi, I forgot my password.

[0076] Interim analysis 1:

[0077] <tool internet provider API>

[0078] What to do if you forget your password

[0079] <tool-output>

[0080] Go to internetprovider.com / password

[0081]

[0082] Output 1:

[0083] Agent: Yes. I'm sorry this happened to you. You

[0084] You can reset your password at internetprovider.com / password

[0085] Context 2:

[0086] Customer: Ok... it worked, but my internet is slow

[0087] Interim analysis 2:

[0088] <tool internet provider API>

[0089] What to do when your internet is slow

[0090] <tool-output>

[0091] Measure your speed with a custom internet provider meter

[0092]

[0093] <tool meter>

[0094] customer id 777

[0095] <tool-output>

[0096] 1234Mbps

[0097]

[0098] <tool cust-acct>

[0099] Customer ID 777

[0100] <tool-output>

[0101] John Doe

[0102] Internet Provider Final Plan

[0103] Expected speed: 1000Mbps

[0104]

[0105] Output 2:

[0106] Agent: Okay. I see you purchased the 1000Mbps package and have 1234Mbps. Everything is going well. Would you like to renew?

[0107] Exemplary Devices and Systems 5A illustrates a block diagram of an exemplary computing system 100 according to an exemplary embodiment of the present disclosure. The system 100 includes a user computing device 102, a server computing system 130, and a training computing system 150, which are communicatively coupled via a network 180.

[0108] The user computing device 102 may be any type of computing device, such as, for example, a personal computing device (e.g., a laptop or desktop), a mobile computing device (e.g., a smartphone or tablet), a gaming console or controller, a wearable computing device, an embedded computing device, or any other type of computing device.

[0109] The user computing device 102 includes one or more processors 112 and memory 114. The one or more processors 112 may be any suitable processing device (e.g., a processor core, a microprocessor, an ASIC, an FPGA, a controller, a microcontroller, etc.) and may be a single processor or multiple operatively connected processors. The memory 114 may include one or more non-transitory computer-readable storage media such as RAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, etc., and combinations thereof. The memory 114 may store data 116 and instructions 118 that are executed by the processor 112 to cause the user computing device 102 to perform operations.

[0110] In some implementations, the user computing system 102 can store or include one or more machine-learned models 120. For example, the machine-learned models 120 can be or otherwise include various machine-learned models, such as neural networks (e.g., deep neural networks) or other types of machine-learned models, including nonlinear and / or linear models. The neural networks can include feedforward neural networks, recurrent neural networks (e.g., long-short-term memory recurrent neural networks), convolutional neural networks, or other forms of neural networks. Some exemplary machine-learned models can leverage attention mechanisms, such as self-attention. For example, some exemplary machine-learned models can include multi-head self-attention models (e.g., Transformer models). Exemplary machine-learned models 120 are discussed with reference to FIGS. 1-4.

[0111] In some implementations, one or more machine-learned models 120 may be received from server computing system 130 over network 180, stored in user computing device memory 114, and then used or otherwise implemented by one or more processors 112. In some implementations, user computing device 102 may implement multiple parallel instances of a single machine-learned model 120 (e.g., to perform parallel language generation across multiple instances of a language generation task).

[0112] Additionally or alternatively, one or more machine-learned models 140 may be included in or otherwise stored and implemented by a server computing system 130 that communicates with the user computing device 102 according to a client-server relationship. For example, the machine-learned models 140 may be implemented by the server computing system 130 as part of a web service (e.g., a language generation service such as a question-answering service, a dialogue service (e.g., used by a “chatbot” or digital assistant), etc.). Thus, one or more models 120 may be stored and implemented at the user computing device 102, and / or one or more models 140 may be stored and implemented at the server computing system 130. The models 120 and / or 140 may be used by any language generation service, such as a question-answering service, a dialogue service (e.g., used by a “chatbot” or digital assistant), etc.

[0113] The user computing device 102 may also include one or more user input components 122 that receive user input. For example, the user input component 122 may be a touch-sensitive component (e.g., a touch-sensitive display screen or touchpad) that is sensitive to the touch of a user input object (e.g., a finger or stylus). The touch-sensitive component may be useful for implementing a virtual keyboard. Other exemplary user input components include a microphone, a traditional keyboard, or other means by which a user can provide user input.

[0114] The server computing system 130 includes one or more processors 132 and memory 134. The one or more processors 132 may be any suitable processing device (e.g., a processor core, a microprocessor, an ASIC, an FPGA, a controller, a microcontroller, etc.) and may be a single processor or multiple operatively connected processors. The memory 134 may include one or more non-transitory computer-readable storage media such as RAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, etc., and combinations thereof. The memory 134 may store data 136 and instructions 138 that are executed by the processor 132 to cause the server computing system 130 to perform operations.

[0115] In some implementations, server computing system 130 includes or is otherwise implemented by one or more server computing devices. In instances in which server computing system 130 includes multiple server computing devices, such server computing devices may operate according to a sequential computing architecture, a parallel computing architecture, or some combination thereof.

[0116] As described above, the server computing system 130 may store or otherwise include one or more machine-learned models 140. For example, the models 140 may be or otherwise include various machine-learned models. Exemplary machine-learned models include neural networks or other multi-layer nonlinear models. Exemplary neural networks include feedforward neural networks, deep neural networks, recurrent neural networks, and convolutional neural networks. Some exemplary machine-learned models may utilize attention mechanisms such as self-attention. For example, some exemplary machine-learned models may include multi-head self-attention models (e.g., Transformer models). Exemplary models 140 are discussed with reference to FIGS. 1-4.

[0117] The user computing device 102 and / or the server computing system 130 can train the models 120 and / or 140 by interacting with a training computing system 150 that is communicatively coupled via a network 180. The training computing system 150 may be separate from the server computing system 130 or may be part of the server computing system 130.

[0118] The training computing system 150 includes one or more processors 152 and memory 154. The one or more processors 152 may be any suitable processing device (e.g., a processor core, a microprocessor, an ASIC, an FPGA, a controller, a microcontroller, etc.) and may be a single processor or multiple operably connected processors. The memory 154 may include one or more non-transitory computer-readable storage media such as RAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, etc., and combinations thereof. The memory 154 may store data 156 and instructions 158 that are executed by the processor 152 to cause the training computing system 150 to perform operations. In some implementations, the training computing system 150 includes or is otherwise implemented by one or more server computing devices.

[0119] The training computing system 150 may include a model trainer 160 that trains the machine-learned models 120 and / or 140 stored at the user computing device 102 and / or the server computing system 130 using various training or learning techniques, such as, for example, backpropagation. For example, a loss function may be backpropagated through the model to update one or more parameters of the model (e.g., based on the gradient of the loss function). Various loss functions may be used, such as mean squared error, likelihood loss, cross-entropy loss, hinge loss, and / or various other loss functions. Gradient descent techniques may be used to iteratively update the parameters for several training iterations.

[0120] In some implementations, performing backpropagation may include performing abbreviated temporal backpropagation. The model trainer 160 may implement several generalization techniques (e.g., weight decay, dropout, etc.) to improve the generalization ability of the trained model.

[0121] In particular, model trainer 160 can train machine-learned model 120 and / or 140 based on a set of training data 162. Training data 162 can include, for example, multiple training tuples. Each training tuple can include an example context text string including one or more context text tokens, one or more example intermediate text strings including one or more intermediate text tokens, and an example output text string including one or more output text tokens.

[0122] In some implementations, if the user provides consent, the training examples may be provided by the user computing device 102. Thus, in such implementations, the model 120 provided to the user computing device 102 may be trained by the training computing system 150 against user-specific data received from the user computing device 102. In some cases, this process may be referred to as personalizing the model.

[0123] Model trainer 160 includes computer logic used to provide the desired functionality. Model trainer 160 can be implemented in hardware, firmware, and / or software controlling a general-purpose processor. For example, in some implementations, model trainer 160 includes program files stored on a storage device, loaded into memory, and executed by one or more processors. In other implementations, model trainer 160 includes one or more sets of computer-executable instructions stored on a tangible computer-readable storage medium, such as RAM, a hard disk, or an optical or magnetic medium.

[0124] Network 180 may be any type of communications network, such as a local area network (e.g., an intranet), a wide area network (e.g., the Internet), or some combination thereof, and may include any number of wired or wireless links. Generally, communications over network 180 may be carried over any type of wired and / or wireless connection, using a wide variety of communications protocols (e.g., TCP / IP, HTTP, SMTP, FTP), encodings or formats (e.g., HTML, XML), and / or protection schemes (e.g., VPN, Secure HTTP, SSL).

[0125] 5A illustrates one exemplary computing system that may be used to implement the present disclosure. Other computing systems may be used as well. For example, in some implementations, the user computing device 102 may include a model trainer 160 and a training dataset 162. In such implementations, the model 120 may be both trained and used locally on the user computing device 102. In some such implementations, the user computing device 102 may implement the model trainer 160 to personalize the model 120 based on user-specific data.

[0126] 5B illustrates a block diagram of an exemplary computing device 10 for performing according to an exemplary embodiment of the present disclosure. The computing device 10 may be a user computing device or a server computing device.

[0127] The computing device 10 includes several applications (e.g., applications 1-N). Each application includes its own machine learning library and machine-learned model. For example, each application may include a machine-learned model. Exemplary applications include a text messaging application, an email application, a dictation application, a virtual keyboard application, a browser application, etc.

[0128] 5B, each application may communicate with several other components of the computing device, such as one or more sensors, a context manager, a device state component, and / or additional components. In some implementations, each application may communicate with each device component using an API (e.g., a public API). In some implementations, the API used by each application is specific to that application.

[0129] 5C illustrates a block diagram of an exemplary computing device 50 for performing according to an exemplary embodiment of the present disclosure. Computing device 50 may be a user computing device or a server computing device.

[0130] Computing device 50 includes several applications (e.g., applications 1-N). Each application communicates with a central intelligence layer. Exemplary applications include a text messaging application, an email application, a dictation application, a virtual keyboard application, a browser application, etc. In some implementations, each application can communicate with the central intelligence layer (and the models stored therein) using an API (e.g., a common API across all applications).

[0131] The central intelligence layer includes several machine-learned models. For example, as shown in FIG. 5C , a respective machine-learned model may be provided for each application and managed by the central intelligence layer. In other implementations, two or more applications may share a single machine-learned model. For example, in some implementations, the central intelligence layer may provide a single model for all applications. In some implementations, the central intelligence layer is included in or otherwise implemented by the operating system of the computing device 50.

[0132] The central intelligence layer can communicate with a central device data layer, which can be a centralized repository of data for computing device 50. As shown in FIG. 5C , the central device data layer can communicate with several other components of the computing device, such as one or more sensors, a context manager, a device state component, and / or additional components. In some implementations, the central device data layer can communicate with each device component using an API (e.g., a private API).

[0133] Additional Disclosures The technology discussed herein refers to servers, databases, software applications, and other computer-based systems, as well as actions taken and information sent to and from such systems. The flexibility inherent in computer-based systems allows for a wide variety of possible configurations, combinations, and divisions of tasks and functions among components. For example, the processes described herein may be implemented using a single device or component, or multiple devices or components acting in combination. Databases and applications may be implemented on a single system or distributed across multiple systems. Distributed components may operate sequentially or in parallel.

[0134] While the present subject matter has been described in detail with respect to various specific exemplary embodiments thereof, each example is provided by way of explanation, not limitation, of the present disclosure. Those skilled in the art, upon understanding the above, will be able to readily create modifications, variations, and equivalents of such embodiments. Accordingly, the present disclosure does not exclude the inclusion of such modifications, variations, and / or additions to the present subject matter as would be readily apparent to one skilled in the art. For example, features shown or described as part of one embodiment may be used with another embodiment to yield yet a further embodiment. Accordingly, it is intended that the present disclosure cover such modifications, variations, and equivalents. [Explanation of symbols]

[0135] 10. Computing Devices 12 Contextual Text Strings 14 Language Models 15 Structural Tools 16 Intermediate Text Strings 18 Output Text String 50 computing devices 100 Computing Systems 102 User Computing Devices 112 processors 114 memory 116 Data 118 Command 120 machine learning models 122 User Input Components 130 Server Computing System 132 processors 134 memory 136 Data 138 Command 140 machine learning models 150 Training Computing System 152 processors 154 memory 156 Data 158 Command 160 Model Trainer 162 training data sets 180 Network 202 Base Language Model 204 base output 312 training text tokens 314 Language Models 316 Next predicted text token 318 Loss Function 412 Context Tokens 414 Machine-learned language models 415 Structural Tools 416 Output Token 418 Reward Function < / output>

Claims

1. 1. A computing system for generating contextual text with improved interpretability, comprising: one or more processors; and one or more non-transitory computer-readable media, wherein the one or more non-transitory computer-readable media: a machine-learned language model that performs text analysis in a contextual text generation service; and instructions that, when executed by the one or more processors, cause the computing system to perform operations, the operations comprising: obtaining a contextual text string comprising one or more contextual text tokens; processing the contextual text string with the machine-learned language model to generate one or more intermediate text strings comprising one or more intermediate text tokens; processing the one or more intermediate text strings with the machine-learned language model to generate an output text string comprising one or more output text tokens; The one or more intermediate text strings include text analysis of the contextual text strings to support the output text string.

2. 10. The computing system of claim 1, wherein the one or more intermediate text tokens include at least one tool token that triggers the use of a structural tool to access additional information not included in the contextual text string.

3. The computing system of claim 2 , wherein the structural tool includes a database search to access additional information from a database.

4. 4. The computing system of claim 2 or 3, wherein the structural tool includes an application programming interface (API) call to request and receive additional information via the API.

5. 5. The computing system of claim 2, wherein the structural tool comprises a programming language interpreter that performs a sequence of one or more operations on input text tokens.

6. 6. The computing system of claim 2, wherein the structural tool comprises a query service that queries results from a search engine, a knowledge graph, or a digital assistant.

7. When the machine-learned language model generates the tool token, the operation interrupting the machine-learned language model; executing the structure tool to access the additional information; appending said additional information to a current version of said one or more intermediate text strings; and resuming text generation using the machine-learned language model based on the current version of the one or more intermediate text strings and the added additional information.

8. 8. The computing system of claim 1, further comprising processing the output text string with a speech synthesis system to generate an audio output.

9. 9. The computing system of claim 1, wherein the machine-learned language model operates on a token-by-token basis and recursively receives as input each generated intermediate text token when generating the one or more intermediate text strings.

10. processing the contextual text string with the machine-learned language model to generate one or more intermediate text strings comprising one or more intermediate text tokens; For the first iteration, processing the contextual text string with the machine-learned language model to generate a first intermediate text string including one or more intermediate text tokens; appending the first intermediate text string to the contextual text string to generate an updated contextual text string; for each of one or more additional iterations until the machine-learned language model outputs a closure token; processing the updated contextual text string with the machine-learned language model to generate an additional intermediate text string including one or more intermediate text tokens; and appending the additional intermediate text string to the updated contextual text string to generate the updated contextual text string for a next iteration.

11. 11. The computing system of claim 1, wherein the machine-learned language model is trained on a plurality of training tuples, each training tuple including an example context text string, one or more example intermediate text strings, and an example output text string.

12. The computing system of claim 11 , wherein at least the one or more example intermediate text strings are generated by a human labeler.

13. 13. The computing system of claim 1, wherein the machine-learned language model comprises a question-answering model and the contextual text string comprises a question.

14. 14. The computing system of claim 1, wherein the machine-learned language model comprises a dialogue model and the contextual text string comprises a dialogue history.

15. The machine-learned language model is Recurrent neural networks, Multi-head self-attention model, or 15. A computing system according to any one of claims 1 to 14, comprising a sequence-to-sequence model.

16. 16. The computing system of claim 1, wherein at least a portion of the contextual text string includes text entered by a user, and wherein the actions further include providing at least the output text string for display to the user.

17. 17. The computing system of claim 1, wherein the contextual text string comprises the original contextual text string concatenated with a base output generated by a machine-learned language model configured to generate the base output directly from the original contextual text string without generating intermediate text strings.

18. 1. A computer-implemented method for improved contextual text generation, the method comprising: obtaining a plurality of training tuples, each training tuple including an example context text string including one or more context text tokens, one or more example intermediate text strings including one or more intermediate text tokens, and an example output text string including one or more output text tokens; For each training tuple, inputting at least a portion of the training tuples into a language model; receiving a predicted next token as an output of the language model, the predicted next token being generated by the language model by processing the portion of the training tuples; evaluating a loss function that compares the predicted next tokens generated by the language model with the actual next tokens included in the training tuples; and modifying one or more values ​​of one or more parameters of the language model based on the evaluation of the loss function.

19. 20. The computer-implemented method of claim 18, wherein for each training tuple, the inputting, receiving, evaluating, and modifying steps are performed for each token included in the one or more exemplary intermediate text strings and the exemplary output text string.

20. 20. The computer-implemented method of claim 18 or 19, wherein for at least one of the training tuples, the one or more intermediate text tokens include at least one tool token that triggers the use of a structural tool to access additional information not included in the example contextual text string, the additional information being included in the one or more intermediate text tokens.

Images