Machine learning language model generating intermediate text analytics to serve contextual text generation

Abstract

The invention relates to a machine learning language model generating intermediate text analysis to serve contextual text generation. For example, a computing system may obtain a contextual text string comprising one or more contextual text lems. The computing system may process the contextual text string using a machine learning language model to generate one or more intermediate text strings including one or more intermediate text lems. The computing system may process the one or more intermediate text strings using a machine learning language model to generate an output text string that includes one or more output text lems. The one or more intermediate text strings may include text analysis of contextual text strings that support the output text string.

Description

Description of the case

[0001] This application is a divisional application of Chinese invention patent application 202280027602.9, with a filing date of May 20, 2022. Related Applications

[0002] This application claims priority to and the benefit of U.S. Provisional Patent Application No. 63 / 191,563, filed on May 21, 2021. The entire contents of U.S. Provisional Patent Application No. 63 / 191,563 are incorporated herein by reference. Technical Field

[0003] The present disclosure generally relates to the use of machine learning for language modeling. More specifically, the present disclosure relates to generating a machine learning language model that serves intermediate text analysis (e.g., including using structural tools such as APIs) for contextual text generation. Background Art

[0004] Natural language processing (NLP) has made rapid progress in recent years, and such progress is mainly attributed to improvements in learning-based algorithms and other aspects of machine or "neural" learning. A specific task within the field of NLP is contextual text generation. In a contextual text generation task, an agent (e.g., a machine learning model) is tasked with generating output text from a given context. For example, a given context may include one or more context text strings. Thus, in some example approaches to the contextual text generation task, a text-to-text model reads an input context text and then directly produces an output text.

[0005] An example of a contextual text generation task is a question answering task. In a question answering task, the question is the input context and the desired output is the answer to the question. Another example of a contextual text generation task is dialogue generation. In dialogue generation, the input context is the conversation history record and the desired output is the next utterance, where the next utterance is responsive to the context of the conversation history record or otherwise makes sense within the context of the conversation history record.

[0006] Current state-of-the-art models for contextual text generation tend to be Transformer-based neural models, which: or left-to-right language models like GPT3 (Brown et al. Language Models are Few-ShotLearners, arXiv:2005.14165), where “ <input> <output>(<input><output>)” is treated as a sequence; or a sequence-to-sequence model, like the original Transformer (Vaswani et al., Attention is All You Need, arXiv:1706.03762).

[0007] However, neural language models such as GPT3 and Transformers suffer from a number of shortcomings. Specifically, while neural language models display significant intelligence, their knowledge is limited to information contained in (and learned from) their training datasets and / or information introduced within contextual text inputs. As a result, their knowledge of factual information is severely limited and is often frozen in time. As a result, when requested to produce an output containing factual information, the model often either produces incorrect facts or supplies outdated information. Reliance on incorrect factual information can lead to inefficiencies, where incorrect actions (e.g., computerized actions) are taken and need to be corrected or otherwise remedied, resulting in redundant and unnecessary use of resources (e.g., computing resources).

[0008] As another example shortcoming of neural language models, it is difficult to interpret or understand their output. Specifically, because such models typically generate output directly from input, it is difficult to understand exactly why such models generated the output or what aspects of the input led to the output. The lack of interpretability of language model outputs may lead to a lack of confidence or reliance on the model output, which may lead to unnecessary overhead or other efforts (e.g., computerized operations) attempting to "double-check" the accuracy or usefulness of the model output. Summary of the invention

[0009] Aspects and advantages of the embodiments of the present disclosure will be set forth in part in the description which follows, or may be learned from the description, or may be learned through practice of the embodiments.

[0010] An example aspect of the present disclosure relates to a computing system for contextual text generation with improved interpretability. The computing system includes one or more processors and one or more non-transitory computer-readable media, which together store: a machine learning language model that performs text analysis to serve contextual text generation; and instructions that cause the computing system to perform operations when executed by the one or more processors. These operations include: obtaining a context text string including one or more context text tokens. These operations include: processing the context text string using the machine learning language model to generate one or more intermediate text strings including one or more intermediate text tokens. These operations include: processing the one or more intermediate text strings using the machine learning language model to generate an output text string including one or more output text tokens. The one or more intermediate text strings include text analysis of the context text string that supports the output text string.

[0011] Another example aspect of the present disclosure relates to a computer-implemented method for improved contextual text generation.

[0012] The method includes: obtaining a plurality of training tuples, each training tuple including: an example context text string including one or more context text tokens; one or more example intermediate text strings including one or more intermediate text tokens; and an example output text string including one or more output text tokens. The method includes, for each training tuple: inputting at least a portion of the training tuple into a language model; receiving a predicted next token as an output of the language model, the predicted next token being generated by the language model by processing a portion of the training tuple; evaluating a loss function, the loss function comparing the predicted next token generated by the language model with an actual next token included in the training tuple; and modifying one or more values ​​of one or more parameters of the language model based on the evaluation of the loss function.

[0013] Other aspects of the disclosure relate to various systems, apparatus, non-transitory computer-readable media, user interfaces, and electronic devices.

[0014] These and other features, aspects and advantages of various embodiments of the present disclosure will be better understood with reference to the following description and appended claims.The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate example embodiments of the present disclosure and, together with the description, serve to explain relevant principles. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] A detailed discussion of embodiments for those of ordinary skill in the art is set forth in the specification with reference to the accompanying drawings, in which:

[0016] Figure 1 Depicted is a block diagram of an example machine learning language model for generating text analysis for contextual text generation according to an example embodiment of the present disclosure.

[0017] Figure 2 Depicted is a block diagram of an example machine learning language model for generating text analysis for contextual text generation according to an example embodiment of the present disclosure.

[0018] Figure 3 A block diagram of an example training process for generating a machine learning language model for text analysis serving contextual text generation according to an example embodiment of the present disclosure is depicted.

[0019] Figure 4 A block diagram of an example training process for generating a machine learning language model for text analysis serving contextual text generation according to an example embodiment of the present disclosure is depicted.

[0020] Figure 5A A block diagram of an example computing system is depicted according to an example embodiment of the present disclosure.

[0021] Figure 5B A block diagram of an example computing device is depicted according to an example embodiment of the present disclosure.

[0022] Figure 5C A block diagram of an example computing device is depicted according to an example embodiment of the present disclosure.

[0023] Reference numerals that are repeated across multiple drawings are intended to identify like features in the various embodiments. DETAILED DESCRIPTION Overview

[0024] In general, the present disclosure relates to systems and methods that include and / or make full use of one or more machine learning language models, which generate intermediate text analysis that serves contextual text generation (e.g., including using structural tools such as APIs). For example, a computing system can obtain a context text string including one or more context text tokens. The computing system can process the context text string using a machine learning language model to generate one or more intermediate text strings including one or more intermediate text tokens. The computing system can process the one or more intermediate text strings using a machine learning language model to generate an output text string including one or more output text tokens. The one or more intermediate text strings may include text analysis of the context text string that supports the output text string.

[0025] Therefore, aspects of the present disclosure improve the knowledge, foundation, and interpretability of machine learning language models by teaching the model to generate textual analysis before (e.g., serving) generating output text (e.g., generating a response to a question or a previous conversation) in response to contextual text input. The generation of such intermediate textual analysis can improve the interpretability of the model output. In particular, the intermediate textual analysis can be reviewed or checked to explain or understand how the model generates output in response to contextual input. This can also facilitate the evaluation of the reliability and / or applicability of the output for serving a specific task.

[0026] According to another aspect of the present disclosure, in some embodiments, text analysis may include and / or make full use of the use of structural tools that provide access to additional information. For example, one or more intermediate text tokens included in the intermediate text analysis may include at least one tool token that calls the use of structural tools to access additional information that is not included in the context text string and / or within the training data on which the model is trained. Therefore, the language model can call on and use such structural tools to have access to additional information that may be up-to-date, factual, domain-specific, client- or user-specific, etc. This improves the knowledge available to the language model when formulating text output, and further improves the flexibility of the system by enabling the introduction of various information sources for various use cases. The means of the present disclosure can achieve improved or optimized integration with external tools, because machine learning processes can be applied to minimize the computational overhead of calling such services; for example, tool tokens and the order in which they are generated can be adjusted to minimize computational overhead, such as waiting time and / or network usage.

[0027] As examples, the structural tools to which a machine learning language model may have access include: database lookups for accessing additional information from a database; application programming interface (API) calls for requesting and receiving additional information via the API; a programming language interpreter that performs a sequence of one or more operations on input text tokens; a query service that queries results from a search engine, knowledge graph, or digital assistant; a communication client that creates a communication (e.g., an email, a short message service message, a multimedia messaging service message, an application-based chat message, etc.) and transmits it to another device or user; and / or, various other forms of structural tools that generate additional information or otherwise provide access to additional information. Thus, structural tools are not limited to finding information, but may also have side effects or result in actions (e.g., booking a meeting, purchasing something, submitting a ticket to a human, etc.).

[0028] The machine learning language model described herein can be trained by a variety of different means. In one example, a human volunteer or group worker can generate example intermediate analysis text for multiple (e.g., thousands) examples. For example, a human worker can be given a pair of context input text and output text, and the human worker can generate an intermediate analysis text that demonstrates an analysis of the context input text that leads to or otherwise supports the output text. Human workers can be given access to structural tools and their use of such tools, and the corresponding information obtained can be included in the example intermediate analysis text.

[0029] The intermediate analysis (in the training examples or generated by the model) may contain step-by-step logic in human-readable form in some instances, such as a multi-step solution to an algebraic problem. It may also contain the use of external text-to-text tools, such as databases, Python interpreters, search engines, etc., as described elsewhere herein. In some embodiments, the use of tools in the intermediate analysis section may be marked and / or triggered by specific tags that specify which tool to use and describe the input and output of the tool. The intermediate text may contain multiple instances of tool use, as well as any amount of free-form text.

[0030] Thus, intermediate analysis may include using tools (e.g., APIs) that take a structured list of input parameters and return a structured output (e.g., do_thing(a: int, b: List[str]) --> response_object). However, any structured input / output can also be serialized / parsed to / from free text using some serialization method - such as Google Protos or JSON's text serialization. From this perspective, the text-to-text interface may be a superset of the structured interface.

[0031] To generate an example training data set, the example intermediate analysis text generated by the human annotator can be combined with the context input text and output text pairs to form a training tuple, which includes: an example context text string including one or more context text tokens; one or more example intermediate text strings including one or more intermediate text tokens; and, an example output text string including one or more output text tokens. Therefore, in some embodiments, the human annotator can be given an example context text string and an example output text string, and the human annotator can generate an example intermediate text string. In other embodiments, the human annotator can be given only an example context text string, and the human annotator can generate both an example intermediate text string and an example output text string.

[0032] A training data set such as that described above can be used to train a language model. For example, a training data set can be used to fine-tune a model that has been pre-trained on trillions of unsupervised data. As an example, a model can be trained using the model to predict the next token contained in a training tuple (e.g., the next intermediate text token or the next output text token). A loss function can be used to evaluate the model's ability to predict the next token. The parameters of the model can be updated based on the loss function (e.g., via a back-propagation-based technique). In some embodiments, training the model on each training tuple can include iteratively training each token contained in the intermediate text string on a token-by-token basis, followed by each token contained in the output text string.

[0033] In another example, a language model can be trained using reinforcement learning to generate intermediate text that serves contextual language generation. For example, various aspects of the intermediate text and / or output text generated by the model can be evaluated by an objective function to determine a reward, which can then be used to update model parameters.

[0034] Then, at inference time, the language model can be used to generate intermediate analyses given the input. In some embodiments, the intermediate analyses can be generated one token at a time. In some embodiments, each time the model finishes generating input to an external tool, the tool itself is called with the input to generate tool output, which is appended to the intermediate text, and the model continues generating from there.

[0035] Therefore, aspects of the present disclosure propose to expand the (input, output) training examples and language generation paradigm to have an intermediate analysis part, which is also text. In this way, instead of simply generating an output given an input, a language model can learn to generate an intermediate analysis given an input, and then generate an output given the input and the intermediate analysis.

[0036] Thus, while a conversational agent (or any other contextual text generation model) is typically trained on (context, response) pairs so that it can directly generate responses for a given context, in example embodiments of the present disclosure, a language model may instead be trained on (context, intermediate analysis, response) tuples, and the model learns to generate (intermediate analysis | context) and (response | context, intermediate analysis).

[0037] In some embodiments, the output text generated as described herein can be further processed using a text-to-speech system to generate an audio output. As another example, a speech-to-text system can be used to generate input text from an audio input. For example, a virtual assistant can interact with a user via audio input and output, and the conversion of audio / speech to text can be used to enable the processing of the virtual assistant to occur in the text domain, as described herein.

[0038] The systems and methods of the present disclosure provide many technical effects and benefits. As an example, the proposed model demonstrates improved interpretability. For example, the intermediate text analysis generated by the model can be reviewed or checked to explain or understand how the model generates output in response to contextual input. Improved interpretability can lead to more efficient use of computing resources, such as processor usage, memory usage, etc. For example, the lack of interpretability on the language model output may lead to a lack of confidence or reliance on the model output, which can lead to unnecessary overhead or other work (such as computerized operations) in attempting to "double-check" the accuracy or practicality of the model output. By improving interpretability, confidence in computerized systems can be improved. Specifically, the reliability of the model output can be verified and / or evaluated to establish the usability of the system for a specific task.

[0039] As another example technical effect and benefit, the proposed means enables the language model to utilize structural tools to access additional information, such as additional factual information. Thus, the language model can call upon and use such structural tools to have access to additional information, which may be up-to-date, factual, domain-specific, client- or user-specific, etc. This improves the knowledge available to the language model when formulating text output, and further improves the flexibility of the system by enabling the introduction of various information sources for various use cases.

[0040] In addition to improving the quality of the model output, the use of the proposed structural tools also leads to savings in computing resources, such as processor usage, memory usage, network bandwidth, etc. Specifically, the knowledge available to previous language models is limited to the information contained in (and learned from) their training datasets and / or the information introduced in the contextual text input. Therefore, their understanding of factual information is severely limited and is often frozen in time. As a result, when asked to produce output containing factual information, the model often either produces incorrect facts or supplies outdated information. Therefore, it is necessary to retrain the entire language model so that the language model remains up to date with changing real-world facts, port the language model to a new domain or set of user information, or otherwise deploy the model in a new situation where new information becomes a problem. Retraining of the language model requires the use of computing resources, such as processor usage, memory usage, network bandwidth, etc.

[0041] However, the use of the structural tools proposed by the present disclosure eliminates the need to retrain the model in order to keep the language model up to date with changing real-world facts, to port the language model to a new domain or set of user information, or to otherwise deploy the model in a new situation where new information becomes a problem. Instead, the model can simply be given access to additional information (e.g., via the structural tools), which may be up-to-date, factual, domain-specific, client- or user-specific, etc. Thus, the model can be easily ported to different domains, uses, users, etc. and / or can provide responses that take advantage of the latest factual information without having to retrain the model, thereby significantly saving computing resources. By encoding the context in the form of an intermediate analysis that can interact with (potentially external) information sources, the process can help address technical limitations in providing information and / or functionality.

[0042] Similarly, another example technical effect is derived from the model's ability to fully exploit external sources to obtain information, rather than needing to store (e.g., in the form of learned relationships) all the information needed to respond to various inputs. In particular, past approaches required the storage and use of large models of sufficient size (e.g., number of parameters) (e.g., on user devices with limited memory and / or battery availability) to learn and store relationships among various inputs and outputs. In contrast, some example embodiments of the present disclosure may enable "thin" (smaller) models to exist on a user device or other mobile client or browser. Thin models may fully exploit various structural tools (e.g., cloud services) to save battery, computation, storage, updates, and the like. Therefore, smaller models with access to structural tools may achieve similar or better performance than large self-contained models, thereby saving computing resources, such as memory usage, network bandwidth, energy consumption, and the like.

[0043] Referring now to the accompanying drawings, example embodiments of the present disclosure will be discussed in greater detail. Generate an example language model for intermediate text analysis

[0044] Figure 1 A block diagram of an example machine learning language model 14 for generating text analysis serving context text generation according to an example embodiment of the present disclosure is depicted. Specifically, the language model 14 may receive a context text string 12 including one or more context text tokens. The language model 14 may process the context text string 12 to generate one or more intermediate text strings 16 including one or more intermediate text tokens. The language model 14 may process the one or more intermediate text strings 16 to generate an output text string including one or more output text tokens 18. The one or more intermediate text strings 16 may include text analysis of the context text string 12 that supports, causes, proves, or otherwise demonstrates a logical analysis of the context text string 12 to generate an output text string 18.

[0045] According to one aspect of the present disclosure, in some embodiments, one or more intermediate text tokens 16 may include at least one tool token that invokes the use of a structure tool 15 to access additional information that is not included in the context text string 12. In some embodiments, the structure tool 15 may include a database lookup to access additional information from a database. In some embodiments, the structure tool 15 may include an application programming interface (API) call to request and receive additional information via an API. In some embodiments, the structure tool 15 may include a programming language interpreter that performs a sequence of one or more operations on an input text token. In some embodiments, the structure tool 15 may include a query service that queries results from a search engine, a knowledge graph, or a digital assistant. In addition to tool tokens, one or more intermediate text tokens 16 may further include at least one natural language text token.

[0046] In some embodiments, when the machine learning language model 14 generates tool words, the computing system may: pause the machine learning language model; execute the structure tool 15 to access additional information; append the additional information to the current version of one or more intermediate text strings 16; and resume text generation using the machine learning language model 14 based on the current version of the one or more intermediate text strings 16 and the appended additional information.

[0047] In some embodiments, the machine learning language model 14 operates on a token-by-token basis. In some such embodiments, when generating one or more intermediate text strings 16, the language model 16 receives each generated intermediate text token 16 as input in a recursive manner.

[0048] Therefore, in some embodiments, processing the context text string 12 with the machine learning language model 14 to generate one or more intermediate text strings 16 including one or more intermediate text tokens can be performed through multiple iterations. At the first iteration, the computing system can utilize the machine learning language model 14 to process the context text string 12 to generate a first intermediate text string 16 including one or more intermediate text tokens. Then, the computing system can append the first intermediate text string 16 to the context text string 12 to generate an updated context text string. Then, for each of one or more additional iterations, and until the machine learning language model outputs an end token, the computing system can utilize the machine learning language model 14 to process the updated context text string to generate an additional intermediate text string 16, which includes one or more intermediate text tokens. The computing system can append the additional intermediate text string to the updated context text string to generate an updated context text string for the next iteration.

[0049] The machine learning language model 14 may be or include various types of models, including, by way of example, a recurrent neural network; a multi-head self-attention model; a sequence-to-sequence model; and / or other forms of language models. The language model may be a cloze model or a left-to-right model. The language model may optionally have an encoder-decoder architecture.

[0050] In some example implementations, the machine learning language model 14 may be a question answering model, and the context text string 12 may be or include a question. In some example implementations, the machine learning language model 14 may be a conversation model, and the context text string 12 may be or include a conversation history.

[0051] In some implementations, at least a portion of context text string 12 includes or corresponds to text entered by a user. In some implementations, the computing system may provide at least output text string 18 for display to a user.

[0052] Figure 2 A block diagram of an example machine learning language model 14 for generating text analysis serving contextual text generation according to an example embodiment of the present disclosure is depicted. Specifically, Figure 2 Similar to Figure 1 , except in Figure 2 In the example, the context text string 12 is additionally input into the basic language model 202. In some embodiments, the basic language model 202 may be configured to generate the basic output 204 directly from the context text string 12 without generating an intermediate text string. Figure 2 As shown, the base output 204 is combined with the context text string 12 (eg, appended to or concatenated with the context text string 12 ), and the combined string is then input into the machine learning language model 14 .

[0053] Using the base language model 202 in this manner can allow the role of the machine learning language model 14 to shift to an error correction or "fact checking" role. Figure 1 In , model 14 is mainly responsible for generating output text 18. In contrast, Figure 2 In the example, the role of the model 14 may be to supplement or correct the facts contained within the base output 204. In this way, an existing base language model 202 may be extended or fully utilized by adding a model 14 having access to the structural tools 15.

[0054] This may, for example, enable the application of the models 14 and tools 15 to any number of different existing-based models that may have been trained for different tasks, contexts, domains, users, applications, etc. Thus, any application that already has a custom language model may be combined with the additional model 14 to provide improved use of factual or up-to-date information when generating contextual language output.

[0055] Figure 3 A block diagram of an example training process for generating a machine learning language model for text analysis serving contextual text generation according to an example embodiment of the present disclosure is depicted. Specifically, Figure 3 A supervised training approach is shown.

[0056] like Figure 3 As shown, a plurality of training text tokens 312 may be obtained. A portion of the training text tokens 312 may be input into a language model 314. The model 314 may predict a next predicted text token 316 of the training text tokens 312. For example, the next predicted text token may be an example intermediate text token, or may be an example output token. The next predicted text token 316 may be compared with the true value text tokens contained in the training text tokens 312 using a loss function 318. The parameters of the model 314 may be updated based on the loss function 318 (e.g., a logarithmic loss function or the like). The above process may be iterated and sequentially performed for each text token contained in the training text tokens 312. Figure 3 For example, one way to achieve this is to use the word sequence [ <tokenized-context>, EOS, <tokenized-reasoning>, EOS, <tokenized-response>, EOS]([<lemmatization context>, EOS, <lemmatization reasoning>, EOS, <lemmatization response>, EOS]). This process can be performed on many different training examples.

[0057] In some examples, training data can be collected via a volunteer / group worker interface. As an example, a human annotator can interact with a base language model. After the base language model issues a response, the human annotator can open a feedback interface that allows the user to input an "intermediate analysis" and / or edit the output of the response of the base language model. The "intermediate analysis" can start with the call to the base language tool, and its output can include several sample responses to the current context generated by the base language model. Then, the human annotator can add an intermediate analysis, which may include additional tool use. In order to facilitate the use of the tool, a form can be provided that allows the human annotator to query the tool, and it has a button that appends the tool input / output to the intermediate analysis. When the human annotator finishes adding the intermediate analysis, the human annotator optionally modifies the response of the base model and clicks "save", which returns the human annotator to the conversation interface. These changes are reflected in the response of the base model, and it is the human annotator's turn to say something. In some embodiments, the base model can be simply regarded as another tool used.

[0058] Figure 4 A block diagram of an example training process for generating a machine learning language model for text analysis serving contextual text generation according to an example embodiment of the present disclosure is depicted. Specifically, Figure 4 Intensive training methods are shown.

[0059] like Figure 4 As shown, one or more context words 412 may be input into a Figure 1 and Figure 2 . Model 414 can generate an intermediate text string (e.g., which can include accessing or making full use of structure tools 415). Ultimately, model 414 can generate one or more output tokens 416. Reward function 418 can determine a reward based on output tokens 416. Reward function 418 can determine the extent to which output tokens 416 satisfy or lead to satisfying a certain goal (e.g., user satisfaction). Optionally, output tokens 416 can be supplied to a user or other interactive agent to generate additional, new context tokens, which can restart the process shown. Example text generation using intermediate analysis

[0060] This section provides some examples of intermediate text analysis, which demonstrate various example applications.

[0061] Calling in the base agent, then doing fact checking / supplementation:

[0062] Context:

[0063] Agent: hi (Agent: Hi)

[0064] User: hi, what can you tell me about Mexico? (User: Hi, what can you tell me about Mexico?)

[0065] Intermediate analysis:

[0066] <tool: base-pass-context> (<tool: base-pass-context>) # Automatically obtain context

[0067] < tool-output > (<tool-output>)

[0068] - Mexico, officially the United Mexican States, is a country in the southern portion of North America. (- Mexico, officially the United Mexican States, is a country in the southern portion of North America.)

[0069] - Mexico is bordered to the north by the United States; to the south and west by the Pacific Ocean; to the southeast by Guatemala, Belize, and the Caribbean Sea; and to the east by the Gulf of Mexico. (- Mexico is bordered to the north by the United States; to the south and west by the Pacific Ocean; to the southeast by Guatemala, Belize, and the Caribbean Sea; and to the east by the Gulf of Mexico.)

[0070] - The president of Mexico is the head of state and head of government of Mexico. (- The president of Mexico is the head of state and head of government of Mexico.)

[0071] ()< / tool>

[0072] < tool: assistant > (<tool: assistant>)

[0073] current president of Mexico

[0074] < tool-output >

[0075] Andrés Manuel López Obrador

[0076]

[0077] response:

[0078] Mexico, officially the United Mexican States, is a country in the southern portion of North America. The current president of Mexico is Andrés Manuel López Obrador.

[0079] End-to-end customer support

[0080] Context 1:

[0081] Cust: hi. I lost my password

[0082] Intermediate analysis 1:

[0083] <tool internet provider API> (<Tools Internet Provider API>)

[0084] how to retrieve lost password

[0085] < tool-output >

[0086] You need to go to internetprovider.com / password

[0087]

[0088] Output 1:

[0089] Agent: Hi. I'm sorry this happened to you! You can reset your

[0090] password at internetprovider.com / password (Agent: Hi. I'm sorry this happened to you! You can reset your password at internetprovider.com / password)

[0091] Context 2:

[0092] Cust: ah ok… so i recovered it, but my internet is slow

[0093] Intermediate analysis 2:

[0094] <tool internet provider API>

[0095] What to do when internet is slow

[0096] <tool-output>

[0097] you just need to measure the speed with the custom Internet Providermeter

[0098]

[0099] <tool meter> (<Tools and Instruments>)

[0100] Customer ID 777

[0101] <tool-output>

[0102] 1234 Mbps

[0103]

[0104] <tool cust-acct> (<Tool Customer Account>)

[0105] Customer id 777

[0106] <tool-output>

[0107] John Doe

[0108] Internet Provider Ultimate Plan

[0109] Expected speed: 1000Mbps

[0110]

[0111] Output 2:

[0112] Agent: ok. I'm seeing that you have bought the 1000 Mbps package, and you're getting 1234 Mbps, so everything is working as intended. Would you like to upgrade? Example devices and systems

[0113] Figure 5A A block diagram of an example computing system 100 is depicted, in accordance with an example embodiment of the present disclosure. System 100 includes a user computing device 102, a server computing system 130, and a training computing system 150 communicatively coupled via a network 180.

[0114] The user computing device 102 may be any type of computing device, such as, for example, a personal computing device (e.g., a laptop or desktop), a mobile computing device (e.g., a smartphone or tablet), a game console or controller, a wearable computing device, an embedded computing device, or any other type of computing device.

[0115] The user computing device 102 includes one or more processors 112 and a memory 114. The one or more processors 112 may be any suitable processing device (e.g., a processor core, a microprocessor, an ASIC, an FPGA, a controller, a microcontroller, etc.) and may be one processor or a plurality of processors operatively connected. The memory 114 may include one or more non-transitory computer-readable storage media, such as RAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, etc., and combinations thereof. The memory 114 may store data 116 and instructions 118 that are executed by the processor 112 to cause the user computing device 102 to perform operations.

[0116] In some implementations, the user computing device 102 may store or include one or more machine learning models 120. For example, the machine learning model 120 may be or may otherwise include various machine learning models, such as a neural network (e.g., a deep neural network) or other types of machine learning models, including nonlinear models and / or linear models. The neural network may include a feedforward neural network, a recurrent neural network (e.g., a long short-term memory recurrent neural network), a convolutional neural network, or other forms of neural networks. Some example machine learning models may make full use of attention mechanisms, such as self-attention. For example, some example machine learning models may include a multi-head self-attention model (e.g., a transformer model). Reference Figure 1-4 Discuss example machine learning model 120.

[0117] In some implementations, one or more machine learning models 120 may be received from the server computing system 130 via the network 180, stored in the user computing device memory 114, and then used or otherwise implemented by the one or more processors 112. In some implementations, the user computing device 102 may implement multiple parallel instances of a single machine learning model 120 (e.g., to perform parallel language generation with multiple instances of a cross-language generation task).

[0118] Additionally or alternatively, one or more machine learning models 140 may be included in or otherwise stored and implemented by a server computing system 130 that communicates with the user computing device 102 according to a client-server relationship. For example, the machine learning models 140 may be implemented by the server computing system 140 as part of a web service (e.g., a language generation service, such as a question-answering service, a conversational service (e.g., used by a "chatbot" or digital assistant), etc.). Thus, one or more models 120 may be stored and implemented at the user computing device 102, and / or one or more models 140 may be stored and implemented at the server computing system 130. Models 120 and / or 140 may be used by any language generation service, such as a question-answering service, a conversational service (e.g., used by a "chatbot" or digital assistant), etc.

[0119] The user computing device 102 may also include one or more user input components 122 that receive user input. For example, the user input component 122 may be a touch-sensitive component (e.g., a touch-sensitive display or touchpad) that is sensitive to the touch of a user input object (e.g., a finger or a stylus). The touch-sensitive component may be used to implement a virtual keyboard. Other example user input components include a microphone, a traditional keyboard, or other means by which a user may provide user input.

[0120] The server computing system 130 includes one or more processors 132 and memory 134. The one or more processors 132 may be any suitable processing device (e.g., a processor core, a microprocessor, an ASIC, an FPGA, a controller, a microcontroller, etc.) and may be one processor or a plurality of processors operatively connected. The memory 134 may include one or more non-transitory computer-readable storage media, such as RAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, etc., and combinations thereof. The memory 134 may store data 136 and instructions 138 that are executed by the processor 132 to cause the server computing system 130 to perform operations.

[0121] In some implementations, server computing system 130 includes or is otherwise implemented by one or more server computing devices. In instances where server computing system 130 includes multiple server computing devices, such server computing devices may operate according to a sequential computing architecture, a parallel computing architecture, or some combination thereof.

[0122] As described above, the server computing system 130 may store or otherwise include one or more machine learning models 140. For example, the model 140 may be or may otherwise include a variety of machine learning models. Example machine learning models include neural networks or other multi-layer nonlinear models. Example neural networks include feedforward neural networks, deep neural networks, recurrent neural networks, and convolutional neural networks. Some example machine learning models may make full use of attention mechanisms, such as self-attention. For example, some example machine learning models may include a multi-head self-attention model (e.g., a transformer model). Reference Figure 1-4 Discuss example model 140 .

[0123] User computing device 102 and / or server computing system 130 may train models 120 and / or 140 via interaction with training computing system 150 communicatively coupled via network 180. Training computing system 150 may be separate from server computing system 130 or may be part of server computing system 130.

[0124] The training computing system 150 includes one or more processors 152 and a memory 154. The one or more processors 152 may be any suitable processing device (e.g., a processor core, a microprocessor, an ASIC, an FPGA, a controller, a microcontroller, etc.) and may be one processor or a plurality of processors operably connected. The memory 154 may include one or more non-transitory computer-readable storage media, such as RAM, ROM, EEPROM, EPROM, flash memory devices, disks, etc. and combinations thereof. The memory 154 may store data 156 and instructions 158 that are executed by the processor 152 to cause the training computing system 150 to perform operations. In some embodiments, the training computing system 150 includes or is otherwise implemented by one or more server computing devices.

[0125] The training computing system 150 may include a model trainer 160 that trains the machine learning models 120 and / or 140 stored at the user computing device 102 and / or the server computing system 130 using various training or learning techniques such as, for example, back propagation of errors. For example, a loss function may be back propagated through the model to update one or more parameters of the model (e.g., based on the gradient of the loss function). Various loss functions may be used, such as mean squared error, likelihood loss, cross entropy loss, hinge loss, and / or various other loss functions. Gradient descent techniques may be used to iteratively update the parameters over multiple training iterations.

[0126] In some implementations, performing back-propagation of errors may include performing truncated back-propagation through time.The model trainer 160 may perform a variety of generalization techniques (eg, weight decay, dropout, etc.) to improve the generalization capabilities of the model being trained.

[0127] In particular, model trainer 160 can train machine learning models 120 and / or 140 based on a set of training data 162. Training data 162 can include, for example, a plurality of training tuples. Each training tuple can include an example context text string including one or more context text tokens, one or more example intermediate text strings including one or more intermediate text tokens, and an example output text string including one or more output text tokens.

[0128] In some implementations, if the user has provided consent, the training examples may be provided by the user computing device 102. Thus, in such implementations, the model 120 provided to the user computing device 102 may be trained by the training computing device 150 on user-specific data received from the user computing device 102. In some instances, this process may be referred to as personalizing the model.

[0129] The model trainer 160 includes computer logic for providing the desired functionality. The model trainer 160 can be implemented in hardware, firmware, and / or software that controls a general purpose processor. For example, in some embodiments, the model trainer 160 includes a program file stored on a storage device, loaded into a memory, and executed by one or more processors. In other embodiments, the model trainer 160 includes one or more sets of computer executable instructions stored in a tangible computer readable storage medium such as RAM, a hard disk, or an optical or magnetic medium.

[0130] Network 180 may be any type of communication network, such as a local area network (e.g., an intranet), a wide area network (e.g., the Internet), or some combination thereof, and may include any number of wired or wireless links. In general, communications on network 180 may be carried via any type of wired and / or wireless connection using a variety of communication protocols (e.g., TCP / IP, HTTP, SMTP, FTP), encodings or formats (e.g., HTML, XML), and / or protection schemes (e.g., VPN, secure HTTP, SSL).

[0131] Figure 5A An example computing system that can be used to implement the present disclosure is illustrated. Other computing systems may also be used. For example, in some embodiments, the user computing device 102 may include a model trainer 160 and a training data set 162. In such embodiments, the model 120 may be trained and used locally at the user computing device 102. In some embodiments of such embodiments, the user computing device 102 may implement the model trainer 160 to personalize the model 120 based on user-specific data.

[0132] Figure 5B A block diagram of an example computing device 10 performing in accordance with an example embodiment of the present disclosure is depicted. Computing device 10 may be a user computing device or a server computing device.

[0133] Computing device 10 includes multiple applications (e.g., applications 1 to N). Each application contains its own machine learning library and machine learning model. For example, each application can include a machine learning model. Example applications include text messaging applications, email applications, dictation applications, virtual keyboard applications, browser applications, etc.

[0134] like Figure 5B As shown, each application can communicate with multiple other components of the computing device, such as, for example, one or more sensors, context managers, device state components, and / or additional components. In some embodiments, each application can communicate with each device component using an API (e.g., a public API). In some embodiments, the API used by each application is specific to the application.

[0135] Figure 5C Depicted is a block diagram of an example computing device 50 that performs in accordance with an example embodiment of the present disclosure. Computing device 50 may be a user computing device or a server computing device.

[0136] The computing device 50 includes a plurality of applications (e.g., applications 1 to N). Each application communicates with a central intelligence layer. Example applications include text messaging applications, email applications, dictation applications, virtual keyboard applications, browser applications, etc. In some embodiments, each application can communicate with the central intelligence layer (and the models stored therein) using an API (e.g., a common API across all applications).

[0137] The central intelligence layer includes multiple machine learning models. For example, Figure 5C As shown, a corresponding machine learning model can be provided for each application and managed by the central intelligence layer. In other embodiments, two or more applications can share a single machine learning model. For example, in some embodiments, the central intelligence layer can provide a single model for all applications. In some embodiments, the central intelligence layer is included in the operating system of the computing device 50 or is otherwise implemented by the operating system of the computing device 50.

[0138] The central intelligence layer can communicate with the central device data layer. The central device data layer can be a centralized data repository for computing devices 50. Figure 5C As shown, the central device data layer can communicate with multiple other components of the computing device, such as, for example, one or more sensors, context managers, device state components, and / or additional components. In some embodiments, the central device data layer can communicate with each device component using an API (e.g., a private API). Additional disclosure

[0139] The technology discussed herein refers to servers, databases, software applications and other computer-based systems and the actions taken and the information sent to and from such systems. The inherent flexibility of computer-based systems allows for multiple possible configurations, combinations and divisions of tasks and functionality between and among components. For example, a single device or component or multiple devices or components working in combination can be used to implement the processes discussed herein. Databases and applications can be implemented on a single system or distributed across multiple systems. Distributed components can operate sequentially or in parallel.

[0140] Although the subject matter has been described in detail with respect to various specific example embodiments of the subject matter, each example is provided by way of explanation, rather than limitation of the present disclosure. Those skilled in the art, after understanding the foregoing, can easily produce changes, variations, and equivalents to such embodiments. Therefore, the subject matter disclosure does not exclude such modifications, variations, and / or additions to the subject matter, which are apparent to those of ordinary skill in the art. For example, a feature illustrated or described as part of an embodiment can be used together with another embodiment to produce yet further embodiments. Therefore, the present disclosure is intended to cover such changes, variations, and equivalents. < / output>

Claims

1. A computing system for contextual text generation with improved interpretability, the computing system comprising: one or more processors; as well as One or more non-transitory computer-readable media that collectively store: a machine learning language model that performs text analysis in service of contextual text generation; as well as instructions that, when executed by the one or more processors, cause the computing system to perform operations comprising: Obtaining a context text string including one or more context text tokens; Processing the context text string using the machine learning language model to generate one or more intermediate text strings including one or more intermediate text tokens, wherein the one or more intermediate text tokens include at least one tool token that invokes the use of a structural tool to access additional information not included in the context text string; processing the one or more intermediate text strings using the machine learning language model to generate an output text string; and The output text string is processed using a text-to-speech system to generate an audio output.

2. The computing system of claim 1, wherein: The structural means include a database lookup for accessing additional information from a database, and / or Wherein, the structural tool includes an application programming interface API call for requesting and receiving additional information via the API, and / or Wherein, the structural tool comprises a programming language interpreter that performs a sequence of one or more operations on input text words, and / or Among them, the structural tools include query services for querying results from search engines, knowledge graphs or digital assistants.

3. The computing system of claim 1, wherein: When the machine learning language model generates the tool word, the operation includes: pausing the machine learning language model; executing the structure tool to access the additional information; appending the additional information to a current version of the one or more intermediate text strings; and Based on the current version of the one or more intermediate text strings and the appended additional information, text generation is resumed using the machine learning language model.

4. The computing system of claim 1, wherein: The machine learning language model operates on a token-by-token basis and, when generating the one or more intermediate text strings, recursively receives each generated intermediate text token as input.

5. The computing system of claim 1, wherein: Processing the context text string using the machine learning language model to generate one or more intermediate text strings including one or more intermediate text tokens includes: For the first iteration: Processing the context text string using the machine learning language model to generate a first intermediate text string including one or more intermediate text tokens; and appending the first intermediate text string to the context text string to generate an updated context text string; and For each of one or more additional iterations, until the machine learning language model outputs an end token: processing the updated context text string using the machine learning language model to generate an additional intermediate text string including one or more intermediate text tokens; and The additional intermediate text string is appended to the updated context text string to generate the updated context text string for the next iteration.

6. The computing system of claim 1, wherein: The machine learning language model has been trained on a plurality of training tuples, each training tuple comprising an example context text string, one or more example intermediate text strings, and an example output text string.

7. The computing system of claim 1, wherein: The machine-learned language model comprises a question answering model, and wherein the context text string comprises a question, and / or Wherein the machine learning language model comprises a conversation model, and wherein the context text string comprises a conversation history.

8. The computing system of claim 1, wherein: The machine learning language model includes: Recurrent Neural Networks; Multi-head self-attention model; or Sequence-to-sequence models.

9. The computing system of claim 1, wherein: At least a portion of the context text string includes text input by a user, and wherein the operations further include providing at least the output text string for display to the user.

10. The computing system of claim 1, wherein: The context text string includes an original context text string that has been concatenated with a basic output generated by a machine learning language model, and the machine learning language model is configured to generate the basic output directly from the original context text string without generating an intermediate text string.

Images