Hint generation for guiding custom machine learning collaboration

Abstract

Systems and methods relate to performing a task based on prompt generation and collaborative interaction with a user using a machine learning model. The machine language model generates a question set based on the task request. And the user interactively answers the question. A task processor generates a set of question and answer pairs based on questions generated by a machine learning model and answers given by a user. A machine learning model generates a task-specific output based on the set of question and answer pairs. The machine learning model represents a large language model with deep learning. Simple question and answer cues enable a non-expert user to indicate a machine learning model with information sufficient to perform a task without making the user unable to operate. The machine learning model uses the answers to accurately execute the task, thereby providing the efficacy of the prompt technology.

Description

Background Art

[0001] It is already common for people to interact with smart devices and computers to perform tasks ranging from simple to complex. Part of interacting with smart devices includes instructing one or more machine learning models implemented in the device to perform tasks. User interaction with machine learning models is based on various prompting techniques used by the device. In practice, it is often difficult for non-expert users to efficiently and accurately instruct machine learning models to perform tasks. The difficulty often stems from the problem that non-expert users are unaware of the specific type of content or instructions required to instruct the machine learning model to generate output that meets the user's expectations. The type and content and instructions vary depending on the specific task.

[0002] In a typical scenario, a user issues a command with some associated data to a device using a machine learning model to perform a task. The user may further instruct the machine learning model to modify (e.g., remove and / or add) output content generated by the machine learning model based on the task. The need to modify the output is often time consuming. Therefore, there is a need for improved prompting techniques to improve the efficacy of using machine learning models to perform tasks, especially when the type of tasks varies.

[0003] With respect to these and other general considerations, various aspects of the disclosure herein have been made.In addition, although relatively specific problems may be discussed, it should be understood that the examples should not be limited to solving specific problems identified in the background or elsewhere in this disclosure. Summary of the invention

[0004] Aspects of the present disclosure relate to performing tasks for generating content, such as documents, videos, emails. In particular, the disclosed technology relates to automatically generating multiple questions based on a given area of ​​the task.

[0005] Various aspects of the present disclosure utilize machine learning models to generate question prompts based on specific tasks. Upon identifying a task, the machine learning model generates task-related questions to collect information from the user that can be used to complete the task. Various aspects of the present disclosure can be used to generate outputs related to various different tasks based on answers received in response to generating question prompts.

[0006] The machine learning model generates a set of question-answer pairs based on a given specific task, and then extracts questions from the set of question-answer pairs to generate question prompts related to the specific task. The task processor provides the question prompt. And receives one or more answers to the question prompt. The task processor generates a modified set of question-answer pairs. The machine learning model uses the modified set of question-answer pairs to generate task-specific outputs.

[0007] This summary is provided to introduce a selection of concepts in simplified form, which will be further described in the following detailed description. This summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter. Additional aspects, features, and / or advantages of the examples will be set forth in part in the following description, and in part will be apparent from the description, or may be learned through practice of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Non-limiting and non-exhaustive examples are described with reference to the following figures.

[0009] Figure 1 An overview of an example system for performing a task by generating a plurality of questions associated with the task in accordance with aspects of the present disclosure is shown.

[0010] Figure 2 Examples of methods for using machine learning models to generate task-specific outputs in accordance with aspects of the present disclosure are shown.

[0011] Figure 3A An example of a method for generating a plurality of questions according to aspects of the present disclosure is shown.

[0012] Figure 3B Examples of methods for generating task-specific outputs according to aspects of the present disclosure are shown.

[0013] Figure 4A An exemplary user interface for displaying question prompts is depicted.

[0014] Figure 4B An exemplary user interface in which answers to question prompts may be received is depicted.

[0015] Figure 4C An exemplary user interface that displays task-specific output in response to receiving an answer is depicted.

[0016] Figure 5 is a block diagram illustrating example physical components of a computing device with which aspects of the present disclosure may be practiced.

[0017] Figure 6 is a simplified block diagram of a mobile computing device with which aspects of the present disclosure may be practiced. DETAILED DESCRIPTION

[0018] Various aspects of the present disclosure are described more fully below with reference to the accompanying drawings, which form a part of the present disclosure and illustrate specific example aspects. However, different aspects of the present disclosure can be implemented in many different ways and should not be construed as being limited to the aspects set forth herein; on the contrary, these aspects are provided so that the present disclosure will be thorough and complete, and will fully convey the scope of the various aspects to those skilled in the art. The various aspects of practice may be methods, systems, or devices. Therefore, the various aspects may take the form of a hardware implementation, a complete software implementation, or an implementation combining software and hardware aspects. Therefore, the following detailed description should not be considered in a limiting sense.

[0019] The effectiveness of the prompting technique depends on the specific type of associated task. Typically, applications or users that employ prompting techniques when working with a machine learning model expect the machine learning model to solve the task as completely as possible. However, most applications or users do not have the expertise required to develop a machine learning model, nor do they have the expertise required to correct the model when the machine learning model produces incorrect output. Lack of expertise limits its adoption to applications involving customization / personalization that require a lot of expert input. In addition, when a machine learning model has been extensively trained for general purposes, it is time-consuming to train a machine learning model for a specific purpose or a specific task.

[0020] Various aspects of the present disclosure relate to using machine learning models to generate question prompts based on specific tasks. When identifying a task, the machine learning model generates task-related questions to collect information from the user that can be used to complete the task. Various aspects disclosed herein can use various different types of generated machine learning models, such as transformer models, autoregressive language models, logic learning machine (LLM) models, etc. Various aspects of the present disclosure can be used to generate outputs related to various different tasks based on answers received in response to question prompts. Examples of tasks include, but are not limited to, generating user biographies, generating travel plans, generating conversations, generating written works (e.g., poems, stories, etc.), generating event summaries, and the like. It will be understood by those skilled in the art that although specific tasks are described herein, various aspects disclosed herein are operable to perform other types of tasks using unsupervised generation models. However, various aspects disclosed herein can also be employed using task-specific models. That is, without departing from the scope of the present disclosure, various different machine learning models trained to perform specific tasks or multiple tasks can be employed to generate any type of task-specific output.

[0021] Aspects of the present disclosure utilize generative models that are trained using a general unsupervised training process. That is, in the examples disclosed herein, a generative machine learning model is employed to perform tasks that the generative model is not specifically trained to perform a specific requested task. That is, the generative model employed herein is operable to perform tasks that the generative model is not specifically trained to perform during the unsupervised training process.

[0022] Figure 1 An overview of an example system 100 for performing a task by generating a plurality of questions associated with the task according to aspects of the present disclosure is shown. The system 100 includes a client computing device 102, a task processor 104, and a network 106 connecting the client computing device 102 and the task processor 104. The task processor 104 is connected to a language model 130. The task processor 104 includes a task request receiver 110, a task prompt generator 112, a question generator 114, an answer receiver 116, a revised question-answer pair generator 118, a task result receiver 120, and a task result transmitter 122.

[0023] The language model 130 can be a large language model, including a question-answer pair generator 132 and a task-specific output generator 134. In an example, the large language model includes an autoregressive model using deep learning based on a transformer network, however, various types of machine learning models can be employed without departing from the scope of the present disclosure. The language model is trained to perform various tasks, including summarizing text, generating question-answer pairs, and answering questions. The training data can include hundreds of billions of tokens.

[0024] The client computing device 102 communicates with the task processor 104 to request the execution of a task. Examples of tasks include generating one or more documents, including but not limited to event notifications, biographies, essays, etc. The client computing device 102 interactively receives a request from a user.

[0025] The task processor 104 performs the task using the language model 130 in response to receiving a request to perform a task from the client computing device 102. The task request receiver 110 receives the task request. The task request specifies the type of task requested to be performed. The task prompt generator 112 generates prompts to the user using the client computing device 102. The prompts may describe the next course of action when performing the task and / or the request information associated with the task. For example, when the task request is to generate a wedding invitation, the prompt is "I am an expert in generating wedding invitations. I ask questions to collect information. Then, I use the information to generate a wedding invitation for you." The task prompt generator 112 may store templates for generating prompts. The task prompt generator 112 sends the generated prompts to the client computing device 102 for display.

[0026] The question generator 114 instructs the language model 130 to generate a set of question-answer pairs associated with the task. For example, when the task is to generate a wedding invitation, the question generator 114 instructs the language model 130 to generate a list of question-answer pairs based on the knowledge base of the language model 130. In the example, the list is exhaustive and the question-answer pairs are mutually exclusive. In various aspects, the language model 130 represents a large natural language processing model, including a deep learning model that has been trained using hundreds of millions of tokens as training data. In various aspects, the question-answer pair generator 132 of the language model 130 generates a set of question-answer pairs that fully covers all aspects of the training data. For example, based on the large training data, the set of question-answer pairs generated for the task of the wedding invitation fully covers the elements that are usually in the wedding invitation. Examples of elements associated with wedding invitations include the names of the bride and groom, the date and location of the wedding, dress codes, etc.

[0027] The question generator 114 receives the set of question-answer pairs from the language model 130 and generates a set of questions from the set of question-answer pairs by extracting only the questions. The question generator 114 then sends the generated list of questions to the client computing device 102 for display and interactive input of answers to the corresponding questions. In various aspects, the questions in the list of questions can be displayed all at once to the user on the client computing device 102. Additionally or alternatively, the questions can be displayed one by one or in a group of questions in a predetermined order. The user is asked to enter the answer to the question.

[0028] The answer receiver 116 receives answers to the corresponding questions from the client computing device 102. In various aspects, the answer receiver 116 receives all answers to all questions in the question set from the client computing device 102 at one time. In some other aspects, the answer receiver 116 receives one or more answers to a part of the question set at one time. Additionally or alternatively, the answer receiver 116 requests answers to one or more unanswered questions with incomplete answers. The answer receiver 116 can use a predetermined rule set to determine whether the received answer is sufficient to be used as part of a question-answer pair to instruct the language model 130 to perform a task. For example, when the answer is irrelevant to the topic within a predetermined level of similarity of the semantics of the topic associated with the question, the answer receiver 116 can determine that the answer is sufficient. In various aspects, the answer receiver determines that the answer is within a predetermined level of similarity of the semantics of the topic by using a language model. In another example, the answer receiver 116 can determine an answer that is not in a predetermined type (e.g., a number) of the question as an incomplete answer to the question.

[0029] The revised question-answer pair generator 118 generates a set of question-answer pairs based on the set of questions and corresponding answers that have been received from the client computing device 102. In various aspects, the questions in the generated set of questions are substantially the same as those in the set of question-answer pairs that have been originally generated by the language model 130. The revised question-answer pair generator 118 generates the set of question-answer pairs by replacing the answers of the originally generated set of question-answer pairs with the answers that have been received from the client computing device 102. Therefore, the revised question-answer pair generator 118 sends the revised set of question-answer pairs to the language model 130 to generate a task-specific output.

[0030] The task-specific output generator 134 receives a revised set of question-answer pairs including answers received from the client computing device 102 (e.g., answers received via a user interface), and generates a task-specific output. For example, the task-specific output generator 134 receives a set of question-answer pairs including answers from a user about a wedding invitation, and generates a wedding invitation as a task-specific output. The language model 130 uses a machine learning model to generate a task-specific output corresponding to a probability with a confidence level above a predetermined threshold. The task-specific output can be in the form of a natural language. For example, the task-specific output generator 134 generates a wedding invitation expressed in a natural language based on the received list of question-answer pairs.

[0031] The task result receiver 120 receives the task-specific output from the task-specific output generator 134 of the language model 130. For example, the task result receiver 120 receives a wedding invitation in natural language text.

[0032] The task result sender 122 sends the task-specific output to the client computing device 102 for display via the network 106. For example, the task result sender 122 sends text data representing a wedding invitation for display on the client computing device 102.

[0033] Figure 2 An example of a method for generating task-specific outputs using a machine learning model in accordance with aspects of the present disclosure is shown. Flow At operation 202, a task-specific request is received. The task-specific request identifies a task to be performed by the machine learning model. For example, the task may be, but is not limited to, generating a user biography, generating a travel itinerary, generating a conversation, generating a written work, generating an event summary, etc. In an example, the task-specific information received at operation 202 of receiving the task-specific request is analyzed to determine the task based on parameters of the request and / or based on a contextual analysis of the request.

[0034] Additionally or alternatively, the operation 202 of receiving a task-specific request receives one or more keywords associated with the task being requested. The one or more keywords improve the accuracy of the list of questions generated by the language model by clarifying the terminology used to specify the task. For example, for a task requesting the generation of a wedding invitation, examples of keywords may include "ceremony" and "reception" to clarify that the invitation is for both the wedding ceremony and the reception. The language model can generate a list of question-answer pairs by considering the keywords.

[0035] At generate question operation 204, information from the received task request and / or context information derived from the task-related request is analyzed using a machine learning model such as a transformer model, an LLM model, etc. The type of model used may differ based on the type of content being generated. In addition, multimodal models may be used, for example, to generate images, audio, video, or other types of content. The machine learning model uses this information to generate a series of questions related to the task. Specifically, the machine learning model generates a list of question-answer pairs related to the task. In doing so, the machine learning model identifies the type of information related to the task and generates questions and answers to the questions based on the relevant type of information. Generate question operation 204 continues to generate a series of questions by extracting questions from the set of question-answer pairs. In the example, the generate question operation discards the answers to the questions because a new set of answers to the questions will be provided by the user. APPENDIX Figure 3A Additional details are provided regarding generating questions based on the requested task.

[0036] In various aspects, at generate answer option list operation 206, answer options associated with the question of the question-answer pair are generated. The answer options can be generated based at least on the answer portion of the question-answer pair. The task performer can present the answer options and their corresponding questions for the user to select. Having answer options enables receiving answers to the questions within a range of widths from the user, thereby enabling the language model to generate accurate and consistent task-specific outputs.

[0037] At receive one or more questions operation 208, method 200 receives one or more questions from the language model for display. In various aspects, one or more questions operation 208 also receives a list of answer options for the one or more questions generated by generate answer option list operation 206.

[0038] At display question operation 210, one or more question prompts including questions are displayed to the user via the client computing device. Figure 4A, an exemplary user interface 400A is provided, which shows a question prompt that can be generated at the generate question operation 204 and displayed at the display question operation 208. In the exemplary user interface 400A, a task request to generate a wedding invitation is received. Based on the exemplary task request, a series of related questions are generated and displayed in the user interface 400A.

[0039] return Figure 1 , the process continues with receive answer operation 212, wherein, in response to displaying one or more question prompts related to the task, method 200 receives answers to the questions. In one example, the answers can be received via user input entered into a user interface. For example, now turning to Figure 4B , shows an exemplary user interface 400B, in which an answer question prompt can be received. As shown in user interface 400B, in response to displaying the task-related question prompt, a text-based answer is received via user interface 400B. Return again Figure 1 , the received answers are aggregated by the device performing method 200 to generate a task-specific output. In some examples, the received answers can be verified. That is, the received answers can be compared with information of the expected type to determine whether the answers are relevant to the question prompt. If not, the user can be prompted to answer the question again until a relevant answer is received. Alternatively or additionally, if no relevant answer is received, the answer can be discarded so that it is not used to perform the requested task.

[0040] Upon receiving an answer to the question prompt, the process continues to operation 214 of generating a revised list of question-answer pairs, wherein a set of question-answer pairs is generated based on the questions posed to the user and the answers to the questions received from the user. In various aspects, the questions are substantially the same as the questions generated by the language model. The answers that have been received from the user replace the answers generated by the language model to generate a revised set of question-answer pairs.

[0041] When the revised list of question-answer pairs is generated, the process continues to generate task-specific output operation 216, where the revised list of question-answer pairs is used to generate task-specific output based on the requested task. In one example, the received answer is provided to the machine learning model. Alternatively or additionally, the machine learning model can also receive task-related parameters from the task request received at operation 202. In one example, the same machine learning model as the model used to generate the question prompt can receive the answer. Alternatively, a different machine learning model trained to perform a specific task can be used at operation 216. In response to receiving the answer, a task-specific output is generated.

[0042] The task-specific output can be extended upon receiving an answer to add additional content that was not specified by the user when generating the task-specific output. For example, go to Figure 4C , an exemplary user interface 400C is provided that displays a task-specific output in response to receiving an answer. As shown in user interface 400C, text for a wedding invitation is generated based on the received answer. The wedding invitation text generated by the machine learning model includes additional content, thereby converting the answer received in response to the question prompt into the requested task-specific output (e.g., a wedding invitation in the depicted example).

[0043] The task-specific output is received by the task processor at receive task-specific output 218. For example, receive task-specific output 218 receives text data representing a wedding invitation.

[0044] Additionally or alternatively, the language model may generate (222) additional questions for refining the task-specific output, and send the additional questions for requesting further answers from the user.

[0045] Additionally or alternatively, the language model may receive (224) user feedback about the task-specific output for modifications desired by the user. The feedback may be in the form of instructions or additional answers to the questions. Additionally or alternatively, the language model may revise (226) the task-specific output based on the answers to the additional questions and / or the user feedback.

[0046] Figure 3A An example of a method for generating multiple questions according to aspects of the present disclosure is shown. Figure 2 Method 3A is performed at operation 210 to generate a question prompt. The process begins at operation 304, where one or more stop conditions are determined based on the task. The stop condition can be a parameter used by the machine learning model that generates the question prompt to determine the question related to the task.

[0047] At operation 306, task-related information (such as information received with the task request, context information derived from the task request, etc.) and / or a stop condition can be provided as input to the machine learning model. For example, the stop condition can include the time when the machine learning model generates more than one question of the same question in the question-answer pair. In other aspects, the stop condition includes the passage of a predetermined time from the start of generating one or more of the question-answer pairs.

[0048] At operation 308, the machine learning model generates one or more questions in a question-answer pair related to the task based on the task information and / or stop condition received as input. The machine learning model analyzes the requested task and determines the type of information that can be used to complete the task. Upon determining the type of relevant information, the machine learning model generates one or more question prompts that can be used to request the relevant information.

[0049] At operation 310, the question prompts generated by the machine learning model can be analyzed to determine their relevance to the task. For example, the questions generated by the machine learning model can be analyzed to determine whether the questions are repeated, redundant, or irrelevant to the requested task. At decision operation 312, method 200 determines whether the questions generated by the machine learning model match the expected output based on the analysis. For example, if the questions generated by the model are irrelevant to the requested task (e.g., asking for information that is not required to perform the task), it is determined that the model does not produce the expected output. In the described scenario, the process branches "no" to operation 314, where the settings of the model are adjusted. For example, the settings of the model can be adjusted by adjusting the control temperature for the model, adjusting the maximum output size for the model, adding additional task-specific instructions to the model (e.g., providing additional information from the task request), adding example questions related to the task as input to the model, etc. The process then returns to operation 206, and the model is executed again to generate task-related question prompts based on the adjusted settings.

[0050] Returning to decision operation 312 , if the model generates the expected output, flow branches “yes” to operation 316 , and the set of question prompts generated by the model is output to an application or displayed to a user (eg, as shown in exemplary user interface 400A).

[0051] Figure 3B An example of a method for generating task-specific outputs according to aspects of the present disclosure is shown. The process begins at start operation 330, followed by operation 332, where the user is prompted with a task-specific question. As described above, the task-specific question can be generated by a machine learning model. It is worth noting that aspects of the present disclosure can be practiced with a general language model. That is, the model used to generate the question prompted at operation 332 does not have to be trained for the specific task requested to be performed. In response to prompting the user (or another application) with the task-specific question, the process continues to operation 334, where an answer is received in response to prompting the user (or another application) with the question. For example, the user can be prompted via a user interface (such as Figure 4B User interface 400B depicted in FIG. 4A ) receives answers to questions.

[0052] At operation 336, the received answers may be analyzed. In one example, the answers may be analyzed to determine whether they are responsive to the prompt question. That is, the answers may be analyzed to determine whether they provide information relevant to the prompted question. The analysis may be performed by comparing the received answers to expected answers or expected types of answers. Additionally or alternatively, the received answers may be analyzed to determine whether two or more of the answers are dependent on each other. Dependent answers may be grouped together when processed to generate task-specific outputs.

[0053] At operation 338, a stop condition is determined based on the type of task. The stop condition determined at operation 338 can be provided as an input to a machine learning model for generating task-specific outputs to determine when the requested task is completed. For example, the stop condition can be a task-related parameter processed by the machine learning model to generate task-specific outputs. By utilizing the generated stop condition and the answer received based on the question prompt, a general machine learning model (e.g., a general language model) can be used to generate task-specific outputs. That is, aspects of the present disclosure are operable to generate task-specific outputs using a machine learning model that is not specifically trained to perform the requested task.

[0054] At operation 340, the answer and the stopping condition are provided as input to a machine learning model to be used to generate a task-specific output. In one example, the machine learning model can be the same model used to generate the question prompt described above. Alternatively, at operation 342, a different machine learning model can receive the answer and stopping condition information. The process then continues to operation 344, where the machine learning model generates a task-specific output in response to receiving the answer and the stopping condition.

[0055] At decision operation 346, a determination is made as to whether the model generated the expected output at operation 344. For example, if the output generated by the model is not relevant to the requested task, then it is determined that the model does not generate the expected output. If the model does not generate the expected output, then the process branches "no" to operation 348, where the settings of the model are adjusted. For example, the settings of the model can be adjusted by adjusting the control temperature of the model, adjusting the maximum output size for the model, adding example task-specific outputs, adding example questions related to the task as input to the model, etc. The process then returns to operation 346, and the model is executed again to generate the task-specific output. Returning to decision operation 346, if the model generates the expected output, then the process branches "yes" to operation 350, where the output is provided to a user or another application. For example, referring again to Figure 4C , task-specific output (e.g., wedding invitations) can be displayed to the user who requested the task.

[0056] Figure 4A An exemplary user interface for displaying a question prompt is depicted. User interface 400A shows "Wedding Invitations" as the requested task. User interface 400A also includes a prompt to the user: "I am an expert at generating wedding invitations. I ask questions to gather information. Then, I use the information to generate a wedding invitation for you." In various aspects, the prompt sets expectations for the user and facilitates communication between the user and the computing device (and therefore the task processor) in natural language. User interface 400A also includes a list of questions that the user is requested to answer. In the example of a task for generating wedding invitations, the question set includes: 1. Who is hosting the wedding? 2. What is the groom's name? 3. What is the bride's name? 4. When is the wedding held? 5. Where is the wedding held? 6. What is the dress code for the wedding? A) Formal; B) Semi-formal; C) Casual". The computing device asks questions to the user to enter answers.

[0057] Additionally or alternatively, questions may be presented with a list of answer options. For example, Question 6, "What is the dress code for a wedding?" may have a list of answer options (e.g., A) formal; B) semi-formal; C) casual). Including a list of answer options enables the user to limit the answer to one of the predetermined phrases in the answer options. In various aspects, the answer choices convey the meaning of the answer more clearly than the answer in free-style text.

[0058] In various aspects, a machine learning model generates a set of question-answer pairs for a given task. A task processor generates a list of questions by extracting questions from the set of question-answer pairs. The task processor may discard answers from the generated set of question-answer pairs. The list of questions is consistent with the execution task because the list of questions is based on the question-answer pairs that the trained machine learning model has generated according to the given task.

[0059] Figure 4B 400B illustrates an example user interface in which answer question prompts may be received. User interface 400B shows an example of a screen in which a user has entered answers to a list of questions associated with a task of generating wedding invitations. Figure 4AIn addition to the prompts shown, the screen also indicates "Please answer the following questions: 1. Who is officiating the wedding? John Doe. 2. What is the groom's name? Jeal Anderson. 3. What is the bride's name? John Doe. 4. When will the wedding be held? August 14, 2022 at 10 am. 5. Where will the wedding be held? Seattle wedding venue. 6. What is the dress code for the wedding? A) Formal". After the questions and corresponding answers, the user interface 400B also indicates the user's command to the task processor: "Command: Write a wedding invitation based on the above questions and answers". In the example, the user has selected "A) Formal" from the three answer options presented to the user by the accompanying questions (e.g., Question 6).

[0060] Figure 4C An exemplary user interface that displays task-specific outputs in response to receiving answers is depicted. User interface 400C illustrates an example command for a task to generate a wedding invitation and task-specific outputs: "Command: Compose a wedding invitation based on the above questions and answers. Task-specific output: John Doe and Jeal Anderson and family invite you to their wedding at a Seattle wedding venue at 10 a.m. on August 14, 2022. Reception to follow. Formal attire (e.g., morning coat) is required."

[0061] In various aspects, the task-specific output includes an enhanced text representation based on the answer given to the question. For example, the language model adds the phrase "morning dress" to the sentence "Formal attire (e.g., morning dress) is required." Since the answer to the question already indicates that the wedding event starts at a morning time (e.g., 10 a.m.), the language model adds "morning dress" as a further inference of formal attire for this particular wedding event.

[0062] Figure 5 is a block diagram illustrating the physical components (e.g., hardware) of a computing device 500 with which aspects of the present disclosure may be practiced. The computing device components described below may be applicable to the computing devices described above. In a basic configuration, the computing device 500 may include at least one processing unit 502 and a system memory 504. Depending on the configuration and type of the computing device, the system memory 504 may include, but is not limited to, volatile storage devices (e.g., random access memory), non-volatile storage devices (e.g., read-only memory), flash memory, or any combination of such memories. The system memory 504 may include an operating system 505 and one or more program tools 506 suitable for performing various aspects disclosed herein. The operating system 505 may, for example, be suitable for controlling the operation of the computing device 500. In addition, aspects of the present disclosure may be practiced in conjunction with a graphics library, other operating systems, or any other application, and is not limited to any particular application or system. The basic configuration is Figure 5508. The computing device 500 may have additional features or functionality. For example, the computing device 500 may also have additional data storage devices (removable and / or non-removable), such as magnetic disks, optical disks, or tapes. Such additional storage devices may be used to store data or other data. Figure 5 , which is illustrated by a removable storage device 509 and a non-removable storage device 510.

[0063] As described above, a plurality of program tools and data files may be stored in system memory 504. When executed on at least one processing unit 502, program tools 506 (e.g., applications 520) may perform processes including, but not limited to, various aspects as described herein. Applications 520 include prompt generator 530, one or more machine learning models 532, and / or prompt user interface 534, as well as instructions for performing various processes disclosed herein. Other program tools that may be used in accordance with various aspects of the present disclosure may include email and contact applications, word processing applications, spreadsheet applications, database applications, slide presentation applications, drawing or computer-aided applications, and the like.

[0064] In addition, various aspects of the present disclosure may be implemented in circuits including discrete electronic components, packaged or integrated electronic chips containing logic gates, circuits utilizing microprocessors, or on a single chip containing electronic components or microprocessors. For example, various aspects of the present disclosure may be practiced via a system on a chip (SOC), wherein Figure 5 Each or more of the components shown may be integrated onto a single integrated circuit. Such a SOC device may include one or more processing units, a graphics unit, a communication unit, a system virtualization unit, and various application functions, all of which are integrated (or "burned") onto a chip substrate as a single integrated circuit. When operated via a SOC, the functions described herein regarding the ability to switch client protocols may be operated via dedicated logic integrated with other components of the computing device 500 on a single integrated circuit (chip). Various aspects of the present disclosure may also be practiced using other technologies capable of performing logical operations, such as AND, OR, and NOT, including but not limited to mechanical, optical, fluid, and quantum technologies. In addition, various aspects of the present disclosure may be practiced within a general-purpose computer or any other circuit or system.

[0065] The computing device 500 may also have one or more input devices 512, such as a keyboard, a mouse, a pen, a sound or voice input device, a touch or sliding input device, etc. Output devices 514 may also be included, such as a display, a speaker, a printer, etc. The aforementioned devices are examples, and other devices may be used. The computing device 500 may include one or more communication connections 516 that allow communication with other computing devices 550. Examples of communication connections 516 include, but are not limited to, radio frequency (RF) transmitters, receivers, and / or transceiver circuits; universal serial bus (USB), parallel and / or serial ports.

[0066] The term "computer-readable medium" as used herein may include computer storage media. Computer storage media may include volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, or program tools). System memory 504, removable storage device 509, and non-removable storage device 510 are all examples of computer storage media (e.g., memory storage devices). Computer storage media may include RAM, ROM, electrically erasable read-only memory (EEPROM), flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical storage device, cassette, magnetic tape, magnetic disk storage or other magnetic storage device, or any other article that can be used to store information and can be accessed by computing device 500. Any such computer storage media may be part of computing device 500. Computer storage media do not include carrier waves or other propagated or modulated data signals.

[0067] Communication media may be embodied by computer readable instructions, data structures, program tools, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and includes any information delivery media. The term "modulated data signal" may describe a signal that has one or more characteristics set or changed in a manner that encodes information in the signal. By way of example, and not limitation, communication media may include wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, radio frequency (RF), infrared, and other wireless media.

[0068] Figure 6 A system 602 of computing devices is shown. Examples of computing devices include, but are not limited to, mobile phones, smart phones, wearable computers (such as smart watches), tablet computers, laptop computers, etc., in which aspects of the present disclosure may be practiced. In some aspects, the client utilized by the user may be a mobile computing device. Figure 6602 is a block diagram illustrating the architecture of one aspect of a computing device, server, mobile computing device, etc. System 602 may be implemented as a "smart phone" capable of running one or more applications (e.g., browser, email, calendar, contact manager, messaging client, game, and media client / player). In some aspects, system 602 is integrated as a computing device, such as an integrated digital assistant (PDA) and a wireless phone.

[0069] One or more application programs 666 may be loaded into the memory 662 and run on or in association with the operating system 664. Examples of application programs include a phone dialer program, an email program, a information management (PIM) program, a word processing program, a spreadsheet program, an Internet browser program, a messaging program, and the like. The system 602 also includes a non-volatile storage area 668 within the memory 662. The non-volatile storage area 668 may be used to store persistent information that should not be lost when the system 602 loses power. The application programs 666 may use and store information in the non-volatile storage area 668, such as emails or other messages used by the email application, and the like. A synchronization application (not shown) also resides on the system 602 and is programmed to interact with a corresponding synchronization application residing on the host computer to keep the information stored in the non-volatile storage area 868 synchronized with the corresponding information stored at the host computer. It should be understood that other applications may be loaded into the memory 662 and run on the computing devices described herein.

[0070] System 602 has a power source 670, which can be implemented as one or more batteries. Power source 670 can also include an external power source, such as an AC adapter or a powered docking station that replenishes or recharges the batteries.

[0071] The system 602 may also include a radio interface layer 672 that performs the functions of sending and receiving radio frequency communications. The radio interface layer 672 facilitates wireless connectivity between the system 602 and the "outside world" via a communications carrier or service provider. Transmissions to and from the radio interface layer 672 are performed under the control of the operating system 664. In other words, communications received by the radio interface layer 672 may be propagated to the application 666 via the operating system 664, and vice versa.

[0072] Visual indicator 620 (e.g., LED) can be used to provide visual notification, and / or audio interface 674 can be used to generate audible notification via audio transducer 625. In the configuration shown, visual indicator 620 is a light emitting diode (LED), and audio transducer 625 is a speaker. These devices can be directly coupled to power supply 670 so that when activated, they remain turned on for the duration indicated by the notification mechanism, even if processor 660 and other components may be turned off to save battery power. LED can be programmed to remain on indefinitely until the user takes action to indicate the power-on state of the device. Audio interface 674 is used to provide audible signals to the user and receive audible signals from the user. For example, in addition to being coupled to audio transducer 625, audio interface 674 can also be coupled to a microphone to receive audible input, such as to facilitate telephone conversations. According to aspects of the present disclosure, the microphone can also be used as an audio sensor to facilitate the control of notifications, as described below. System 602 can also include a video interface 676, which enables the operation of the device connected to peripheral device port 630 to record still images, video streams, etc.

[0073] A computing device implementing system 602 (e.g., Figure 1 The client computing device 102 shown may have additional features or functionality. For example, the computing device may also include additional data storage devices (removable and / or non-removable), such as magnetic disks, optical disks, or tapes. Such additional storage devices may be used to store data or other data. Figure 6 , shown by non-volatile storage area 668.

[0074] As described above, data / information generated or captured by a computing device and stored via system 602 may be stored locally on the computing device, or the data may be stored on any number of storage media that can be accessed by the device via radio interface layer 672 or via a wired connection between the computing device and a separate computing device associated with the computing device (e.g., a server computer in a distributed computing network such as the Internet). It should be understood that such data / information may be accessed by the computing device via radio interface layer 672 or via a distributed computing network. Similarly, such data / information may be easily transferred between computing devices for storage and use according to well-known data / information transmission and storage components, including email and collaborative data / information sharing systems.

[0075] The present disclosure relates to systems and methods for generating task-specific outputs based on answers to one or more prompts. A method includes: receiving a task-specific request; using a machine learning model, generating one or more question prompts based on the task-specific request, wherein the machine learning model is a generative model trained using a general unsupervised training process; displaying one or more question prompts; receiving one or more answers corresponding to the one or more question prompts; and generating task-specific answers based on the one or more question prompts. Generating one or more question prompts also includes: using the machine learning model, generating one or more question-answer pairs based on the task-specific request; and extracting one or more question prompts from the one or more question-answer pairs. Generating one or more question prompts also includes: using the machine learning model, generating one or more question-answer pairs based on the task-specific request; based on the answers in the one or more question-answer pairs, generating one or more answer options associated with the questions corresponding to the answers; and generating one or more question prompts, wherein the one or more question prompts include a question set and one or more answer options associated with the questions. The machine learning model includes a trained language model. The task-specific request includes a keyword that further specifies the task in the task-specific request. The method also includes: determining a level of semantic similarity between one or more answers and a question associated with the answer; determining, based on the semantic similarity, that the answer is insufficient to generate a task-specific answer; generating additional questions associated with the question; and displaying the additional questions. The method also includes: receiving feedback on the task-specific answer; and updating at least a portion of the content of the task-specific answer based on the feedback.

[0076] Another aspect of the present technology relates to a method. The method includes: providing task information related to a requested task as input to a machine learning model; using the machine learning model, generating one or more question prompts related to the requested task, wherein the one or more question prompts relate to information used to complete the requested task, and wherein the machine learning model is a generative model trained using a general unsupervised training process; and using the machine learning model, based on one or more answers associated with the one or more question prompts, generating an output of the requested task. The method also includes: providing one or more stop conditions to the machine learning model. The method also includes: analyzing the one or more question prompts to determine whether the one or more question prompts are related to the requested task. Generating one or more question prompts also includes: using the machine learning model, generating one or more question-answer pairs based on the requested task; and extracting one or more question prompts from the one or more question-answer pairs. Generating one or more question prompts also includes: using the machine learning model, generating one or more question-answer pairs based on the requested task; based on the answers in the one or more question-answer pairs, generating one or more answer options associated with the questions corresponding to the answers; and generating one or more question prompts, wherein the one or more question prompts include a question set and one or more answer options associated with the questions. The machine learning model includes a trained language model. The task information includes keywords, which further specify the requested task. The method also includes: receiving one or more answers; determining a level of semantic similarity between the answer to the one or more answers and the question associated with the answer; determining, based on the semantic similarity, that the answer is insufficient to generate a task-specific answer; generating additional questions associated with the question; and providing the additional questions. The method also includes: receiving feedback on the one or more answers; and updating at least a portion of the content of the one or more answers based on the feedback.

[0077] On the other hand, the present technology relates to a method. The method includes: receiving an answer to a task-specific question prompt; determining one or more stop conditions related to the requested task; and generating a task-specific output using a machine learning model, wherein the machine learning model receives the answer and the task-specific question prompt as input and generates a task-specific output, wherein the machine learning model is a generative model trained using a general unsupervised training process. The machine learning model is not trained to perform the requested task. The method also includes: using the machine learning model to generate one or more question-answer pairs based on the requested task; extracting task-specific question prompts from the one or more question-answer pairs; modifying one or more question-answer pairs by replacing each answer in the one or more question-answer pairs with one or more answers in the received answers; and generating a task-specific output using the machine learning model based on the modified one or more question-answer pairs. The method also includes: using the machine learning model to generate a task-specific question prompt based on the requested task; and displaying the task-specific question prompt.

[0078] The description and illustration of one or more aspects provided in this application are not intended to limit or restrict the scope of the disclosure claimed for protection in any way. The disclosure claimed for protection should not be interpreted as being limited to any aspect or detail provided in the application, for example. Whether combined or shown and described separately, various features (both structure and method) are intended to be selectively included or omitted to produce an embodiment with a specific feature set. After the description and illustration of the application have been provided, those skilled in the art can envision changes, modifications and alternative aspects within the spirit of the broader aspects of the general inventive concept embodied in the application, which do not depart from the broader scope of the disclosure claimed for protection.

[0079] Any one of the one or more aspects described above in combination with any other one of the one or more aspects. Any one of the one or more aspects as described herein.

Claims

1. A method for generating task-specific outputs based on answers to one or more prompts, comprising: Receive task-specific requests; generating one or more question prompts based on the task-specific request using a machine learning model, wherein the machine learning model is a generative model trained using a general unsupervised training process; displaying the one or more question prompts; receiving one or more answers corresponding to the one or more question prompts; as well as A task-specific answer is generated based on the one or more question prompts.

2. The method according to claim 1, wherein generating one or more question prompts further comprises: generating one or more question-answer pairs based on the task-specific request using the machine learning model; as well as The one or more question prompts are extracted from the one or more question-answer pairs.

3. The method according to claim 1, wherein generating one or more question prompts further comprises: generating one or more question-answer pairs based on the task-specific request using the machine learning model; Based on the answers in the one or more question-answer pairs, generating one or more answer options associated with the questions corresponding to the answers; as well as The one or more question prompts are generated, wherein the one or more question prompts include a set of questions and one or more answer options associated with the questions.

4. The method of claim 1, wherein the machine learning model comprises a trained language model. The method of claim 1 , wherein the task-specific request includes a keyword that further specifies a task in the task-specific request.

6. The method according to claim 1, further comprising: determining a level of semantic similarity between an answer to the one or more answers and a question associated with the answer; Based on the semantic similarity, determining that the answer is insufficient to generate the task-specific answer; generating additional questions associated with the question; and The additional questions are displayed.

7. The method according to claim 1, further comprising: receiving feedback on said task-specific answers; as well as Based on the feedback, at least a portion of the task-specific answer is updated.

8. A method comprising: providing task information related to the requested task as input to the machine learning model; generating, using the machine learning model, one or more question prompts related to the requested task, wherein the one or more question prompts relate to information used to complete the requested task, and wherein the machine learning model is a generative model trained using a general unsupervised training process; as well as Using the machine learning model, an output of the task requested is generated based on one or more answers associated with the one or more question prompts.

9. The method of claim 8, further comprising providing one or more stopping conditions to the machine learning model.

10. The method according to claim 8, further comprising: The one or more question prompts are analyzed to determine whether the one or more question prompts are relevant to the requested task.

11. The method of claim 8, wherein generating one or more question prompts further comprises: Using the machine learning model, generating one or more question-answer pairs based on the requested task; as well as The one or more question prompts are extracted from the one or more question-answer pairs.

12. The method of claim 8, wherein generating the one or more question prompts further comprises: Using the machine learning model, generating one or more question-answer pairs based on the requested task; Based on the answers in the one or more question-answer pairs, generating one or more answer options associated with the questions corresponding to the answers; as well as The one or more question prompts are generated, wherein the one or more question prompts include a set of questions and one or more answer options associated with the questions.

13. A method comprising: receive answers to task-specific question prompts; determining one or more stop conditions associated with the requested task; as well as A task-specific output is generated using a machine learning model, wherein the machine learning model receives the answer and the task-specific question prompt as input and generates the task-specific output, wherein the machine learning model is a generative model trained using a general unsupervised training process.

14. The method of claim 13, wherein the machine learning model is not trained to perform the requested task.

15. The method according to claim 13, further comprising: Using the machine learning model, generating one or more question-answer pairs based on the requested task; extracting the task-specific question prompt from the one or more question-answer pairs; modifying the one or more question-answer pairs by replacing each answer in the one or more question-answer pairs with one or more of the received answers; as well as Based on the modified one or more question-answer pairs, the task-specific output is generated using the machine learning model.

Images