Simulating cue generation for fine tuning of machine learning models

Abstract

Aspects of the present disclosure relate to systems and methods for generating one or more cues based on an input and a semantic context associated with the input. In an example, prompts may be provided as inputs to one or more generic ML models to provide semantic contexts around the inputs and / or outputs of the models. The cue simulates specialization of training and fine tuning of the generic ML model without the need to use a fine tuning process to actually train the generic ML model into a fine-tuned state. Additionally, the model output may be evaluated for the responsiveness of the input before being returned to the user. The advantage of the present disclosure is that a universal ML model is allowed to be applied to multiple applications without requiring expensive and time-consuming training to trim the ML model.

Description

Background Art

[0001] The applications of machine learning (ML) models are diverse and increasing over time. In fact, there seem to be few aspects of life where innovation leveraging ML models has not emerged. However, general ML models typically need to be trained on specific training datasets to be effective in specific scenarios or for specific industries. Without training, an ML model may not produce relevant, repeatable, and consistent results for users. While it is possible to train general ML models, the training process can be very costly and time-consuming, which may limit the timeliness of adopting fine-tuned ML models in specific applications. In some cases, cost and time constraints may make it less likely that fine-tuned ML models will be utilized, which restricts the potential for innovation.

[0002] Embodiments have been described with respect to these and other general considerations. Moreover, although relatively specific problems have been discussed, it should be understood that embodiments should not be limited to solving the specific problems identified in the background art. Summary of the Invention

[0003] Aspects of the present disclosure relate to systems and methods for generating one or more prompts based on an input and a semantic context or task associated with the input. In an example, the prompt can be provided as an input to one or more ML models, such as a generative large language model (LLM), to provide semantic context around the input and / or output of the model. The prompt emulates the specialization of training and fine-tuning of general ML models without the need to use the fine-tuning process to train a general ML model to perform a specific task. Additionally, the model output can be evaluated for responsiveness to the input before being returned to the user. One advantage of the present disclosure is that it allows general ML models to be applied to perform multiple tasks without the need for expensive and time-consuming training to fine-tune the ML models.

[0004] The present summary is provided to introduce a selection of concepts that are further described below in the detailed description in a simplified form. The present summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter. Additional aspects, features, and / or advantages of the examples will be set forth in part in the description that follows, and in part will be obvious from the description, or may be learned by practice of the present disclosure. Brief Description of the Drawings

[0005] Non-limiting examples and non-exhaustive examples are described with reference to the following drawings.

[0006] Figure 1 is a diagram showing a system for generating one or more prompts for a machine learning model to generate a model output according to aspects described herein.

[0007] Figure 2 is a block diagram showing a method for generating one or more prompts for generating a model output for a machine learning model according to aspects described herein.

[0008] Figure 3 is a block diagram showing a method for generating prompts using embeddings according to aspects described herein.

[0009] Figure 4 is a block diagram showing a method for evaluating a model output for responsiveness to an input according to aspects described herein.

[0010] Figure 5A and 5B shows an overview of an example generative machine learning model that can be used according to aspects described herein.

[0011] Figure 6 shows a block diagram of example physical components of a computing device in which aspects of the present disclosure can be practiced.

[0012] Figure 7 shows a simplified block diagram of a computing device in which aspects of the present disclosure can be practiced.

[0013] Figure 8 is a simplified block diagram of a distributed computing system in which aspects of the present disclosure can be practiced. Detailed Description

[0014] In the following detailed description, reference is made to the accompanying drawings, which form a part hereof, and in which are shown by way of illustration specific embodiments or examples. Aspects may be combined, other aspects may be utilized, and structural changes may be made without departing from the present disclosure. Embodiments may be practiced as a method, system, or device. Thus, embodiments may take the form of a hardware implementation, a fully software implementation, or an implementation combining software and hardware aspects. Accordingly, the following detailed description should not be taken in a limiting sense, and the scope of the present disclosure is defined by the appended claims and their equivalents.

[0015] Figure 1FIG. 0 is a diagram of a system 100 for generating one or more prompts for a machine learning model according to aspects described herein. In an example, system 100 includes a computing device 102, one or more data stores 106, a response engine 120, and a communication network 150. The computing device 102 may include one or more applications 104. The response engine 120 may include a request wrapper 122, a model repository 130, and a response evaluator 132. The request wrapper 122 may include a request processor 124, a task objective module 126, and a prompt generator 128. The applications 104 and data stores 106 are referred to in the plural because in some embodiments it may be preferable to include more than one of these elements to accommodate different types and amounts of applications. However, for ease of discussion, the description herein refers to each element in the singular, but the characteristics and examples of each element apply to multiple instances and embodiments.

[0016] The computing device 102, applications 104, data stores 106, response engine 120, request wrapper 122, request processor 124, task objective module 126, prompt generator 128, model repository 130, and response evaluator 132 communicate via the network 150. The network 150 may include one or more networks, such as a local area network (LAN), a wide area network (WAN), an enterprise network, the Internet, etc., and may include one or more of wired, wireless, and / or optical components.

[0017] The computing device 102 may be any of a variety of computing devices, including but not limited to a mobile computing device, a laptop computing device, a tablet computing device, a desktop computing device, and / or a virtual reality computing device. The computing device 102 may be configured to execute one or more applications 104 and / or services and / or manage hardware resources (e.g., processors, memory, etc.) that may be utilized by a user of the computing device 102. The application(s) 104 may be native applications or web-based applications. The application(s) 104 may operate substantially locally on the computing device 102 or may operate in accordance with a server / client paradigm in conjunction with one or more servers (not shown). The application 104 may be used for communication across the network 150 to provide input and receive and view output from the request engine 120.

[0018] In an example, input may be provided at the computing device 102 and / or application 104 and transmitted to the response engine 120 for the purpose of generating a model output in response to the input. In an example, the input may be received, such as in a chat function of the application(s) 104 for interacting with an ML model (such as a generative large language model (LLM)) in the model repository 130. In some aspects, the input may be via Figure 1Input devices of computing device 102 not shown (e.g., microphone, camera, keyboard, uploading images or videos from local storage or data storage 106, etc.) are provided as natural language (NL) input provided as voice input, text input, and / or as any one of various other inputs (e.g., text-based, image, video, etc.). Alternatively, the input can be programmatically generated by application 104, can be based on the content of a file or electronic communication, can include images, other data types, and / or a number of other examples that will be understood by those skilled in the art. In some cases, the input can refer to previously created entities or known entities (e.g., as can be stored within data storage 106). It should be understood that the input does not have to be in a specific format, contain proper grammar or syntax, or include a complete description of the model output that the user intends for the ML model to generate. Although the amount of detail provided with the input can improve the resulting model output, sparse input is sufficient.

[0019] Request wrapper 122 is used to receive the input, generate a task request, determine one or more task goals based on the input and the task request, and generate one or more prompts for the ML model based on the task goals, the task request, and / or the input. Request processor 124 receives the input and analyzes the input to determine the task request based on the input. The task request can include one or more subtasks, and these subtasks can be sorted into a task request that can be processed by the ML model. In some examples, request processor 124 can utilize one or more NL processing tools, rule-based analysis, and / or an ML model to determine the task request based on one or more portions of the input. In an example, the request wrapper can be a software object that abstracts the ML model and exposes the ML model in a way that makes it seem that the ML model is trained or tuned for a specific purpose or task.

[0020] Additionally, once the task request is generated, request processor 124 can also evaluate the task request in combination with the input and the task goals to evaluate whether a part or all of the task request is a known task request. The known portion of the task request can have one or more associated prompts stored in data storage 106, and / or the general ML model may be able to directly process the task request without a prompt. If a known prompt is needed, request processor 124 can retrieve the known prompt from data storage 106, parameterize the known prompt based on user-specific and / or session-specific context, and pass the known prompt to the ML model for processing the task request. In some examples where multiple known prompts are retrieved, the known prompts can be appropriately linked to process the request. If no additional prompt is needed, request processor 124 can pass the known task request to the ML model for processing the task request.

[0021] In addition, the request processor 124 can store one or more of the task request, task goal, prompt, prompt template, and model output in the data store 106 as semantic context and / or known entities that can be used for subsequent inputs. Additionally, the request processor can provide the model output to the application 104 after the response evaluator 132 generates and / or evaluates the model output. In an example, the request processor 124 can be a machine learning model trained to identify a specific task or intent, a rule-based process that identifies a task based on the received input, or any other type of application or process capable of parsing and analyzing the input to determine a task, request, and / or intent based on the input.

[0022] The task goal module 126 receives the task request and determines the task goal of the task request. Once determined, the task goal encapsulates the general intent of the input, the requested task, and / or the specific meaning, and can be used to help generate one or more prompts that are particularly relevant to that goal. To determine the task goal, the task goal module 126 analyzes one or both of the input and the task request. In an example, the task goal module 126 can use a rule-based approach where the input and the task request are analyzed based on a series of rules to determine the task goal. On the other hand, a semantic encoding model can be used to determine the semantic context associated with the intent and the task request and determine the task goal. The semantic encoding model can determine one or more semantic parts of the input and the task request, and process the semantic parts to generate an overall task goal that describes the potential intent of the input. For example, if the user's task request is "Tell me about my most recent work files on the semantic memory", the ML model may not have been fine-tuned to have context related to the user's work communication. However, the semantic encoding model can appropriately retrieve relevant emails, summarize them, and automatically inject them into the task goal for use as a prompt for the context of the ML model. In this way, the ML model obtains the necessary context to understand the user's work communication and provides a more complete and relevant output in response to the task request. In another example, natural language (NL) processing tools can analyze the input and the task request to determine the task goal based on the language used in both. In an example, the task goal can be determined by an application that processes the input and the task request. In additional embodiments, one or more embeddings can be used to determine the task goal. The embeddings can be generated jointly for the input and the task request such that a single embedding describes both the input and the task request, or can be generated separately and / or for one or more parts of each of the input and the task request based on the desired granularity within the system. The embeddings can then be used to identify semantically related task goals from a data store such as the data store 106 configured as an embedding object memory. The semantically related task goals can then be analyzed and fine-tuned to determine the task goal for the input.

[0023] The prompt generator 128 receives a task objective and a task request and utilizes them to generate one or more prompts for the ML model. The prompt generator 128 can generate one or more prompts that provide sufficient context for the ML model (such as a generative large language model (LLM)) when processing the ML model to generate a model output in response to the task objective associated with the input. That is, the generated prompts enable the ML model to understand the context surrounding the input, which has a task objective and task request (e.g., intent or specific meaning) unknown to the general ML model previously utilized from the model repository 130. Thus, the one or more prompts contain the semantic context of the task objective and task request such that the ML model can generate a model output in response to the requested task and / or intent without the need for additional training or fine-tuning of the model prior to generating the model output in response to the task or intent. It should be understood that the prompt can include multiple prompt templates. The prompt template can include any one of various types of data, including but not limited to natural language, image data, audio data, video data, and / or binary data, etc. In an example, the type of data can depend on the type of ML model that will be utilized to respond to the received input. One or more fields, regions, and / or other portions of the prompt can be filled with one or more prompt templates containing the input and / or context, thereby generating a prompt that can be processed by the ML model of the model repository 130 according to the aspects described herein.

[0024] In an additional example, the prompt template includes known entities previously created or input into the system 100, enabling the user to reference any of the previously created model outputs and / or various other content. For example, the data store 106 can include one or more embeddings associated with the previously generated model outputs and / or previously processed inputs, enabling semantic retrieval of the prompt template and associated context (e.g., such that the previously generated model outputs can be iterated).

[0025] In some aspects, the prompt generator 128 can utilize one or more prompts in place of the input to the ML model. In another example, in addition to the input, the prompt generator 128 can also provide one or more prompts. The prompts can be generated in various ways. In one example, the applications and / or ML models from the model repository 130 can analyze the task objective and the input to select from one or more prompt templates stored in the data store 106 and utilize these prompt templates to fill the prompt. In an alternative example, NL processing tools can be used to analyze the input and the task objective to determine one or more associated prompt templates and / or fill the prompt. In another example, the prompt generator 128 can directly use the ML model to write prompts, rewrite prompts, and / or expand seed prompts to determine one or more associated prompt templates and / or fill the prompt.

[0026] In another aspect, the prompt generator 128 can associate one or more pre-configured prompt templates with at least a portion of the intent and task request and populate each prompt template to generate one or more prompts accordingly. The prompt templates can include code and / or instructions to trigger the recall and / or retrieval of semantic information, which can be injected into the prompts to encapsulate the specific context that the ML model needs to generate a model output in response to the input. One or more prompt templates can be retrieved by the prompt generator 128 from the data store 106. In another example, embeddings can be generated individually and / or jointly for the input and / or task request. One or more embeddings can be stored in the data store 106 for future use. Semantically related prompt templates are used to generate one or more prompts that will be processed by one or more ML models from the model repository 130. In additional embodiments, the ML models stored in the model repository 130 can be trained to output prompts. The trained ML models can be used to process the intent and task objectives and output one or more prompts in response to the input.

[0027] The prompts can be provided as input to one or more ML models in the model repository 130 to generate a model output in response to the input. The model repository 130 can include any one of a variety of ML models. The generative models used in accordance with the aspects described herein can generate any output type of a variety of output types (and thus can be multimodal generative models in some examples) and can be generative transformer models, large language models (LLMs), and / or generative image models, etc. Example ML models include but are not limited to Generative Pretrained Transformer 3 (GPT-3), BigScience BLOOM (Large Open Science Open Access Multilingual Language Model), DALL-E, DALL-E2, Stable Diffusion, or Jukebox. Additional examples of these aspects are discussed below with respect to Figures 5A - 5B the generative ML models shown in. Additionally or alternatively, one or more recognition models (or any one of any other types of ML models) can produce their own outputs to be processed in accordance with the aspects described herein. Additionally, the model repository can include one or more user-specific ML models, meaning that they have been trained on information that is particularly relevant to a specific user (e.g., chat history, browsing history, purchase history, previous inputs, user preferences from an online profile, other data particularly relevant to the user, etc.).

[0028] The response evaluator 132 can process the model output to determine whether the model output is responsive to the input. The response evaluator 132 can evaluate the model output, which can include any one of various types of content (e.g., text, image, procedural output, code, instructions for 3D printed objects, etc.), and can be returned to the user, executed, parsed, and / or otherwise processed (e.g., as one or more API calls or function calls) to verify functionality. In some aspects, based on determining that the model output does not meet or exceeds a predetermined responsiveness threshold, based on the score generated for the response, an indication of an error in the model output, and / or processing of at least a portion of the model output failure (e.g., which can be the case when the model output includes code or other output determined to be syntactically or semantically incorrect) and other examples, the model output can be unresponsive. In some aspects, if the model output is determined to be unresponsive to the input, the response evaluator 132 can re-initiate the process for generating the model output such that another model output is created before the response is returned to the application 104, such that the system can generate multiple potential model outputs before returning the response to the application 104. For example, the prompt generator 128 can utilize the generated prompt and additionally utilize an embedded question to query the ML model to indicate whether it would benefit from more context to provide the model output, more information to provide the model output, and / or respond in the case where some aspect of the prompt is unclear. If the ML model would benefit from additional context, information, and / or clarity, the ML model can provide an intermediate output, including one or more requests for additional lookups by the prompt generator 128, retrieval of additional context, information, prompt templates, etc., and / or other instructions for generating the model output. In this way, the ML model can utilize these requests to generate an intermediate output, rather than the user receiving an unresponsive output and having to determine through trial and error what can improve the ML model output. Once the additional context / information is collected, an updated prompt will be generated by the prompt generator 128, and then the ML model can process the updated prompt based on its own requests. In some examples, the ML model can place the requested context / information into a context store for subsequent calls. Once the ML model has received sufficient additional information, prompt templates, context, etc., the ML model can produce a final output, and the response evaluator 132 will determine that the final output is responsive to the input and return the final output to the user. In this way, the model can improve the response success rate. In other aspects, the response evaluator 132 can provide a failure indication to the user, such as indicating that the user can retry or reformulate the input, that the input was not correctly understood, or that the requested functionality may not be available.

[0029] In various aspects, computing device 102 can be any device that can receive, process, modify, and communicate content over network 150. Examples of computing device 102 include mobile computers or mobile computing devices (e.g., devices, laptops, notebooks, tablet computers, netbooks, etc.), fixed computing devices such as desktop computers or PCs (personal computers), telephones, mobile devices, virtual reality devices, gaming devices, vehicle computers, and / or wireless devices. Computing device 102 can be configured to execute one or more design applications (or “apps”), such as app 104 and / or services and / or manage hardware resources (e.g., processors, memory, etc.), which can be utilized by a user of computing device 102. App 104 can be a native app or a web-based app. App 104 can operate substantially locally on computing device 102 or can operate in conjunction with one or more servers (not shown) according to a server / client paradigm. App 104 can be used for communication across network 150 to provide input for a user and receive and view model output from response engine 120.

[0030] Computing device 102 can receive, send, and receive content data as input or output, which can be, for example, from a microphone, camera, global positioning system (GPS), etc., which sends content data, a computer-executed program that generates content data, and / or a memory that stores data corresponding to the content data. Content data can include visual content data, audio content data (e.g., speech or ambient noise), user input (such as voice queries, text queries, etc.), images, actions performed by a user and / or device, computer commands, procedural evaluation gaze content data, calendar entries, emails, document data (e.g., virtual documents), weather data, news data, blog data, encyclopedia data, and / or other types of private and / or public data that can be recognized by one of ordinary skill in the art. In some examples, content data can include text, source code, commands, skills, or procedural evaluations.

[0031] Computing device 102 and request engine 120 can include at least one processor that executes software and / or firmware stored in memory, such as request processor 124. The software / firmware code includes instructions that, when executed by the processor, cause the control logic to perform the functions described herein. As used herein, the term “logic” or “control logic” can include software and / or firmware executed on one or more programmable processors, application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), digital signal processors (DSPs), hardwired logic, or combinations thereof. Thus, according to embodiments, the various logics can be implemented in any suitable manner and will be maintained according to the embodiments disclosed herein.

[0032] According to some aspects, the computing device 102 and the response engine 120 can access data included in the data store 106 and the ability to store data in the data store 106. The data store 106 can include multiple contents related to generating outputs and providing data to the ML model. The data store 106 can be a web server, a cloud server, a network-attached storage (“NAS”) device, or another suitable computing device. The data store 106 can include one or more of any type of storage mechanism or memory, including disks (e.g., in a hard disk drive), optical discs (e.g., in an optical disc drive), magnetic tapes (e.g., in a magnetic tape drive), memory devices such as random access memory (RAM) devices, read-only memory (ROM) devices, etc., and / or any other suitable type of storage medium. Although only one instance of the data store 106 is shown in Figure 1 , the system 100 can include two, three, or more similar instances of the data store 106. Additionally, in some embodiments, the network 150 can provide access to other data stores similar to the data store 106 located outside the system 100.

[0033] In some examples, the network 150 can be any suitable communication network or combination of communication networks. For example, the network 150 can include a Wi-Fi network (which can include one or more wireless routers, one or more switches, etc.), a peer-to-peer network (e.g., a Bluetooth network), a cellular network (e.g., a 3G network, a 4G network, a 5G network, etc., compliant with any suitable standard), a wired network, etc. In some examples, the network 150 can be a local area network (LAN), a wide area network (WAN), a public network (e.g., the Internet), a private or semi-private network (e.g., a corporate or university intranet), any other suitable type of network, or any suitable combination of networks. Figure 1 The illustrated communication links (arrows) can each be any suitable communication link or combination of communication links, such as a wired link, an optical fiber link, a Wi-Fi link, a Bluetooth link, a cellular link, etc.

[0034] It should be understood that the various methods, devices, applications, nodes, features, etc. described with respect to Figure 1 or any of the figures described herein are not intended to limit the system to being performed by the specific applications and features described. Thus, the methods and systems herein can be practiced with additional configurations and / or the described features and applications can be excluded without departing from the methods and systems disclosed herein.

[0035] Figure 2 is a block diagram showing a method for generating one or more prompts for a machine learning model to generate a model output according to aspects described herein. The general order of operations for method 200 is inFigure 2 is shown. Generally, method 200 begins at operation 202 and ends at operation 216. Method 200 may include more or fewer steps, or may have the steps arranged in a different order than Figure 2 shown. Method 200 may be executed as computer-executable instructions executed by a computer system and encoded or stored on a computer-readable medium or other non-transitory computer storage medium. Additionally, method 200 may be executed by gates or circuits associated with a processor, ASIC, FPGA, SOC, or other hardware device. Hereinafter, method 200 will be explained with reference to the Figure 1 , Figure 3 , Figure 4 , Figure 5A , Figure 5B , Figure 6 , Figure 7 and Figure 8 described systems, components, devices, modules, software, data structures, data feature representations, signaling diagrams, methods, etc.

[0036] At operation 202, an input corresponding to a task request is received. The input may be received from an application (e.g., application 104) on a computing device (e.g., computing device 102). The input may indicate a request for a model output.

[0037] At operation 204, a task objective may be determined, for example, by a task objective module (e.g., task objective module 126) based on the input and the task request. The task objective may encapsulate the general intent or specific meaning of the input and may be used to help generate one or more prompts that are particularly relevant to that objective.

[0038] At operation 206, it is determined whether the task objective is a known task request. A known task request is a task request that has known prompts associated with it, or a task request for which a prompt is not required for the ML model to generate a model output in response to the input. An unknown task request is any other task request that is not a known task request. The determination of a known task request is based on an analysis of the input, the task request, and the task objective. If the task request is known, the flow proceeds to operation 210, where the known task request is processed as a direct input and / or the known prompts associated with the task request. If the task request is unknown, the flow proceeds to operation 208, where one or more prompts are generated by a prompt generator (e.g., prompt generator 128) based on the task objective and the task request. The prompts are used to provide sufficient context to a general ML model such that it can generate a model output based on the input in response to the requested task.

[0039] At operation 210, the input and / or one or more prompts are processed by an ML model (e.g., a generative large language model (LLM)) that is part of a model repository (e.g., model repository 130). In some cases, as described above, the input can be processed alone. In other cases, the input and the prompt are processed together to generate a model output. Alternatively, the prompt can be processed without an input to the ML model. The model output can be evaluated at operation 212 by a response evaluator (e.g., response evaluator 132) to determine whether the model output is responsive to the input. Operation 212 is shown in dashed lines to indicate that this step is optional and can be omitted in certain aspects.

[0040] If the model output is not responsive, the process proceeds to operation 208, where a new prompt can be generated. As described above, the new prompt can be fine-tuned by a new method of generating the prompt and / or by widening or narrowing the parameters of the method of generating the prompt previously used. The new prompt will be input to the ML model to generate a new model output. This loop will continue until it is determined that the model output is responsive to the input. If the model output is responsive, the process proceeds to operation 214, where the model output is provided to the user by a request processor (e.g., request processor 124).

[0041] At operation 216, one or more of the input, the task request, the task goal, the prompt, and / or the model output are stored by a request processor (e.g., request processor 124) in a data store (e.g., data store 106). Operation 216 is shown in dashed lines to indicate that this step is optional and can be omitted in certain embodiments.

[0042] Figure 3 is a block diagram showing a method for generating prompts using embeddings according to aspects described herein. The general order of operations for method 300 is shown in Figure 3 Typically, method 300 begins at operation 302 and ends at operation 306. Method 300 can include more or fewer steps, or can arrange the order of steps differently than Figure 3 shown. Method 300 can be performed as computer-executable instructions executed by a computer system and encoded or stored on a computer-readable medium or other non-transitory computer storage medium. Additionally, method 300 can be performed by gates or circuits associated with a processor, ASIC, FPGA, SOC, or other hardware device. Hereinafter, reference will be made in connection with Figure 1 、 Figure 2 、 Figure 4 Figure 5、 Figure 6 、 Figure 7 and Figure 8The described systems, components, devices, modules, software, data structures, data representations of data characteristics, signaling diagrams, methods, etc. are used to explain Method 300.

[0043] At operation 302, embeddings for the input, task request, and task goal can be generated, for example, by a prompt generator (e.g., prompt generator 128). The embeddings can be created individually for each of the input, task request, and task goal, or jointly for one or more of the input, task request, and task goal. Once created, the embeddings can be inserted into an embedding object memory (e.g., data store 106 configured as an embedding object memory). At operation 304, one or more semantically related prompt templates can be determined by the prompt generator (e.g., prompt generator 128). At operation 306, one or more semantically related prompt templates are filled into the prompts to be processed by the ML model from the model repository (e.g., model repository 130).

[0044] Figure 4 is a block diagram showing a method for responsive evaluation model output for an input according to aspects described herein. The general order of operations for Method 400 is shown in Figure 4 as follows. Generally, Method 400 begins at operation 402 and ends at operation 412. Method 400 can include more or fewer steps, or can have the steps arranged in a different order than Figure 4 shown. Method 400 can be executed as computer-executable instructions executed by a computer system and encoded or stored on a computer-readable medium or other non-transitory computer storage medium. Additionally, Method 400 can be executed by gates or circuits associated with a processor, ASIC, FPGA, SOC, or other hardware device. Hereinafter, Method 400 will be explained with reference to the systems, components, devices, modules, software, data structures, data representations of data characteristics, signaling diagrams, methods, etc. described in conjunction with Figure 1 , Figure 2 , Figure 3 , Figure 5, Figure 6 , Figure 7 and Figure 8 The described systems, components, devices, modules, software, data structures, data representations of data characteristics, signaling diagrams, methods, etc. are used to explain Method 400.

[0045] At operation 402, the model output is received by a response evaluator (e.g., response evaluator 132). At operation 404, a score can be generated by the response evaluator for the response. The score can be a determination of the model output relative to the response to the input. At operation 406, it is determined by the response evaluator whether the score is below a threshold. The threshold can be determined by the system developer as a design choice, or can be determined by the response evaluator based on the nature and complexity of the user input. If the score is below the threshold, the process proceeds to operation 408, where, as described above with respect to Figure 2As described in operation 212, new prompts and model outputs are generated substantially similarly. This method is performed in a loop until it is determined that the score meets or exceeds a threshold. If this occurs, the process proceeds to operation 410, where the model output is provided by a request processor (e.g., request processor 124) to an application (e.g., application 104).

[0046] Figure 5A and 5B shows an overview of an example generative machine learning model that can be used in accordance with aspects described herein. First, referring to Figure 5A , the conceptual diagram 500 depicts an overview of a pre-trained generative model package 504 in terms of processing an input and a prompt 502 to generate a model output 506 described herein. Example ML models include, but are not limited to, Generative Pretrained Transformer 3 (GPT-3), BigScience BLOOM (Large Open Science Open Access Multilingual Language Model), DALL-E, DALL-E2, Stable Diffusion, or Jukebox.

[0047] In an example, the generative model package 504 is pre-trained based on various inputs (e.g., various human languages, various programming languages, and / or various content types), and thus does not need to be fine-tuned or trained for a specific scenario. Instead, the generative model package 504 can be pre-trained more generally such that the input 502 includes prompts that are generated, selected, or otherwise designed to induce the generative model package 504 to produce a specific generative model output 506. It should be understood that the input 502 and the generative model output 506 can each include any one of various content types, including but not limited to text output, image output, audio output, video output, procedural output, and / or binary output, among other examples. In an example, the input 502 and the generative model output 506 can have different content types, as can be the case when the generative model package 504 includes a generative multimodal machine learning model.

[0048] Thus, the generative model package 504 can be used in any of a variety of scenarios, and further, different generative model packages can be used in place of the generative model package 504 with substantially no modification to other associated aspects (e.g., similar to herein with respect to Figure 1 , Figure 2 , Figure 3 and Figure 4The aspects described). Thus, the generative model package 504 operates as a tool for performing machine learning processing, where a specific input 502 to the generative model package 504 is programmatically generated or otherwise determined such that the generative model package 504 produces a model output 506, which can subsequently be used for further processing.

[0049] The generative model package 504 can be provided or otherwise used according to any of a variety of paradigms. For example, the generative model package 504 can be used locally on a computing device (e.g., Figure 1 the computing device 102 in), or can be accessed remotely from a machine learning service (e.g., the response engine 120). In other examples, aspects of the generative model package 504 are distributed across multiple computing devices. In some instances, the generative model package 504 can be accessed via an application programming interface (API), such as can be provided by the operating system of a computing device and / or by other examples such as a machine learning service.

[0050] Referring now to the illustrated aspects of the generative model package 504, the generative model package 504 includes an input tokenization 508, an input embedding 510, a model layer 512, an output layer 514, and an output decoding 516. In an example, the input tokenization 508 processes the input 502 to generate the input embedding 510, which includes a sequence of symbolic representations corresponding to the input 502. Thus, the input embedding 510 is processed by the model layer 512, the output layer 514, and the output decoding 516 to produce the model output 506. Figure 5B An example architecture corresponding to the generative model package 504 is depicted in, and this example architecture is discussed in further detail below. Even so, it should be understood that the architectures shown and described herein should not be taken in a limiting sense, and in other examples, any of a variety of other architectures can be used.

[0051] Figure 5B is a conceptual diagram depicting an example architecture 550 of a pre-trained generative machine learning model that can be used according to the aspects described herein. As noted above, without departing from the aspects described herein, any of a variety of alternative architectures and corresponding ML models can be used in other examples.

[0052] As shown, the architecture 550 processes the input 502 to produce a generative model output 506, aspects of which were discussed above with respect to Figure 5A The architecture 550 is depicted as a transformer model including an encoder 552 and a decoder 554. The encoder 552 processes the input embedding 558 (aspects of which can be similar to Figure 5AThe input embedding 510), where the input embedding includes a sequence of symbolic representations corresponding to the input 556. In the example, the input 556 includes the input for generation and the prompt 502 (e.g., skills corresponding to a skill chain).

[0053] In addition, the positional encoding 560 can introduce information about the relative and / or absolute positions of the tokens of the input embedding 558. Similarly, the output embedding 574 includes a sequence of symbolic representations corresponding to the output 572, and the positional encoding 576 can similarly introduce information about the relative and / or absolute positions of the tokens of the output embedding 574.

[0054] As shown, the encoder 552 includes an example layer 570. It should be understood that any number of such layers can be used, and the depicted architecture is simplified for illustrative purposes. The example layer 570 includes two sub-layers: a multi-head attention layer 562 and a feed-forward layer 566. In the example, residual connections are included around each of the layers 562, 566, followed by a normalization layer 564 and a normalization layer 568, respectively.

[0055] The decoder 554 includes an example layer 590. Similar to the encoder 552, any number of such layers can be used in other examples, and the architecture of the depicted decoder 554 is simplified for illustrative purposes. As shown, the example layer 590 includes three sub-layers: a masked multi-head attention layer 578, a multi-head attention layer 582, and a feed-forward layer 586. Aspects of the multi-head attention layer 582 and the feed-forward layer 586 can be similar to those discussed above with respect to the multi-head attention layer 562 and the feed-forward layer 566, respectively. Additionally, the masked multi-head attention layer 578 performs multi-head attention on the output of the encoder 552 (e.g., the output 572). In the example, the masked multi-head attention layer 578 prevents the positions from attending to subsequent positions. This mask in combination with the offset embedding (e.g., offset by one position, as shown in the multi-head attention layer 582) can ensure that the prediction for a given position depends on the known outputs of one or more positions less than the given position. As shown, residual connections are also included around the layers 578, 582, and 586, followed by a normalization layer 580, a normalization layer 584, and a normalization layer 588, respectively.

[0056] The multi-head attention layer 562, the multi-head attention layer 578, and the multi-head attention layer 582 can each linearly project the query, key, and value to corresponding dimensions using a set of linear projections. Each linear projection can be processed using an attention function (e.g., dot product or additive attention) to produce an n-dimensional output value for each linear projection. The resulting values can be concatenated and projected again such that the values are then processed as Figure 5B shown (e.g., by the corresponding normalization layer 564, normalization layer 580, or normalization layer 584).

[0057] The feed-forward layer 566 and the feed-forward layer 586 can each be a fully-connected feed-forward network applied to each position. In an example, the feed-forward layer 566 and the feed-forward layer 586 each include a plurality of linear transformations with rectified linear unit activations between the linear transformations. In an example, each linear transformation is the same across different positions and can use different parameters compared to other linear transformations of the feed-forward network.

[0058] Additionally, aspects of the linear transformation 592 can be similar to the linear transformations discussed above with respect to the multi-head attention layer 562, the multi-head attention layer 578, the multi-head attention layer 582, and the feed-forward layer 566 and the feed-forward layer 586. The softmax function 594 can also convert the output of the linear transformation 592 into predicted next-token probabilities, as shown by the output probabilities 596. It should be understood that the illustrated architectures are provided as examples, and in other examples, any of a variety of other model architectures can be used in accordance with the disclosed aspects.

[0059] Thus, the output probabilities 596 can thus form the model output 506 in accordance with the aspects described herein such that the output of the generative ML model defines an output corresponding to the input. For example, the model output 506 can be associated with a corresponding application and / or data format such that the model output is processed to display the output to a user and / or to fabricate a physical object, etc.

[0060] Figures 6 - 8 and the associated description provide a discussion of various operating environments in which aspects of the present disclosure can be practiced. However, the devices and systems shown and discussed are for purposes of example and illustration and do not limit the large number of computing device configurations that can be used to practice the aspects of the present disclosure described herein. Figures 6 - 8 The devices and systems shown and discussed are for purposes of example and illustration and do not limit the large number of computing device configurations that can be used to practice the aspects of the present disclosure described herein.

[0061] Figure 6 is a block diagram showing the physical components (e.g., hardware) of a computing device 600 in which aspects of the present disclosure can be practiced. The computing device components described below can be applicable to the computing devices described above, including Figure 1 the computing device 102 in. In a basic configuration, the computing device 600 can include at least one processing unit 602 and a system memory 604. Depending on the configuration and type of the computing device, the system memory 604 can include, but is not limited to, volatile storage (e.g., random access memory), non-volatile storage (e.g., read-only memory), flash memory, or any combination of these memories.

[0062] System memory 604 may include an operating system 605 and one or more program modules 606 suitable for running software applications 620, such as one or more components supported by the systems described herein. As an example, system memory 604 may store an embedded object memory insertion engine or component 624 and / or an embedded object memory retrieval engine or component 626. The operating system 605 may be suitable for controlling the operation of the computing device 600, for example.

[0063] Additionally, aspects of the present disclosure may be practiced in conjunction with a graphics library, other operating systems, or any other application programs and are not limited to any particular application or system. This basic configuration is shown by those components within dashed line 608. The computing device 600 may have additional features or functionality. For example, the computing device 600 may also include additional data storage devices (removable and / or non-removable), such as magnetic disks, optical disks, or tapes. Such additional storage is shown by removable storage device 609 and non-removable storage device 610 in Figure 6 As described above, multiple program modules and data files may be stored in the system memory 604. When executed on the processing unit 602, the program modules 606 (e.g., applications 620) may perform processes including but not limited to aspects described herein. Other program modules that may be used in accordance with aspects of the present disclosure may include email and contact applications, word processing applications, spreadsheet applications, database applications, slide presentation applications, drawing or computer-aided applications, etc. Figure 6 shown in

[0064] Moreover, aspects of the present disclosure may be practiced on the following circuits: including discrete electronic elements, a packaged or integrated electronic chip including logic gates, a circuit utilizing a microprocessor, or a single chip including electronic elements or a microprocessor. For example, aspects of the present disclosure may be practiced via a system-on-a-chip (SOC), where

[0065] each or many of the components shown may be integrated onto a single integrated circuit. Such an SOC device may include one or more processing units, graphics units, communication units, system virtualization units, and various application functions, all of which are integrated (or "burned") onto a chip substrate as a single integrated circuit. When operating via an SOC, the functions described herein regarding the capabilities of the client switching protocol may be operated via dedicated logic integrated on a single integrated circuit (chip) with other components of the computing device 600. Some aspects of the present disclosure may also be practiced using other technologies capable of performing logical operations (e.g., AND, OR, and NOT), including but not limited to mechanical, optical, fluidic, and quantum technologies. Additionally, some aspects of the present disclosure may be practiced within a general-purpose computer or in any other circuit or system. Figure 6 shown in

[0066] The computing device 600 may also have one or more input devices 612, such as a keyboard, mouse, pen, voice or speech input device, touch or swipe input device, etc. (Multiple) output devices 614, such as a display, speaker, printer, etc., may also be included. The above devices are examples, and other devices may be used. The computing device 600 may include one or more communication connections 616 that allow communication with other computing devices 650. Examples of suitable communication connections 616 include, but are not limited to, radio frequency (RF) transmitter, receiver, and / or transceiver circuitry; universal serial bus (USB), parallel, and / or serial ports.

[0067] As used herein, the term computer-readable medium may include computer storage media. Computer storage media may include volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information, such as computer-readable instructions, data structures, or program modules. System memory 604, removable storage device 609, and non-removable storage device 610 are all examples of computer storage media (e.g., memory storage). Computer storage media may include RAM, ROM, electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile disk (DVD) or other optical storage, magnetic tape cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other article that can be used to store information and can be accessed by the computing device 600. Any such computer storage media may be part of the computing device 600. Computer storage media does not include carrier waves or other propagated or modulated data signals.

[0068] Communication media may be embodied by computer-readable instructions, data structures, program modules, or other data in a modulated data signal, such as a carrier wave or other transmission mechanism, and includes any information delivery media. The term "modulated data signal" may describe a signal having one or more characteristics set or changed in such a manner as to encode information in the signal. By way of example and not limitation, communication media may include wired media such as a wired network or direct wired connection, and wireless media such as acoustic, radio frequency (RF), infrared, and other wireless media.

[0069] Figure 7 is a block diagram of an architecture showing one aspect of a computing device. That is, the computing device may incorporate a system (e.g., an architecture) 702 to implement some aspects. In some examples, the system 702 is implemented as a "smartphone" capable of running one or more applications (e.g., browser, email, calendar, contact manager, messaging client, game, and media client / player). In some aspects, the system 702 is integrated into a computing device, such as an integrated personal digital assistant (PDA) and wireless phone.

[0070] One or more applications 766 may be loaded into the memory 762 and run on or in association with the operating system 764. Examples of applications include a telephone dialer, an email program, a personal information management (PIM) program, a word processing program, a spreadsheet program, an Internet browser program, a messaging program, and the like. The system 702 also includes a non-volatile storage area 768 within the memory 762. The non-volatile storage area 768 may be used to store persistent information that should not be lost when the system 702 is powered down. The applications 766 may use and store information in the non-volatile storage area 768, such as email or other messages used by an email application or the like. A synchronization application (not shown) also resides on the system 702 and is programmed to interact with a corresponding synchronization application residing on a host computer to keep the information stored in the non-volatile storage area 768 synchronized with the corresponding information stored at the host computer. It should be understood that other applications may be loaded into the memory 762 and run on the mobile computing device 700 described herein (e.g., an embedded object memory insertion engine, an embedded object memory retrieval engine, etc.).

[0071] The system 702 has a power supply 770, which may be implemented as one or more batteries. The power supply 770 may also include an external power supply, such as an AC adapter or a power docking cradle that supplements or recharges the battery.

[0072] The system 702 may also include a radio interface layer 772 that performs the functions of transmitting and receiving radio frequency communications. The radio interface layer 772 facilitates a wireless connection between the system 702 and the "outside world" via a communication carrier or service provider. Transmissions to and from the radio interface layer 772 occur under the control of the operating system 764. In other words, communications received by the radio interface layer 772 may be propagated to the applications 766 via the operating system 764, and vice versa.

[0073] The visual indicator 720 can be used to provide visual notifications, and / or the audio interface 774 can be used to generate audible notifications via the audio transducer 725. In the example shown, the visual indicator 720 is a light-emitting diode (LED), and the audio transducer 725 is a speaker. These devices can be directly coupled to the power supply 770 such that when activated, these devices remain on for a duration specified by the notification mechanism even if the processor 760 and / or the dedicated processor 761 and other components may be turned off to conserve battery power. The LED can be programmed to remain on indefinitely until the user takes an action to indicate the powered-on state of the device. The audio interface 774 is used to provide audible signals to the user and receive audible signals from the user. For example, in addition to being coupled to the audio transducer 725, the audio interface 774 can also be coupled to a microphone to receive audible input, such as to facilitate a telephone conversation. According to aspects of the present disclosure, the microphone can also be used as an audio sensor to facilitate control of the notifications, as will be described below. The system 702 can also include a video interface 776 that enables operation of the on-board camera 730 to record still images, video streams, and the like.

[0074] The computing device implementing the system 702 can have additional features or functionality. For example, the computing device can also include additional data storage devices (removable and / or non-removable), such as magnetic disks, optical disks, or tapes. Such additional storage is Figure 7 illustrated by the non-volatile storage area 768.

[0075] As described above, the data / information generated or collected by the computing device and stored via the system 702 can be stored locally on the computing device, or the data can be collected and stored on any number of storage media that can be accessed by the device via the radio interface layer 772 or via a wired connection between the computing device and a separate computing device associated with the computing device (e.g., a server computer in a distributed computing network such as the Internet). It should be understood that such data / information can be accessed via the computing device via the radio interface layer 772 or via a distributed computing network. Similarly, such data / information can be easily transferred between computing devices for storage and use in accordance with well-known data / information transfer and storage components, including email and collaborative data / information sharing systems.

[0076] Figure 8Shows an aspect of the architecture of a system for processing data received at a computing system from a remote source, such as a personal computer 804, a tablet computing device 806, or a mobile computing device 808, as described above. The content displayed at the server device 802 can be stored in different communication channels or other storage types. For example, various documents can be stored using a directory service 824, a web portal 825, a mailbox service 826, an instant messaging store 828, or a social networking site 830.

[0077] The application 820 (e.g., similar to the application 620) can be adopted by a client communicating with the server device 802. Additionally or alternatively, the embedded object memory insertion engine 821 and / or the embedded object memory retrieval engine 822 can be adopted by the server device 802. The server device 802 can provide data to and receive data from client computing devices via the network 815, such as a personal computer 804, a tablet computing device 806, and / or a mobile computing device 808 (e.g., a smart phone). As an example, the computer system described above can be embodied in a personal computer 804, a tablet computing device 806, and / or a mobile computing device 808 (e.g., a smart phone). In addition to receiving graphical data that can be used for preprocessing at a graphical source system or postprocessing at a receiving computing system, any of these examples of computing devices can obtain content from the storage 816.

[0078] For example, aspects of the present disclosure have been described above with reference to block diagrams and / or operational illustrations of methods, systems, and computer program products according to aspects of the present disclosure. The functions / actions recited in the blocks may not occur in any order shown in any flowchart. For example, depending on the functions / actions involved, two consecutive blocks shown may actually be executed substantially simultaneously, or the blocks may sometimes be executed in the reverse order.

[0079] The description and illustration of one or more aspects provided in this application are not intended to limit or define the scope of the present disclosure in any way. The aspects, examples, and details provided in this application are considered sufficient to convey possession and enable others to make and use the claimed aspects of the present disclosure. The claimed disclosure should not be construed as limited to any aspect, example, or detail provided in this application. Whether shown and described combinatorially or individually, various features (both structures and methods) are intended to be selectively included or omitted to produce embodiments having a particular set of features. Having provided the description and illustration of this application, those skilled in the art can envision variations, modifications, and alternative aspects that fall within the spirit of the broader aspects embodied in this application, which do not depart from the broader scope of the claimed disclosure.

Claims

1. A system, comprising: at least one processor; and a memory storing instructions that, when executed by the at least one processor, cause the system to perform a set of operations, the set of operations including: receiving an input; generating a task request based on the received input; determining a task objective based on the input and the task request; generating one or more prompts based on the task objective and the task request, wherein the one or more prompts include additional context related to the input for a machine learning model; processing the one or more prompts using the machine learning model to generate a model output; and providing the model output to an application of a computing device.

2. The system according to claim 1, further comprising: storing one or more of the following: the input, the task request, the task objective, the prompt, and the model output.

3. The system according to claim 1, further comprising: embedding one or more questions for the machine learning model in the one or more prompts; receiving, in response to the one or more questions, one or more requests from the machine learning model for additional context, information, or prompt templates; generating one or more updated prompts using the requested context, information, or prompt templates; and processing the one or more updated prompts using the machine learning model to generate a model output.

4. The system according to claim 1, wherein generating one or more prompts further comprises: generating one or more embeddings for one or more of the input, the task request, and the task objective; inserting the one or more embeddings into an embedding object memory; determining one or more prompt templates semantically associated with the one or more embeddings; and populating the one or more prompt templates into the prompts.

5. The system according to claim 1, further comprising: evaluating whether the model output is responsive to the input.

6. The system according to claim 5, further comprising: receiving the model output; generating a score for the model output; determining whether the score is below a threshold; and generating an updated prompt when the score is below the threshold.

7. The system according to claim 6, further comprising: providing the model output to an application of the computing device when the score meets or exceeds the threshold.

8. The system according to claim 1, wherein the machine learning model is a large language generative transformer model.

9. A method, comprising: receiving an input; generating a task request based on the received input; determining a task objective based on the input and the task request; generating one or more prompts based on the task objective and the task request, wherein the one or more prompts include additional context related to the input for a machine learning model; processing the one or more prompts using the machine learning model to generate a model output; if the model output is responsive to the input, providing the model output to an application of a computing device.

10. The method according to claim 9, further comprising: Store one or more of the following: the input, the task request, the task objective, the prompt, and the model output.

11. The method according to claim 9, further comprising: Embedding one or more questions for the machine learning model in the one or more prompts; In response to the one or more questions, receiving from the machine learning model one or more requests for additional context, information, or prompt templates; Generating one or more updated prompts using the requested context, information, or prompt templates; And Processing the one or more updated prompts using the machine learning model to generate a model output.

12. The method according to claim 9, wherein generating one or more prompts further comprises: Generating one or more embeddings for one or more of the input, the task request, and the task objective; Inserting the one or more embeddings into an embedding object memory; Determining one or more prompt templates semantically associated with the one or more embeddings; And Populating the one or more prompt templates into the prompts.

13. The method according to claim 9, further comprising: Evaluating whether the model output is in response to the input.

14. The method according to claim 13, further comprising: Receiving the model output; Generating a score for the model output; Determining whether the score is below a threshold; And When the score is below the threshold, generating an updated prompt.

15. A computer storage medium comprising instructions that, when executed by a processor, cause the processor to: Receive an input; Generate a task request based on the received input; Determine a task objective based on the input and the task request; Generate one or more prompts based on the task objective and the task request, wherein the one or more prompts include additional context related to the input for the machine learning model; Process the one or more prompts using the machine learning model to generate a model output; And Provide the model output to an application of a computing device.

Images