Generating a diverse set of LLM responses

Abstract

Systems and methods for generating a set of diverse LLM responses can include processing input data to generate a first response, updating an output response corpus based on the first response, and processing the input data with a sampling method that biases sampling of additional token sequences associated with additional token sequences based on a distance metric. A second sampling method selects additional tokens that are different from tokens included in the output response corpus based on the distance metric. The systems and methods can generate a set of diverse responses based on the additional tokens.

Description

Technical Field

[0001] This disclosure generally relates to causing artificial intelligence agents to generate diverse outputs. More specifically, this disclosure relates to improving the semantic diversity between the output responses of machine learning models. Background Technology

[0002] A computer can receive input. It can execute instructions to process the input using a parameterized model to generate output. The computer can use the model to obtain feedback on its performance in generating output. The computer can generate feedback by evaluating its performance. The computer can receive feedback from external sources. Based on this feedback, the computer can update the model's parameters to improve its performance. In this way, the computer can iteratively "learn" to generate the desired output. The resulting model is often called a machine learning model. Summary of the Invention

[0003] Aspects and advantages of embodiments of this disclosure will be set forth in part in the description which follows, or may be learned from the description, or may be learned by practice of the embodiments.

[0004] One example aspect of this disclosure relates to a computer-implemented method. The method may include processing input data using a sequence processing model to generate a first response as the output of the sequence processing model, wherein the first response includes a first lexical sequence. The method may include updating an output response corpus based on the first response. The method may also include processing the input data using the sequence processing model to generate an additional response as an additional output of the sequence processing model. The additional response may include an additional lexical sequence. The sequence processing model may utilize a sampling method that biases the sampling of the additional lexical sequence based on a distance metric, wherein a second sampling method selects one or more additional lexicals that are different from those contained in the output response corpus based on the distance metric.

[0005] In some implementations, the method includes updating the output response corpus based on the additional response.

[0006] In some implementations, the method includes providing the output response corpus as additional output of the sequence processing model.

[0007] In some implementations, the distance metric includes the Levenshtein distance.

[0008] In some implementations, the sampling method that biases the sampling of the additional lexical sequence includes penalizing candidate lexicals based on the distance metric to modify the probability distribution of the candidate lexicals, wherein the modified probability distribution reduces the probability that the penalized candidate lexicals are included in the additional response.

[0009] In some implementations, these penalized candidate lexical units are associated with candidate lexical units that do not meet a distance metric threshold. In some implementations, this distance metric threshold indicates the degree of semantic difference between these candidate lexical units and the lexical units contained in the output response corpus.

[0010] In some implementations, one or more additional lexical units that differ from those contained in the output response corpus include one or more semantic differences.

[0011] In some implementations, the method includes analyzing the output response corpus to determine one or more basic lexical units. In some implementations, these basic lexical units indicate probability distribution thresholds.

[0012] In some implementations, this probability distribution threshold is associated with the degree of correctness.

[0013] In some implementations, the additional response includes one or more basic terms.

[0014] In some implementations, the method includes generating another response as another output of the sequence processing model while generating the first response, wherein the other response includes another sequence of terms. In some implementations, the method includes updating the output response corpus in real time based on the other response.

[0015] In some implementations, the sequence processing model includes at least one of the following: (i) a large language model (LLM) or (ii) a large multimodal model (LMM).

[0016] In some implementations, the one or more additional lexical units include text data.

[0017] In some implementations, the method includes training the sequence processing model using one or more additional lexical units that are different from those contained in the output response corpus.

[0018] Another exemplary aspect of this disclosure relates to a computing system. The system may include one or more processors and one or more non-transitory computer-readable media storing instructions that, when implemented, cause the computing system to perform operations. These operations may include processing input data using a sequence processing model to generate a first response as the output of the sequence processing model, wherein the first response includes a first lexical sequence. These operations may include updating an output response corpus based on the first response. These operations may include processing the input data using the sequence processing model to generate an additional response as an additional output of the sequence processing model. The additional response may include an additional lexical sequence. The sequence processing model may utilize a sampling method that biases the sampling of candidate lexical units associated with the additional lexical sequence based on a distance metric, wherein a second sampling method selects one or more additional lexical units that are different from those contained in the output response corpus based on the distance metric.

[0019] In some implementations, these operations further include updating the output response corpus based on the additional response.

[0020] In some implementations, these operations further include providing the output response corpus as additional output of the sequence processing model.

[0021] In some implementations, the distance metric includes the Levenshtein distance.

[0022] In some implementations, these operations further include analyzing the output response corpus to determine one or more basic lexical units. In some implementations, these basic lexical units indicate probability distribution thresholds.

[0023] Another example aspect of this disclosure relates to a non-transitory computer-readable medium storing one or more instructions that, when implemented, cause one or more processors to perform operations. These operations include processing input data using a sequence processing model to generate a first response as the output of the sequence processing model, wherein the first response includes a first lexical sequence. These operations include updating an output response corpus based on the first response. These operations also include processing the input data using the sequence processing model to generate an additional response as an additional output of the sequence processing model. The additional response may include an additional lexical sequence. The sequence processing model may utilize a sampling method that biases the sampling of candidate lexical units associated with the additional lexical sequence based on a distance metric, wherein a second sampling method selects one or more additional lexical units that are different from those contained in the output response corpus based on the distance metric.

[0024] Other exemplary aspects of this disclosure relate to other systems, methods, apparatuses, tangible non-transitory computer-readable media, and devices for performing the functions described herein.

[0025] These and other features, aspects, and advantages of the various implementations will be better understood by referring to the following description and the appended claims. The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate implementations of this disclosure and, together with the description, help to explain the relevant principles.

[0026] For example, various aspects of this disclosure can provide numerous technical effects and benefits, such as reducing computational resources for machine learning models, improving the quality of information available to users, and, as further described herein, efficiently decoding the diverse outputs of machine learning models in a single invocation. Attached Figure Description

[0027] Referring to the accompanying drawings, a detailed discussion of embodiments is set forth in this specification for those skilled in the art, in which:

[0028] Figure 1 A block diagram of an example computing system according to an example embodiment of the present disclosure is depicted;

[0029] Figure 2 A block diagram of an example computing system according to an example embodiment of the present disclosure is depicted;

[0030] Figure 3 A flowchart is depicted for an example method for generating a diverse set of output responses according to an example embodiment of the present disclosure;

[0031] Figure 4A An illustration depicts example input data according to an example embodiment of the present disclosure;

[0032] Figure 4B Illustrations depicting several example output responses according to embodiments of the present disclosure;

[0033] Figure 5 A block diagram of an example computing system according to an example embodiment of the present disclosure is depicted;

[0034] Figure 6 This is a flowchart illustrating an example method for training a machine learning model, according to various aspects of this disclosure;

[0035] Figure 7 It is a block diagram of an example processing flow for using a machine learning model to process input to generate output, based on example implementations of various aspects of this disclosure;

[0036] Figure 8 This is a block diagram of an example sequence processing model based on various aspects of the present disclosure;

[0037] Figure 9 It is a block diagram of an example technique for filling example input sequences for processing by a sequence processing model, based on example implementations of various aspects of this disclosure;

[0038] Figure 10 This is a block diagram of an example model development platform based on example implementations of various aspects of this disclosure;

[0039] Figure 11 This is a block diagram of an example training workflow for training a machine learning model, based on example implementations of various aspects of this disclosure;

[0040] Figure 12 This is a block diagram of an inference system for performing inference by operating one or more machine learning models, based on example implementations of various aspects of this disclosure.

[0041] Figure 13 This is a block diagram of an example networked computing system based on various aspects of the present disclosure;

[0042] Figure 14 This is a block diagram of an example computing device implementing various aspects of this disclosure; and

[0043] Figure 15 This is a block diagram of an example computing device that embodies various aspects of this disclosure.

[0044] The repeated reference numerals across multiple figures are intended to identify the same features in various implementations. Detailed Implementation

[0045] Generally, this disclosure relates to systems and methods for improving the generation of a diverse set of outputs from a machine learning model. More specifically, a challenge in the field of machine learning is that machine learning models often generate multiple output responses that are semantically similar to each other or otherwise repetitive for a specific input (e.g., a user query or prompt). In some instances, the user may expect consistency among the multiple output responses. For example, when a machine learning model is used to generate outputs (e.g., responses or completions) for the same input at multiple times, the model may often generate outputs that repeat each other, for example, because the model decodes only the most probable output value (e.g., a word) conditioned on the input. Specifically, traditional decoding methods for large language models (LLMs) use sampling techniques that are not aimed at optimizing the decoding of semantically diverse responses. However, in some settings, it may be desirable to obtain multiple distinct outputs for a given input. Therefore, a technical problem in the field of machine learning is how to control the generation of multiple distinct outputs, each of high quality and different from each other, by the machine learning model.

[0046] In view of the above challenges, the system and method described in this paper improve the generation of a diverse set of possible output responses by selectively biasing the sampling of additional lexical sequences associated with the additional responses and selecting lexicals that are different from those contained in the output response corpus generated by the machine learning model. In the implementation, diverse lexicals can be added to the output response corpus, and a machine learning model can be trained on the output response corpus to generate diverse output responses over time.

[0047] For example, in some implementations, the input data associated with a user query can be processed by a machine learning sequence processing model to generate one or more output responses. The initial response may include a sequence of terms, which are decoded using greedy decoding or sampling to select the terms most likely to be included in the sequence. This sequence of terms can be added to an output response corpus. For example, the output response corpus may include multiple candidate responses (e.g., K candidates), which are typically the possible output responses to the input data.

[0048] In one example, the model may subsequently or in parallel generate additional output responses. The model may then process the input data again. However, during this processing, the model may bias the sampling of lexical units by imposing a penalty on them based on a distance metric—an indicator of the similarity between two lexical units or sequences of lexical units. For example, the Levenshtein distance can be used to determine the distance representing the similarity between sequences of lexical units contained in the output response corpus and the additional candidate responses. Alternatively, other similarity metrics (e.g., semantic similarity) may be used. The penalty may be applied selectively or consistently to individual lexical units or sequences of lexical units. For example, the model may modify lexical units or sequences of lexical units with lower determinism in the additional output responses based on contextual information available in the overall output. In an embodiment, the penalty may be applied only to lexical units or sequences of lexical units that include lower determinism because these lexical units or sequences of lexical units are less fundamental indicators of the output response compared to high-determinism lexical units. Additional words or word sequences that are within a threshold range (e.g., based on a distance metric) of the words in the output response corpus can be penalized to prevent the selection of similar words in the additional output response. Additional words that are more than a threshold distance can be sampled without any bias.

[0049] In this embodiment, terms associated with high certainty may be penalized in a consistent manner with terms with lower certainty. For example, terms associated with higher certainty may be fundamental to the output response due to, for example, the degree of correctness associated with them (e.g., providing the correct name, etc.). However, terms with higher certainty may be penalized due to their higher probability of being valid, while maintaining higher certainty compared to other candidate terms when penalties are consistently applied. This bias sampling process can be repeated until K output responses are generated. Because the validity of a term can depend not only on preceding terms but also on subsequent terms, the model can provide improved diversity and accuracy of the output response when considering the overall output.

[0050] In this way, these additional lexical units, which differ from those in the output response corpus, provide a method for generating semantically diverse output responses without penalizing "correct" or base lexical units. At each generation loop, the diversified output from the lexical sequence processing model provides the user with alternative output response options that include semantic differences. For example, when the model generates an initial output response and additional output responses, the model's output can indicate an increase in diversity associated with each lexical unit and / or the overall additional output response. It may be desirable to provide multiple suggested responses simultaneously or sequentially as responses to the user's query, thereby improving the diversity of information available from the output without substantially altering accuracy.

[0051] Example aspects of this disclosure provide systems and methods for generating improved, diversified output responses from machine learning models. For example, the systems and methods described herein can more efficiently diversify the information contained in the output response by penalizing overly closely related lexical units without increasing the computational cost associated with computational distance during decoding over time. For example, aspects of this disclosure can provide a model that can generate diversified output responses in a single invocation by distilling each generated additional output response back into the model. Furthermore, a guiding vector can be applied to the model to encourage it to generate different types of output responses.

[0052] In one aspect, the systems and methods described herein provide the ability to process input data using a machine learning model sequence processing model (e.g., via a computing system including one or more computing devices) to generate a first response as output. This first response may include a sequence of lexical units containing one or more lexical units. A lexical unit may be a portion of a larger response or answer generated by the sequence processing model. For example, in some example implementations, the model may generate responses such as blocks or paragraphs of text comprising multiple lexical units. A lexical unit can be any suitable lexical unit, such as, for example, textual lexical units, including textual data such as words, phrases, sentences, letters, or other suitable textual descriptions.

[0053] In some implementations, the output may be obtained from a machine learning lexical generation model. The machine learning lexical generation model can be any suitable model. For example, in some implementations, an agent system (e.g., an "artificial intelligence agent" or "AI agent") may employ one or more machine learning models to generate output in response to a query from a user. As an example, the agent system may be or include a computing system comprising one or more machine learning lexical generation models, wherein the computing system is configured to receive input from a user device or calling device and provide an output comprising a plurality of lexical units in response to input to the user device or calling device. As an example, the plurality of lexical units may be or may include text data (such as words), and / or the plurality of lexical units may collectively form a text paragraph or block displayed to the user. For example, the plurality of lexical units may collectively describe an answer or response to user input. Furthermore, in some implementations, the plurality of lexical units may be or may include (e.g., or may be converted to) spoken data, such as audio data that can be spoken to the user. In some implementations, machine learning lexical generation models can output values ​​such as confidence scores or certainty scores, intonation scores, and / or other suitable scores and / or values ​​associated with multiple lexical units, respectively.

[0054] In some implementations, the agent system can be or can be implemented as a multimodal agent (e.g., a multimodal AI agent). For example, a multimodal agent can process input from one or more data modalities. In some implementations, the agent system can be implemented as a "contextual agent." The term contextual agent refers to an environment in which the agent system shares one or more perceptual inputs with a human user. For example, a contextual agent can receive and process various data inputs that can also be observed by a human user, including video, audio, and / or text data. The agent system can process these inputs to generate responses relevant to the user's physical or digital environmental context, such as enabling the agent system to generate dialogues or other responses or outputs that help the user understand the environment and / or navigate within it.

[0055] In one respect, the systems and methods described herein can provide an output response corpus that is updated (e.g., by computation) based on a first response. The output response corpus may include a set of text data on which a sequence processing model is trained. For example, in some implementations, the systems and methods described herein can update the output response corpus by adding the first response and additional responses corresponding to multiple lexical units in the output. This corpus can be iteratively updated and made accessible to the sequence processing model during each decoding loop of the output response.

[0056] As an example, in some implementations, a sequence of lexical terms may be associated with a certainty score or confidence score associated with each lexical term. The certainty score may represent the certainty of the lexical generation model associated with the lexical term, such as the certainty that the lexical term is correct, the certainty that the lexical term will be included in the final output, the certainty that the lexical term is the best option among several possible options, or other certainty that typically reflects the confidence of the lexical term as output. For example, in some implementations, the lexical generation model is configured to select each lexical term from multiple candidate lexical terms based on a probability distribution associated with multiple candidate lexical terms. For example, the model may select the lexical term with the highest probability. The certainty score may be determined based on the probability distribution associated with multiple candidate lexical terms. For example, in some implementations, the certainty score may be the probability of the probability distribution associated with the lexical term selected from multiple candidate lexical terms. Therefore, a lexical term with a higher probability may be more likely to be decoded into the output response generated by the sequence processing model, and / or a lexical term with a lower probability may be less likely to be decoded into the output response generated by the sequence processing model. Furthermore, in some implementations, models other than the lexical generation model (e.g., adversarial models) can generate deterministic scores.

[0057] As another example, in some implementations, the machine learning lexical generation model may be or may include a sequence processing model. A sequence processing model can be configured to generate lexical sequences individually. For example, the output at each iteration or update loop may be or may include multiple lexical units (e.g., a sequence of lexical units). Examples of sequence processing models include, but are not limited to, large language models (LLMs) or large multimodal models (LMMs). For example, a sequence processing model may generate lexical units sequentially. Instead of providing fixed-length lexical blocks, the sequence processing model may generate each lexical unit as the model evaluates. EOS lexical units may signal the model to stop generating lexical units. For example, the model's output may contain fewer lexical units than the maximum length. An iteration (e.g., an update loop) of a sequence processing model can refer to multiple "passes" through a lexical sequence, where lexical units from the sequence of one update loop and / or their corresponding rendered values ​​are provided as input to subsequent update loops to improve the model's understanding of the entire lexical sequence at subsequent update loops.

[0058] In one aspect, the systems and methods described herein can provide processing (e.g., via a computational system) of input data to generate one or more additional responses as additional outputs of a sequence processing model. These additional responses may include additional lexical sequences. To generate additional responses, the sequence processing model may utilize a sampling method that biases the sampling of the additional lexical sequences. For example, the sequence processing model may use a distance metric to select one or more additional lexicals that are different from those contained in the output response corpus. The sequence processing model may compute the distance between the first lexical sequence associated with the first output response and the additional (e.g., candidate) lexical sequences associated with the additional responses. For lexicals or lexical sequences with distances below a threshold distance, the sequence processing model may impose a penalty to further reduce the deterministic score of the penalized lexical. For example, the penalty may modify the probability distribution of the penalized lexical such that the penalized lexical has a lower probability of being selected within the additional lexical sequences associated with the additional output responses. In this way, diverse output responses can be generated during the decoding of lexicals from the lexical generation model, thereby reducing complexity and computational cost.

[0059] In one aspect, the systems and methods described herein can provide an output response corpus that updates (e.g., via a computational system) based on additional responses. For example, a sequence processing model can add or store additional output responses into the output response corpus. The user interface can define lexical locations configured to receive, store, and / or otherwise accept values ​​or data from lexical units in the output response. As an example, these lexical locations can receive characters or words defined by lexical units. As another example, these lexical locations can be placeholder elements in the user interface. For example, in some implementations, generating the user interface may include replacing placeholder elements with values ​​defined by lexical units in the output response within the output response corpus. In some implementations, generating the user interface may include generating data that can be implemented by a computer system to display the user interface. Such data may include, for example, Extensible Markup Language (XML) data, Hypertext Markup Language (HTML) data, text file data, slideshow data, spreadsheet data, and / or any other suitable data.

[0060] In some implementations, the systems and methods described herein can provide (e.g., through a computational system) analysis of an output response corpus to determine one or more basic lexical units. As used herein, a "basic lexical unit" is a lexical unit within the output response that is identified by a computational process as relevant to the core informational content of the response, thereby contributing to the meaning or accuracy of the response within the context established by the input data and the output response corpus. In some embodiments, the computational process may include probabilistic analysis based on the output of a sequence processing model. In such embodiments, lexical units with probabilities exceeding a predetermined threshold or a dynamically adjusted threshold may be considered relevant. Alternatively or additionally, the computational process may include natural language processing analysis of the relationships between lexical units in the response and other lexical units. In such cases, lexical units identified as having a direct syntactic or semantic relationship with key elements of the input or output response corpus are considered relevant. As a further alternative or supplement, the computational process may include analysis of the contribution of lexical units to the factual accuracy of the response. Here, lexical units that can be verified by external knowledge sources or by logical reasoning based on the input data and the output response corpus are considered relevant.

[0061] For example, the output response corpus may include a first response output and additional response outputs indicating consistent lexical terms. Consistent lexical terms appearing in the output responses contained across the corpus may be associated with high-determinism lexical terms and indicate that these lexical terms are fundamental to the output response. For example, basic lexical terms may be associated with words such as "yes" for binary ("yes / no") questions. Thus, each of the output responses may include basic lexical terms beginning with "yes," "yeah," "yep," etc., to indicate an affirmative response to the user query. In an embodiment, basic lexical terms may indicate a high-probability distribution that is likely to be included in the additional output response, regardless of the sampling bias applied during decoding. As another example, basic lexical terms may be associated with factual information that directly responds to a particular query. For example, a lexical term corresponding to the word "George Washington" is very useful, if not absolutely necessary, for providing an accurate answer to the query "Who was the first president of the United States?" Therefore, some example implementations may include identifying certain basic terms and excluding those terms from the penalty techniques described herein, thereby enabling the model to create diverse but factual or other accurate responses.

[0062] Another example aspect of this disclosure provides (e.g., via a computational system) the ability to train a sequence processing model using additional output responses that differ from the first response output. For example, at each decoding loop, the additional diversified output responses can be distilled back into the sequence processing model, allowing the model to be trained to generate diversified output responses by caching distance metrics computed between previous lexical units in the previous sequence. The cached distance metric can be used to apply penalties more efficiently when needed. Furthermore, the sequence processing model can be trained to avoid penalizing underlying lexical units with consistent output responses across the corpus. On the other hand, diversified responses can be used to generate preference pairs or tuples, for which human preference labels can be collected in a reinforcement learning (RLHF) training scheme via human feedback.

[0063] In some implementations, the systems and methods described herein may provide rendered lexical units and one or more additional lexical units representing alternatives to lexical units in the output of a machine learning lexical unit generation model. For example, the additional lexical unit sequence may be displayed simultaneously with the first lexical unit sequence as alternatives (e.g., in a stacked or similar adjacent arrangement) in response to a user command (e.g., by requesting a new response) or in other suitable manner. For example, in some implementations, formatting properties such as the size of the lexical units and the rendering value corresponding to each lexical unit may represent a deterministic score associated with each lexical unit. In this way, a more deterministic lexical unit sequence may be displayed in a larger size and adjacent to alternatives, which are displayed in a smaller size. In some implementations, the systems and methods described herein may determine the rendered alternative lexical units based on some ratio or relationship (e.g., distance) between the first lexical unit sequence and the additional lexical unit sequence.

[0064] In one aspect, the systems and methods described herein can provide a means for a computing system to display a user interface. As an example, in some implementations, causing a computing system to display a user interface may include transmitting the user interface (e.g., data describing the user interface) to a display device. This display device may be located at the computer system and / or an external computer system. For example, in some implementations, transmitting the user interface may include transmitting the user interface via one or more communication networks.

[0065] Furthermore, in some implementations, the display user interface can be rendered via a display device based on the user interface, rendering one or more user interface elements. The display device can be configured to render and / or (e.g., to a user or other observer) display the user interface. For example, the display device can control one or more display elements (e.g., pixels) based on a user interface including rendered lexical elements, such that the user interface is presented to the user. According to this disclosure, any suitable method for displaying a user interface can be employed. As an example, a graphics processing unit (GPU) or similar computing device can calculate a pixel intensity matrix representing the user interface and provide the pixel intensity matrix to a display (e.g., a screen or monitor), which is configured to interpret the pixel intensity matrix and illuminate corresponding pixels in the display according to the pixel intensity matrix.

[0066] The exemplary aspects of this disclosure offer numerous technical effects and benefits, including improvements in computational techniques. As an example, providing a diverse set of output responses in response to a user query improves the diversity of training samples used to train a machine learning model to generate diverse responses. For instance, the model can easily interpret distance metrics between word sequences and be trained to diversify output responses while maintaining accuracy within the responses. In this way, the user does not need to repeatedly request the regeneration of output responses, for example, due to the model generating duplicate or redundant responses. This, in turn, reduces the computational resources used by the model.

[0067] As another example, providing a diverse set of output responses can improve the quality of information available in the user interface. This, in turn, allows for the delivery of comparable output responses of better quality to the user more quickly, thereby reducing the computational resources required to receive, generate, render, and / or display additional output responses via the user interface. In applications with limited network connectivity, such as, for example, in areas with high network traffic, these time and computational savings can significantly improve the usability and user experience of the systems and methods according to this disclosure.

[0068] As yet another example, aspects of this disclosure implement novel operating modes for computing devices. For instance, aspects of this disclosure include the diversification output at the progressive decoding loop independently of the model's lexical generation process. This information can be readily communicated to the user. For example, while decoding the lexical from the first response output, additional output response lexicals can also be decoded in subsequent generation loops, making the selection of output responses for the user to choose from more robust. This information might otherwise be unavailable to the user, or could not be communicated in this way due to the computational cost associated with computing a distance metric involving a large number of candidate lexical values. However, according to an example aspect of this disclosure, by distilling the diversified output response back and subsequently providing the cached distance metric to the machine learning sequence processing model, the computing system can be made to decode the diversified output more efficiently in a single call, rather than requiring the output to be decoded first and then diversified.

[0069] Therefore, the above-described exemplary aspects of this disclosure provide that a computing device can modify the decoding of a lexical sequence or bias the decoding of a lexical sequence in response to the detection of semantic similarity. This can save a significant amount of computational resources that would otherwise be wasted on repeatedly decoding responses that are no more diverse than previous responses and / or receiving new prompts or user queries to facilitate the generation of diverse responses.

[0070] Exemplary embodiments of this disclosure will now be discussed in further detail with reference to the accompanying drawings.

[0071] Figure 1 A block diagram of an example computing system 100 according to an exemplary embodiment of the present disclosure is depicted. In some implementations, the computing system is configured to receive and / or obtain input data 102 describing a user prompt indicating an expected response to a statement and / or question, and, upon receiving the input data 102, generate, determine, and / or provide one or more refined output responses 114, the one or more refined output responses including a sequence of terms determined to be in response to the input data 102. Thus, in some implementations, the computing system 100 may include a generative model 104 operable to generate predicted outputs describing a first response output 108 and a second response output 116 that may be semantically different from each other.

[0072] Specifically, the computing system 100 may obtain input data 102 from a user computing device. Input data 102 may include text data, image data, audio data, latently encoded data, multimodal data, and / or other data. Input data 102 may include requests for multiple different responses to the same question or statement (e.g., draft responses, alternative response options, etc.).

[0073] The computing system 100 can process input data 102 to generate a first response output 106. The first response output 106 may include a sequence of lexical terms obtained from a lexical corpus available from the generative model 104. A first sampling method 106 may be used to determine the first lexical sequence, which employs semantic analysis to determine semantic relationships between lexical terms within the lexical corpus and selects the lexical terms with the highest probability or likelihood of being used together in similar contexts of the input data 102 to include in the first output response 108. The first sampling method 106 may include greedy sampling, random sampling, top-K sampling, bundle search sampling, or other sampling techniques for selecting the most probable lexical terms for the lexical sequence.

[0074] Generative model 104 may use a first sampling method 106 to process input data 102 to generate a first response output 108. Generative model 106 may include an autoregressive language model. Generative model 106 may include an encoder to encode input data 102 and a decoder to employ the sampling method to generate an output sequence of lexical units contained in output response 110.

[0075] Based on the first output response 108, the output response corpus 110 can be updated to include the first decoded response to the input data 102. The output response corpus 110 may include a collection of response output data (e.g., lexical sequences) generated in response to the input data 102. The generative model 104 may use a second sampling method 112 based on the lexical sequences contained in the output response corpus 110 to simultaneously and / or sequentially generate a second response output 116 (e.g., additional lexical sequences). The second sampling method 112 may cause the second response output 116 to be semantically different from the first response output 108 and other lexical sequences contained in the output response corpus 110.

[0076] For example, the generative model 104 may employ a second sampling method 112 to decode the second response output 116, thereby selecting sample candidate lexical units from a semantically diverse lexical corpus to include in the second response output 116. The second sampling method 112 may penalize lexical units determined to be semantically similar to lexical units or sequences of lexical units contained in the output response corpus 110. The penalty may be based on a distance metric indicating the similarity or proximity of two lexical units or sequences of lexical units. The computational system 100 may determine the distance metric between the candidate lexical units of the second response output 116 and the lexical units contained in the output response corpus 110 during each decoding step. For example, the distance metric may be determined using methods such as Levenshtein distance calculation, triple comparison, Jaro-Winkler algorithm calculation, latent space lexical embedding, normalized distance, etc.

[0077] For example, given a set of tokens, the Levenshtein distance metric between two strings a and b (of lengths |a| and |b|, respectively) is given by lev(a,b), where

[0078]

[0079] Here, the tail of a string x is a string consisting of all characters except the first character of x (i.e., tail(x0x1...x)). n )=x1x2...x n ), and head(x) is the first character of x (i.e., head(x0x1...x)). n = x0. The symbol x[n] or x n Used to refer to the nth character of string x, counting from 0, so head(x) = x0 = x[0]. The first element in the minimum value corresponds to deletion (from a to b), the second element corresponds to insertion, and the third element corresponds to replacement.

[0080] For example, the Levenshtein distance between the lexical units “kitten” and “sitting” is 3 because the following 3 edits transform one lexical unit into the other, and cannot be achieved with fewer than 3 edits: kitten → sitten (replacing “k” with “s”), sitten → sitt (replacing “e” with “i”), sittin → sitting (inserting “g” at the end). An example of deletion can be seen in “uninformed” and “uniformed”, where the distance is 1: uninformed → uniform (deleting “n”).

[0081] Lexical terms that do not meet the distance metric threshold (e.g., less than 3, 4, etc.) can be penalized using the second sampling method 112 because they are semantically too similar to lexical terms included in the lexical sequence contained in the output response corpus 110. Penalization can modify the probability distribution of candidate lexical terms, which can cause penalized candidate lexical terms to exhibit a lower probability of being used together in similar contexts. For example, the generative model 104 can select lexical terms to include in the lexical sequence included in the second response output 116. The selected lexical terms included in the second response output 116 can include candidate lexical terms that have the highest probability of being used together in similar contexts after (e.g., via the second sampling method 112) penalty has been applied.

[0082] For example, a deterministic score for each lexical unit can be determined based on a modified (e.g., penalized) probability distribution. The deterministic score can be an updated or modified probability distribution associated with a lexical unit selected from a plurality of candidate lexical units based on a penalty. Once a penalty has been applied, more diverse lexical units with higher probabilities are more likely to be decoded into the second response output 116, and / or less diverse lexical units with lower probabilities are therefore less likely to be decoded into the second response output. In an embodiment, lexical units or sequences of lexical units that are fundamental to responding to the input data 102 (e.g., correct answer, name, location, etc.) may be exempt from penalty. Reference Figure 5 Further examples describing the basic lexical units.

[0083] In response to imposing a penalty on candidate lexical units, a second response output 116 may be generated. The output response corpus 110 may be updated based on the second response output 116. A set of refined output responses 114 may be generated based on the output response corpus 110. This set of refined output responses 114 may include multiple response options or drafts in response to the input data 102. The refined output responses 114 may be provided back to the user.

[0084] Figure 2 This is a block diagram of an example computing system 200 according to an example embodiment of the present disclosure. The computing system 200 and... Figure 1 The computing system 200 is similar to the computing system 100, except that the computing system 200 further includes a decoder model 208, bias sampling methods 210-1 and 210-2, and candidate lexical units 212-1 and 212-2.

[0085] Specifically, the computing system 200 may obtain input data 202 from a user computing device. Input data 202 may include text data, image data, audio data, latently encoded data, multimodal data, and / or other data. Input data 202 may include requests for multiple different responses to the same question or statement (e.g., draft responses, alternative response options, etc.).

[0086] The computing system 200 can process input data 202 to generate a set of refined output responses 214. This set of refined output responses 214 may include a set of output responses based on word sequences contained in the output response corpus 212. The word sequences can be decoded using a first sampling method 208 and bias sampling methods 210-1, 210-2.

[0087] For example, generative model 204 can process input data 202 and can use a first sampling method 208 to generate a first output response, which selects lexical units to be included in the lexical sequence associated with the first output response. In response to the first sampling method 208, the output response corpus 212 can be updated by adding the first output response to the output response corpus 212. In some implementations, the output response corpus 212 can be updated by processing input data 202 using generative model 204 and decoding the lexical sequence associated with the output response using decoder model 206 and / or any other machine learning model.

[0088] Decoder model 206 may be separate from, part of, or identical to generative model 204. Decoder model 206 may include a natural language processing model (e.g., an autoregressive language model) tuned to implement various sampling methods for decoding the output response. First sampling method 208 may employ semantic analysis to determine semantic relationships between lexical units within a lexical corpus and select lexical units that indicate the highest probability or likelihood of being used together in similar contexts to include in a first lexical unit sequence. First sampling method 208 may include greedy sampling, random sampling, top-K sampling, bundle search sampling, or other sampling techniques for selecting the most probable lexical units for the lexical unit sequence.

[0089] Based on the first sampling method 208, the output response corpus 212 can be updated to include the first output response in response to the input data 202. The decoder model 206 can analyze the output response corpus 212 at each decoding step for additional output responses (e.g., draft responses) to determine the distance metric between candidate lemmas 212-1, 212-2 and the lemmas contained in the output response corpus 212.

[0090] For example, decoder model 206 can parse candidate lemmas 210-1 and 210-2 output by generator model 204 and generate distance metrics between candidate lemmas 210-1 and 210-2 and lemmas contained in output response corpus 212. For example, decoder model 206 can determine the distance between a first lemma associated with a first lemma sequence contained in output response corpus 212 and additional candidate lemmas in candidate lemmas 212-1, either in parallel or sequentially.

[0091] Based on a distance metric, decoder model 206 may employ a bias sampling method 210-1. The bias sampling method may include a modified first sampling method 208. For example, bias sampling method 210-1 may penalize lemmas or lemma sequences that do not meet a distance metric threshold. For instance, decoder model 206 may be configured to enforce a distance metric threshold that penalizes candidate lemmas 212-1 that are determined to be 2 or less away from lemmas contained in the output response corpus 212. The penalty may modify the probability distribution of candidate lemmas 212-1 during decoding. For example, candidate lemmas 212-1 that are highly likely to be decoded in the output response may be penalized because they do not meet the distance metric threshold, thus reducing their probability of being decoded in the output response. Therefore, bias sampling method 210-1 may penalize decoded output responses that are semantically too similar to output responses already contained in the output response corpus 212. This encourages computational system 200 to decode diverse outputs.

[0092] Based on bias sampling method 210-1, additional output responses (e.g., associated with candidate lexical units 212-1) can be generated. These additional output responses can be added to the output response corpus 212.

[0093] In an embodiment, the generative model 204 may use a first sampling method 208, a second sampling method 210-1, 210-2, and / or other sampling methods to simultaneously generate multiple output responses. For example, the generative model 204 may generate another output response while generating a first output response, in order to provide multiple refined output responses 214 as a set of draft responses. Reference Figures 4A to 4B Further description of an example of a draft response.

[0094] For example, generative model 204 can process input data 202 and use a first sampling method 208, a second sampling method 210-1, etc., to simultaneously generate a first output response and another output response. For example, the first output response can be partially decoded while the other output response is being decoded. Based on the partially decoded response, which includes one or more word sequences, generative model 204 can begin using bias sampling methods 210-1, 210-2 to decode the remainder of the first output response and / or the other output response. For example, while decoding the first output response, selected words from the partial response can be added to the output response corpus 212.

[0095] In an embodiment, decoder model 206 can monitor output response corpus 212 to detect lexical sequences while they are being decoded, and simultaneously begin using bias sampling methods 210-1 and 210-2 in real time to decode another output response based on lexical sequences associated with the partially decoded first output response. For example, output response corpus 212 can be updated after a threshold number k candidate lexical sequences 212-1 and 212-2 have been selected to ensure sufficient context for simultaneous or subsequent decoding steps using the second sampling methods 210-1 and 210-2. In this way, output response corpus 212 can be updated in real time and simultaneously based on multiple output responses while maintaining semantic diversity among them.

[0096] In an embodiment, the distance metric of candidate lexical 212-1 may be cached and used to determine the distance between an additional output response (e.g., associated with candidate lexical 212-1) and another candidate lexical in candidate lexical 212-2. For example, candidate lexical 212-1 and candidate lexical 212-2 may be associated with corresponding output responses (e.g., different draft responses) generated in response to input data 202, in addition to being associated with the first output response.

[0097] In response to the distance metric of the cache associated with additional output responses (e.g., candidate lexical unit 212-1) contained in the output response corpus 212, decoder model 206 can determine a second distance metric between candidate lexical unit 212-2 and the lexical unit associated with the additional output response. Based on a comparison of the second distance metric associated with candidate lexical unit 212-2 and a distance metric threshold, decoder model 206 can employ bias sampling method 210-2 to penalize lexical units in candidate lexical unit 212-2 that do not meet the distance metric threshold. Therefore, decoder model 206 can iteratively or continuously generate output responses that differ from those previously or simultaneously decoded.

[0098] While the examples herein describe static distance metrics, this disclosure is not limited to such embodiments. At each decoding step, the distance metric can be dynamically adjusted within a certain range to provide flexibility in the penalty scheme and prevent diversification of output responses that produce inaccurate or poorly expressed nuances. Furthermore, steering vectors can be used to encourage the decoder model 206 to generate outputs that differ from each other.

[0099] Based on bias sampling methods 210-1 and 210-2, the output response corpus 212 can be updated, and multiple refined output responses 214 can be generated and provided to the user.

[0100] Figure 3 A flowchart depicts an example method for generating a diverse set of output responses according to an example embodiment of this disclosure. Although Figure 3 The steps performed in a particular order are depicted for illustrative and discussion purposes, but the method of this disclosure is not limited to the order or arrangement specifically shown. The various steps of method 300 may be omitted, rearranged, combined and / or adapted in various ways without departing from the scope of this disclosure.

[0101] At point 302, the computing system may use a sequence processing model to process the input data to generate a first response as the output of the sequence processing model. The first response may include a first sequence of terms. For example, the input data may include text data, image data, audio data, latently encoded data, multimodal data, and / or other data. The input data may include requests for multiple different responses to the same question or statement (e.g., draft responses, alternative response options, etc.). The input data may be obtained from a user computing device associated with the user.

[0102] The computational system may include a sequence processing model configured to process input data to generate a first response output. The first response output may include a sequence of lexical terms obtained from a lexical corpus available from a large language model. A first sampling method may be used to determine the first lexical sequence, employing semantic analysis to determine semantic relationships between lexical terms within the lexical corpus and selecting the lexical terms with the highest probability or likelihood of being used together in similar contexts of the input data to be included in the first output response. The first sampling method may include greedy sampling, random sampling, top-K sampling, bundle search sampling, or other sampling techniques.

[0103] At 304, the computational system may update the output response corpus based on the first response. For example, based on the first output response, the output response corpus may be updated to include the first response decoded in response to the input data. The output response corpus may include a collection of response output data (e.g., word sequences) generated in response to the input data.

[0104] In an embodiment, a first response may be added to the output response corpus at each decoding step. For example, the computing system may generate multiple output responses sequentially and / or simultaneously in response to input data.

[0105] At 306, the computational system can use a sequence processing model to process the input data to generate additional responses as additional outputs of the sequence processing model. For example, a large language model can use a second sampling method to generate a second response output (e.g., an additional lexical sequence) simultaneously and / or sequentially based on the lexical sequences contained in the output response corpus.

[0106] At 308, the additional response may include an additional lexical sequence. For example, the additional response may be associated with candidate lexical units selected to be included in the lexical sequence representing the additional response. In an embodiment, a second sampling method may be used to select the lexical sequence associated with the additional response.

[0107] At 310, the sequence processing model can utilize a sampling method that biases the sampling of additional lexical sequences to select one or more additional lexical units that are different from those contained in the output response corpus. For example, a second sampling method could cause the additional response to be semantically different from the first response and other responses contained in the output response corpus.

[0108] In an embodiment, the sequence processing model can analyze the output response corpus to determine a distance metric between candidate nodules associated with the additional response and nodules associated with the first response contained in the output response corpus. For example, the sequence processing model can parse candidate nodules and generate a distance metric between the candidate nodules and nodules of the first response contained in the output response corpus.

[0109] In an embodiment, the sequence processing model can determine the distance between the first word in the first word sequence contained in the output response corpus and the additional candidate words associated with the additional responses, either in parallel or sequentially. Based on the distance metric, the sequence processing model can employ a bias sampling method.

[0110] In embodiments, the bias sampling method may include a modified first sampling method. For example, the bias sampling method may penalize lemmas or lemma sequences that do not meet a distance metric threshold. The penalty may modify the probability distribution of candidate lemmas with respect to the additional response during decoding. For example, candidate lemmas of an additional response that are highly likely to be decoded in the additional response may be penalized because they do not meet the distance metric threshold, thus reducing their probability of being decoded in the additional response.

[0111] Therefore, the bias sampling method penalizes decoded additional responses that are semantically too similar to the first response, causing the sequence processing model to select additional lexical units that are different from those contained in the output response corpus. Based on the bias sampling method, additional responses (e.g., associated with candidate lexical units) can be generated. These additional responses can be added to the output response corpus.

[0112] In an embodiment, the distance metric of candidate nouns can be cached and used to determine the distance between an additional output response (e.g., associated with a candidate noun) and another candidate noun among the candidate nouns. For example, the candidate noun associated with the additional response and the other candidate noun can be associated with different draft responses generated in response to the input data, in addition to being associated with the first response.

[0113] Figure 4AAn illustration depicts example input data according to an exemplary embodiment of this disclosure. Specifically, Figure 4A Example user input 402 is depicted. For example, user input 402 may be a question provided via a user interface of a user computing device and processed to generate one or more responses. Input data 402 may be encoded into a format that can be processed by a generative model.

[0114] Figure 4B Illustrations depicting several example output responses according to embodiments of the present disclosure are provided. Specifically, Figure 4B An example output response 404 generated in response to input data 402 is depicted. For example, the generative model may receive encoded input data 402 and generate a first response 404A and additional responses 404B-C in response to input data 402.

[0115] In embodiments, additional responses 404B-C may be generated sequentially and / or simultaneously with the first response 404A. For example, as depicted in the figures, the first response 404A and additional responses 404B-C may be corresponding drafts or alternative options generated in response to input data 402. The first response 404A and additional responses 404B-C may be semantically different from each other in order to provide the user with alternative response options, each of which includes the correct information in response to input data 402, but differs in how the response is expressed.

[0116] Figure 5 A block diagram of an example computing system 500 according to an exemplary embodiment of the present disclosure is depicted. Specifically, the computing system 500 may analyze the lexical units contained in the output response corpus 504 to iteratively determine basic lexical units that should be exempted from penalty by the decoder model 502 in subsequent decoding steps.

[0117] For example, decoder model 502 (e.g., a large language model tuned to decode diverse outputs) may generate output response 504-1 based on input data provided to computing system 500. Output response 504-1 may include multiple lexical units and / or sequences of lexical units. Output response 504-1 including multiple lexical units and / or sequences of lexical units may then be added to output response corpus 504. Output response corpus 504 may store a collection of output responses 504-1 generated in response to input data. In an embodiment, decoder model 502 may also update output response corpus 504 to include context data 504-2. Context data 504-2 may include relevant data or context used by computing system 500 to determine candidate lexical units to be included in output response 504-1. Therefore, the output response corpus 504 may include contextual information associated with each of the output responses 504-1, such that the keyword meta-identifier 506 can determine the fundamental lexical and / or lexical sequences for the output response 504-1 generated in response to the input data.

[0118] At each decoding step for output response 504-1 added to output response corpus 504, the keyword meta-identifier can analyze output response corpus 504 to determine the underlying lexical units that can meet the penalty exemption conditions of the additional output response 504-1. Keyword meta-identifier 506 may include an autoregressive language model.

[0119] Keyword meta-identifier 506 can determine the underlying lexical units contained in the output response corpus 504-1 based on the corresponding probability distribution of each lexical unit and / or lexical unit sequence associated with the output response 504-1. For example, lexical units and / or lexical unit sequences can be ranked based on the confidence level (e.g., probability distribution) of the corresponding lexical units / lexical unit sequences in the output response 504-1, which may be included within the indicated context data 504-2.

[0120] In an embodiment, the keyword meta-identifier 506 may determine the base lexical units and / or lexical sequences based on a probability distribution that exceeds a probability distribution threshold. For example, lexical units associated with correct names, locations, numbers, etc., may be associated with a higher probability distribution compared to other lexical units, because correct names, locations, numbers, etc., are likely to be included in the output response 504-1 generated in response to input data querying such names, locations, numbers, etc. Therefore, the keyword meta-identifier 506 may determine base lexical units that should be exempt from penalty by the decoder model 502 in subsequent decoding steps for the additional output response 504-1.

[0121] For example, decoder model 502 may employ a bias sampling method to generate additional output responses 504-1 that are semantically different from the output responses 504-1 contained in the output response corpus 504. The bias sampling method penalizes lemmas and / or lemma sequences 504 that are overly similar to lemmas or lemma sequences contained in the output response corpus. Keyword meta-identifiers 506 exempt the underlying lemmas from penalty, thus discouraging decoder model 502 from decoding inaccurate or irrelevant output responses 504-1.

[0122] Figure 6 A flowchart depicts a method 600 for training one or more machine learning models according to various aspects of this disclosure. For example, an example machine learning model may include a decoder model.

[0123] One or more portions of Example Method 600 may be implemented by a computing system comprising one or more computing devices, such as the computing system described, for example, with reference to other accompanying figures. Each corresponding portion of Example Method 600 may be performed by any one (or any combination of) of one or more computing devices. Furthermore, one or more portions of Example Method 600 may be implemented on hardware components of the apparatus described herein, for example, to train one or more systems or models. Figure 6 For illustrative and discussion purposes, elements executed in a particular order are depicted. Those skilled in the art will understand using the disclosure provided herein that elements of any method discussed herein can be adapted, rearranged, extended, omitted, combined, or modified in various ways without departing from the scope of this disclosure. Figure 6 The description is for illustrative purposes only and with reference to elements / terms described with respect to other systems and figures, and is not intended to be limiting. One or more portions of example method 600 may be performed additionally or alternatively by other systems.

[0124] At 602, example method 600 may include obtaining training instances. A set of training data may include multiple training instances partitioned across multiple datasets (e.g., training datasets, validation datasets, or test datasets). Training instances may be labeled or unlabeled. Although referred to as “training” instances in example method 600, it should be understood that runtime inference may form training instances when a model is trained (e.g., online training / learning) using an evaluation of the model’s performance on that runtime instance. Example data types of training instances and various tasks associated with them are described throughout this disclosure.

[0125] At 604, example method 600 may include using one or more machine learning models to process training instances to generate output. This output may be obtained directly from the one or more machine learning models, or it may be a downstream result of a processing operation chain that includes the output of one or more machine learning models.

[0126] At 606, example method 600 may include receiving an evaluation signal associated with the output. The evaluation signal can be obtained using a loss function. Various losses can be used, such as mean squared error, likelihood loss, cross-entropy loss, hinge loss, contrastive loss, or various other loss functions. The evaluation signal can be computed using known baseline true labels (e.g., supervised learning), predicted or estimated labels (e.g., semi-supervised or self-supervised learning), or without labels (e.g., unsupervised learning). The evaluation signal can be a reward (e.g., for reinforcement learning). The reward can be computed using a machine learning reward model configured to generate a reward based on the received output. The reward can also be computed using feedback data describing human feedback to the output.

[0127] At 608, example method 600 may include using an evaluation signal to update a machine learning model. For example, in some embodiments, various training or learning techniques, such as backpropagation, may be used to learn the values ​​of the parameters of the machine learning model. For example, the evaluation signal may be backpropagated from the output (or another source of the evaluation signal) through the machine learning model to update one or more parameters of the model (e.g., based on the gradient of the evaluation signal relative to the parameter values). For example, a system containing one or more machine learning models may be trained in an end-to-end manner. Gradient descent techniques may be used to iteratively update the parameters over multiple training iterations. In some implementations, performing backpropagation of error may include performing truncated backpropagation over time. Example method 600 may include implementing various generalization techniques (e.g., weight decay, dropout, etc.) to improve the generalization ability of the model being trained.

[0128] In some implementations, example method 600 can be implemented to train a machine learning model from an initial state to a fully trained state (e.g., when the model exhibits a desired performance profile, such as based on accuracy, precision, recall, etc.).

[0129] In some implementations, example method 600 can be implemented for a specific stage of the training process. For example, in some implementations, example method 600 can be implemented for pre-training a machine learning model. Pre-training can include, for example, large-scale training on potentially noisy data to achieve a broad performance level base across multiple tasks / data types.

[0130] In some implementations, Example Method 600 can be implemented for fine-tuning a machine learning model. Fine-tuning can include smaller-scale training on, for example, higher-quality (e.g., labeled, curated, etc.) data. Fine-tuning can affect all or part of the parameters of the machine learning model. For example, parts of the machine learning model can be “frozen” for certain training phases. For example, parameters associated with the embedding space can be “frozen” during fine-tuning (e.g., to preserve information learned from a broader domain than that present in the fine-tuned dataset). In some implementations, Example Method 600 uses an adapter module. The adapter can be a small trainable layer inserted between pre-existing layers of the pre-trained model. During the fine-tuning process, the original parameters of the pre-trained model are typically frozen, and only the parameters of the adapter are updated.

[0131] In some implementations, example method 600 can be implemented to perform parameter-efficient fine-tuning methods such as Residual Hierarchical Optimization (LoRA). LoRA can refine a pre-trained model by making minimal adjustments to the original parameters. This can be achieved by introducing trainable low-rank matrices that modify the behavior of the pre-trained weights without directly changing them. In some implementations, only these auxiliary matrices are updated during fine-tuning, which significantly reduces the number of parameters being trained.

[0132] Example fine-tuning methods include reinforcement learning. Reinforcement learning can be based on user feedback on model performance during use.

[0133] Figure 7 This is a block diagram of an example processing flow for using machine learning model 1 to process input 2 to generate output 3.

[0134] Machine learning model 1 can be or includes one or more machine learning models or model components. Example machine learning models can include neural networks (e.g., deep neural networks). Example machine learning models can include non-linear or linear models. Example machine learning models can use other architectures to replace or supplement neural networks. Example machine learning models can include decision tree-based models, support vector machines, hidden Markov models, Bayesian networks, linear regression models, k-means clustering models, etc.

[0135] Machine learning model 1 may be, include, or otherwise represent any one or more of the machine learning models described above with respect to the foregoing figures. For example, machine learning model 1 may be, include, or otherwise represent any one or more of the decoder model 206, etc. Although the various features, variations, and implementations described below are described with respect to machine learning model 1, it should be understood that such features, variations, and implementations should be understood to be described with respect to each of the decoder models, etc., and any other machine learning components described herein.

[0136] Example neural networks can include feedforward neural networks, recurrent neural networks (RNNs) (including recurrent neural networks based on long short-term memory (LSTM), convolutional neural networks (CNNs), diffusion models, generative adversarial networks, or other forms of neural networks. Example neural networks can be deep neural networks. Some example machine learning models can utilize attention mechanisms, such as self-attention. For example, some example machine learning models can include multi-head self-attention models.

[0137] Machine learning model 1 may include one or more instances of the same model configured to operate on data from input 2. Machine learning model 1 may also include multiple different models or multiple different model parts configured to operate on data from input 2.

[0138] Machine learning model 1 can include an ensemble of different models that can collaborate and interact to process data from input 2. For example, the model ensemble can include multiple models with different properties (e.g., different architectures, trained using different schemes, etc.). The ensemble can output an overall output based on the individual outputs of the constituent models. In this way, for example, different constituent models can work together to provide system-level robustness by effectively aggregating the individual strengths and weaknesses of any given model. The corresponding individual outputs can be combined using weighted combinations, voting or routing mechanisms, or learned output layers (e.g., one or more feedforward or fully connected layers).

[0139] Machine learning model 1 can employ a hybrid expert structure. For example, see Zhou et al., Mixture-of-Experts with Expert Choice Routing, arXiv:2202.09368v2 (October 14, 2022). Different parts of the model can learn (explicitly or implicitly) different expert domains, where the path through the model is selected by a learned routing mechanism that uses the appropriate expert for a given input (e.g., a given part of the input, such as on a per-word basis). For example, the feedforward network can be sparsely activated for that part of the input based on the output of the routing mechanism that processes that part. In this way, the recombined activated weights can form the “expert” selected by the router. In each forward pass, only a subset of the total model weights can be used, thus reducing the amount of operations performed to process a given input compared to a densely activated model. In this way, for example, expressiveness and interpretability of high-parameter counting models can be achieved through computationally more efficient forward passes.

[0140] Input 2 can typically include or otherwise represent various types of data. Input 2 can include one type or many different types of data. Output 3 can be data of the same type or a different type compared to Input 2. Output 3 can include one type or many different types of data.

[0141] Example data types for input 2 or output 3 include natural language text data, software code data (e.g., source code, object code, machine code, or any other form of computer-readable instruction or programming language), machine code data (e.g., binary code, assembly code, or other forms of machine-readable instruction that can be executed directly by a computer's central processing unit), assembly code data (e.g., a low-level programming language that uses a symbolic representation of machine code instructions to program the processing unit), genetic data or other chemical or biochemical data, image data, audio data, audiovisual data, tactile data, biometric data, medical data, financial data, statistical data, geographic data, astronomical data, historical data, and generally sensor data (e.g., digital or analog values, such as voltage or other absolute or relative level measurements from real or artificial inputs, such as from audio sensors, light sensors, displacement sensors, etc.). Data can be raw or processed and can be in any format or mode.

[0142] In multimodal input 2 or output 3, example combinations of data types include image data and audio data, image data and natural language data, natural language data and software code data, image data and biometric data, sensor data and medical data, etc. It should be understood that any combination of data types in input 2 or output 3 can exist.

[0143] Example input 2 may include one or more data types, such as the example data types indicated above. Example output 3 may include one or more data types, such as the example data types indicated above. The data type of input 2 may be the same as or different from the data type of output 3. It should be understood that the example data types indicated above are provided for illustrative purposes only. The data types contemplated within the scope of this disclosure are not limited to those examples indicated above.

[0144] Figure 8 This is a block diagram illustrating an example implementation of an example machine learning model configured to process information sequences. For example, an example implementation of machine learning model 1 could include machine learning sequence processing model 4. The example system could pass input 2 to sequence processing model 4. Sequence processing model 4 could include one or more machine learning components. Sequence processing model 4 could process the data from input 2 to obtain input sequence 5. Input sequence 5 could include one or more input elements 5-1, 5-2, ..., 5-M, etc., obtained from input 2. Sequence processing model 4 could use prediction layer 6 to process input sequence 5 to generate output sequence 7. Output sequence 7 could include one or more output elements 7-1, 7-2, ..., 7-N, etc., generated based on input sequence 5. The system could generate output 3 based on output sequence 7.

[0145] Sequence processing models 4 may include one or more machine learning model components configured to ingest, generate, or otherwise infer sequences of information. For example, some example sequence processing models in the text domain are referred to as “large language models” or LLMs. See, for example, the PaLM 2 technical report, Google, https: / / ai.google / static / documents / palm2techreport.pdf (nd). Other example sequence processing models can operate in other domains, such as the image domain (see, for example, Dosovitskiy et al., An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale), arXiv:2010.11929v2 (June 3, 2021); the audio domain (see, for example, Agostinelli et al., AudioLM: Language Modeling Approach to Audio Generation), arXiv:2209.03143v2 (July 26, 2023); and the biochemical domain (see, for example, Jumper et al., Highly accurate protein structure prediction with AlphaFold), 596 Nature 583 (August 26, 2021). Sequence processing model 4 can process one or more types of data simultaneously. Sequence processing model 4 can include relatively large models (e.g., more parameters, high computational cost), relatively small models (e.g., fewer parameters, lightweight computation), or both.

[0146] Generally, sequence processing model 4 can use data from input 2 to obtain input sequence 5. For example, input sequence 5 may include a representation of the data from input 2 in a format understood by sequence processing model 4. One or more machine learning components of sequence processing model 4 may ingest data from input 2, parse the data into fragments compatible with the processing architecture of sequence processing model 4 (e.g., via “word segmentation”), and project the fragments into the input space associated with prediction layer 6 (e.g., via “embedding”).

[0147] Sequence processing model 4 can ingest data from input 2 and parse the data into a sequence of elements to obtain input sequence 5. For example, a portion of the input data from input 2 can be decomposed into segments, which collectively represent the content of said portion of the input data. The segments can provide the elements of the sequence.

[0148] In some cases, elements 5-1, 5-2, ..., 5-M can represent building blocks used to capture or express meaningful information in a specific data domain. For example, elements can describe "atomic units" across one or more domains. For instance, for a text input source, elements can correspond to a group of one or more word or sub-word components (such as a set of one or more characters).

[0149] For example, elements 5-1, 5-2, ..., 5-M can represent tokens obtained using a tokenizer. A tokenizer can process a given portion of the input source and output a series of tokens representing that portion of the input source (e.g., corresponding to the input elements 5-1, 5-2, ..., 5-M). Various tokenization methods can be used. For example, byte-pair encoding (BPE) can be used to tokenize a text input source. For example, see Kudo et al., SentencePiece: Asimple and language independent subword tokenizer and detokenizer for NeuralText Processing, Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing (System Demonstrations), pp. 66–71 (October 31–November 4, 2018), https: / / aclanthology.org / D18-2012.pdf. Tokenization of image-based input sources can be performed by extracting and serializing fragments from images.

[0150] Generally speaking, any data type can be serialized and processed into the input sequence 5. It should be understood that... Figure 8 The elements 5-1, 5-2, ..., 5-M described in the text can be lexical units or their embedded representations.

[0151] Prediction layer 6 can predict one or more output elements 7-1, 7-2, ..., 7-N based on the input elements. Prediction layer 6 may include one or more machine learning model architectures, such as one or more learning parameter layers, which manipulate and transform the input to extract higher-order meaning from the input elements 5-1, 5-2, ..., 5-M and the relationships between the input elements. In this way, for example, example prediction layer 6 can predict new output elements based on the context provided by the input sequence 5.

[0152] Prediction layer 6 can evaluate the associations between portions of the input sequence 5 and specific output elements. These associations can inform predictions about the likelihood that a particular output follows the input context. For example, consider the text snippet “The carpenter's toolbox was small and heavy. It was full of ___.” Example prediction layer 6 can identify that “It” refers back to “toolbox” by determining the relationships between corresponding embeddings. Example prediction layer 6 can also associate “it” with attributes of the toolbox, such as “small” and “heavy.” Based on these associations, for example, prediction layer 6 can assign a higher probability to the word “nails” than to the word “sawdust.”

[0153] A transformer is an example architecture that can be used in prediction layer 4. See, for example, Vaswani et al., Attention Is All You Need, arXiv:1706.03762v7 (August 2, 2023). A transformer is an example of a machine learning model architecture that uses an attention mechanism to compute associations between items within a context window. The context window can include a sequence containing an input sequence 5 and potentially one or more output elements 7-1, 7-2, ..., 7-N. A transformer block can include one or more attention layers and one or more post-attention layers (e.g., feedforward layers, such as a multilayer perceptron).

[0154] Prediction layer 6 can include other machine learning model architectures to complement or replace transformer-based architectures. For example, recurrent neural networks (RNNs) and long short-term memory (LSTM) models, as well as convolutional neural networks (CNNs), can also be used. In general, prediction layer 6 can leverage various artificial neural networks that can understand or generate sequences of information.

[0155] The output sequence 7 may include or otherwise represent the same or different data type as the input sequence 5. For example, the input sequence 5 may represent text data, and the output sequence 7 may represent text data. The input sequence 5 may represent image, audio, or audiovisual data, and the output sequence 7 may represent text data (e.g., describing image, audio, or audiovisual data). It should be understood that the prediction layer 6 and any other gap model components of the sequence processing model 4 can be configured to receive multiple data types from the input sequence 5 and output multiple data types in the output sequence 7.

[0156] Output sequence 7 can have various relationships with input sequence 5. Output sequence 7 can be a continuation of input sequence 5. Output sequence 7 can be a supplement to input sequence 5. Output sequence 7 can translate, transform, expand, or otherwise modify input sequence 5. Output sequence 7 can answer, evaluate, acknowledge, or otherwise respond to input sequence 5. Output sequence 7 can implement instructions provided via input sequence 5 (or describe instructions for implementing said instructions).

[0157] Output sequence 7 can be generated autoregressively. For example, for some applications, the output of one or more prediction layers 6 can be passed through one or more output layers (e.g., softmax layers) to obtain a probability distribution of an output vocabulary (e.g., a text or symbol vocabulary) conditioned on a set of input elements in a context window. In this way, output sequence 7 can be generated autoregressively, for example, by sampling possible next output elements; adding that element to the context window and regenerating the probability distribution based on the updated context window; and sampling possible next output elements, etc.

[0158] Output sequence 7 can also be generated non-autoregressively. For example, multiple output elements of output sequence 7 can be predicted together without explicit order conditions. See, for example, Saharia et al., Non-Autoregressive Machine Translation with Latent Alignments, arXiv:2004.07437v3 (November 16, 2020).

[0159] Output sequence 7 may include one or more parts or elements. In the example content generation configuration, output sequence 7 may include multiple elements corresponding to multiple parts of the generated output sequence (e.g., text sentences, values ​​of discrete waveforms, computer code, etc.). In the example classification configuration, output sequence 7 may include a single element associated with the classification output. For example, the output "vocabulary" may include a set of classes to which the input sequence will be classified. For example, a visual transformer block may pass latent state information to a multilayer perceptron, which outputs possible class values ​​associated with the input image.

[0160] Figure 9 This is a block diagram of an example technique for populating example input sequence 8. Input sequence 8 may include various functional elements forming part of the model infrastructure, such as element 8-0 obtained from task indicator 9, which signals to any model processing input sequence 8 that a specific task is being performed (e.g., helping to adapt the model's performance to that specific task). Input sequence 8 may include various data elements from different data modalities. For example, input modality 10-1 may include one data modality. Data-to-sequence model 11-1 may process the data from input modality 10-1 to project the data into a format compatible with input sequence 8 (e.g., determining one or more vectors of dimensions based on the dimensions of input sequence 8) to obtain elements 8-1, 8-2, 8-3. Another input modality 10-2 may include a different data modality. Data-to-sequence model 11-2 may project the data from input modality 10-2 into a format compatible with input sequence 8 to obtain elements 8-4, 8-5, 8-6. Another input modality 10-3 may include yet another different data modality. The data to sequence model 11-3 can project the data from the input mode 10-3 into a format compatible with the input sequence 8 to obtain elements 8-7, 8-8, and 8-9.

[0161] Input sequence 8 may be the same as or different from input sequence 5. Input sequence 8 may be a multimodal input sequence containing elements representing data from different modalities using a common dimensional representation. For example, the embedding space may have P dimensions. Input sequence 8 may be configured to contain multiple elements with P dimensions. In this way, for example, the example implementation can facilitate information extraction and inference across different data modalities by projecting data onto elements in the same embedding space to compare, combine, or otherwise compute between them.

[0162] For example, elements 8-0, ..., 8-9 can indicate specific locations within a multidimensional embedding space. Some elements can map to a discrete set of locations in the embedding space. For example, elements corresponding to discrete members in a predefined lexicon can map to discrete locations in the embedding space associated with those lexicons. Other elements can be continuously distributed across the embedding space. For example, some data types can be decomposed into continuously defined parts (e.g., image patches), which can be described using continuously distributed locations within the embedding space.

[0163] In some implementations, the expressive power of the embedding space may not be limited to meanings associated with any particular set of lexical units or other building blocks. For example, a contiguous embedding space can encode a series of higher-order information. Individual segments of information (e.g., lexical units) can be mapped to specific points in the space: for example, lexical units of the word "dog" can be projected to embedding values ​​that point to specific locations in the embedding space associated with dog-related information. Similarly, image patches of a dog on grass can be projected into the embedding space. In some implementations, the projection of the dog image can resemble the projection of the word "dog" and also resemble the projection of the word "grass," but potentially differ from both. In some implementations, the projection of the image patch may not be perfectly aligned with any single projection of a single word. In some implementations, the projection of the image patch can be aligned with a combination of the projections of the words "dog" and "grass." In this way, for example, a higher-order embedding space can encode information that is independent of the data modality expressing the information.

[0164] Task indicator 9 may include a model or model component configured to identify the task being performed and inject the input value represented by element 8-0 into input sequence 8, which signals which task is being performed. For example, the input value may be provided as a data type associated with an input modality and projected along with that input modality (e.g., the input value may be a text task label embedded along with other text data in the input; the input value may be a pixel-based representation of the task embedded along with other image data in the input; etc.). The input value may be provided as a data type different from or at least independent of other inputs. For example, the input value represented by element 8-0 may be learned within a contiguous embedding space.

[0165] Input modes 10⁻¹, 10⁻², and 10⁻³ can be associated with a variety of different data types (e.g., as described above with respect to input 2 and output 3).

[0166] Data-to-sequence models 11-1, 11-2, and 11-3 can be the same as or different from each other. Data-to-sequence models 11-1, 11-2, and 11-3 can be adapted to each corresponding input modality 10-1, 10-2, and 10-3. For example, a text data-to-sequence model can subdivide a portion of the input text and project the subdivisions onto elements in input sequence 8 (e.g., elements 8-1, 8-2, 8-3, etc.). An image data-to-sequence model can subdivide the input image and project these subdivisions onto elements in input sequence 8 (e.g., elements 8-4, 8-5, 8-6, etc.). An arbitrary data type data-to-sequence model can subdivide the input of that arbitrary data type and project the subdivisions onto elements in input sequence 8 (e.g., elements 8-7, 8-8, 8-9, etc.).

[0167] Data-to-sequence models 11-1, 11-2, and 11-3 can form part of machine learning sequence processing model 4. Data-to-sequence models 11-1, 11-2, and 11-3 can be trained jointly with machine learning sequence processing model 4 or independently of it. Data-to-sequence models 11-1, 11-2, and 11-3 can be trained end-to-end with machine learning sequence processing model 4.

[0168] Figure 10 This is a block diagram of an example model development platform 12, which facilitates the creation, adaptation, and refinement of example machine learning models (e.g., machine learning model 1, sequence processing model 4, etc.). The model development platform 12 can provide several different toolkits that developer systems can use to develop new or adapted machine learning models.

[0169] The model development platform 12 can provide one or more model libraries 13 containing building blocks for new models. Model libraries 13 can include one or more pre-trained base models 13-1 that can provide a backbone of processing power across a variety of tasks. Model libraries 13 can include one or more pre-trained expert models 13-2 that can focus on performance in a specific domain. Model libraries 13 can include various model primitives 13-3 that can provide low-level architectures or components (optionally pre-trained) that can be assembled in various arrangements as needed. Model primitives 13-3 can include libraries of pre-trained adapters or LoRA modules that can adapt the baseline base model to align its output with a desired performance profile, enhance model capabilities (e.g., to adapt to different input modalities, etc.), etc.

[0170] The model development platform 12 can receive selections of various model components 14. The model development platform 12 can transfer the selected model components 14 to the workbench 15, which combines the selected model components 14 into the development model 16.

[0171] Workbench 15 can facilitate further refinement and adaptation of the development model 16 by utilizing multiple different toolkits integrated with the model development platform 12. For example, workbench 15 can facilitate the use of model alignment toolkit 17 to align the development model 16 with expected performance profiles for various tasks.

[0172] The model alignment toolkit 17 can provide multiple tools for causing the development model 16 to generate outputs aligned with desired behavioral characteristics. Alignment can include increasing the accuracy, precision, recall, etc., of the model output. Alignment can include enforcing output styles, patterns, or other preferred characteristics of the model output. Alignment can be general or domain-specific. For example, the pre-trained base model 13-1 can start from an initial performance level across multiple domains. Alignment of the pre-trained base model 13-1 can include improving performance in a specific information or task domain (e.g., even at the expense of performance in another information or task domain).

[0173] The model alignment toolkit 17 can integrate one or more datasets 17-1 used to align development models 16. Selected datasets 17-1 may include labeled or unlabeled training data. Datasets 17-1 can be obtained from public domain datasets. Datasets 17-1 can also be obtained from private datasets associated with one or more developer systems for aligning customized machine learning models tailored for private use cases.

[0174] The pre-training pipeline 17-2 may include a machine learning model training workflow configured to update the development model 16 on a large-scale, potentially noisy dataset. For example, pre-training may utilize unsupervised learning techniques (e.g., denoising, etc.) to process a large number of training instances to update model parameters from an initial state and achieve the desired baseline performance. The pre-training pipeline 17-2 may utilize the unlabeled dataset in dataset 17-1 to perform pre-training. Workbench 15 may implement the pre-training pipeline 17-2 to pre-train the development model 16.

[0175] The fine-tuning pipeline 17-3 may include a machine learning model training workflow configured to refine the model parameters of the development model 16 using higher-quality data. The fine-tuning pipeline 17-3 can update the development model 16 through supervised training using the labeled dataset in dataset 17-1. The fine-tuning pipeline 17-3 can also update the development model 16 through reinforcement learning using reward signals from user feedback. Workbench 15 enables the fine-tuning pipeline 17-3 to fine-tune the development model 16.

[0176] Hint library 17-4 may include a set of inputs configured to elicit behavior aligned with desired performance criteria. Hint library 17-4 may include few-shot hints (e.g., inputs that provide examples of the desired model output to be added before the desired runtime query), thought chain hints (e.g., inputs that provide step-by-step inference within a paradigm to facilitate comprehensive inference by the model), and so on.

[0177] Sample hints can be retrieved from the available repository of hint library 17-4. Sample hints can be facilitated by one or more developer systems using workbench 15.

[0178] In some implementations, pre-trained or fine-tuned models can achieve satisfactory performance even when there are no paradigms in the input. For example, zero-shot hints can include inputs lacking paradigms. Zero-shot hints can be within the domain of the training dataset or outside the training domain.

[0179] Hint library 17-4 may include one or more hint engineering tools. Hint engineering tools can provide workflows for retrieving or learning optimized hint values. Hint engineering tools can facilitate the direct learning of hint values ​​(e.g., input element values) based on one or more training iterations. Workbench 15 can implement the hint engineering tools within development model 16.

[0180] Hint library 17-4 may include a pipeline for hint generation. For example, input can be generated using development model 16 itself or other machine learning models. In this way, for example, a first model can process information about the task and output input for a second model to process in order to perform steps of the task. The second model may be the same as or different from the first model. Workbench 15 can implement the hint generation pipeline within development model 16.

[0181] Hint library 17-4 may include a pipeline for context injection. For example, if additional context is provided for performing a specific task, the performance of development model 16 on that task can be improved. Hint library 17-4 may include software components configured to identify the desired context, retrieve the context from external sources (e.g., databases, sensors, etc.), and add the context to the input hints. Workbench 15 can implement the context injection pipeline in development model 16.

[0182] Although the various training examples described herein regarding model development platform 12 involve "pre-training" and "fine-tuning," it should be understood that the model alignment toolkit 17 can generally support a wide variety of training techniques suitable for training a wide range of machine learning models. Example training techniques may correspond to the example training method 600 described above.

[0183] The model development platform 12 may include a model plug-in toolkit 18. The model plug-in toolkit 18 may include a variety of tools configured to enhance the functionality of the machine learning model by integrating it with other systems, devices, and software components. For example, the machine learning model can use tools to improve performance quality where appropriate. For instance, deterministic tasks can be offloaded to dedicated tools instead of performing tasks probabilistically when the risk of error increases. For example, instead of autoregressively predicting solutions to a system of equations, the machine learning model can identify the tools invoked to obtain solutions and pass the system of equations to the appropriate tool. This tool can be a conventional equation solver that operates deterministically to solve the system of equations. The tool's output can be returned in response to the original query. In this way, tool usage can allow some example models to focus on the strengths of the machine learning model—e.g., understanding the intent in unstructured requests for a task—while enhancing model performance by offloading certain tasks to more focused tools to mechanically apply deterministic algorithms to well-defined problems.

[0184] The model plugin toolkit 18 may include a validation tool 18-1. The validation tool 18-1 may include tools capable of parsing and verifying the output of a machine learning model. The validation tool 18-1 may include engineered heuristics that establish certain thresholds applied to the model output. For example, the validation tool 18-1 may base the output of the machine learning model on a structured data source (e.g., to mitigate "illusion").

[0185] The model plugin toolkit 18 may include a toolkit 18-2 for implementing one or more tools, which may include scripts or other executable code that can be executed with the development model 16. The toolkit 18-2 may include one or more inputs configured to cause a machine learning model to implement the tools (e.g., few-shot hints that cause the model to output tool calls with correct syntax). For example, the toolkit 18-2 may include fine-tuned training data for training the model to use the tools.

[0186] The model plugin toolkit 18 may include interfaces for calling external application programming interfaces (APIs) 18-3. For example, as a supplement or alternative to directly implementing tool calls or tool code using development model 16, development model 16 may be aligned with output instructions that initiate API calls to send or obtain data via external systems.

[0187] The model plugin toolkit 18 can be integrated with the hint library 17-4 to create a catalog of available tools for use with the development model 16. For example, the model can receive a catalog of available tools in its input, and the model can generate output that selects a tool from the available tools and initiates a tool call for using that tool.

[0188] Model development platform 12 may include a suite of computational optimization tools 19 for optimizing the computational performance of development model 16. For example, tools for model compression 19-1 may allow development model 16 to be reduced in size while maintaining the desired performance level. For example, model compression 19-1 may include quantization workflows, weight pruning, and sparsification techniques. Tools for hardware acceleration 19-2 may facilitate the configuration of model storage and execution formats for optimal operation on different hardware resources. For example, hardware acceleration 19-2 may include tools for optimally sharding the model for distributed processing across multiple processing units to increase bandwidth, reduce uniform memory requirements, etc. Tools for distillation 19-3 may provide the ability to train a lighter model based on knowledge encoded in development model 16. For example, development model 16 may be a large, high-performance machine learning model optimized using model development platform 12. To obtain a lightweight model for operation in resource-constrained environments, the smaller model may be a “student model” that learns from and imitates development model 16 as the “teacher model.” In this way, for example, the investment in learning and developing the parameters and configuration of model 16 can be efficiently transferred to a smaller model for more efficient inference.

[0189] Workbench 15 can implement one or more of the toolset implemented in model development platform 12, or none of the toolset. Workbench 15 can output output model 20 based on development model 16. Output model 20 can be a deployment version of development model 16. Output model 20 can be a development or training checkpoint of development model 16. Output model 20 can be a distilled version, compressed version, or otherwise optimized version of development model 16.

[0190] Figure 11 This is a block diagram of an example training process for training a machine learning development model 16. One or more parts of the example training process can be implemented by a computing system (e.g., a computing system such as that described with reference to other figures) including one or more computing devices. Each corresponding part of the example training process can be performed by any one (or any combination of) of the one or more computing devices. Furthermore, one or more parts of the example training process can be implemented on the hardware components of the apparatus described herein, for example, to train one or more systems or models. Figure 11 For illustrative and discussion purposes, elements executed in a particular order are depicted. Those skilled in the art will understand using the disclosure provided herein that elements of any method discussed herein can be adapted, rearranged, extended, omitted, combined, or modified in various ways without departing from the scope of this disclosure. Figure 11 The descriptions are for illustrative purposes only and refer to elements / terms related to other systems and diagrams, and are not intended to be limiting. Additionally or alternatively, one or more parts of the example training process may be performed by other systems.

[0191] Initially, development model 16 can be kept in its initial state as initialization model 21. Development model 16 can be initialized using weight values. The initial weight values ​​can be random or based on an initialization pattern. The initial weight values ​​can be based on previous pre-training for the same or different models.

[0192] The initialization model 21 can undergo pre-training in the pre-training phase 22. The pre-training phase 22 can be implemented using one or more pre-training pipelines 17-2 on data from dataset 17-1. For example, if the initialization model 21 has already been pre-trained (e.g., the development model 16 contains, is, or is based on a pre-trained base model or expert model), pre-training can be omitted.

[0193] Then, the pre-trained model 23 can be a new version of the development model 16, which can remain as the development model 16 or a new development model. If the development model 16 has already been pre-trained, the pre-trained model 23 can be in its initial state. The pre-trained model 23 can undergo fine-tuning in the fine-tuning phase 24. The fine-tuning phase 24 can be implemented using one or more fine-tuning pipelines 17-3 on data from dataset 17-1. For example, fine-tuning can be omitted if the pre-trained model has satisfactory performance, if the model has already been fine-tuned, or if other tuning methods are preferred.

[0194] The fine-tuned model 29 can then be a new version of the development model 16, which can be maintained as either the development model 16 or a new development model. If the development model 16 has already been fine-tuned, the fine-tuned model 29 can be in its initial state. The fine-tuned model 29 can undergo refinement 26 using user feedback. For example, refinement 26 using user feedback can optionally include reinforcement learning based on human feedback from human users of the fine-tuned model 25. Since reinforcement learning can take the form of fine-tuning, it should be understood that the fine-tuning phase 24 can include a phase for refinement 26 using user feedback. Refinement 26 using user feedback can produce a refined model 27. The refined model 27 can be output to the downstream system 28 for deployment or further development.

[0195] In some implementations, computational optimization operations can be applied before, during, or after each stage. For example, the initialization model 21 can undergo computational optimization 29-1 (e.g., using computational optimization toolkit 19) before the pre-training stage 22. The pre-trained model 23 can undergo computational optimization 29-2 (e.g., using computational optimization toolkit 19) before the fine-tuning stage 24. The fine-tuned model 25 can undergo computational optimization 29-3 (e.g., using computational optimization toolkit 19) before the refinement 26 utilizing user feedback. The refined model 27 can undergo computational optimization 29-4 (e.g., using computational optimization toolkit 19) before outputting to the downstream system 28. Computational optimizations 29-1, ..., 29-4 can all be the same, all be different, or include at least some different optimization techniques.

[0196] Figure 12 This is a block diagram of an inference system used to operate one or more machine learning models 1 to perform inference (e.g., for training, for deployment, etc.). Model host 31 can receive machine learning model 1. Model host 31 can host one or more model instances 31-1, which can be one or more instances of one or more models. Model host 31 can use available computing resources 31-2 associated with model host 31 to host model instances 31-1.

[0197] Model host 31 can perform inference on behalf of one or more clients 32. Client 32 can transmit input request 33 to model host 31. Using input request 33, model host 31 can obtain input 2 to feed into machine learning model 1. Machine learning model 1 can process input 2 to generate output 3. Using output 3, model host 31 can return output payload 34 in response to input request 33 from client 32. Output payload 34 can include or be based on output 3.

[0198] Model host 31 can utilize various other resources and tools to enhance the inference task. For example, model host 31 can communicate with tool interface 35 to facilitate the use of tools by model instance 31-1. Tool interface 35 may include local or remote APIs. Tool interface 35 may include integrated scripts or other software functions. Model host 31 can use online learning interface 36 to facilitate continuous improvement of machine learning model 1. For example, online learning interface 36 can be used in reinforcement learning loops to retrieve user feedback on inference served by model host 31. Model host 31 can access runtime data source 37 to enhance input 2 using additional contextual information. For example, runtime data source 37 may include knowledge graph 37-1 that facilitates structured information retrieval for information associated with input request 33 (e.g., search engine service). Runtime data source 37 may include public or private, external or local database 37-2 that can store information associated with input request 33 for enhancing input 2. The runtime data source 37 may include account data 37-3, which can be retrieved in association with the user account corresponding to the client 32 to customize the behavior of the model host 31 accordingly.

[0199] The model host 31 may be implemented by one or more computing devices or systems. The client 2 may be implemented by one or more computing devices or systems, which may include computing devices or systems shared with the model host 31.

[0200] For example, model host 31 may operate on a server system that provides machine learning services (e.g., via a local area network or wide area network) to client devices operating client 32. The client device may be an end-user device used by an individual. The client device may also be a server system that operates client 32 to provide various functionalities as services to downstream end-user devices.

[0201] In some implementations, model host 31 may operate on the same device or system as client 32. Model host 31 may be a machine learning service that runs on the device to provide machine learning capabilities to one or more applications operating on the client device, which may include the application implementing client 32. Model host 31 and client 32 may be part of the same application. For example, model host 31 may be a subroutine or method implemented as part of the application, and client 32 may be another subroutine or method that uses model host 31 to perform inference functions within the application. It should be understood that model host 31 and client 32 may have various different configurations.

[0202] Model instance 31-1 may include one or more machine learning models that can be used to perform inference. Model instance 31-1 may include weights or other model components stored in persistent storage, temporary cache, or loaded into high-speed memory. Model instance 31-1 may include multiple instances of the same model (e.g., for parallel execution of more requests on the same model). Model instance 31-1 may include instances of different models. Model instance 31-1 may include cached intermediate states of active or inactive models, which are used to accelerate inference for those models. For example, an inference session with a particular model can generate a significant amount of computational results that can be reused for future inference runs (e.g., using a KV cache for a transformer-based model). These computational results can be stored in association with the inference session, allowing for more efficient execution when the session resumes.

[0203] Computing resource 31-2 may include one or more processors (central processing unit, graphics processing unit, tensor processing unit, machine learning accelerator, etc.) connected to one or more memory devices. Computing resource 31-2 may include a dynamic pool of available resources shared with other processes. Computing resource 31-2 may include a memory device large enough to fit an entire model instance into a single memory instance. Computing resource 31-2 may also share model instances across multiple memory devices (e.g., using data parallelization or tensor parallelization). Doing so can increase parallelization or execute large models using multiple memory devices that, individually, might not be able to fit the entire model into memory.

[0204] Input request 33 may include data for input 2. Model host 31 can process input request 33 to obtain input 2. Input 2 can be obtained directly from input request 33 or retrieved using input request 33. Input request 33 can be submitted to model host 31 via API.

[0205] Model host 31 can perform inference on multiple batch input requests 33 in parallel. For example, model instance 31-1 can be configured with an input structure having a batch dimension. Individual inputs 2 can be distributed across the batch dimension (e.g., rows of an array). Individual inputs 2 can include completely different contexts. Individual inputs 2 can be multiple inference steps for the same task. Individual inputs 2 can be interleaved in the input structure, such that any given inference loop can operate on different parts of the corresponding inputs 2. In this way, for example, model host 31 can perform inference on batches in parallel, such that output 3 can also contain the batch dimension and return the inference results of the batched inputs 2 in parallel. In this way, for example, multiple batch input requests 33 can be processed in parallel to achieve higher throughput of output payload 34.

[0206] The output payload 34 may include or be based on the output 3 from the machine learning model 1. The model host 31 may process the output 3 to obtain the output payload 34. This may include linking multiple rounds of inference (e.g., iteratively, recursively, across the same or different models) to obtain the final output of the task to be returned in the output payload 34. The output payload 34 may be transferred to the client 32 via an API.

[0207] Online learning interface 36 can facilitate reinforcement learning for machine learning model 1. Online learning interface 36 can facilitate reinforcement learning with human feedback (RLHF). Online learning interface 36 can facilitate federated learning for machine learning model 1.

[0208] The model host 31 can access libraries of pre-trained adapters or LoRA modules that can adapt the baseline model to align its output with a desired performance profile, enhance model capabilities (e.g., to adapt to different input modalities), etc. For example, the model host 31 can receive input requests to load a custom model, and the model host 31 can retrieve one or more components to adapt the baseline model to a custom profile. The model host 31 can determine which specific functionality is required for a specific task (e.g., based on the output of the model with pre-processed input) and retrieve pre-trained components accordingly.

[0209] Model host 31 can execute machine learning model 1 to perform inference for various tasks using various types of data. For example, various inputs 2 and outputs 3 can be used for various different tasks. In some implementations, input 2 may be or otherwise represent image data. Machine learning model 1 can process image data to generate outputs. As an example, machine learning model 1 can process image data to generate image recognition outputs (e.g., image data identification, latent embedding of image data, encoded representation of image data, hashing of image data, etc.). As another example, machine learning model 1 can process image data to generate image segmentation outputs. As another example, machine learning model 1 can process image data to generate image classification outputs. As another example, machine learning model 1 can process image data to generate image data modification outputs (e.g., image data alterations, etc.). As another example, machine learning model 1 can process image data to generate encoded image data outputs (e.g., encoded and / or compressed representations of image data, etc.). As another example, machine learning model 1 can process image data to generate augmented image data outputs. As another example, machine learning model 1 can process image data to generate prediction outputs.

[0210] In some implementations, the task is a computer vision task. In some cases, input 2 includes pixel data from one or more images, and the task is an image processing task. For example, an image processing task could be image classification, where the output is a set of scores, each score corresponding to a different object class and representing the probability that one or more images depict an object belonging to that object class. An image processing task could be object detection, where the image processing output identifies one or more regions in one or more images, and for each region, identifies the probability that the region depicts an object of interest. As another example, an image processing task could be image segmentation, where the image processing output defines a corresponding probability for each class in a predetermined set of categories for each pixel in one or more images. For example, this set of categories could be foreground and background. As another example, this set of categories could be object classes. As another example, an image processing task could be depth estimation, where the image processing output defines a corresponding depth value for each pixel in one or more images. As another example, an image processing task could be motion estimation, where the network input includes multiple images, and the image processing output defines the motion of the scene depicted at each pixel in one of the input images between the images in the network input.

[0211] In some implementations, input 2 can be or otherwise represent natural language data. Machine learning model 1 can process the natural language data to generate output. As an example, machine learning model 1 can process natural language data to generate language-encoded output. As another example, machine learning model 1 can process natural language data to generate latent text embedding output. As another example, machine learning model 1 can process natural language data to generate translation output. As another example, machine learning model 1 can process natural language data to generate classification output. As another example, machine learning model 1 can process natural language data to generate text segmentation output. As another example, machine learning model 1 can process natural language data to generate semantic intent output. As another example, machine learning model 1 can process natural language data to generate expanded text or natural language output (e.g., text or natural language data of higher quality than the input text or natural language, etc.). As another example, machine learning model 1 can process natural language data to generate predictive output (e.g., the next part of one or more predictions of natural language content).

[0212] In some implementations, input 2 can be or otherwise represent speech data (e.g., data describing spoken natural language, such as audio data, text data, etc.). Machine learning model 1 can process the speech data to generate output. As an example, machine learning model 1 can process speech data to generate speech recognition output. As another example, machine learning model 1 can process speech data to generate speech translation output. As another example, machine learning model 1 can process speech data to generate latent embedding output. As another example, machine learning model 1 can process speech data to generate encoded speech output (e.g., encoded and / or compressed representations of speech data, etc.). As another example, machine learning model 1 can process speech data to generate amplified speech output (e.g., speech data of higher quality than the input speech data, etc.). As another example, machine learning model 1 can process speech data to generate text representation output (e.g., text representations of the input speech data, etc.). As another example, machine learning model 1 can process speech data to generate predictive output.

[0213] In some implementations, input 2 can be or otherwise represent latent encoded data (e.g., a latent space representation of the input). Machine learning model 1 can process the latent encoded data to generate an output. As an example, machine learning model 1 can process the latent encoded data to generate an identification output. As another example, machine learning model 1 can process the latent encoded data to generate a reconstruction output. As another example, machine learning model 1 can process the latent encoded data to generate a search output. As another example, machine learning model 1 can process the latent encoded data to generate a re-clustering output. As yet another example, machine learning model 1 can process the latent encoded data to generate a prediction output.

[0214] In some implementations, input 2 may be or otherwise represent statistical data. Statistical data may be, represent, or otherwise include data calculated and / or computed from another data source. Machine learning model 1 can process the statistical data to generate output. As an example, machine learning model 1 can process the statistical data to generate an identification output. As another example, machine learning model 1 can process the statistical data to generate a prediction output. As another example, machine learning model 1 can process the statistical data to generate a classification output. As another example, machine learning model 1 can process the statistical data to generate a segmentation output. As another example, machine learning model 1 can process the statistical data to generate a visualization output. As another example, machine learning model 1 can process the statistical data to generate a diagnostic output.

[0215] In some implementations, input 2 can be or otherwise represent sensor data. Machine learning model 1 can process the sensor data to generate output. As an example, machine learning model 1 can process sensor data to generate identification output. As another example, machine learning model 1 can process sensor data to generate prediction output. As another example, machine learning model 1 can process sensor data to generate classification output. As another example, machine learning model 1 can process sensor data to generate segmentation output. As another example, machine learning model 1 can process sensor data to generate visualization output. As another example, machine learning model 1 can process sensor data to generate diagnostic output. As another example, machine learning model 1 can process sensor data to generate detection output.

[0216] In some implementations, machine learning model 1 can be configured to perform tasks including encoding input data to achieve reliable and / or efficient transmission or storage (and / or corresponding decoding). For example, the task could be an audio compression task. The input could include audio data, and the output could include compressed audio data. In another example, the input includes visual data (e.g., one or more images or videos), the output includes compressed visual data, and the task is a visual data compression task. In another example, the task could include generating embeddings for input data (e.g., input audio or visual data). In some cases, the input includes audio data representing spoken utterances, and the task is a speech recognition task. The output could include text output mapped to spoken utterances. In some cases, the task includes encrypting or decrypting the input data. In some cases, the task includes microprocessor performance tasks such as branch prediction or memory address translation.

[0217] In some implementations, the task is a generation task, and machine learning model 1 can be configured to output content generated based on input 2. For example, input 2 can be, or otherwise represent, data of one or more modalities, which encodes the context used to generate additional content.

[0218] In some implementations, the task can be a text completion task. Machine learning model 1 can be configured to process input 2 representing text data and generate output 3, whereby the output represents additional text data following the text sequence of input 2. For example, machine learning model 1 can be configured to generate output 3 to complete a sentence, paragraph, or section of text following a portion of the text represented by input 2.

[0219] In some implementations, the task can be an instruction-following task. Machine learning model 1 can be configured to process input 2 representing instructions for performing a function and generate output 3 that advances towards a goal satisfying the instruction function (e.g., at least one step of a multi-step process for performing the function). Output 3 can represent data of the same or different modality as input 2. For example, input 2 can represent text data (e.g., natural language instructions for a task to be performed), and machine learning model 1 can process input 2 to generate output 3 representing text data in response to the instructions (e.g., natural language response, programming language response, machine language response, etc.). Input 2 can represent image data (e.g., image-based instructions for a task to be performed, optionally accompanied by text instructions), and machine learning model 1 can process input 2 to generate output 3 representing text data in response to the instructions (e.g., natural language response, programming language response, machine language response, etc.). One or more outputs 3 can be generated iteratively or recursively to process and complete steps toward completing the requested function in sequence. For example, the initial output can be executed by an external system or processed by machine learning model 1 to complete the initial steps of performing the function. Multiple steps can be performed, in which the final output is obtained in response to the initial instruction.

[0220] In some implementations, the task can be a question-answering task. Machine learning model 1 can be configured to process input 2 representing a question to be answered and generate output 3 that advances towards a goal of returning an answer to the question (e.g., at least one step of a multi-step process for performing the function). Output 3 can represent data of the same or different modality as input 2. For example, input 2 can represent text data (e.g., natural language instructions for a task to be performed), and machine learning model 1 can process input 2 to generate output 3 representing text data in response to the question (e.g., a natural language response, a programming language response, a machine language response, etc.). Input 2 can represent image data (e.g., image-based instructions for a task to be performed, optionally accompanied by text instructions), and machine learning model 1 can process input 2 to generate output 3 representing text data in response to the question (e.g., a natural language response, a programming language response, a machine language response, etc.). One or more outputs 3 can be generated iteratively or recursively to process and complete steps toward answering the question sequentially. For example, the initial output can be executed by an external system or processed by a machine learning model 1 to complete the initial steps to obtain an answer to the question (e.g., querying a database, performing calculations, executing scripts, etc.). Multiple steps can be performed to obtain the final output in response to the question.

[0221] In some implementations, the task can be an image generation task. Machine learning model 1 can be configured to process input 2, which represents context regarding a desired portion of the image content. Context can include text data, image data, audio data, etc. Machine learning model 1 can be configured to generate output 3, which represents image data depicting the image in context. For example, machine learning model 1 can be configured to generate pixel data of an image. The values ​​of the channels associated with pixels in the pixel data can be selected based on context (e.g., based on probabilities determined according to the context).

[0222] In some implementations, the task can be an audio generation task. Machine learning model 1 can be configured to process input 2, which represents context regarding a desired portion of the audio content. Context can include text data, image data, audio data, etc. Machine learning model 1 can be configured to generate output 3, which represents audio data relevant to the context. For example, machine learning model 1 can be configured to generate waveform data in the form of an image (e.g., a spectrogram). The values ​​of channels associated with pixels in the image can be selected based on the context. Machine learning model 1 can be configured to generate waveform data in the form of a sequence of discrete samples of a continuous waveform. The values ​​of the sequence can be selected based on the context (e.g., based on probabilities determined according to the context).

[0223] In some implementations, the task can be a data generation task. Machine learning model 1 can be configured to process input 2, which represents context regarding a desired portion of the data (e.g., data from various data domains, such as sensor data, image data, multimodal data, statistical data, etc.). For example, the desired data can be synthetic data used to train other machine learning models. The context can include any data type. Machine learning model 1 can be configured to generate output 3, which represents data aligned with the desired data. For example, machine learning model 1 can be configured to generate data values ​​to populate a dataset. The values ​​of data objects can be selected based on context (e.g., based on probabilities determined according to the context).

[0224] Figure 13This is a block diagram of an example networked computing system capable of implementing aspects of the exemplary implementations of this disclosure. The system may include multiple computing devices and systems communicatively coupled via network 49. Example computing device 50 is described as an example of a computing device capable of implementing any aspect of this disclosure (e.g., implementing model host 31, client 32, or both). Example server computing system 60 is described as an example of a server computing system capable of implementing any aspect of this disclosure (e.g., implementing model host 31, client 32, or both). Computing device 50 and server computing system 60 may interact collaboratively (e.g., via network 49) to implement any aspect of this disclosure (e.g., implementing model host 31, client 32, or both). Model development platform system 70 is an example system that can host or provide a model development platform 12 for developing machine learning models. Third-party system 80 is an example system that any of computing device 50, server computing system 60, or model development platform system 70 can interact with when performing various aspects of this disclosure (e.g., using third-party tools, accessing third-party databases or other resources, etc.).

[0225] Network 49 can be any type of communication network, such as a local area network (e.g., intranet), a wide area network (e.g., the Internet), or some combination thereof, and can include any number of wired or wireless links. Generally, communication over network 49 can be conducted using a wide variety of communication protocols (e.g., TCP / IP, HTTP, SMTP, FTP), encodings or formats (e.g., HTML, XML), or protection schemes (e.g., VPN, Secure HTTP, SSL) via any type of wired or wireless connection. Network 49 can also be implemented via a system bus. For example, Figure 13 One or more devices or systems may be located in the same place as, contained in, or otherwise integrated into one or more other devices or systems.

[0226] Computing device 50 can be any type of computing device, such as a personal computing device (e.g., a laptop or desktop computer), a mobile computing device (e.g., a smartphone or tablet computer), a game console or controller, a wearable computing device, an embedded computing device, a server computing device, a virtual machine operating on a host device, or any other type of computing device. Computing device 50 can be a client computing device. Computing device 50 can be an end-user computing device. Computing device 50 can be a computing device that provides services to an end user (who may use another computing device to interact with computing device 50).

[0227] Computing device 50 may include one or more processors 51 and memory 52. ​​Processor 51 may be any suitable processing device (e.g., processor core, microprocessor, ASIC, FPGA, controller, microcontroller, etc.) and may be a single processor or multiple processors operatively connected. Memory 52 may include one or more non-transitory computer-readable storage media, such as HBM, RAM, ROM, EEPROM, EPROM, flash memory devices, disks, etc., and combinations thereof. Memory 52 may store data 53 and instructions 54, which may be executed by processor 51 to cause computing device 50 to perform operations. These operations may implement any or more features described herein. These operations may implement the exemplary methods and techniques described herein.

[0228] The computing device 50 may also include one or more input components for receiving user input. For example, the user input component may be a touch-sensitive component (e.g., a touch-sensitive display or touchpad) that is sensitive to the touch of a user input object (e.g., a finger or stylus). The touch-sensitive component can be used to implement a virtual keyboard. Other example user input components include a microphone, camera, LiDAR, physical keyboard or other buttons, or other components through which the user can provide input.

[0229] Computing device 50 may store or include one or more machine learning models 55. Machine learning model 55 may include one or more machine learning models 1, such as sequence processing model 4. Machine learning model 55 may include one or more model instances 31-1. Machine learning model 55 may be received from server computing system 60, model development platform system 70, third-party system 80 (e.g., application distribution platform), or developed locally on computing device 50. Machine learning model 55 may be loaded into memory 52 and used by processor 51 or otherwise implemented. Computing device 50 may implement multiple parallel instances of machine learning model 55.

[0230] Server computing system 60 may include one or more processors 61 and memory 62. Processor 61 may be any suitable processing device (e.g., processor core, microprocessor, ASIC, FPGA, controller, microcontroller, etc.) and may be a single processor or multiple processors operatively connected. Memory 62 may include one or more non-transitory computer-readable storage media, such as HBM, RAM, ROM, EEPROM, EPROM, flash memory devices, disks, etc., and combinations thereof. Memory 62 may store data 63 and instructions 64, which may be executed by processor 61 to cause server computing system 60 to perform operations. These operations may implement any or more features described herein. These operations may implement the exemplary methods and techniques described herein.

[0231] In some implementations, server computing system 60 includes one or more server computing devices or is otherwise implemented by one or more server computing devices. In instances where server computing system 60 includes multiple server computing devices, such server computing devices may operate according to a sequential computing architecture, a parallel computing architecture, or some combination thereof.

[0232] Server computing system 60 may store or otherwise include one or more machine learning models 65. Machine learning model 65 may be the same as or different from machine learning model 55. Machine learning model 65 may include one or more machine learning models 1, such as sequence processing model 4. Machine learning model 65 may include one or more model instances 31-1. Machine learning model 65 may be received from computing device 50, model development platform system 70, third-party system 80, or developed locally on server computing system 60. Machine learning model 65 may be loaded into memory 62 and used by processor 61 or otherwise implemented. Server computing system 60 may implement multiple parallel instances of machine learning model 65.

[0233] In the example configuration, machine learning model 65 may be included in or otherwise stored and implemented by server computing system 60 to establish a client-server relationship with computing device 50 for providing model inference. For example, server computing system 60 may implement model host 31 on behalf of client 32 on computing device 50. For example, machine learning model 65 may be implemented by server computing system 60 as part of a web service (e.g., a remote machine learning model hosting service, such as an online interface for performing machine learning model operations on server computing system 60 over a network). For example, server computing system 60 may communicate with computing device 50 via a local intranet or internet connection. For example, computing device 50 may be a workstation or endpoint communicating with server computing system 60, where the implementation of machine learning model 65 is managed by server computing system 60 to remotely perform inference (e.g., for runtime or training operations), where output is returned (e.g., projected, streamed, etc.) to computing device 50. Machine learning model 65 may work collaboratively or interoperably with machine learning model 55 on computing device 50 to perform various tasks.

[0234] The model development platform system 70 may include one or more processors 71 and memory 72. Processor 71 may be any suitable processing device (e.g., processor core, microprocessor, ASIC, FPGA, controller, microcontroller, etc.) and may be a single processor or multiple processors operatively connected. Memory 72 may include one or more non-transitory computer-readable storage media, such as HBM, RAM, ROM, EEPROM, EPROM, flash memory devices, disks, etc., and combinations thereof. Memory 72 may store data 73 and instructions 74, which may be executed by processor 71 to cause the model development platform system 70 to perform operations. These operations may implement any one or more features described herein. These operations may implement the example methods and techniques described herein. Example operations include the functionality described herein with respect to model development platform 12. This functionality, and other functionalities, may be implemented by developer tools 75.

[0235] The third-party system 80 may include one or more processors 81 and memory 82. Processor 81 may be any suitable processing device (e.g., processor core, microprocessor, ASIC, FPGA, controller, microcontroller, etc.) and may be a single processor or multiple processors operatively connected. Memory 82 may include one or more non-transitory computer-readable storage media, such as HBM, RAM, ROM, EEPROM, EPROM, flash memory devices, disks, etc., and combinations thereof. Memory 82 may store data 83 and instructions 84, which may be executed by processor 81 to cause the third-party system 80 to perform operations. These operations may implement any one or more features described herein. These operations may implement the example methods and techniques described herein. Example operations include the functionality described herein with respect to tools and other external resources (e.g., third-party resource 85) invoked when training machine learning models 1, 4, 16, 20, 55, 65, etc., or performing inference using the machine learning models.

[0236] Figure 14An example arrangement of a computing system that can be used to implement the computing system disclosed herein is shown. Other computing system configurations may also be used. For example, in some implementations, one or both of computing system 50 or server computing system 60 may implement all or part of the operation of model development platform system 70. For example, computing system 50 or server computing system 60 may implement developer tool 75 (or extensions thereof) to develop, update / train, or refine machine learning models 1, 4, 16, 20, 55, 65, etc., using one or more techniques described herein with respect to model alignment toolkit 17. In this way, for example, computing system 50 or server computing system 60 may develop, update / train, or refine machine learning models based on local datasets (e.g., for model personalization / customization, as permitted by user data preference selection).

[0237] Figure 15 This is a block diagram of an example computing device 98 implemented according to an example embodiment of the present disclosure. The computing device 98 may be a user computing device or a server computing device (e.g., computing device 50, server computing system 60, etc.). The computing device 98 may implement model host 31. For example, the computing device 98 may include multiple applications (e.g., application 1 to N). Each application may contain its own machine learning library and machine learning model. For example, each application may include a machine learning model. Example applications include text messaging applications, email applications, dictation applications, virtual keyboard applications, browser applications, etc. Figure 14 As shown, each application can communicate with multiple other components of the computing device, such as, for example, one or more sensors, a context manager, a device state component, or additional components. In some implementations, each application can communicate with each device component using an API (e.g., a public API). In some implementations, the API used by each application is application-specific.

[0238] Figure 15 This is a block diagram of an example computing device 99 implemented according to an example embodiment of the present disclosure. Computing device 99 may be the same as or different from computing device 98. Computing device 99 may be a user computing device or a server computing device (e.g., computing device 50, server computing system 60, etc.). Computing device 98 may implement model host 31. For example, computing device 99 may include multiple applications (e.g., applications 1 to N). Each application may communicate with a central intelligence layer. Example applications include text messaging applications, email applications, dictation applications, virtual keyboard applications, browser applications, etc. In some implementations, each application may use an API (e.g., a common API across all applications) to communicate with the central intelligence layer (and the model stored therein).

[0239] The central intelligence layer can include multiple machine learning models. For example, such as Figure 15 As shown, a corresponding machine learning model can be provided for each application, and this corresponding machine learning model is managed by a central intelligent layer. In other implementations, two or more applications can share a single machine learning model. For example, in some implementations, the central intelligent layer can provide a single model for all applications. In some implementations, the central intelligent layer is included within the operating system of the computing device 99 or otherwise implemented by the operating system.

[0240] The central intelligence layer can communicate with the central device data layer. The central device data layer can be a centralized data storage repository for computing device 99. For example... Figure 15 As shown, the central device data layer can communicate with multiple other components of the computing device, such as, for example, one or more sensors, a context manager, a device state component, or additional components. In some implementations, the central device data layer can communicate with each device component using an API (e.g., a private API).

[0241] This paper discusses technologies related to servers, databases, software applications, and other computer-based systems, as well as the actions taken and the information sent to and from such systems. The inherent flexibility of computer-based systems allows for a wide range of possible configurations, combinations, and partitions of tasks and functionality between and within components. For example, the processes discussed in this paper can be implemented using a single device or component, or multiple devices or components working in combination. Databases and applications can be implemented on a single system or distributed across multiple systems. Distributed components can operate sequentially or in parallel.

[0242] While the subject matter has been described in detail with respect to various specific example embodiments, each example is provided by way of explanation and not limitation. Modifications, alterations, and equivalents to such embodiments will be readily apparent to those skilled in the art upon understanding the foregoing. Therefore, this disclosure does not exclude such modifications, alterations, or additions to the subject matter that will be readily understood by those of ordinary skill in the art. For example, features shown or described as part of an embodiment may be used with another embodiment to produce yet another embodiment. Therefore, this disclosure is intended to cover such modifications, alterations, and equivalents.

[0243] Various aspects of this disclosure have been described with reference to their illustrative embodiments. Any and all features of the appended claims can be combined or rearranged in any possible manner, including combinations of claims not expressly listed together, because the illustrative claims dependencies listed herein should not be construed as limiting the scope of possible combinations of features disclosed herein. Therefore, the scope of this disclosure is illustrative rather than limiting, and this disclosure does not exclude such modifications, alterations, or additions to the subject matter that will be readily understood by one of ordinary skill in the art. Furthermore, terms are described herein using lists of illustrative elements connected by conjunctions such as “and,” “or,” “but,” etc. It should be understood that such conjunctions are provided for illustrative purposes only. For example, a sequence of terms and other items connected by a specific conjunction such as “or” can refer to “and / or,” “at least one of,” “any combination of,” etc., of the illustrative elements listed therein. Terms such as “based on” should be understood as “at least partially based on.”

[0244] The term "possibly" should be understood as referring to the possibility of a feature in various implementations, rather than a capability that must exist in every implementation. For example, the phrase "X can execute Y" should be understood as indicating that in various implementations, X may be configured to execute Y, rather than indicating that Y must always be able to execute in every instance of X. It should be understood that in various implementations, X may not be able to execute Y and is still within the scope of this disclosure.

[0245] The term "may" should be understood as referring to the possibility of a feature in various implementations, rather than a capability that must exist in every implementation. For example, the phrase "X can execute Y" should be understood as indicating that in various implementations, X may be configured to execute Y, rather than indicating that X must always be able to execute Y in every instance. It should be understood that in various implementations, X may not be able to execute Y and is still within the scope of this disclosure.

Claims

1. A computer-implemented method, comprising: The input data is processed using a sequence processing model to generate a first response as the output of the sequence processing model, wherein the first response includes a first word sequence; Update the output response corpus based on the first response; and The sequence processing model is used to process the input data to generate an additional response as an additional output of the sequence processing model, wherein: The additional response includes an additional lexical sequence; and The sequence processing model utilizes a bias sampling method, which biases the sampling of the additional lexical sequence based on a distance metric. The bias sampling method selects one or more additional lexical units that are different from those contained in the output response corpus based on the distance metric.

2. The computer-implemented method as described in claim 1, further comprising: The output response corpus is updated based on the additional response.

3. The computer-implemented method as described in claim 2, further comprising: The output response corpus is provided as an additional output of the sequence processing model.

4. The computer-implemented method of claim 1, wherein, The distance metric includes the Levenshtein distance.

5. The computer-implemented method as described in claim 1, wherein, The bias sampling method for biasing the sampling of the additional lexical sequence includes: The candidate nouns are penalized based on the distance metric to modify their probability distribution, wherein the modified probability distribution reduces the probability that the penalized candidate noun is included in the additional response.

6. The computer-implemented method of claim 5, wherein, The penalized candidate noun is associated with a candidate noun that does not meet a distance metric threshold, which indicates the degree of semantic difference between the candidate noun and the noun contained in the output response corpus.

7. The computer-implemented method of claim 1, wherein, The one or more additional lexical units that are different from those contained in the output response corpus include one or more semantic differences.

8. The computer-implemented method of claim 1, further comprising: The output response corpus is analyzed to determine one or more basic lexical units, which indicate probability distribution thresholds.

9. The computer-implemented method of claim 8, wherein, The probability distribution threshold is related to the degree of correctness.

10. The computer-implemented method of claim 8, wherein, The additional response includes one or more of the basic terms.

11. The computer-implemented method of claim 1, further comprising: While generating the first response, another response is generated as another output of the sequence processing model, wherein the other response comprises another word sequence; and The output response corpus is updated in real time based on the other response.

12. The computer-implemented method of claim 1, wherein, The sequence processing model includes at least one of the following: (i) a large language model (LLM) or (ii) a large multimodal model (LMM).

13. The computer-implemented method of claim 1, wherein, The one or more additional lexical units include text data.

14. The computer-implemented method of claim 1, further comprising training the sequence processing model using one or more additional lexical units that are different from those contained in the output response corpus.

15. A computing system, comprising: One or more processors; as well as One or more non-transitory computer-readable media storing instructions, which, when implemented, cause the one or more processors to perform operations including: The input data is processed using a sequence processing model to generate a first response as the output of the sequence processing model, wherein the first response includes a first word sequence; Update the output response corpus based on the first response; and The sequence processing model is used to process the input data to generate an additional response as an additional output of the sequence processing model, wherein: The additional response includes an additional lexical sequence; and The sequence processing model utilizes a bias sampling method, which biases the sampling of candidate lexical units associated with the additional lexical unit sequence based on a distance metric. The bias sampling method selects one or more additional lexical units that are different from those contained in the output response corpus based on the distance metric.

16. The computing system of claim 14, wherein, The operation further includes: The output response corpus is updated based on the additional response.

17. The computing system of claim 15, wherein, The operation further includes: The output response corpus is provided as an additional output of the sequence processing model.

18. The computing system of claim 17, wherein, The distance metric includes the Levenshtein distance.

19. The computing system of claim 14, wherein, The operation further includes: The output response corpus is analyzed to determine one or more basic lexical units, which indicate probability distribution thresholds.

20. A non-transitory computer-readable medium storing one or more instructions, said instructions, when implemented, causing one or more processors to perform operations, said operations including: The input data is processed using a sequence processing model to generate a first response as the output of the sequence processing model, wherein the first response includes a first word sequence; Update the output response corpus based on the first response; and The sequence processing model is used to process the input data to generate an additional response as an additional output of the sequence processing model, wherein: The additional response includes an additional lexical sequence; and The sequence processing model utilizes a bias sampling method, which biases the sampling of candidate lexical units associated with the additional lexical unit sequence based on a distance metric. The bias sampling method selects one or more additional lexical units that are different from those contained in the output response corpus based on the distance metric.

Images