Variable precision tokenization for adaptive machine-learned models

Abstract

An example computer-implemented method includes receiving runtime content for rendering using a user interface, wherein the runtime content is selectable at a first precision corresponding to individual content elements that compose the runtime content. The example computer-implemented method includes receiving data describing a runtime input that instructs selection of a portion of the runtime content. The example computer-implemented method includes tokenizing, using a variable precision tokenizer, the runtime content to obtain a tokenized representation of the runtime content. The example computer-implemented method includes processing, using a machine-learned model, the tokenized representation of the runtime content to predict a predicted selection boundary, wherein the predicted selection boundary defines a selection at a second precision corresponding to individual tokens that compose the tokenized representation of the runtime content. The example computer-implemented method includes outputting the predicted selection boundary.

Description

VARIABLE PRECISION TOKENIZATION FORADAPTIVE MACHINE-LEARNED MODELSFIELD

[0001] The present disclosure relates generally to machine learning processes and machine-learned devices and systems. More particularly, the present disclosure relates to variable precision tokenization.BACKGROUND

[0002] A computer can receive input(s). The computer can execute instructions to process the input(s) to generate output(s) using a parameterized model. The computer can obtain feedback on its performance in generating the outputs with the model. The computer can generate feedback by evaluating its performance. The computer can receive feedback from an external source. The computer can update parameters of the model based on the feedback to improve its performance. In this manner, the computer can iteratively “learn” to generate the desired outputs. The resulting model is often referred to as a machine-learned model.SUMMARY

[0003] Aspects and advantages of embodiments of the present disclosure will be set forth in part in the following description, or can be learned from the description, or can be learned through practice of the embodiments.

[0004] In an aspect, the present disclosure provides a first example computer- implemented method. In some implementations, the first example computer-implemented method includes obtaining plurality of selection records. In some implementations of the first example computer-implemented method, obtaining a respective selection record includes obtaining content rendered using a user interface, wherein the content is selectable at a precision corresponding to individual content elements that compose the content. In some implementations of the first example computer-implemented method, obtaining a respective selection record includes tokenizing the content using a tokenizer to obtain a tokenized representation of the content, wherein one or more tokens of the tokenized representation each represent a plurality of the individual content elements. In some implementations of the first example computer-implemented method, obtaining a respective selection record includesobtaining a ground truth selection boundary corresponding to a selection instruction that selects a portion of the content, wherein the ground truth selection boundary is between two adjacent content elements that were represented together by a respective token in the tokenized representation such that the ground truth selection boundary divides the content elements represented by the respective token into a respective first group on a first side of the boundary and a respective second group on a second side of the boundary. In some implementations of the first example computer-implemented method, obtaining a respective selection record includes storing, in the respective selection record, a subtoken value indicating the respective first group or the respective second group. In some implementations, the first example computer-implemented method includes processing the plurality of selection records to determine a recurrence measure associated with the subtoken value. In some implementations, the first example computer-implemented method includes based on the recurrence measure, updating the tokenizer to use a token representing the subtoken value.

[0005] In an aspect, the present disclosure provides a second example computer- implemented method. In some implementations, the second example computer-implemented method includes obtaining plurality of selection records. In some implementations of the second example computer-implemented method, obtaining a respective selection record includes obtaining content rendered using a user interface, wherein the content is selectable at a precision corresponding to individual content elements that compose the content. In some implementations of the second example computer-implemented method, obtaining a respective selection record includes tokenizing the content using a tokenizer to obtain a tokenized representation of the content, wherein tokens of the tokenized representation each represent one or more of the individual content elements. In some implementations of the second example computer-implemented method, obtaining a respective selection record includes obtaining a ground truth selection boundary corresponding to a selection instruction that selects a portion of the content. In some implementations of the second example computer-implemented method, obtaining a respective selection record includes storing, in the respective selection record, a representation of a contiguous sequence of two or more tokens within the selected portion, wherein the two or more tokens collectively represent a sequence of individual content elements. In some implementations, the second example computer-implemented method includes processing the plurality of selection records to determine a recurrence measure associated with the contiguous sequence. In some implementations, the second example computer-implemented method includes based on therecurrence measure, updating the tokenizer to use a single token that represents the sequence of individual content elements.

[0006] In an aspect, the present disclosure provides a third example computer- implemented method. In some implementations, the third example computer-implemented method includes receiving runtime content for rendering using a user interface, wherein the runtime content is selectable at a first precision corresponding to individual content elements that compose the runtime content. In some implementations, the third example computer- implemented method includes receiving data describing a runtime input that instructs selection of a portion of the runtime content. In some implementations, the third example computer-implemented method includes tokenizing, using a tokenizer that was updated according to the method of any of the preceding claims, the runtime content to obtain a tokenized representation of the runtime content. In some implementations, the third example computer-implemented method includes processing, using a machine-learned model, the tokenized representation of the runtime content to predict a predicted selection boundary, wherein the predicted selection boundary defines a selection at a second precision corresponding to individual tokens that compose the tokenized representation of the runtime content. In some implementations, the third example computer-implemented method includes outputting the predicted selection boundary.

[0007] In an aspect, the present disclosure provides one or more example non- transitory, computer readable media storing instructions that, when executed by one or more processors, cause a computing system to perform operations, wherein the operations include any implementation of the first example method, the second example method, or the third example method.

[0008] In an aspect, the present disclosure provides an example computing system. The example computing system includes one or more processors. The example computing system includes one or more non-transitory, computer readable media storing instructions that, when executed by the one or more processors, cause the example computing system to perform operations, wherein the operations include any implementation of the first example method, the second example method, or the third example method.

[0009] Other example aspects of the present disclosure are directed to other systems, methods, apparatuses, tangible non-transitory computer-readable media, computer program products, and devices for performing functions described herein. These and other features, aspects, and advantages of various implementations will become better understood with reference to the following description and appended claims. The accompanying drawings,which are incorporated in and constitute a part of this specification, illustrate implementations of the present disclosure and, together with the description, help explain the related principles.BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 is a block diagram of an example system according to example implementations of aspects of the present disclosure.

[0011] Figure 2 is a block diagram of an example system according to example implementations of aspects of the present disclosure.

[0012] Figure 3 is a block diagram of an example system according to example implementations of aspects of the present disclosure.

[0013] Figure 4 is a block diagram of an example system according to example implementations of aspects of the present disclosure.

[0014] Figure 5 is a flow chart diagram illustrating an example method for training a machine-learned model according to example implementations of aspects of the present disclosure.

[0015] Figure 6 is a block diagram of an example processing flow for using machine- learned model(s) to process input(s) to generate output(s) according to example implementations of aspects of the present disclosure.

[0016] Figure 7 is a block diagram of an example sequence processing model according to example implementations of aspects of the present disclosure.

[0017] Figure 8 is a block diagram of an example technique for populating an example input sequence for processing by a sequence processing model according to example implementations of aspects of the present disclosure.

[0018] Figure 9 is a block diagram of an example model development platform according to example implementations of aspects of the present disclosure.

[0019] Figure 10 is a block diagram of an example training workflow for training a machine-learned model according to example implementations of aspects of the present disclosure.

[0020] Figure 11 is a block diagram of an inference system for operating one or more machine-learned model(s) to perform inference according to example implementations of aspects of the present disclosure.

[0021] Figure 12 is a block diagram of an example networked computing system according to example implementations of aspects of the present disclosure.

[0022] Figure 13 is a block diagram of an example computing device according to example implementations of aspects of the present disclosure.

[0023] Figure 14 is a block diagram of an example computing device according to example implementations of aspects of the present disclosure.

[0024] Figure 15 is a flow chart diagram illustrating an example method for updating a tokenizer according to example implementations of aspects of the present disclosure.

[0025] Figure 16 is a flow chart diagram illustrating an example method for updating a tokenizer according to example implementations of aspects of the present disclosure.

[0026] Figure 17 is a flow chart diagram illustrating an example method for using a tokenizer according to example implementations of aspects of the present disclosure.DETAILED DESCRIPTION

[0027] Generally, the present disclosure is directed to techniques for variable precision tokenization of data. Various types of content can be represented in sequences. Tokenization can involve decomposing content into a sequence of component elements that represent pieces of the content. Tokenization can be performed at varying levels of precision: for instance, a text string can be coarsely split into sentences or finely subdivided into individual characters. Tokenization precision can affect a computational cost of downstream processes (e.g., with more individual tokens to store, process, etc.). Example implementations of the present disclosure provide a variable precision tokenizer that can adjust its tokenization precision to adapt to its implementation environment. For instance, an example tokenizer can learn from how users subdivide content and update its tokenization scheme to match, such as by increasing precision (e.g., splitting one token into two, if user interactions indicate that a sub-token portion is independently meaningful) or decreasing precision (e.g., combining multiple tokens into one if user interactions indicate that multiple tokens are more meaningful as a group).

[0028] Prior techniques for tokenization may either overgeneralize or overspecialize, leading to inefficiencies (e.g., overloading downstream systems with overly precise tokenization) or inaccuracies (e.g., obscuring useful information within overly general tokens). For generally static, open-loop systems, these compromises may be locked in from the start. Further, some types of content (e.g., logographic writing systems) can present special challenges in constructing an optimal tokenization scheme a priori (e.g., due to insufficient data, regional use variations, etc.).

[0029] Advantageously, the techniques described in the present disclosure address these issues by providing an adaptive, closed-loop tokenizer. For instance, an example implementation includes an on-device adaptive tokenization process that evolves based on actual user selections. The precision of the tokenization can be updated in view of actual usage to adapt the precision to fit the current use profile.

[0030] More particularly, the present disclosure provides example techniques for adaptively increasing the precision of tokenization by splitting tokens based on user interaction data. In various implementations, the system can obtain a plurality of selection records from user interactions with content displayed via a user interface, where the content is selectable at a precision that corresponds to individual content elements. These elements may include textual characters, image pixels, or video frames, depending on the nature of the content. The selection records can include data describing users’ actual selections of portions of the content at a precision that corresponds to individual content elements.

[0031] The data describing users’ actual selections can form ground truth data for evaluating a performance of a tokenizer. For instance, a tokenizer can tokenize the content into a tokenized representation, where each token may represent one or multiple individual content elements. A ground truth selection boundary is determined based on the user’s selection. The boundary may divide the content elements represented by a respective token into two groups: a first group that precedes the boundary and a second group that follows it.

[0032] A recurrence measure can characterize how often the first group or the second group is split by a selection boundary. If the recurrence measure meets certain criteria (e.g., minimum occurrence threshold, etc.), the system can update the tokenizer to split the respective token into two or more tokens. This adaptive refinement allows for increased precision in tokenization.

[0033] The present disclosure also provides techniques for adaptively decreasing the precision of tokenization by merging tokens based on user interaction data. In such implementations, the system similarly obtains a plurality of selection records and processes these to identify common patterns or sequences of token selections. Contiguous sequence of tokens that are routinely selected together can be identified as candidates for merging (e.g., to reduce a precision into larger tokens). A recurrence measure can be computed for a given contiguous sequence. Based on such recurrence measures, the system can update the tokenizer to merge these tokens into a single token, thereby reducing the number of tokens used in the tokenized representation of the content.

[0034] Example implementations of the present disclosure can be integrated into systems and devices to support a variety of processing tasks that use tokenized content representations.

[0035] In an example implementation, the present disclosure can be applied to enhance predictive selection systems that receive user selection inputs and predict the target portion of content that was intended to be selected. For example, a user may select a portion of text using a cursor or touch interface, and the system attempts to understand and predict the exact bounds of the text the user intended to select. This prediction can be particularly useful in cases where the user’s selection is imprecise, for example, due to a small screen size of mobile devices, noise in an input signal, or limitations in the user’s ability to precisely control the selection tool, etc.

[0036] To predict bounds of a selection, an example predictive selection system can use a tokenizer to tokenize content and process the content to generate a prediction of what content would be selected together (e.g., outputting a string of text that is selected, outputting timestamps associated with a video timeline selection, etc.). In some cases, the predictive precision is tied to the precision of the tokenizer. For instance, if the predictive selection system has an output space that selects from coarse tokens, then the output may be limited in precision to those coarse tokens. For example, in text, if an entire phrase is represented by a single token, then a predictive selection system may not have sufficient precision to predict a selection that includes only a portion of the phrase. In timeline-based selections, for instance, if a longer interval is represented by a single token, then a predicted selection system may not have sufficient precision to predict a selection that includes only a portion of the interval.This limitation might not align well with user needs, especially in scenarios requiring finegrained selection, such as editing or annotating specific words or phrases.

[0037] Advantageously, the techniques described in the present disclosure can improve a precision of predictive text selection systems in an efficient manner. The precision of tokenization can be dynamically adjusted based on actual user interactions. For instance, if the system observes that users frequently attempt to select sub-portions of a tokenized unit, it may automatically increase the granularity of tokenization for that unit. Conversely, if it finds that certain smaller tokens are typically selected together as a group, it might aggregate these into a larger single token.

[0038] Example implementations of the present disclosure can provide a number of technical effects and benefits.

[0039] An example technical effect of example implementations of the present disclosure is the potential reduction in the use of computing resources, memory usage, and latency in systems that utilize tokenized content (e.g., machine-learned systems, database systems, etc.). This can be achieved by adapting the tokenizer to use token vocabularies (e.g., sets of candidate tokens) that are specifically adapted to a particular task. For instance, a text selection task may involve both subword selections and selections that do not split words. Naively specifying a constant precision of all tokens to split into subwords or individual characters may be inefficient if many words are unlikely to be split at that level of detail. The task-adaptive approach of the present disclosure allows certain tokens to be split to increase precision where needed while allowing other tokens to remain less granular. This variable precision approach can lead to smaller token sets, which may subsequently require less computational power and memory for processing and storage. In scenarios where content needs to be processed in real-time or near real-time, such as in live communication applications or streaming services, the use of smaller token sets can decrease processing latency, thereby enhancing the responsiveness of the system.

[0040] Improved resource efficiency can benefit both powerful server systems as well as resource-constrained devices (e.g., mobile devices, edge devices, etc.). For instance, in environments where server resources are shared across multiple clients or services, such as in cloud computing platforms or large-scale data centers, the ability to minimize resource consumption and thermal output while maintaining high responsiveness is desirable. By dynamically adjusting the tokenization precision based on user interaction data, the system may reduce the number of tokens processed, generated, or otherwise stored in memory for a given task. This can lead to a decrease in the computational load and memory usage on the server. This reduction may, in turn, free up resources that can be allocated to handling additional client requests or improving service responsiveness. Furthermore, in scenarios involving thin-client devices that rely on server-side processing for content interaction tasks, such as text editing or media streaming, the efficiency improvements on the server side can directly translate to faster response times and smoother interactions on the client side. This is particularly relevant for devices with limited processing capabilities or in situations where network latency is a concern. By optimizing the tokenization process, the server can process requests more quickly and return results to the client in a shorter time frame.

[0041] An example technical effect of example implementations of the present disclosure is an improved noise tolerance for human-machine interfaces. For instance, some input interfaces, such as small screens, coarse input controls, or voice-only input methods,can be associated with noisy inputs. For instance, in environments where users interact with content on devices with small screens, the precision of user selections can often be compromised due to the limited display area and the difficulty in making precise selections with fingers or styluses. Similarly, devices with coarse input controls (e.g., wearables with simple button-based interfaces) may not always capture the user’s intent accurately due to the limited expressive capacity of these input methods. By improving a tokenizer to better match actual selections, example implementations of the present disclosure can provide for improved prediction selection models that can generate selection boundaries that more often align with a selection that a given input was intended to convey. By refining the tokenizer to recognize and predict selection boundaries more accurately, the system may reduce the need for users to manually adjust or refine their selections, or to repeat an input or request, expending additional processing time.

[0042] Such improvements to human-machine interfaces can make such interfaces more accessible to all users. In addition to making such devices easier to use for users without impairments, notably such improvements may make such devices easier to use for users with various impairments, such as decreased motor precision or impaired eyesight, which might otherwise render such limited interfaces impractical or impossible to use.

[0043] A technical effect of example implementations of the present disclosure is increased energy efficiency in performing operations using machine-learned models, thereby improving the functioning of computers implementing such models. For instance, example implementations can provide for more energy-efficient runtime execution or inference. In some scenarios, increased energy efficiency can provide for less energy to be used to perform a given task (e.g., less energy expended to maintain the model in memory, less energy expended to perform calculations within the model, etc.). In some scenarios, increased energy efficiency can provide for more task(s) to be completed for a given energy budget (e.g., a larger quantity of tasks, more complex tasks, the same task but with more accuracy or precision, etc.).

[0044] In another example aspect, example implementations can provide for more energy-efficient training operations or model updates. In some scenarios, increased energy efficiency can provide for less energy to be used to perform a given number of update iterations (e.g., less energy expended to maintain the model in memory, less energy expended to perform calculations within the model, such as computing gradients, backpropagating a loss, etc.). In some scenarios, increased energy efficiency can provide for more update iterations to be completed for a given energy budget (e.g., a larger quantity of iterations, etc.).In some scenarios, greater expressivity afforded by model architectures and training techniques of the present disclosure can provide for a given level of functionality to be obtained in fewer training iterations, thereby expending a smaller energy budget. In some scenarios, greater expressivity afforded by model architectures and training techniques of the present disclosure can provide for an extended level of functionality to be obtained in a given number of training iterations, thereby more efficiently using a given energy budget.

[0045] In this manner, for instance, the improved energy efficiency of example implementations of the present disclosure can reduce an amount of pollution or other waste associated with implementing machine-learned models and systems, thereby advancing the field of machine-learning and artificial intelligence as a whole. The amount of pollution can be reduced in toto (e.g., an absolute magnitude thereof) or on a normalized basis (e.g., energy per task, per model size, etc.). For example, an amount of CO2 released (e.g., by a power source) in association with training and execution of machine-learned models can be reduced by implementing more energy-efficient training or inference operations. An amount of heat pollution in an environment (e.g., by the processors / storage locations) can be reduced by implementing more energy-efficient training or inference operations.

[0046] Various example implementations are described herein with respect to the accompanying Figures.

[0047] Figure 1 is a block diagram of an example system 100 for variable precision tokenization. A tokenizer 102 can process content data 104-1 to generate tokenized content data 106-1 that chunks content data 104-1 into token representations. System 100 can store attributes characterizing a selection of a portion of content data 104-1. For example, system 100 can receive a user input that instructs selection of a portion of content data 104-1.Characteristics associated with a boundary of the selection can be stored in selection record 108-1. The characteristics can include an alignment of the boundary and one or more tokens of tokenized content data 106-1. Similarly, system 100 can receive a user input that instructs selection of a portion of content data 104-N. Characteristics associated with a boundary of the selection can be stored in selection record 108-N. The characteristics can include an alignment of the boundary and one or more tokens of tokenized content data 106-N. Selection records 108-1, . . . , 108-N can be cached in selection record cache 108. Evaluation system 110 can process the records in selection record cache 108 to compute a recurrence measure that characterizes how often over-precision indicators or under-precision indicators occur in the selections. Based on recurrence measure 112, evaluation system 110 can provide token updates 114 to tokenizer 102 to update the precision of the tokens used by tokenizer 102.

[0048] Tokenizer 102 can be or include a component within system 100 that processes various types of content data 104-1, . . . , 104-N to generate tokenized content data 106-1, . . . , 106-N. This processing can involve the conversion of raw content data into a structured format that segments the data (or representations thereof) into discrete units referred to herein as tokens.

[0049] Tokenizer 102 can support tokenization of different data types, such as text, images, video, audio, etc. In general, to tokenize arbitrary data objects, tokenizer 102 can map data into a discrete space that can be represented with a vocabulary of tokens. For example, tokenizer 102 can process input data to recognize meaningfully distinct or predetermined segments that can be processed in downstream processes as a unit. That unit can be extracted or otherwise designated as a token.

[0050] Tokens or token embeddings can latently encode information (e.g., an undesignated token with an embedding that has implicitly learned to represent features associated with dogs). For instance, an arbitrary datatype can be tokenized using a learned tokenization scheme. For instance, input data objects can be encoded using a pool of K token embeddings (e.g., a codebook). Values for the embeddings can be learned to facilitate reconstruction of the data objects (e.g., using a reconstruction loss on an output of a decoder) or to facilitate selection prediction over the data objects (e.g., using a loss quantifying an error in a predicted selection). Segments or portions of the input data objects can be represented by different tokens. The mapping between data object portions (e.g., patches of an image, fields of a structured data record, subwords, etc.) can be facilitated by a learned model that receives the input and outputs one or more tokens or token embeddings from the codebook or vocabulary. In this manner, for instance, increasing or decreasing a precision of a token can include adjusting or adding an entry in a codebook and re-training the system to match a desired precision (e.g., a desired location of a selection boundary).

[0051] Tokens can correspond to explicit categories or classes. For instance, a token can explicitly correspond to strings, content categories, attributes, etc. (e.g., a token designated to represent “Hello”, a token designated to represent “dog”).

[0052] For textual content data, tokenizer 102 can employ natural language processing (NLP) techniques to segment text into meaningful units or tokens. For example, input text can be processed to identify chunks that form units of semantic meaning or other linguistic structures, such as sentences or phrases. Pretrained natural language processing models can be used to recognize tokens in textual content data. Regular expressions or other deterministic parsing methods can be used. Tokenizer 102 can be configured to identifysubwords, words, clauses, phrases, sentences, paragraphs, or other linguistic structures as tokens, depending on the requirements of the application.

[0053] For image content, tokenizer 102 can employ techniques that segment images into patches, which may or may not overlap depending on the specific requirements of the application. The size and overlap of patches can be configured to capture relevant visual information while maintaining efficient processing capabilities. Each patch can be mapped to a token in a token vocabulary for predicting what portion of an image may be selected. For instance, one or multiple patches may depict a people standing together. Tokenizer 102 can process the image to recognize that there are people in the image and associate each patch depicting the people with a tag “people.” In this manner, a “people” token can be associated with each patch depicting the people. A predictive selection model can process the tokens and predict that a selection could include all the “dog” tokens. If a ground truth image selection splits the “people” token (e.g., selects one person in the patch), a more granular “person” token can be used instead to predict selection of individual persons. If selections never or rarely select individual persons, then “person” can be replaced with “people.”

[0054] For video content, tokenizer 102 can segment videos into tokens that represent time intervals or frames. Each frame can be processed individually as an image, as noted above. Additionally, the video can be chunked over time. Each chunk can be mapped or otherwise categorized (e.g., with a machine-learned video processing model) to a token value. For instance, a video may be represented as a sequence of token, “introduction,” “dialog,” “action sequence,” etc. A predictive selection model can process the tokens to predict that a selection could include the dialog token.

[0055] For audio content, tokenizer 102 can tokenize the data into time intervals that capture distinct sound bites or acoustic features (e.g., as recognized by an audio processing model). These intervals can be determined based on the characteristics of the audio signal, such as changes in frequency or amplitude, which may correspond to different sounds or spoken words. Each chunk can be mapped to a token, and a sequence of tokens can be ingested by a predictive selection model to predict a likely selection. For instance, tokens can include “intro,” “stanza,” “chorus,” “bridge,” “riff,” etc. Higher precision tokens can include “chord,” “note,” etc.

[0056] In general, for timeline-based content (e.g., video, audio, slideshow, etc.), tokens can correspond to intervals. For example, each token can represent a designated interval of the content. A predictive selection model can process the tokenized intervals tooutput a predicted selected set of one or more intervals. In this manner, for instance, a predicted selection can correspond to a temporal precision of the interval tokens.

[0057] In an example, timeline-based content can be tokenized with a temporal precision based on a type of segment of the chunk. For instance, a video can be analyzed to determine that a segment corresponds to an initial title sequence of a show. It may be unlikely that a user will select a subportion within the title sequence, so that segment can be tokenized with low temporal precision (e.g., one token for the whole segment). The video can be analyzed to determine that a segment corresponds to a dialog sequence. Such a segment may be expected (e.g., based on a training corpus) to be associated with more precise selections, so that segment may be tokenized with more precision.

[0058] Tokenizer 102 can output tokens in various different formats. In an example, a token can include a discrete unit of content data that has been segmented during the tokenization process. A token can be stored in association with a token embedding. A token embedding can be a numerical representation of tokenized content. Embeddings can capture semantic or contextual relationships between tokens in a way that can be processed by machine learning models. A token can be stored in associated with a token identifier, such as a unique identifier that identifies the token.

[0059] Content data 104-1 can be or include a variety of data formats and types that are suitable for processing by tokenizer 102. Content data 104-1 can include textual data, images, video content, audio streams, or any combination thereof.

[0060] Content data 104-1 can include textual documents such as books, articles, emails, or any other form of written communication. This textual data can be sourced from various inputs, including digital files, public domain online content, or scanned documents that have been converted into digital format. The textual data might include multiple languages and scripts, ranging from widely used scripts such as Latin or Cyrillic to more specialized scripts like Arabic or Hanzi. Individual content elements can include characters, words, sentences, paragraphs, etc.

[0061] Content data 104-1 can include image data, which may consist of digital photographs, scanned images, or artwork. These images can be in various formats such as JPEG, PNG, or TIFF. Image data can include raster or vector files. The image content could include naturally captured images (e.g., photographs), imaged sensor data (e.g., X-ray scans, spectrograms, or other medical or scientific imaging, etc.), synthetic images (e.g., computergenerated graphics), or graphical representations such as charts and graphs. Individual content elements can include pixels, layers, etc.

[0062] Content data 104-1 can include video data. The video data may include different formats such as AVI, MP4, or MPEG. Video data can include time-varying pixelbased image content (optionally aligned with an audio track). Individual content elements can include pixels, frames, audio signals, video interval s / segments, layers, timestamps, etc.

[0063] Furthermore, content data 104-1 can also include audio data. Audio data can include voice recordings, music files, or any other sound data. Audio data can be stored in formats such as MP3, WAV, or AAC. Individual content elements can include samples, an interval of samples, timestamps, etc.

[0064] In some implementations, content data 104-1 can also include arbitrary data types such as structured data from databases, software applications, etc. This data can be in the form of JSON objects, XML documents, or any other structured data format that is amenable to tokenization by the system. Individual content elements can include file objects, folder objects, spreadsheet cells, lines of code, etc.

[0065] Content data 104-1 can be pre-processed before tokenization to enhance the quality and consistency of the data. For example, textual data might undergo normalization processes such as removing extraneous whitespace, converting all characters to a uniform case, or eliminating punctuation. Image data might be pre-processed to adjust brightness or contrast, or to crop the images to a standard size. Video and audio data might be presegmented into clips or excerpts that are of a manageable duration for processing.

[0066] Tokenized content data 106-1, . . . , 106-N can be or include the tokens output produced by tokenizer 102 from the based on the input provided by content data 104-1, . . . , 104-N.

[0067] Tokenized content data 106-1, . . . , 106-N can include textual tokens derived from textual content data. Tokenized content data 106-1, . . . , 106-N can include image tokens. These image tokens might be segments or patches of the original images, determined based on color, texture, or object detection algorithms. Tokenized content data 106-1, . . . , 106-N can include video tokens, which can include frames or groups of frames segmented based on temporal and visual characteristics. Tokenized content data 106-1, . . . , 106-N can include audio tokens, such as segments of the audio stream that contain distinct sound elements such as syllables, words, or phrases in speech, or notes and beats in music. Tokenized content data 106-1, . . . , 106-N can include tokens derived from arbitrary data objects, where each token represents a structured data element such as a database record, a JSON object, or any other discrete unit of data.

[0068] Selection records 108-1, . . . , 108-N can store data that captures user interactions with content in a structured data object. These records can store information that describes how users interact with content rendered via a user interface. Each selection record can include input signal(s) representing the selection actions performed by users, the context of the selections, and the precise content elements involved in each selection instance.

[0069] Selection records 108-1, . . . , 108-N can include data that specifies one or more boundaries of user selections within the content. These boundaries can be defined using the content elements that are selected by the user. For instance, if the content comprises text, the selection records can store a selected text string. The boundaries can be defined using indices associated with content selected by or excluded from the content boundaries. For instance, for a text selection, a record can store data indicating the start and end points of the text selected by the user in terms of character positions or word indices, or similarly a start point that immediately precedes the selection.

[0070] Selection records 108-1, . . . , 108-N can describe the alignment of user- selected boundaries with the tokens generated by the tokenizer. Alignment data can include an indicator of whether the selection boundary falls between individual content elements represented by a single token. Alignment data can include a record of a number of tokens preceding a selection boundary or following a selection boundary. This alignment information can be used to determine whether a tokenization scheme appropriately matches the granularity of user interactions. For example, if a user consistently selects portions of content that align with the boundaries of existing tokens, this might indicate that the tokenization granularity is well-suited to user needs. Conversely, frequent selections that include the same contiguous set of tokens or that divide single tokens can suggest a need for adjusting the tokenization precision.

[0071] Selection records 108-1, . . . , 108-N can include metadata related to the user interaction, such as the time of the interaction, the device used for the interaction, and the application in which the interaction occurred.

[0072] Selection records 108-1, . . . , 108-N can be used to identify user interaction patterns over time. By analyzing a series of selection records, the system can identify trends in selection patterns. In this manner, for instance, token updates can be computed to reflect consistent trends rather than responsive to individual edge case scenarios.

[0073] Selection records 108-1, . . . , 108-N can include the entirety of selected context. Selection records 108-1, . . . , 108-N can include any predicted selections generated by a selection prediction system (e.g., as described below with respect to Figure 4).

[0074] Selection record cache 108 can store selection records 108-1, . . . , 108-N. Selection record cache 108 can be persistent in memory or on disk. Selection record cache 108 can be stored in a variety of different data formats, including a database, text file, etc.

[0075] Selection record cache 108 can be stored on-device. For instance, system 100 can be an on-device implementation that performs processing on a single device (e.g., mobile device). System 100 can use multiple devices or systems, such as cloud compute, to perform various operations. Selection record cache 108 can be encrypted and locked to a user account associated with a device.

[0076] Selection record cache 108 can implement a rolling buffer mechanism to manage the storage of selection records dynamically. This rolling buffer can allow the cache to maintain a fixed number of the most recent selection records, automatically discarding older records as new ones are added.

[0077] Evaluation system 110 can be or include processing logic configured to process records from selection record cache 108 to compute token updates. Evaluation system 110 can determine updates to a token set (e.g., a list of candidate tokens) responsive to a trigger condition. The trigger condition can be based on a recurrence measure.

[0078] Evaluation system 110 can periodically analyze the records in selection record cache 108 to compute new values for one or more recurrence measures 112. For instance, evaluation system 110 can evaluate the records each time a new selection record is added to the cache or based on a temporal frequency (e.g., once a day, etc.).

[0079] Recurrence measures 112 can quantify the effectiveness and precision of the tokenization process applied by tokenizer 102. Recurrence measures 112 can indicate a frequency at which a user ground truth selection boundary aligns with a boundary token (e.g., a token adjacent to or intersecting a ground truth selection boundary) in a particular manner. Different alignment events can indicate under- and over-precision.

[0080] Recurrence measures 112 can quantify a frequency of over-precision indicators. An over-precision indicator can include an occurrence of a selection of contiguous tokens that could be more efficiently represented as a single token. In an example, user interactions may involve the selection of a sequence of tokens (adjacent to a boundary or otherwise). Evaluation system 110 can process a selection record indicating selection of the sequence. After processing multiple other selection records indicating selection of the same sequence, evaluation system 110 can compute a recurrence measure 112 can quantify a frequency with which the particular selection pattern is occurring. If the particular selectionpattern occurs with sufficient frequency, it may be an opportunity to introduce a decreased precision token that represents the sequence in one token.

[0081] Recurrence measures 112 can quantify a frequency of under-precision indicators. An under-precision indicator can include an occurrence of a selection boundary that splits a token. In an example, user interactions may involve a selection boundary that lies between individual content elements that are represented by the same token (e.g., the boundary splits the token into two portions). Evaluation system 110 can process a selection record indicating the split to extract a subtoken value. After processing multiple other selection records indicating split tokens that share at least one same subtoken value, evaluation system 110 can compute a recurrence measure 112 can quantify a frequency with which the subtoken value is being split from a larger token. If the particular selection pattern occurs with sufficient frequency, it may be an opportunity to introduce an increased precision token that represents the subtoken value.

[0082] Recurrence measures 112 can also include an evaluation of a frequency of selection patterns that do not conform to the identified over-precision or under-precision indicator. For instance, when merging tokens together, evaluation system 110 can determine whether the individual tokens are ever or often used separately. For instance, evaluation system 110 can determine whether a threshold number of selection records correspond to a ground truth selection boundary that divides the two or more tokens. If there are sufficient selection records that indicate that some selection boundaries do fall between the tokens, then the individual tokens may be retained (e.g., not deleted). If there are less than the threshold number of selection records (e.g., less than 10, less than 1, or zero), then the individual tokens may be deleted or deactivated. The thresholds for token addition or deactivation can be tuned hyperparameters (e.g., in an optimization loop to decrease overall error in the downstream system(s)).

[0083] Token updates 114 can instruct tokenizer 102 to add a token to a set of candidate tokens (e.g., a “vocabulary”). For instance, when evaluation system 110 identifies a recurring selection pattern that aligns with a sub-token portion of existing tokens, token updates 114 can include adding a new token to represent this frequently selected sub-token. Token updates 114 can include adding a new token to represent a commonly selected contiguous sequence of tokens.

[0084] Token updates 114 can instruct tokenizer 102 to delete, remove, or otherwise deactivate a token. Deactivating a token can include removing it from a set of possibleoutputs from tokenizer 102. This can include removing it from a set of candidate tokens, deactivating tokenizing logic associated with parsing that particular token, etc.

[0085] Figure 2 is a block diagram of an example implementation of system 100 in which evaluation system 110 detects instances of under-precision indicators. For example, content data 104-1 can include individual content elements 200 (e.g., discrete units of data such as textual characters, image pixels, or other data types). Tokenizer 102 can process individual content elements 200 to output tokens 202 that form tokenized content data 106-1. A ground truth selection 204-1 can be obtained for content data 104-1. A selection boundary 206-1 from the ground truth selection 204-1 can split a token 202 into a first portion 208-1-1 and a second portion 208-1-2. This split can be recorded in selection record 108-1. Selection record 108-1 can describe one or more subtoken value(s) 210-1 associated with the split.

[0086] Similarly, tokenizer 102 can process individual content elements 200 of content data 104-N to generate tokens 202 for tokenized content data 106-N. A ground truth selection 204-N can be obtained for content data 104-N. A selection boundary 206-N from the ground truth selection 204-N can split a token 202 into a first portion 208-N-l and a second portion 208-N-2. This split can be recorded in selection record 108-N. Selection record 108-N can describe one or more subtoken value(s) 210-N associated with the split.

[0087] Evaluation system 110 can process selection records in selection record cache 108 to identify a recurrence of common splits. For instance, selection records 108-1 and 108- N may be associated with the same subtoken value(s). For instance, both first portions 208-1- 1 and 208-N-l correspond to the same two individual content elements 200 (as graphically indicated with comer markings). Second portions 208-1-2 and 208-N-2 can be the same or different. As such, evaluation system 110 can determine that a sufficient number of selection records indicate the same selection pattern having a split of those individual content elements 200. Recurrence measure 112 can satisfy a threshold. Token updates 114 can include a new token for tokenizer 102 to use to represent first portion 208-1-1.

[0088] Individual content elements 200 can represent units of data used to compose content data 104-1, . . . , 104-N. These elements can be the smallest discrete units that can be individually selected and manipulated within the content data 104-1, . . . , 104-N using a given selection interface. The granularity of individual content elements 200 can vary depending on the type of content being processed and the specific requirements of the application. For textual content, individual content elements 200 can include characters, such as letters, numbers, punctuation marks, and other typographic symbols. For image content, individual content elements 200 can include pixels or small groups of pixels. For videocontent, individual content elements 200 can include frames or segments of video and any associated audio segments. For audio content, individual content elements 200 can include audio samples or small time slices of audio data. For structured data more generally, such as databases or spreadsheets, individual content elements 200 can include data fields or cells.

[0089] Individual tokens 202 can be generated by the tokenizer 102 and serve as the structured units of tokenized content data 106-1, . . . , 106-N. Examples of tokens generated by tokenizer 102 are provided above.

[0090] Individual tokens 202 can be stored with attributes or other data fields for downstream processing. A token data object can include a token identifier (token ID) which uniquely identifies each token within the system. A token data object can include token content, or the actual data or information represented by the token. A token data object can include a token embedding. A token embedding can be a numerical representation that captures semantic or contextual relationships of the token in a high-dimensional space, facilitating machine learning operations and other advanced data processing tasks.

[0091] Ground truth selection 204-1, . . . , 204-N can include data describing one or more selection boundaries received from a user input interface or other input modality.Ground truth selection 204-1, . . . , 204-N can represent segments of content that users select using various input methods, such as a touchscreen, mouse, stylus, or other pointing devices. In implementations involving a touchscreen interface, users can directly interact with the displayed content by touching the screen to select specific content elements. Similarly, in environments equipped with a mouse or a stylus, users can select content by pointing and clicking or dragging over the desired content elements.

[0092] Ground truth selection 204-1, . . . , 204-N can be facilitated through voice- controlled interfaces. Users can provide vocal commands to select content. A speech processing system can interpret spoken commands that specify the content to be selected, such as “select the next paragraph” or “highlight the word ‘tokenization’.” The system can then convert these commands into corresponding actions that modify the ground truth selection boundaries according to the user’s spoken instructions.

[0093] Ground truth selection 204-1, . . . , 204-N can be generated using predictive selection mechanisms to resolve imprecise inputs. For instance, when a user attempts to select a line of text on a small mobile screen using a finger, the actual touch input coordinates may not precisely correspond to a target content selection intended by the user or even any selectable content element at all. To overcome the challenges presented by the limitations of such an input mechanism, a device can use a selection prediction system to infer the mostlikely intended selection based on the context of the user’s interaction, the underlying content, and the proximity to selectable content elements. The system can output the ground truth selection boundaries to align with what the user likely intended to select.

[0094] Ground truth selection 204-1, . . . , 204-N can be an output of a selection refinement process. For instance, after receiving initial selection inputs and rendering the interpreted selection for the user, additional user inputs can be received that instruct a change to a selection boundary. For instance, a selection boundary can have editable endpoints that, responsive to a user input, can be moved to refine the selection.

[0095] Selection boundary 206-1, . . . , 206-N can be at least one boundary of ground truth selection 204-1, . . . , 204-N. Selection boundary 206-1, . . . , 206-N can be a start or end of a linear selection. Selection boundary 206-1, . . . , 206-N can be a top, bottom, side, or any portion of an edge of an area selection. Selection boundary 206-1, . . . , 206-N can be any surface of a volume selection (e.g., a selection of an area over time, such as a crop for a video).

[0096] Selection boundary 206-1, . . . , 206-N can be described in one or more selection records along with other data describing the user’s selection. A selection record can describe the alignment of the boundary with the content elements and tokens. This can be indicated explicitly (e.g., an indexed position of a boundary in a string) or implicitly (e.g., storage of one or two substrings that are split by or adjacent to the boundary).

[0097] First portion 208-1-1, . . . , 208-N-l and second portion 208-1-2, . . . , 208-N-2 can represent subdivisions of individual tokens 202 determined by the alignment of a ground truth selection boundary 206-1, . . . , 206-N within the tokenized content data 106-1, . . . , 106-N. First portion 208-1-1, . . . , 208-N-l can represent content elements that are on a first side of the ground truth selection boundary within a token, while second portion 208-1-2, . . . , 208-N-2 can represent those that are on a second side of the boundary. First portion 208-1-1, . . . , 208-N-l can represent content elements that precede the ground truth selection boundary within a token, while second portion 208-1-2, . . . , 208-N-2 can represent those that follow the boundary. First portion 208-1-1, . . . , 208-N-l can represent content elements that follow the ground truth selection boundary within a token, while second portion 208-1-2, . . . , 208- N-2 can represent those that precede the boundary.

[0098] Subtoken value(s) 210-1, . . . , 210-N can represent either the first portion 208- 1-1, . . . , 208-N-l or the second portion 208-1-2, . . . , 208-N-2 of a token that has been divided by a selection boundary 206-1, . . . , 206-N. Selection records 108 can store or otherwise describe subtoken value(s) 210-1, . . . , 210-N.

[0099] Figure 3 is a block diagram of an example implementation of system 100 in which evaluation system 110 detects instances of over-precision indicators. For example, content data 104-1 can include individual content elements 200. Tokenizer 102 can process individual content elements 200 to output tokens 202 that form tokenized content data 106-1. A ground truth selection 204-1 can be obtained for content data 104-1. A selection boundary 206-1 from the ground truth selection 204-1 can occur after contiguous token values 308-1. Selection record 108-1 can store one or more tokens preceding selection boundary 206-1, which can include contiguous token values 308-1.

[0100] Similarly, tokenizer 102 can process individual content elements 200 of content data 104-N to generate tokens 202 for tokenized content data 106-N. A ground truth selection 204-N can be obtained for content data 104-N. A selection boundary 206-N from the ground truth selection 204-N can occur after contiguous token values 308-N. Selection record 108-N can store one or more tokens preceding selection boundary 206-N, which can include contiguous token values 308-N.

[0101] Evaluation system 110 can process selection records in selection record cache 108 to identify a recurrence of common sets of contiguous tokens. For instance, selection records 108-1 and 108-N may be associated with the same contiguous tokens 308-1. For instance, both contiguous token values 308-1 and 308-N correspond to the same two individual tokens 202 (as graphically indicated with comer markings). Although shown here as both preceding a selection boundary, it is to be understood that an example could follow the boundary. Evaluation system 110 can determine that a sufficient number of selection records indicate the same selection pattern having the same sets of contiguous token values. Recurrence measure 112 can satisfy a threshold. Token updates 114 can include a new token for tokenizer 102 to represent the contiguous token values.

[0102] To identify common sets of contiguous token values, selection records 108-1, . . . , 108-N can store a number of tokens contained in a selection. Selection records can store all or part of a selection of content (e.g., the content elements themselves or the tokens corresponding thereto), and evaluation system 110 can parse the records to identify common sets of tokens. Selection records can store a fixed number of tokens. The fixed number of tokens can correspond to a merging limit - for instance, if no more than 10 tokens are to be merged (as a system constraint), then selection records may omit storage of more than 10 tokens.

[0103] Contiguous tokens can be adjacent to a boundary. For instance, contiguous token values 308-1 and 308-N can immediately precede a boundary. Contiguous token values 308-1 and 308-N can immediately follow a boundary.

[0104] Contiguous tokens can fall within a boundary and not be adjacent to a boundary. For instance, over-precision may be detected even if the tokens involved do not contact a boundary. For instance, certain phrases may be included in various textual selections but not at a boundary of a selection. Such phrases might be represented with multiple tokens, even though such phrases might never be split by a boundary. Evaluation system 110 can detect recurrence of such tokenized phrases as well as an occurrence rate (or lack thereof) of such tokenized phrases being split by a boundary. Upon determination of a recurrence measure that indicates such phrases are used repeatedly and are unlikely to be split by a boundary, then tokenizer 102 can be updated to use a single token for such phrase to provide a token vocabulary that is adapted to the selection understanding task.

[0105] Figure 4 is a block diagram of an example selection prediction system according to aspects of the present disclosure. Runtime content data 400 can include individual content elements 200. Runtime content data 400 can be rendered on a user interface on which content elements 200 are selectable. Tokenizer 102 can process runtime content data 400 using candidate tokens 402 to generate tokenized runtime content data 404. Tokenized runtime content data 404 can be processed by selection prediction model 406 to predict selection boundaries for runtime content data 400. The predicted selection boundaries can be based on candidate tokens 408, which can be the same or different from candidate tokens 402. Selection prediction model 406 can output predicted selection 408.

[0106] Runtime content data 400 can have any of the attributes described above with respect to content data 104-1, . . . , 104-N.

[0107] Candidate tokens 402 can include a token “vocabulary” or parsing logic that specifies an output domain for tokenizer 102. Candidate tokens 402 can be updated using the update techniques described above with respect to Figures 1 to 3.

[0108] Runtime tokenized content data 404 can have any of the attributes described above with respect to tokenized content data 106-1, . . . , 106-N.

[0109] Selection prediction model 406 can be a machine-learned model configured to generate selection predictions. Selection prediction model 406 can be a sequence processing model configured to process sequences of input tokens and output sequences of output tokens that describe a selection boundary.

[0110] Candidate tokens 408 can include a token “vocabulary” available to selection prediction model 406 to use for predicting boundaries of selection. Candidate tokens 408 can be updated using the update techniques described above with respect to Figures 1 to 3.

[0111] Candidate tokens 408 can be the same as candidate tokens 402. For instance, an input to selection prediction model can be tokenized using the same tokens as an output of selection prediction model 406. In an example, as candidate tokens 402 are updated by evaluation system 110, candidate tokens 408 are updated, such as by updating a size of one or more input or output layers of selection prediction model 406 to accommodate additional tokens or remove tokens (e.g., to adjust an initial embedding of input tokens or to adjust a probability distribution over output tokens).

[0112] Candidate tokens 408 can be different from candidate tokens 402. For instance, an input to selection prediction model 406 can use different tokenization than an output. For instance, selection prediction model 406 can include an encoder that receives an input sequence according to a first tokenization scheme and a decoder that outputs an output sequence according to a second tokenization scheme. The first tokenization scheme can be higher or lower precision than the second tokenization scheme. In this manner, for instance, selection prediction model 406 can implement variable precision within the model itself to optimize allocation of compute where needed to improve prediction performance.

[0113] Predicted selection 408 can be generated responsive to a trigger. For instance, predicted selection 408 can be generated responsive to a device receiving an input instructing a selection. Selection prediction model 406 can process one or more input signals or data descriptive thereof and generate predicted selection 408 as an estimate of the selection that was targeted by the input signals (e.g., the user’s intended selection). Another trigger can be based on rendering the content. For instance, after rendering content, a device or system can proactively analyze the content to assess what areas of the content the user may be likely to navigate to for further playback or rendering. For instance, this predicted selection can allow for predictively caching content before a user navigates to the content (e.g., selections a point in time on a timeline, selects a file, selects an image for enlarged viewing, etc.).

[0114] Predicted selection 408 can include data describing a selection of one or more individual content elements 200. For instance, selection prediction model 406 can predict a boundary by predicting a sequence of selected tokens. Selection prediction model 406 can predict a boundary by predicting a token that precedes a boundary or immediately follows a boundary. Predicted selection 408 can include data describing the boundary, includinggraphical bounds for a bounding box, content bounds indexing selected content from among content data 400, etc.

[0115] A system can use predicted selection 408 to render a suggested selection indicator. For instance, a user interface rendering content 400 can overlay a graphic indicator suggesting selection according to predicted selection 408 (e.g., highlighting selected text, highlighting a proposed cursor position on a content timeline, etc.). An audio interface can verbally describe predicted selection 408 and ask a user for confirmation.

[0116] Selection prediction model 406 can be used in conjunction with the update techniques described with respect to Figures 1 to 3. For instance, the ground truth selection boundaries can be predicted selections 408 or manually refined instances thereof. For instance, selection prediction model 406 can output predicted selections 408, and a user can adjust the boundaries of the selection to obtain ground truth selection 204-1, . . . , 204-N.

[0117] As tokenizer 102 is updated, then, selection model 406 can be retrained to use the new tokens. During retraining, parts of selection model 406 can be frozen. For instance, only embedding layers might be retrained.

[0118] Selection prediction model 406 can retrain over a standard corpus (e.g., the same used to initially train selection prediction model 406).

[0119] Selection record cache 108 can provide a new, more relevant set of training examples for fine-tuning selection prediction model 406 to use the new tokens. For example, because selection record cache 108 was used to identify the use of the new tokens, selection record cache 108 can provide highly targeted fine-tuning examples to demonstrate usage and meaning of the new token.

[0120] An initial embedding for a new token can be obtained using previously trained weights, adapted as needed to accept an additional input. An initial embedding for a new token can inherit an embedding from a larger token from which it was split. A split token’s embedding can be decomposed into component embeddings for the portions split therefrom, with each portion being assigned the component embedding associated therewith. An initial embedding for a new token can be given an embedding based on a combination of the embeddings for the tokens merged into the new token. The initial embedding can be used to initial learning of the embedding. The initial embedding can be used in lieu of learning an embedding.

[0121] Selection prediction model 406 can predict selections in various different modalities. Various examples are described herein with respect to selection of textual content. Other modalities are contemplated as well.

[0122] For instance, selection prediction model 406 can predict selections for image data. Tokenizer 102 can use an image processing model to identify chunks corresponding to discrete categories. These categories can be mapped to tokens. Example tokens include “face,” “dog,” and other identifiable objects or themes within an image.

[0123] The tokens can be used to predict which tokens might be selected in a user selection. The precision of the tokenization can influence the granularity of the prediction. Coarse tokenization can encompass large chunks with multiple elements. This can limit the specificity of the predicted selection. A finer tokenization can involve smaller, more precisely defined chunks. This can allow for a more detailed predicted selection.

[0124] Selection prediction model 406 can process the tokens and output a prediction for the tokens that would be selected. This prediction can in turn be used to generate a rendering of a bounding box surrounding the portions of the image corresponding to the selected tokens. The bounding box can include all selected tokens, or a bounding box can be generated for each selected token.

[0125] In some implementations, selection prediction model 406 can be used to accelerate data annotation processes. For instance, selection prediction model 406 can predict an area of an image likely to be selected for adding a label. The annotation process can include data labeling for constructing training datasets, medical image review an analysis, tagging family photos, etc.

[0126] For instance, selection prediction model 406 can predict selections for audio data. Tokenizer 102 can use an audio processing model to identify segments corresponding to discrete categories. An audio processing model can be implemented using various techniques. Techniques can include Fourier transforms, which can analyze the frequency components of the audio signal. Techniques can also include Mel-frequency cepstral coefficients (MFCCs), which can provide a representation of the short-term power spectrum of the sound. These techniques can assist in distinguishing various sound types or musical elements within audio data.

[0127] These categories can be mapped to tokens. Example tokens include “speech,” “music,” “ambient noise,” “stanza,” “chord,” etc., or other identifiable sounds or themes within an audio clip. The tokens can be used to predict which tokens might be selected in a user selection. The precision of the tokenization can influence the granularity of the prediction. Coarse tokenization can encompass large segments with multiple sound elements. This can limit the specificity of the predicted selection. A finer tokenization can involvesmaller, more precisely defined segments. This can allow for a more precise predicted selection.

[0128] Selection prediction model 406 can process the tokens and output a prediction for the tokens that would be selected. This prediction can in turn be used to generate a suggested selection for navigating to a portion of an audio track, selecting a segment of an audio track, selecting all segments that contain speech, etc.

[0129] For instance, selection prediction model 406 can predict selections for video data. The predicted selections for video data can be based on image tokens and audio tokens generated for the video over time. Selection prediction model 406 can process the tokens and output a prediction for the tokens that would be selected. This prediction can in turn be used to generate a suggested selection for navigating to a portion of a video, selecting a segment of a video (e.g., a temporal segment, a crop, or both), selecting all segments that contain dialog or action, etc.

[0130] Figure 5 depicts a flowchart of a method 500 for training one or more machine-learned models according to aspects of the present disclosure. For instance, an example machine-learned model can include one or more models of machine-learned models 406.

[0131] One or more portion(s) of example method 500 can be implemented by a computing system that includes one or more computing devices such as, for example, computing systems described with reference to the other figures. Each respective portion of example method 500 can be performed by any (or any combination) of one or more computing devices. Moreover, one or more portion(s) of example method 500 can be implemented on the hardware components of the device(s) described herein, for example, to train one or more systems or models. Figure 5 depicts elements performed in a particular order for purposes of illustration and discussion. Those of ordinary skill in the art, using the disclosures provided herein, will understand that the elements of any of the methods discussed herein can be adapted, rearranged, expanded, omitted, combined, or modified in various ways without deviating from the scope of the present disclosure. Figure 5 is described with reference to elements / terms described with respect to other systems and figures for exemplary illustrated purposes and is not meant to be limiting. One or more portions of example method 500 can be performed additionally, or alternatively, by other systems.

[0132] At 502, example method 500 can include obtaining a training instance. A set of training data can include a plurality of training instances divided between multiple datasets (e.g., a training dataset, a validation dataset, or testing dataset). A training instance can belabeled or unlabeled. Although referred to in example method 500 as a “training” instance, it is to be understood that runtime inferences can form training instances when a model is trained using an evaluation of the model’s performance on that runtime instance (e.g., online training / learning). Example data types for the training instance and various tasks associated therewith are described throughout the present disclosure.

[0133] At 504, example method 500 can include processing, using one or more machine-learned models, the training instance to generate an output. The output can be directly obtained from the one or more machine-learned models or can be a downstream result of a chain of processing operations that includes an output of the one or more machine- learned models.

[0134] At 506, example method 500 can include receiving an evaluation signal associated with the output. The evaluation signal can be obtained using a loss function. Various determinations of loss can be used, such as mean squared error, likelihood loss, cross entropy loss, hinge loss, contrastive loss, or various other loss functions. The evaluation signal can be computed using known ground-truth labels (e.g., supervised learning), predicted or estimated labels (e.g., semi- or self-supervised learning), or without labels (e.g., unsupervised learning). The evaluation signal can be a reward (e.g., for reinforcement learning). The reward can be computed using a machine-learned reward model configured to generate rewards based on output(s) received. The reward can be computed using feedback data describing human feedback on the output(s).

[0135] At 508, example method 500 can include updating the machine-learned model using the evaluation signal. For example, values for parameters of the machine-learned model(s) can be learned, in some embodiments, using various training or learning techniques, such as, for example, backwards propagation. For example, the evaluation signal can be backpropagated from the output (or another source of the evaluation signal) through the machine-learned model(s) to update one or more parameters of the model(s) (e.g., based on a gradient of the evaluation signal with respect to the parameter value(s)). For example, system(s) containing one or more machine-learned models can be trained in an end-to-end manner. Gradient descent techniques can be used to iteratively update the parameters over a number of training iterations. In some implementations, performing backwards propagation of errors can include performing truncated backpropagation through time. Example method 500 can include implementing a number of generalization techniques (e.g., weight decays, dropouts, etc.) to improve the generalization capability of the models being trained.

[0136] In some implementations, example method 500 can be implemented for training a machine-learned model from an initialized state to a fully trained state (e.g., when the model exhibits a desired performance profile, such as based on accuracy, precision, recall, etc.).

[0137] In some implementations, example method 500 can be implemented for particular stages of a training procedure. For instance, in some implementations, example method 500 can be implemented for pre-training a machine-learned model. Pre-training can include, for instance, large-scale training over potentially noisy data to achieve a broad base of performance levels across a variety of tasks / data types.

[0138] In some implementations, example method 500 can be implemented for finetuning a machine-learned model. Fine-tuning can include, for instance, smaller-scale training on higher-quality (e.g., labeled, curated, etc.) data. Fine-tuning can affect all or a portion of the parameters of a machine-learned model. For example, various portions of the machine- learned model can be “frozen” for certain training stages. For example, parameters associated with an embedding space can be “frozen” during fine-tuning (e.g., to retain information learned from a broader domain(s) than present in the fine-tuning dataset(s)). In some implementations, example method 500 uses adapter modules. Adapters can be small trainable layers that are inserted between pre-existing layers of a pre-trained model. During the finetuning process, the original parameters of the pre-trained model are typically frozen, and only the parameters of the adapters are updated.

[0139] In some implementations, example method 500 can be implemented to execute parameter-efficient fine-tuning methods, such as Layerwise Optimization of Residuals (LoRA). LoRA can refine pre-trained models with minimal adjustments to the original parameters. This can be achieved by introducing trainable low-rank matrices that modify the behavior of the pre-trained weights without directly altering them. In some implementations, during fine-tuning, only these auxiliary matrices are updated, which significantly reduces the number of parameters that are trained.

[0140] An example fine-tuning approach includes reinforcement learning. Reinforcement learning can be based on user feedback on model performance during use.

[0141] Figure 6 is a block diagram of an example processing flow for using machine- learned model(s) 1 to process input(s) 2 to generate output(s) 3.

[0142] Machine-learned model(s) 1 can be or include one or multiple machine- learned models or model components. Example machine-learned models can include neural networks (e.g., deep neural networks). Example machine-learned models can include non-linear models or linear models. Example machine-learned models can use other architectures in lieu of or in addition to neural networks. Example machine-learned models can include decision tree based models, support vector machines, hidden Markov models, Bayesian networks, linear regression models, k-means clustering models, etc.

[0143] Machine-learned model(s) 1 can be or include, or otherwise be representative of any one or more of the machine-learned models described above with respect to the preceding figures. For example, machine-learned model(s) 1 can be or include, or otherwise be representative of any one or more of a machine-learned model used in selection prediction model 406.

[0144] Example neural networks can include feed-forward neural networks, recurrent neural networks (RNNs), including long short-term memory (LSTM) based recurrent neural networks, convolutional neural networks (CNNs), diffusion models, generative-adversarial networks, or other forms of neural networks. Example neural networks can be deep neural networks. Some example machine-learned models can leverage an attention mechanism such as self-attention. For example, some example machine-learned models can include multiheaded self-attention models.

[0145] Machine-learned model(s) 1 can include a single or multiple instances of the same model configured to operate on data from input(s) 2. Machine-learned model(s) 1 can include multiple different models or multiple different model portions configured to operate on data from input(s) 2.

[0146] Machine-learned model(s) 1 can include an ensemble of different models that can cooperatively interact to process data from input(s) 2. For example, a model ensemble can include multiple models that have different attributes (e.g., different architectures, trained with different recipes, etc.). The ensemble can output an overall output based on the individual outputs of the constituent models. In this manner, for instance, the diverse constituent models can work together to provide system-level robustness by effectively aggregating over individual strengths and weaknesses of any given model. The respective individual outputs can be combined in a weighted combination, using a voting or routing mechanism, or a learned output layer (e.g., one or more feedforward or fully-connected layers).

[0147] Machine-learned model(s) 1 can employ a mixture-of-experts structure. See, e.g., Zhou et al., Mixture -of -Experts with Expert Choice Routing, ARXIV:2202.09368V2 (Oct. 14, 2022). For example, different portions of a model can learn (explicitly or implicitly) different expertise areas, with pathways through the model being selected by a learnedrouting mechanism that engages the appropriate expert for a given input (e.g., a given portion of an input, such as on a per-token basis). For example, a feedforward network can be sparsely activated for a given portion of an input based on an output of a routing mechanism that processes the portion of the input. In this manner, for instance, the group of activated weights can form an “expert” that is selected by the router. On each forward pass, only a subset of the total model weights may be engaged, thereby decreasing a quantity of operations performed for processing a given input compared to a densely activated model. In this manner, for instance, the expressive and interpretive power of a high-parameter-count model can be achieved with more compute-efficient forward passes.

[0148] Input(s) 2 can generally include or otherwise represent various types of data. Input(s) 2 can include one type or many different types of data. Output(s) 3 can be data of the same type(s) or of different types of data as compared to input(s) 2. Output(s) 3 can include one type or many different types of data.

[0149] Example data types for input(s) 2 or output(s) 3 include natural language text data, software code data (e.g., source code, object code, machine code, or any other form of computer-readable instructions or programming languages), machine code data (e.g., binary code, assembly code, or other forms of machine-readable instructions that can be executed directly by a computer’s central processing unit), assembly code data (e.g., low-level programming languages that use symbolic representations of machine code instructions to program a processing unit), genetic data or other chemical or biochemical data, image data, audio data, audiovisual data, haptic data, biometric data, medical data, financial data, statistical data, geographical data, astronomical data, historical data, sensor data generally (e.g., digital or analog values, such as voltage or other absolute or relative level measurement values from a real or artificial input, such as from an audio sensor, light sensor, displacement sensor, etc.), and the like. Data can be raw or processed and can be in any format or schema.

[0150] In multimodal inputs 2 or outputs 3, example combinations of data types include image data and audio data, image data and natural language data, natural language data and software code data, image data and biometric data, sensor data and medical data, etc. It is to be understood that any combination of data types in an input 2 or an output 3 can be present.

[0151] An example input 2 can include one or multiple data types, such as the example data types noted above. An example output 3 can include one or multiple data types, such as the example data types noted above. The data type(s) of input 2 can be the same as or different from the data type(s) of output 3. It is to be understood that the example data typesnoted above are provided for illustrative purposes only. Data types contemplated within the scope of the present disclosure are not limited to those examples noted above.

[0152] Figure 7 is a block diagram of an example implementation of an example machine-learned model configured to process sequences of information. For instance, an example implementation of machine-learned model(s) 1 can include machine-learned sequence processing model(s) 4. An example system can pass input(s) 2 to sequence processing model(s) 4. Sequence processing model(s) 4 can include one or more machine- learned components. Sequence processing model(s) 4 can process the data from input(s) 2 to obtain an input sequence 5. Input sequence 5 can include one or more input elements 5-1, 5- 2, . . . , 5-A , etc. obtained from input(s) 2. Sequence processing model 4 can process input sequence 5 using prediction layer(s) 6 to generate an output sequence 7. Output sequence 7 can include one or more output elements 7-1, 7-2, . . . , 7 -A, etc. generated based on input sequence 5. The system can generate output(s) 3 based on output sequence 7.

[0153] Sequence processing model(s) 4 can include one or multiple machine-learned model components configured to ingest, generate, or otherwise reason over sequences of information. For example, some example sequence processing models in the text domain are referred to as “Large Language Models,” or LLMs. See, e.g., PaLM 2 Technical Report, GOOGLE, https: / / ai.google / static / documents / palm2techreport.pdf (n.d.). Other example sequence processing models can operate in other domains, such as image domains, see, e.g., Dosovitskiy et al., An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale, ARXIV:2010.11929v2 (Jun. 3, 2021), audio domains, see, e.g., Agostinelli et al., MusicLM: Generating Music From Text, ARXIV:2301.1 1325V1 (Jan. 26, 2023), biochemical domains, see, e.g., Jumper et al., Highly accurate protein structure prediction with AlphaFold, 596 Nature 583 (Aug. 26, 2021), by way of example. Sequence processing model(s) 4 can process one or multiple types of data simultaneously. Sequence processing model(s) 4 can include relatively large models (e.g., more parameters, computationally expensive, etc.), relatively small models (e.g., fewer parameters, computationally lightweight, etc.), or both.

[0154] In general, sequence processing model(s) 4 can obtain input sequence 5 using data from input(s) 2. For instance, input sequence 5 can include a representation of data from input(s) 2 in a format understood by sequence processing model(s) 4. One or more machine- learned components of sequence processing model(s) 4 can ingest the data from input(s) 2, parse the data into pieces compatible with the processing architectures of sequence processing model(s) 4 (e.g., via “tokenization”), and project the pieces into an input space associated with prediction layer(s) 6 (e.g., via “embedding”).

[0155] Sequence processing model(s) 4 can ingest the data from input(s) 2 and parse the data into a sequence of elements to obtain input sequence 5. For example, a portion of input data from input(s) 2 can be broken down into pieces that collectively represent the content of the portion of the input data. The pieces can provide the elements of the sequence.

[0156] Elements 5-1, 5-2, . . . , 5-M can represent, in some cases, building blocks for capturing or expressing meaningful information in a particular data domain. For instance, the elements can describe “atomic units” across one or more domains. For example, for textual input source(s), the elements can correspond to groups of one or more words or sub-word components, such as sets of one or more characters.

[0157] For example, elements 5-1, 5-2, . . . , 5-M can represent tokens obtained using a tokenizer. For instance, a tokenizer can process a given portion of an input source and output a series of tokens (e.g., corresponding to input elements 5-1, 5-2, . . . , 5-A ) that represent the portion of the input source. Various approaches to tokenization can be used. For instance, textual input source(s) can be tokenized using a byte-pair encoding (BPE) technique. See, e.g., Kudo et al., SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing, PROCEEDINGS OF THE 2018 CONFERENCE ON EMPIRICAL METHODS IN NATURAL LANGUAGE PROCESSING (System Demonstrations), pages 66-71 (October 31-November 4, 2018), https: / / aclanthology.org / D18-2012.pdf. Image-based input source(s) can be tokenized by extracting and serializing patches from an image.

[0158] In general, arbitrary data types can be serialized and processed into input sequence 5. It is to be understood that element(s) 5-1, 5-2, . . . , 5-M depicted in Figure 7 can be the tokens or can be the embedded representations thereof.

[0159] Prediction layer(s) 6 can predict one or more output elements 7-1, 7-2, . . . , 7- N based on the input elements. Prediction layer(s) 6 can include one or more machine-learned model architectures, such as one or more layers of learned parameters that manipulate and transform the input(s) to extract higher-order meaning from, and relationships between, input element(s) 5-1, 5-2, . . . , 5-M. In this manner, for instance, example prediction layer(s) 6 can predict new output element(s) in view of the context provided by input sequence 5.

[0160] Prediction layer(s) 6 can evaluate associations between portions of input sequence 5 and a particular output element. These associations can inform a prediction of the likelihood that a particular output follows the input context. For example, consider the textual snippet, “The carpenter’s toolbox was small and heavy. It was full of .” Example prediction layer(s) 6 can identify that “It” refers back to “toolbox” by determining arelationship between the respective embeddings. Example prediction layer(s) 6 can also link “It” to the attributes of the toolbox, such as “small” and “heavy.” Based on these associations, prediction layer(s) 6 can, for instance, assign a higher probability to the word “nails” than to the word “sawdust.”

[0161] A transformer is an example architecture that can be used in prediction layer(s) 4. See, e.g., Vaswani et al., Atention Is All You Need, ARXIV: 1706.03762v7 (Aug. 2, 2023). A transformer is an example of a machine-learned model architecture that uses an attention mechanism to compute associations between items within a context window. The context window can include a sequence that contains input sequence 5 and potentially one or more output element(s) 7-1, 7-2, . . . , 7-N. A transformer block can include one or more attention layer(s) and one or more post-attention layer(s) (e.g., feedforward layer(s), such as a multi-layer perceptron).

[0162] Prediction layer(s) 6 can include other machine-learned model architectures in addition to or in lieu of transformer-based architectures. For example, recurrent neural networks (RNNs) and long short-term memory (LSTM) models can also be used, as well as convolutional neural networks (CNNs). In general, prediction layer(s) 6 can leverage various kinds of artificial neural networks that can understand or generate sequences of information.

[0163] Output sequence 7 can include or otherwise represent the same or different data types as input sequence 5. For instance, input sequence 5 can represent textual data, and output sequence 7 can represent textual data. Input sequence 5 can represent image, audio, or audiovisual data, and output sequence 7 can represent textual data (e.g., describing the image, audio, or audiovisual data). It is to be understood that prediction layer(s) 6, and any other interstitial model components of sequence processing model(s) 4, can be configured to receive a variety of data types in input sequence(s) 5 and output a variety of data types in output sequence(s) 7.

[0164] Output sequence 7 can have various relationships to input sequence 5. Output sequence 7 can be a continuation of input sequence 5. Output sequence 7 can be complementary to input sequence 5. Output sequence 7 can translate, transform, augment, or otherwise modify input sequence 5. Output sequence 7 can answer, evaluate, confirm, or otherwise respond to input sequence 5. Output sequence 7 can implement (or describe instructions for implementing) an instruction provided via input sequence 5.

[0165] Output sequence 7 can be generated autoregressively. For instance, for some applications, an output of one or more prediction layer(s) 6 can be passed through one or more output layers (e.g., softmax layer) to obtain a probability distribution over an outputvocabulary (e.g., a textual or symbolic vocabulary) conditioned on a set of input elements in a context window. In this manner, for instance, output sequence 7 can be autoregressively generated by sampling a likely next output element, adding that element to the context window, and re-generating the probability distribution based on the updated context window, and sampling a likely next output element, and so forth.

[0166] Output sequence 7 can also be generated non-autoregressively. For instance, multiple output elements of output sequence 7 can be predicted together without explicit sequential conditioning on each other. See, e.g., Saharia et al., Non-Autoregressive Machine Translation with Latent Alignments, ARXIV:2004.07437V3 (NOV. 16, 2020).

[0167] Output sequence 7 can include one or multiple portions or elements. In an example content generation configuration, output sequence 7 can include multiple elements corresponding to multiple portions of a generated output sequence (e.g., a textual sentence, values of a discretized waveform, computer code, etc.). In an example classification configuration, output sequence 7 can include a single element associated with a classification output. For instance, an output “vocabulary” can include a set of classes into which an input sequence is to be classified. For instance, a vision transformer block can pass latent state information to a multilayer perceptron that outputs a likely class value associated with an input image.

[0168] Figure 8 is a block diagram of an example technique for populating an example input sequence 8. Input sequence 8 can include various functional elements that form part of the model infrastructure, such as an element 8-0 obtained from a task indicator 9 that signals to any model(s) that process input sequence 8 that a particular task is being performed (e.g., to help adapt a performance of the model(s) to that particular task). Input sequence 8 can include various data elements from different data modalities. For instance, an input modality 10-1 can include one modality of data. A data-to- sequence model 11-1 can process data from input modality 10-1 to project the data into a format compatible with input sequence 8 (e.g., one or more vectors dimensioned according to the dimensions of input sequence 8) to obtain elements 8-1, 8-2, 8-3. Another input modality 10-2 can include a different modality of data. A data-to-sequence model 11-2 can project data from input modality 10-2 into a format compatible with input sequence 8 to obtain elements 8-4, 8-5, 8- 6. Another input modality 10-3 can include yet another different modality of data. A data-to- sequence model 11-3 can project data from input modality 10-3 into a format compatible with input sequence 8 to obtain elements 8-7, 8-8, 8-9.

[0169] Input sequence 8 can be the same as or different from input sequence 5. Input sequence 8 can be a multimodal input sequence that contains elements that represent data from different modalities using a common dimensional representation. For instance, an embedding space can have P dimensions. Input sequence 8 can be configured to contain a plurality of elements that have / Jdimensions. In this manner, for instance, example implementations can facilitate information extraction and reasoning across diverse data modalities by projecting data into elements in the same embedding space for comparison, combination, or other computations therebetween.

[0170] For example, elements 8-0, . . . , 8-9 can indicate particular locations within a multidimensional embedding space. Some elements can map to a set of discrete locations in the embedding space. For instance, elements that correspond to discrete members of a predetermined vocabulary of tokens can map to discrete locations in the embedding space that are associated with those tokens. Other elements can be continuously distributed across the embedding space. For instance, some data types can be broken down into continuously defined portions (e.g., image patches) that can be described using continuously distributed locations within the embedding space.

[0171] In some implementations, the expressive power of the embedding space may not be limited to meanings associated with any particular set of tokens or other building blocks. For example, a continuous embedding space can encode a spectrum of high-order information. An individual piece of information (e.g., a token) can map to a particular point in that space: for instance, a token for the word “dog” can be projected to an embedded value that points to a particular location in the embedding space associated with canine-related information. Similarly, an image patch of an image of a dog on grass can also be projected into the embedding space. In some implementations, the projection of the image of the dog can be similar to the projection of the word “dog” while also having similarity to a projection of the word “grass,” while potentially being different from both. In some implementations, the projection of the image patch may not exactly align with any single projection of a single word. In some implementations, the projection of the image patch can align with a combination of the projections of the words “dog” and “grass.” In this manner, for instance, a high-order embedding space can encode information that can be independent of data modalities in which the information is expressed.

[0172] Task indicator 9 can include a model or model component configured to identify a task being performed and inject, into input sequence 8, an input value represented by element 8-0 that signals which task is being performed. For instance, the input value canbe provided as a data type associated with an input modality and projected along with that input modality (e.g., the input value can be a textual task label that is embedded along with other textual data in the input; the input value can be a pixel-based representation of a task that is embedded along with other image data in the input; etc.). The input value can be provided as a data type that differs from or is at least independent from other input(s). For instance, the input value represented by element 8-0 can be a learned within a continuous embedding space.

[0173] Input modalities 10-1, 10-2, and 10-3 can be associated with various different data types (e.g., as described above with respect to input(s) 2 and output(s) 3).

[0174] Data-to-sequence models 11-1, 11-2, and 11-3 can be the same or different from each other. Data-to-sequence models 11-1, 11-2, and 11-3 can be adapted to each respective input modality 10-1, 10-2, and 10-3. For example, a textual data-to-sequence model can subdivide a portion of input text and project the subdivisions into element(s) in input sequence 8 (e.g., elements 8-1, 8-2, 8-3, etc.). An image data-to-sequence model can subdivide an input image and project the subdivisions into element(s) in input sequence 8 (e.g., elements 8-4, 8-5, 8-6, etc.). An arbitrary datatype data-to-sequence model can subdivide an input of that arbitrary datatype and project the subdivisions into element(s) in input sequence 8 (e.g., elements 8-7, 8-8, 8-9, etc.).

[0175] Data-to-sequence models 11-1, 11-2, and 11-3 can form part of machine- learned sequence processing model(s) 4. Data-to-sequence models 11-1, 11-2, and 11-3 can be jointly trained with or trained independently from machine-learned sequence processing model(s) 4. Data-to-sequence models 11-1, 11-2, and 11-3 can be trained end-to-end with machine-learned sequence processing model(s) 4.

[0176] Figure 9 is a block diagram of an example model development platform 12 that can facilitate creation, adaptation, and refinement of example machine-learned models (e.g., machine-learned model(s) 1, sequence processing model(s) 4, etc.). Model development platform 12 can provide a number of different toolkits that developer systems can employ in the development of new or adapted machine-learned models.

[0177] Model development platform 12 can provide one or more model libraries 13 containing building blocks for new models. Model libraries 13 can include one or more pretrained foundational models 13-1, which can provide a backbone of processing power across various tasks. Model libraries 13 can include one or more pre-trained expert models 13-2, which can be focused on performance in particular domains of expertise. Model libraries 13 can include various model primitives 13-3, which can provide low-level architectures orcomponents (optionally pre-trained), which can be assembled in various arrangements as desired. Model primitives 13-3 can include a library of pre-trained adapters or LoRA modules that can adapt a baseline foundational model to align its outputs with a desired performance profile, augment model capabilities (e.g., to adapt to a different input modality, etc.), and the like.

[0178] Model libraries 13 can include a tokenizer 102 according to the present disclosure. Model libraries 13 can include a selection prediction model trained to use a tokenizer according to the present disclosure.

[0179] Model development platform 12 can receive selections of various model components 14. Model development platform 12 can pass selected model components 14 to a workbench 15 that combines selected model components 14 into a development model 16.

[0180] Workbench 15 can facilitate further refinement and adaptation of development model 16 by leveraging a number of different toolkits integrated with model development platform 12. For example, workbench 15 can facilitate alignment of the development model 16 with a desired performance profile on various tasks using a model alignment toolkit 17.

[0181] Model alignment toolkit 17 can provide a number of tools for causing development model 16 to generate outputs aligned with desired behavioral characteristics. Alignment can include increasing an accuracy, precision, recall, etc. of model outputs. Alignment can include enforcing output styles, schema, or other preferential characteristics of model outputs. Alignment can be general or domain-specific. For instance, a pre-trained foundational model 13-1 can begin with an initial level of performance across multiple domains. Alignment of the pre-trained foundational model 13-1 can include improving a performance in a particular domain of information or tasks (e.g., even at the expense of performance in another domain of information or tasks).

[0182] Model alignment toolkit 17 can integrate one or more dataset(s) 17-1 for aligning development model 16. Curated dataset(s) 17-1 can include labeled or unlabeled training data. Dataset(s) 17-1 can be obtained from public domain datasets. Dataset(s) 17-1 can be obtained from private datasets associated with one or more developer system(s) for the alignment of bespoke machine-learned model(s) customized for private use-cases.

[0183] Pre-training pipelines 17-2 can include a machine-learned model training workflow configured to update development model 16 over large-scale, potentially noisy datasets. For example, pre-training can leverage unsupervised learning techniques (e.g., denoising, etc.) to process large numbers of training instances to update model parameters from an initialized state and achieve a desired baseline performance. Pre-training pipelines 17-2can leverage unlabeled datasets in dataset(s) 17-1 to perform pre-training. Workbench 15 can implement a pre-training pipeline 17-2 to pre-train development model 16.

[0184] Fine-tuning pipelines 17-3 can include a machine-learned model training workflow configured to refine the model parameters of development model 16 with higher- quality data. Fine-tuning pipelines 17-3 can update development model 16 by conducting supervised training with labeled dataset(s) in dataset(s) 17-1. Fine-tuning pipelines 17-3 can update development model 16 by conducting reinforcement learning using reward signals from user feedback signals. Workbench 15 can implement a fine-tuning pipeline 17-3 to finetune development model 16.

[0185] Prompt libraries 17-4 can include sets of inputs configured to induce behavior aligned with desired performance criteria. Prompt libraries 17-4 can include few-shot prompts (e.g., inputs providing examples of desired model outputs for prepending to a desired runtime query), chain-of-thought prompts (e.g., inputs providing step-by-step reasoning within the exemplars to facilitate thorough reasoning by the model), and the like.

[0186] Example prompts can be retrieved from an available repository of prompt libraries 17-4. Example prompts can be contributed by one or more developer systems using workbench 15.

[0187] In some implementations, pre-trained or fine-tuned models can achieve satisfactory performance without exemplars in the inputs. For instance, zero-shot prompts can include inputs that lack exemplars. Zero-shot prompts can be within a domain within a training dataset or outside of the training domain(s).

[0188] Prompt libraries 17-4 can include one or more prompt engineering tools. Prompt engineering tools can provide workflows for retrieving or learning optimized prompt values. Prompt engineering tools can facilitate directly learning prompt values (e.g., input element values) based on one or more training iterations. Workbench 15 can implement prompt engineering tools in development model 16.

[0189] Prompt libraries 17-4 can include pipelines for prompt generation. For example, inputs can be generated using development model 16 itself or other machine- learned models. In this manner, for instance, a first model can process information about a task and output an input for a second model to process in order to perform a step of the task. The second model can be the same as or different from the first model. Workbench 15 can implement prompt generation pipelines in development model 16.

[0190] Prompt libraries 17-4 can include pipelines for context injection. For instance, a performance of development model 16 on a particular task can improve if provided withadditional context for performing the task. Prompt libraries 17-4 can include software components configured to identify desired context, retrieve the context from an external source (e.g., a database, a sensor, etc.), and add the context to the input prompt. Workbench 15 can implement context injection pipelines in development model 16.

[0191] Although various training examples described herein with respect to model development platform 12 refer to “pre-training” and “fine-tuning,” it is to be understood that model alignment toolkit 17 can generally support a wide variety of training techniques adapted for training a wide variety of machine-learned models. Example training techniques can correspond to the example training method 1200 described above.

[0192] Model development platform 12 can include a model plugin toolkit 18. Model plugin toolkit 18 can include a variety of tools configured for augmenting the functionality of a machine-learned model by integrating the machine-learned model with other systems, devices, and software components. For instance, a machine-learned model can use tools to increase performance quality where appropriate. For instance, deterministic tasks can be offloaded to dedicated tools in lieu of probabilistically performing the task with an increased risk of error. For instance, instead of autoregressively predicting the solution to a system of equations, a machine-learned model can recognize a tool to call for obtaining the solution and pass the system of equations to the appropriate tool. The tool can be a traditional system of equations solver that can operate deterministically to resolve the system of equations. The output of the tool can be returned in response to the original query. In this manner, tool use can allow some example models to focus on the strengths of machine-learned models — e.g., understanding an intent in an unstructured request for a task — while augmenting the performance of the model by offloading certain tasks to a more focused tool for rote application of deterministic algorithms to a well-defined problem.

[0193] Model plugin toolkit 18 can include validation tools 18-1. Validation tools 18- 1 can include tools that can parse and confirm output(s) of a machine-learned model. Validation tools 18-1 can include engineered heuristics that establish certain thresholds applied to model outputs. For example, validation tools 18-1 can ground the outputs of machine-learned models to structured data sources (e.g., to mitigate “hallucinations”).

[0194] Model plugin toolkit 18 can include tooling packages 18-2 for implementing one or more tools that can include scripts or other executable code that can be executed alongside development model 16. Tooling packages 18-2 can include one or more inputs configured to cause machine-learned model(s) to implement the tools (e.g., few-shot promptsthat induce a model to output tool calls in the proper syntax, etc.). Tooling packages 18-2 can include, for instance, fine-tuning training data for training a model to use a tool.

[0195] Model plugin toolkit 18 can include interfaces for calling external application programming interfaces (APIs) 18-3. For instance, in addition to or in lieu of implementing tool calls or tool code directly with development model 16, development model 16 can be aligned to output instruction that initiate API calls to send or obtain data via external systems.

[0196] Model plugin toolkit 18 can integrate with prompt libraries 17-4 to build a catalog of available tools for use with development model 16. For instance, a model can receive, in an input, a catalog of available tools, and the model can generate an output that selects a tool from the available tools and initiates a tool call for using the tool.

[0197] Model development platform 12 can include a computational optimization toolkit 19 for optimizing a computational performance of development model 16. For instance, tools for model compression 19-1 can allow development model 16 to be reduced in size while maintaining a desired level of performance. For instance, model compression 19-1 can include quantization workflows, weight pruning and sparsification techniques, etc. Tools for hardware acceleration 19-2 can facilitate the configuration of the model storage and execution formats to operate optimally on different hardware resources. For instance, hardware acceleration 19-2 can include tools for optimally sharding models for distributed processing over multiple processing units for increased bandwidth, lower unified memory requirements, etc. Tools for distillation 19-3 can provide for the training of lighter-weight models based on the knowledge encoded in development model 16. For instance, development model 16 can be a highly performant, large machine-learned model optimized using model development platform 12. To obtain a lightweight model for running in resource-constrained environments, a smaller model can be a “student model” that learns to imitate development model 16 as a “teacher model.” In this manner, for instance, the investment in learning the parameters and configurations of development model 16 can be efficiently transferred to a smaller model for more efficient inference.

[0198] Workbench 15 can implement one, multiple, or none of the toolkits implemented in model development platform 12. Workbench 15 can output an output model 20 based on development model 16. Output model 20 can be a deployment version of development model 16. Output model 20 can be a development or training checkpoint of development model 16. Output model 20 can be a distilled, compressed, or otherwise optimized version of development model 16.

[0199] Figure 10 is a block diagram of an example training flow for training a machine-learned development model 16. One or more portion(s) of the example training flow can be implemented by a computing system that includes one or more computing devices such as, for example, computing systems described with reference to the other figures. Each respective portion of the example training flow can be performed by any (or any combination) of one or more computing devices. Moreover, one or more portion(s) of the example training flow can be implemented on the hardware components of the device(s) described herein, for example, to train one or more systems or models. FIG. 17 depicts elements performed in a particular order for purposes of illustration and discussion. Those of ordinary skill in the art, using the disclosures provided herein, will understand that the elements of any of the methods discussed herein can be adapted, rearranged, expanded, omitted, combined, or modified in various ways without deviating from the scope of the present disclosure. FIG. 17 is described with reference to elements / terms described with respect to other systems and figures for exemplary illustrated purposes and is not meant to be limiting. One or more portions of the example training flow can be performed additionally, or alternatively, by other systems.

[0200] Initially, development model 16 can persist in an initial state as an initialized model 21. Development model 16 can be initialized with weight values. Initial weight values can be random or based on an initialization schema. Initial weight values can be based on prior pre-training for the same or for a different model.

[0201] Initialized model 21 can undergo pre-training in a pre-training stage 22. Pretraining stage 22 can be implemented using one or more pre-training pipelines 17-2 over data from dataset(s) 17-1. Pre-training can be omitted, for example, if initialized model 21 is already pre-trained (e.g., development model 16 contains, is, or is based on a pre-trained foundational model or an expert model).

[0202] Pre-trained model 23 can then be a new version of development model 16, which can persist as development model 16 or as a new development model. Pre-trained model 23 can be the initial state if development model 16 was already pre-trained. Pre-trained model 23 can undergo fine-tuning in a fine-tuning stage 24. Fine-tuning stage 24 can be implemented using one or more fine-tuning pipelines 17-3 over data from dataset(s) 17-1. Fine-tuning can be omitted, for example, if a pre-trained model as satisfactory performance, if the model was already fine-tuned, or if other tuning approaches are preferred.

[0203] Fine-tuned model 29 can then be a new version of development model 16, which can persist as development model 16 or as a new development model. Fine-tunedmodel 29 can be the initial state if development model 16 was already fine-tuned. Fine-tuned model 29 can undergo refinement with user feedback 26. For instance, refinement with user feedback 26 can include reinforcement learning, optionally based on human feedback from human users of fine-tuned model 25. As reinforcement learning can be a form of fine-tuning, it is to be understood that fine-tuning stage 24 can subsume the stage for refining with user feedback 26. Refinement with user feedback 26 can produce a refined model 27. Refined model 27 can be output to downstream system(s) 28 for deployment or further development.

[0204] In some implementations, computational optimization operations can be applied before, during, or after each stage. For instance, initialized model 21 can undergo computational optimization 29-1 (e.g., using computational optimization toolkit 19) before pre-training stage 22. Pre-trained model 23 can undergo computational optimization 29-2 (e.g., using computational optimization toolkit 19) before fine-tuning stage 24. Fine-tuned model 25 can undergo computational optimization 29-3 (e.g., using computational optimization toolkit 19) before refinement with user feedback 26. Refined model 27 can undergo computational optimization 29-4 (e.g., using computational optimization toolkit 19) before output to downstream system(s) 28. Computational optimization(s) 29-1, . . . , 29-4 can all be the same, all be different, or include at least some different optimization techniques.

[0205] Figure 11 is a block diagram of an inference system for operating one or more machine-learned model(s) 1 to perform inference (e.g., for training, for deployment, etc.). A model host 31 can receive machine-learned model(s) 1. Model host 31 can host one or more model instance(s) 31-1, which can be one or multiple instances of one or multiple models. Model host 31 can host model instance(s) 31-1 using available compute resources 31-2 associated with model host 31.

[0206] Model host 31 can perform inference on behalf of one or more client(s) 32. Client(s) 32 can transmit an input request 33 to model host 31. Using input request 33, model host 31 can obtain input(s) 2 for input to machine-learned model(s) 1. Machine-learned model(s) 1 can process input(s) 2 to generate output(s) 3. Using output(s) 3, model host 31 can return an output payload 34 for responding to input request 33 from client(s) 32. Output payload 34 can include or be based on output(s) 3.

[0207] Model host 31 can leverage various other resources and tools to augment the inference task. For instance, model host 31 can communicate with tool interfaces 35 to facilitate tool use by model instance(s) 31-1. Tool interfaces 35 can include local or remote APIs. Tool interfaces 35 can include integrated scripts or other software functionality. Modelhost 31 can engage online learning interface(s) 36 to facilitate ongoing improvements to machine-learned model(s) 1. For instance, online learning interface(s) 36 can be used within reinforcement learning loops to retrieve user feedback on inferences served by model host 31. Model host 31 can access runtime data source(s) 37 for augmenting input(s) 2 with additional contextual information. For instance, runtime data source(s) 37 can include a knowledge graph 37-1 that facilitates structured information retrieval for information associated with input request(s) 33 (e.g., a search engine service). Runtime data source(s) 37 can include public or private, external or local database(s) 37-2 that can store information associated with input request(s) 33 for augmenting input(s) 2. Runtime data source(s) 37 can include account data 37-3 which can be retrieved in association with a user account corresponding to a client 32 for customizing the behavior of model host 31 accordingly.

[0208] Model host 31 can be implemented by one or multiple computing devices or systems. Client(s) 2 can be implemented by one or multiple computing devices or systems, which can include computing devices or systems shared with model host 31.

[0209] For example, model host 31 can operate on a server system that provides a machine-learning service to client device(s) that operate client(s) 32 (e.g., over a local or wide-area network). Client device(s) can be end-user devices used by individuals. Client device(s) can be server systems that operate client(s) 32 to provide various functionality as a service to downstream end-user devices.

[0210] In some implementations, model host 31 can operate on a same device or system as client(s) 32. Model host 31 can be a machine-learning service that runs on-device to provide machine-learning functionality to one or multiple applications operating on a client device, which can include an application implementing client(s) 32. Model host 31 can be a part of a same application as client(s) 32. For instance, model host 31 can be a subroutine or method implemented by one part of an application, and client(s) 32 can be another subroutine or method that engages model host 31 to perform inference functions within the application. It is to be understood that model host 31 and client(s) 32 can have various different configurations.

[0211] Model instance(s) 31-1 can include one or more machine-learned models that are available for performing inference. Model instance(s) 31-1 can include weights or other model components that are stored on in persistent storage, temporarily cached, or loaded into high-speed memory. Model instance(s) 31-1 can include multiple instance(s) of the same model (e.g., for parallel execution of more requests on the same model). Model instance(s) 31-1 can include instance(s) of different model(s). Model instance(s) 31-1 can include cachedintermediate states of active or inactive model(s) used to accelerate inference of those models. For instance, an inference session with a particular model may generate significant amounts of computational results that can be re-used for future inference runs (e.g., using a KV cache for transformer-based models). These computational results can be saved in association with that inference session so that session can be executed more efficiently when resumed.

[0212] Compute resource(s) 31-2 can include one or more processors (central processing units, graphical processing units, tensor processing units, machine-learning accelerators, etc.) connected to one or more memory devices. Compute resource(s) 31-2 can include a dynamic pool of available resources shared with other processes. Compute resource(s) 31-2 can include memory devices large enough to fit an entire model instance in a single memory instance. Compute resource(s) 31-2 can also shard model instance(s) across multiple memory devices (e.g., using data parallelization or tensor parallelization, etc.). This can be done to increase parallelization or to execute a large model using multiple memory devices which individually might not be able to fit the entire model into memory.

[0213] Input request 33 can include data for input(s) 2. Model host 31 can process input request 33 to obtain input(s) 2. Input(s) 2 can be obtained directly from input request 33 or can be retrieved using input request 33. Input request 33 can be submitted to model host 31 via an API.

[0214] Model host 31 can perform inference over batches of input requests 33 in parallel. For instance, a model instance 31-1 can be configured with an input structure that has a batch dimension. Separate input(s) 2 can be distributed across the batch dimension (e.g., rows of an array). The separate input(s) 2 can include completely different contexts. The separate input(s) 2 can be multiple inference steps of the same task. The separate input(s) 2 can be staggered in an input structure, such that any given inference cycle can be operating on different portions of the respective input(s) 2. In this manner, for instance, model host 31 can perform inference on the batch in parallel, such that output(s) 3 can also contain the batch dimension and return the inference results for the batched input(s) 2 in parallel. In this manner, for instance, batches of input request(s) 33 can be processed in parallel for higher throughput of output payload(s) 34.

[0215] Output payload 34 can include or be based on output(s) 3 from machine- learned model(s) 1. Model host 31 can process output(s) 3 to obtain output payload 34. This can include chaining multiple rounds of inference (e.g., iteratively, recursively, across thesame model(s) or different model(s)) to arrive at a final output for a task to be returned in output payload 34. Output payload 34 can be transmitted to client(s) 32 via an API.

[0216] Online learning interface(s) 36 can facilitate reinforcement learning of machine-learned model(s) 1. Online learning interface(s) 36 can facilitate reinforcement learning with human feedback (RLHF). Online learning interface(s) 36 can facilitate federated learning of machine-learned model(s) 1.

[0217] Model host 31 can access a library of pre-trained adapters or LoRA modules that can adapt a baseline model to align its outputs with a desired performance profile, augment model capabilities (e.g., to adapt to a different input modality, etc.), and the like. For instance, model host 31 can receive an input request to load a customized model, and model host 31 can retrieve one or more components to adapt a baseline model to the custom profile. Model host 31 can determine that a particular functionality is needed for a particular task (e.g., based on an output of a model that preprocesses an input) and retrieve a pre-trained component accordingly.

[0218] Model host 31 can execute machine-learned model(s) 1 to perform inference for various tasks using various types of data. For example, various different input(s) 2 and output(s) 3 can be used for various different tasks. In some implementations, input(s) 2 can be or otherwise represent image data. Machine-learned model(s) 1 can process the image data to generate an output. As an example, machine-learned model(s) 1 can process the image data to generate an image recognition output (e.g., a recognition of the image data, a latent embedding of the image data, an encoded representation of the image data, a hash of the image data, etc.). As another example, machine-learned model(s) 1 can process the image data to generate an image segmentation output. As another example, machine-learned model(s) 1 can process the image data to generate an image classification output. As another example, machine-learned model(s) 1 can process the image data to generate an image data modification output (e.g., an alteration of the image data, etc.). As another example, machine- learned model(s) 1 can process the image data to generate an encoded image data output (e.g., an encoded and / or compressed representation of the image data, etc.). As another example, machine-learned model(s) 1 can process the image data to generate an upscaled image data output. As another example, machine-learned model(s) 1 can process the image data to generate a prediction output.

[0219] In some implementations, the task is a computer vision task. In some cases, input(s) 2 includes pixel data for one or more images and the task is an image processing task. For example, the image processing task can be image classification, where the output isa set of scores, each score corresponding to a different object class and representing the likelihood that the one or more images depict an object belonging to the object class. The image processing task may be object detection, where the image processing output identifies one or more regions in the one or more images and, for each region, a likelihood that region depicts an object of interest. As another example, the image processing task can be image segmentation, where the image processing output defines, for each pixel in the one or more images, a respective likelihood for each category in a predetermined set of categories. For example, the set of categories can be foreground and background. As another example, the set of categories can be object classes. As another example, the image processing task can be depth estimation, where the image processing output defines, for each pixel in the one or more images, a respective depth value. As another example, the image processing task can be motion estimation, where the network input includes multiple images, and the image processing output defines, for each pixel of one of the input images, a motion of the scene depicted at the pixel between the images in the network input.

[0220] In some implementations, input(s) 2 can be or otherwise represent natural language data. Machine-learned model(s) 1 can process the natural language data to generate an output. As an example, machine-learned model(s) 1 can process the natural language data to generate a language encoding output. As another example, machine-learned model(s) 1 can process the natural language data to generate a latent text embedding output. As another example, machine-learned model(s) 1 can process the natural language data to generate a translation output. As another example, machine-learned model(s) 1 can process the natural language data to generate a classification output. As another example, machine-learned model(s) 1 can process the natural language data to generate a textual segmentation output. As another example, machine-learned model(s) 1 can process the natural language data to generate a semantic intent output. As another example, machine-learned model(s) 1 can process the natural language data to generate an upscaled text or natural language output (e.g., text or natural language data that is higher quality than the input text or natural language, etc.). As another example, machine-learned model(s) 1 can process the natural language data to generate a prediction output (e.g., one or more predicted next portions of natural language content).

[0221] In some implementations, input(s) 2 can be or otherwise represent speech data (e.g., data describing spoken natural language, such as audio data, textual data, etc.). Machine-learned model(s) 1 can process the speech data to generate an output. As an example, machine-learned model(s) 1 can process the speech data to generate a speechrecognition output. As another example, machine-learned model(s) 1 can process the speech data to generate a speech translation output. As another example, machine-learned model(s) 1 can process the speech data to generate a latent embedding output. As another example, machine-learned model(s) 1 can process the speech data to generate an encoded speech output (e.g., an encoded and / or compressed representation of the speech data, etc.). As another example, machine-learned model(s) 1 can process the speech data to generate an upscaled speech output (e.g., speech data that is higher quality than the input speech data, etc.). As another example, machine-learned model(s) 1 can process the speech data to generate a textual representation output (e.g., a textual representation of the input speech data, etc.). As another example, machine-learned model(s) 1 can process the speech data to generate a prediction output.

[0222] In some implementations, input(s) 2 can be or otherwise represent latent encoding data (e.g., a latent space representation of an input, etc.). Machine-learned model(s) 1 can process the latent encoding data to generate an output. As an example, machine- learned model(s) 1 can process the latent encoding data to generate a recognition output. As another example, machine-learned model(s) 1 can process the latent encoding data to generate a reconstruction output. As another example, machine-learned model(s) 1 can process the latent encoding data to generate a search output. As another example, machine- learned model(s) 1 can process the latent encoding data to generate a reclustering output. As another example, machine-learned model(s) 1 can process the latent encoding data to generate a prediction output.

[0223] In some implementations, input(s) 2 can be or otherwise represent statistical data. Statistical data can be, represent, or otherwise include data computed and / or calculated from some other data source. Machine-learned model(s) 1 can process the statistical data to generate an output. As an example, machine-learned model(s) 1 can process the statistical data to generate a recognition output. As another example, machine-learned model(s) 1 can process the statistical data to generate a prediction output. As another example, machine- learned model(s) 1 can process the statistical data to generate a classification output. As another example, machine-learned model(s) 1 can process the statistical data to generate a segmentation output. As another example, machine-learned model(s) 1 can process the statistical data to generate a visualization output. As another example, machine-learned model(s) 1 can process the statistical data to generate a diagnostic output.

[0224] In some implementations, input(s) 2 can be or otherwise represent sensor data. Machine-learned model(s) 1 can process the sensor data to generate an output. As anexample, machine-learned model(s) 1 can process the sensor data to generate a recognition output. As another example, machine-learned model(s) 1 can process the sensor data to generate a prediction output. As another example, machine-learned model(s) 1 can process the sensor data to generate a classification output. As another example, machine-learned model(s) 1 can process the sensor data to generate a segmentation output. As another example, machine-learned model(s) 1 can process the sensor data to generate a visualization output. As another example, machine-learned model(s) 1 can process the sensor data to generate a diagnostic output. As another example, machine-learned model(s) 1 can process the sensor data to generate a detection output.

[0225] In some implementations, machine-learned model(s) 1 can be configured to perform a task that includes encoding input data for reliable and / or efficient transmission or storage (and / or corresponding decoding). For example, the task may be an audio compression task. The input may include audio data and the output may comprise compressed audio data. In another example, the input includes visual data (e.g. one or more images or videos), the output comprises compressed visual data, and the task is a visual data compression task. In another example, the task may comprise generating an embedding for input data (e.g. input audio or visual data). In some cases, the input includes audio data representing a spoken utterance and the task is a speech recognition task. The output may comprise a text output which is mapped to the spoken utterance. In some cases, the task comprises encrypting or decrypting input data. In some cases, the task comprises a microprocessor performance task, such as branch prediction or memory address translation.

[0226] In some implementations, the task is a generative task, and machine-learned model(s) 1 can be configured to output content generated in view of input(s) 2. For instance, input(s) 2 can be or otherwise represent data of one or more modalities that encodes context for generating additional content.

[0227] In some implementations, the task can be a text completion task. Machine- learned model(s) 1 can be configured to process input(s) 2 that represent textual data and to generate output(s) 3 that represent additional textual data that completes a textual sequence that includes input(s) 2. For instance, machine-learned model(s) 1 can be configured to generate output(s) 3 to complete a sentence, paragraph, or portion of text that follows from a portion of text represented by input(s) 2.

[0228] In some implementations, the task can be an instruction following task. Machine-learned model(s) 1 can be configured to process input(s) 2 that represent instructions to perform a function and to generate output(s) 3 that advance a goal of satisfyingthe instruction function (e.g., at least a step of a multi-step procedure to perform the function). Output(s) 3 can represent data of the same or of a different modality as input(s) 2. For instance, input(s) 2 can represent textual data (e.g., natural language instructions for a task to be performed) and machine-learned model(s) 1 can process input(s) 2 to generate output(s) 3 that represent textual data responsive to the instructions (e.g., natural language responses, programming language responses, machine language responses, etc.). Input(s) 2 can represent image data (e.g., image-based instructions for a task to be performed, optionally accompanied by textual instructions) and machine-learned model(s) 1 can process input(s) 2 to generate output(s) 3 that represent textual data responsive to the instructions (e.g., natural language responses, programming language responses, machine language responses, etc.). One or more output(s) 3 can be iteratively or recursively generated to sequentially process and accomplish steps toward accomplishing the requested functionality. For instance, an initial output can be executed by an external system or be processed by machine-learned model(s) 1 to complete an initial step of performing a function. Multiple steps can be performed, with a final output being obtained that is responsive to the initial instructions.

[0229] In some implementations, the task can be an instruction following task that is to be performed by a robot and / or autonomous vehicle (e.g., navigation task, assembly task, movement or object manipulation task, and / or any other robotic / vehicular task and the like). Machine-learned model(s) 1 can be configured to process input(s) 2 that represent instructions for the robot or autonomous vehicle to perform a function and to generate output(s) 3 that advance a goal of satisfying the instruction function (e.g., at least a step of a multi-step procedure for controlling the robot / autonomous vehicle to perform the function). Output(s) 3 can represent data of the same or of a different modality as input(s) 2. For instance, input(s) 2 can represent speech or textual data (e.g., natural language instructions or spoken request for a task to be performed by the robot / autonomous vehicle) and machine- learned model(s) 1 can process input(s) 2 to generate output(s) 3 that control the robot or autonomous vehicle to be responsive to the instruction function. The output(s) 3 may, without limitation, for example represent textual data and / or computer / machine data responsive to the instructions (e.g., natural language responses, programming language responses, machine language responses, etc.). Input(s) 2 can represent image data (e.g., image-based instructions for a task to be performed, optionally accompanied by textual / spoken instructions) and machine-learned model(s) 1 can process input(s) 2 to generate output(s) 3 that represent textual data and / or machine data responsive to the instructions (e.g., natural language responses, programming language responses, machinelanguage responses, etc.). One or more output(s) 3 can be iteratively or recursively generated to sequentially process and accomplish steps toward controlling the robot and / or autonomous vehicle in accomplishing the requested functionality (e.g., navigation function / task, assembly function / task, movement or object manipulation function / task, and / or any other robotic / vehicular function / task and the like). For instance, an initial output can be executed by an external system or control system of the robot / vehicle or be processed by machine- learned model(s) 1 to complete an initial step of the robot / vehicle performing a function. Multiple steps can be performed for controlling the robot / vehicle, with a final output being obtained that is responsive to the initial instructions.

[0230] In some implementations, the task can be a question answering task. Machine- learned model(s) 1 can be configured to process input(s) 2 that represent a question to answer and to generate output(s) 3 that advance a goal of returning an answer to the question (e.g., at least a step of a multi-step procedure to perform the function). Output(s) 3 can represent data of the same or of a different modality as input(s) 2. For instance, input(s) 2 can represent textual data (e.g., natural language instructions for a task to be performed) and machine- learned model(s) 1 can process input(s) 2 to generate output(s) 3 that represent textual data responsive to the question (e.g., natural language responses, programming language responses, machine language responses, etc.). Input(s) 2 can represent image data (e.g., image-based instructions for a task to be performed, optionally accompanied by textual instructions) and machine-learned model(s) 1 can process input(s) 2 to generate output(s) 3 that represent textual data responsive to the question (e.g., natural language responses, programming language responses, machine language responses, etc.). One or more output(s) 3 can be iteratively or recursively generated to sequentially process and accomplish steps toward answering the question. For instance, an initial output can be executed by an external system or be processed by machine-learned model(s) 1 to complete an initial step of obtaining an answer to the question (e.g., querying a database, performing a computation, executing a script, etc.). Multiple steps can be performed, with a final output being obtained that is responsive to the question.

[0231] In some implementations, the task can be an image generation task. Machine- learned model(s) 1 can be configured to process input(s) 2 that represent context regarding a desired portion of image content. The context can include text data, image data, audio data, etc. Machine-learned model(s) 1 can be configured to generate output(s) 3 that represent image data that depicts imagery related to the context. For instance, machine-learned model(s) 1 can be configured to generate pixel data of an image. Values for channel(s)associated with the pixels in the pixel data can be selected based on the context (e.g., based on a probability determined based on the context).

[0232] In some implementations, the task can be an audio generation task. Machine- learned model(s) 1 can be configured to process input(s) 2 that represent context regarding a desired portion of audio content. The context can include text data, image data, audio data, etc. Machine-learned model(s) 1 can be configured to generate output(s) 3 that represent audio data related to the context. For instance, machine-learned model(s) 1 can be configured to generate waveform data in the form of an image (e.g., a spectrogram). Values for channel(s) associated with pixels of the image can be selected based on the context. Machine- learned model(s) 1 can be configured to generate waveform data in the form of a sequence of discrete samples of a continuous waveform. Values of the sequence can be selected based on the context (e.g., based on a probability determined based on the context).

[0233] In some implementations, the task can be a data generation task. Machine- learned model(s) 1 can be configured to process input(s) 2 that represent context regarding a desired portion of data (e.g., data from various data domains, such as sensor data, image data, multimodal data, statistical data, etc.). The desired data can be, for instance, synthetic data for training other machine-learned models. The context can include arbitrary data type(s). Machine-learned model(s) 1 can be configured to generate output(s) 3 that represent data that aligns with the desired data. For instance, machine-learned model(s) 1 can be configured to generate data values for populating a dataset. Values for the data object(s) can be selected based on the context (e.g., based on a probability determined based on the context).

[0234] Figure 12 is a block diagram of an example networked computing system that can perform aspects of example implementations of the present disclosure. The system can include a number of computing devices and systems that are communicatively coupled over a network 49. An example computing device 50 is described to provide an example of a computing device that can perform any aspect of the present disclosure (e.g., implementing model host 31, client(s) 32, or both). An example server computing system 60 is described as an example of a server computing system that can perform any aspect of the present disclosure (e.g., implementing model host 31, client(s) 32, or both). Computing device 50 and server computing system(s) 60 can cooperatively interact (e.g., over network 49) to perform any aspect of the present disclosure (e.g., implementing model host 31, client(s) 32, or both). Model development platform system 70 is an example system that can host or serve model development platform(s) 12 for development of machine-learned models. Third-party system(s) 80 are example system(s) with which any of computing device 50, servercomputing system(s) 60, or model development platform system(s) 70 can interact in the performance of various aspects of the present disclosure (e.g., engaging third-party tools, accessing third-party databases or other resources, etc.).

[0235] Network 49 can be any type of communications network, such as a local area network (e.g., intranet), wide area network (e.g., Internet), or some combination thereof and can include any number of wired or wireless links. In general, communication over network 49 can be carried via any type of wired or wireless connection, using a wide variety of communication protocols (e.g., TCP / IP, HTTP, SMTP, FTP), encodings or formats (e.g., HTML, XML), or protection schemes (e.g., VPN, secure HTTP, SSL). Network 49 can also be implemented via a system bus. For instance, one or more devices or systems of Figure 12 can be co-located with, contained by, or otherwise integrated into one or more other devices or systems.

[0236] Computing device 50 can be any type of computing device, such as, for example, a personal computing device (e.g., laptop or desktop), a mobile computing device (e.g., smartphone or tablet), a gaming console or controller, a wearable computing device, an embedded computing device, a server computing device, a virtual machine operating on a host device, or any other type of computing device. Computing device 50 can be a client computing device. Computing device 50 can be an end-user computing device. Computing device 50 can be a computing device of a service provided that provides a service to an end user (who may use another computing device to interact with computing device 50).

[0237] Computing device 50 can include one or more processors 51 and a memory 52. Processor(s) 51 can be any suitable processing device (e.g., a processor core, a microprocessor, an ASIC, an FPGA, a controller, a microcontroller, etc.) and can be one processor or a plurality of processors that are operatively connected. Memory 52 can include one or more non-transitory computer-readable storage media, such as HBM, RAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, etc., and combinations thereof. Memory 52 can store data 53 and instructions 54 which can be executed by processor(s) 51 to cause computing device 50 to perform operations. The operations can implement any one or multiple features described herein. The operations can implement example methods and techniques described herein.

[0238] Computing device 50 can also include one or more input components that receive user input. For example, a user input component can be a touch-sensitive component (e.g., a touch-sensitive display screen or a touch pad) that is sensitive to the touch of a user input object (e.g., a finger or a stylus). The touch-sensitive component can serve toimplement a virtual keyboard. Other example user input components include a microphone, camera, LIDAR, a physical keyboard or other buttons, or other means by which a user can provide user input.

[0239] Computing device 50 can store or include one or more machine-learned models 55. Machine-learned models 55 can include one or more machine-learned model(s) 1, such as a sequence processing model 4. Machine-learned models 55 can include one or multiple model instance(s) 31-1. Machine-learned model(s) 55 can be received from server computing system(s) 60, model development platform system 70, third party system(s) 80 (e.g., an application distribution platform), or developed locally on computing device 50. Machine-learned model(s) 55 can be loaded into memory 52 and used or otherwise implemented by processor(s) 51. Computing device 50 can implement multiple parallel instances of machine-learned model(s) 55.

[0240] Server computing system(s) 60 can include one or more processors 61 and a memory 62. Processor(s) 61 can be any suitable processing device (e.g., a processor core, a microprocessor, an ASIC, an FPGA, a controller, a microcontroller, etc.) and can be one processor or a plurality of processors that are operatively connected. Memory 62 can include one or more non-transitory computer-readable storage media, such as HBM, RAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, etc., and combinations thereof. Memory 62 can store data 63 and instructions 64 which can be executed by processor(s) 61 to cause server computing system(s) 60 to perform operations. The operations can implement any one or multiple features described herein. The operations can implement example methods and techniques described herein.

[0241] In some implementations, server computing system 60 includes or is otherwise implemented by one or multiple server computing devices. In instances in which server computing system 60 includes multiple server computing devices, such server computing devices can operate according to sequential computing architectures, parallel computing architectures, or some combination thereof.

[0242] Server computing system 60 can store or otherwise include one or more machine-learned models 65. Machine-learned model(s) 65 can be the same as or different from machine-learned model(s) 55. Machine-learned models 65 can include one or more machine-learned model(s) 1, such as a sequence processing model 4. Machine-learned models 65 can include one or multiple model instance(s) 31-1. Machine-learned model(s) 65 can be received from computing device 50, model development platform system 70, third party system(s) 80, or developed locally on server computing system(s) 60. Machine-learnedmodel(s) 65 can be loaded into memory 62 and used or otherwise implemented by processor(s) 61. Server computing system(s) 60 can implement multiple parallel instances of machine-learned model(s) 65.

[0243] In an example configuration, machine-learned models 65 can be included in or otherwise stored and implemented by server computing system 60 to establish a client-server relationship with computing device 50 for serving model inferences. For instance, server computing system(s) 60 can implement model host 31 on behalf of client(s) 32 on computing device 50. For instance, machine-learned models 65 can be implemented by server computing system 60 as a portion of a web service (e.g., remote machine-learned model hosting service, such as an online interface for performing machine-learned model operations over a network on server computing system(s) 60). For instance, server computing system(s) 60 can communicate with computing device 50 over a local intranet or internet connection. For instance, computing device 50 can be a workstation or endpoint in communication with server computing system(s) 60, with implementation of machine-learned models 65 being managed by server computing system(s) 60 to remotely perform inference (e.g., for runtime or training operations), with output(s) returned (e.g., cast, streamed, etc.) to computing device 50. Machine-learned models 65 can work cooperatively or interoperatively with machine- learned models 55 on computing device 50 to perform various tasks.

[0244] Model development platform system(s) 70 can include one or more processors 71 and a memory 72. Processor(s) 71 can be any suitable processing device (e.g., a processor core, a microprocessor, an ASIC, an FPGA, a controller, a microcontroller, etc.) and can be one processor or a plurality of processors that are operatively connected. Memory 72 can include one or more non-transitory computer-readable storage media, such as HBM, RAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, etc., and combinations thereof. Memory 72 can store data 73 and instructions 74 which can be executed by processor(s) 71 to cause model development platform system(s) 70 to perform operations. The operations can implement any one or multiple features described herein. The operations can implement example methods and techniques described herein. Example operations include the functionality described herein with respect to model development platform 12. This and other functionality can be implemented by developer tool(s) 75.

[0245] Third-party system(s) 80 can include one or more processors 81 and a memory 82. Processor(s) 81 can be any suitable processing device (e.g., a processor core, a microprocessor, an ASIC, an FPGA, a controller, a microcontroller, etc.) and can be one processor or a plurality of processors that are operatively connected. Memory 82 can includeone or more non-transitory computer-readable storage media, such as HBM, RAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, etc., and combinations thereof. Memory 82 can store data 83 and instructions 84 which can be executed by processor(s) 81 to cause third-party system(s) 80 to perform operations. The operations can implement any one or multiple features described herein. The operations can implement example methods and techniques described herein. Example operations include the functionality described herein with respect to tools and other external resources called when training or performing inference with machine-learned model(s) 1, 4, 16, 20, 55, 65, etc. (e.g., third-party resource(s) 85).

[0246] Figure 12 illustrates one example arrangement of computing systems that can be used to implement the present disclosure. Other computing system configurations can be used as well. For example, in some implementations, one or both of computing system 50 or server computing system(s) 60 can implement all or a portion of the operations of model development platform system 70. For example, computing system 50 or server computing system(s) 60 can implement developer tool(s) 75 (or extensions thereof) to develop, update / train, or refine machine-learned models 1, 4, 16, 20, 55, 65, etc. using one or more techniques described herein with respect to model alignment toolkit 17. In this manner, for instance, computing system 50 or server computing system(s) 60 can develop, update / train, or refine machine-learned models based on local datasets (e.g., for model personalization / customization, as permitted by user data preference selections).

[0247] Figure 13 is a block diagram of an example computing device 98 that performs according to example embodiments of the present disclosure. Computing device 98 can be a user computing device or a server computing device (e.g., computing device 50, server computing system(s) 60, etc.). Computing device 98 can implement model host 31. For instance, computing device 98 can include a number of applications (e.g., applications 1 through N). Each application can contain its own machine learning library and machine- learned model(s). For example, each application can include a machine-learned model. Example applications include a text messaging application, an email application, a dictation application, a virtual keyboard application, a browser application, etc. As illustrated in Figure 13, each application can communicate with a number of other components of the computing device, such as, for example, one or more sensors, a context manager, a device state component, or additional components. In some implementations, each application can communicate with each device component using an API (e.g., a public API). In some implementations, the API used by each application is specific to that application.

[0248] Figure 14 is a block diagram of an example computing device 99 that performs according to example embodiments of the present disclosure. Computing device 99 can be the same as or different from computing device 98. Computing device 99 can be a user computing device or a server computing device (e.g., computing device 50, server computing system(s) 60, etc.). Computing device 98 can implement model host 31. For instance, computing device 99 can include a number of applications (e.g., applications 1 through N). Each application can be in communication with a central intelligence layer. Example applications include a text messaging application, an email application, a dictation application, a virtual keyboard application, a browser application, etc. In some implementations, each application can communicate with the central intelligence layer (and model(s) stored therein) using an API (e.g., a common API across all applications).

[0249] The central intelligence layer can include a number of machine-learned models. For example, as illustrated in Figure 14, a respective machine-learned model can be provided for each application and managed by the central intelligence layer. In other implementations, two or more applications can share a single machine-learned model. For example, in some implementations, the central intelligence layer can provide a single model for all of the applications. In some implementations, the central intelligence layer is included within or otherwise implemented by an operating system of computing device 99.

[0250] The central intelligence layer can communicate with a central device data layer. The central device data layer can be a centralized repository of data for computing device 99. As illustrated in Figure 14, the central device data layer can communicate with a number of other components of the computing device, such as, for example, one or more sensors, a context manager, a device state component, or additional components. In some implementations, the central device data layer can communicate with each device component using an API (e.g., a private API).

[0251] Figure 15 depicts a flowchart of a method 1500 for updating a tokenizer according to aspects of the present disclosure.

[0252] One or more portion(s) of example method 1500 can be implemented by a computing system that includes one or more computing devices such as, for example, computing systems described with reference to the other figures. Each respective portion of example method 1500 can be performed by any (or any combination) of one or more computing devices. Moreover, one or more portion(s) of example method 1500 can be implemented on the hardware components of the device(s) described herein, for example, to train one or more systems or models. Figure 15 depicts elements performed in a particularorder for purposes of illustration and discussion. Those of ordinary skill in the art, using the disclosures provided herein, will understand that the elements of any of the methods discussed herein can be adapted, rearranged, expanded, omitted, combined, or modified in various ways without deviating from the scope of the present disclosure. Figure 15 is described with reference to elements / terms described with respect to other systems and figures for exemplary illustrated purposes and is not meant to be limiting. One or more portions of example method 1500 can be performed additionally, or alternatively, by other systems.

[0253] At 1502, example method 1500 can include obtaining plurality of selection records. For instance, the plurality of selection records can be stored using the selection record cache 108.

[0254] At 1502-2 of example method 1500, obtaining a respective selection record (e.g., one of selection records 108-1, . . . , 108-N) can include obtaining content rendered using a user interface. For instance, the content can include content data 104-1, . . . , 104-N. The content can be selectable at a precision corresponding to individual content elements (e.g., content elements 200) that compose the content.

[0255] At 1502-4 of example method 1500, obtaining the respective selection record can include tokenizing the content using a tokenizer (e.g., tokenizer 102) to obtain a tokenized representation of the content (e.g., tokenized content data 104-1, . . . , 104-N), wherein one or more tokens of the tokenized representation each represent a plurality of the individual content elements.

[0256] At 1502-6 of example method 1500, obtaining the respective selection record can include obtaining a ground truth selection boundary (e.g., selection boundary 206-1 corresponding to ground truth selection 204-1) corresponding to a selection instruction that selects a portion of the content, wherein the ground truth selection boundary is between two adjacent content elements that were represented together by a respective token in the tokenized representation such that the ground truth selection boundary divides the content elements represented by the respective token into a respective first group on a first side of the boundary (e.g., first portion 208-1-1) and a respective second group on a second side of the boundary (e.g., second portion 208-1-2).

[0257] At 1502-8 of example method 1500, obtaining a respective selection record can include storing, in the respective selection record, a subtoken value (e.g., subtoken value 210-1) indicating the respective first group or the respective second group.

[0258] At 1504, example method 1500 can include processing the plurality of selection records to determine a recurrence measure (e.g., recurrence measure 112) associated with the sub token value.

[0259] At 1506, example method 1500 can include, based on the recurrence measure, updating the tokenizer to use a token representing the subtoken value (e.g., in a token update 114).

[0260] In some implementations of example method 1500, updating the tokenizer includes adding the token representing the subtoken value to a set of candidate tokens used by the tokenizer to tokenize content.

[0261] In some implementations of example method 1500, the token representing the subtoken value represents the respective first group, and the method includes updating the tokenizer to use an additional token representing the respective second group.

[0262] In some implementations of example method 1500, updating the tokenizer includes deactivating the respective token.

[0263] In some implementations of example method 1500, the subtoken value includes the content elements of the respective first group or the respective second group. In some implementations of example method 1500, the subtoken value includes a token identifier for the respective token and an index value corresponding to a boundary of the subtoken value within the content elements represented by the respective token.

[0264] In some implementations, example method 1500 includes storing, in the respective selection record, a representation of a contiguous sequence of two or more tokens within the selected portion, wherein the two or more tokens collectively represent a sequence of individual content elements. In some implementations, example method 1500 includes processing the plurality of selection records to determine a recurrence measure associated with the contiguous sequence. In some implementations, example method 1500 includes based on the recurrence measure, updating the tokenizer to use a single token that represents the sequence of individual content elements.

[0265] Figure 16 depicts a flowchart of a method 1600 for updating a tokenizer according to aspects of the present disclosure.

[0266] One or more portion(s) of example method 1600 can be implemented by a computing system that includes one or more computing devices such as, for example, computing systems described with reference to the other figures. Each respective portion of example method 1600 can be performed by any (or any combination) of one or more computing devices. Moreover, one or more portion(s) of example method 1600 can beimplemented on the hardware components of the device(s) described herein, for example, to train one or more systems or models. Figure 16 depicts elements performed in a particular order for purposes of illustration and discussion. Those of ordinary skill in the art, using the disclosures provided herein, will understand that the elements of any of the methods discussed herein can be adapted, rearranged, expanded, omitted, combined, or modified in various ways without deviating from the scope of the present disclosure. Figure 16 is described with reference to elements / terms described with respect to other systems and figures for exemplary illustrated purposes and is not meant to be limiting. One or more portions of example method 1600 can be performed additionally, or alternatively, by other systems.

[0267] At 1602, example method 1600 can include obtaining plurality of selection records.

[0268] At 1602-2 of example method 1600, obtaining a respective selection record can include obtaining content rendered using a user interface, wherein the content is selectable at a precision corresponding to individual content elements that compose the content.

[0269] At 1602-4 of example method 1600, obtaining a respective selection record can include tokenizing the content using a tokenizer to obtain a tokenized representation of the content, wherein tokens of the tokenized representation each represent one or more of the individual content elements.

[0270] At 1602-6 of example method 1600, obtaining a respective selection record can include obtaining a ground truth selection boundary corresponding to a selection instruction that selects a portion of the content.

[0271] At 1602-8 of example method 1600, obtaining a respective selection record can include storing, in the respective selection record, a representation of a contiguous sequence of two or more tokens within the selected portion (e.g., contiguous token value(s) 308-1, . . . , 308-N), wherein the two or more tokens collectively represent a sequence of individual content elements.

[0272] At 1604, example method 1600 can include processing the plurality of selection records to determine a recurrence measure associated with the contiguous sequence.

[0273] At 1604, example method 1600 can include, based on the recurrence measure, updating the tokenizer to use a single token that represents the sequence of individual content elements.

[0274] In some implementations of example method 1500 or example method 1600, updating the tokenizer includes adding the single token that represents the sequence of individual content elements to a set of candidate tokens used by the tokenizer to tokenize content.

[0275] In some implementations of example method 1500 or example method 1600, updating the tokenizer includes deactivating the two or more tokens.

[0276] In some implementations of example method 1500 or example method 1600, deactivating the two or more tokens includes deactivating the two or more tokens based on determining that zero selection records of the plurality of selection records correspond to a ground truth selection boundary that divides the two or more tokens.

[0277] In some implementations of example method 1500 or example method 1600, deactivating the two or more tokens includes deactivating the two or more tokens based on determining that less than a threshold number of selection records of the plurality of selection records correspond to a ground truth selection boundary that divides the two or more tokens.

[0278] In some implementations of example method 1500 or example method 1600, the representation of the contiguous sequence includes token identifiers corresponding to the two or more tokens. In some implementations of example method 1500 or example method 1600, the representation of the contiguous sequence includes element identifiers corresponding to the individual content elements in the sequence of individual content elements. In some implementations of example method 1500 or example method 1600, the representation of the contiguous sequence includes the individual content elements in the sequence of individual content elements.

[0279] In some implementations of example method 1500 or example method 1600, the contiguous sequence immediately precedes the selection boundary.

[0280] In some implementations, example method 1500 or example method 1600 includes storing, in the respective selection record, a ground truth portion selected by the ground truth selection boundary.

[0281] In some implementations of example method 1500 or example method 1600, the ground truth portion includes the representation of the contiguous sequence. In some implementations of example method 1500 or example method 1600, the ground truth portion includes the representation of the subtoken value.

[0282] In some implementations of example method 1500 or example method 1600, the ground truth portion includes a predetermined number of tokens, wherein thepredetermined number of tokens corresponds to a configurable hyperparameters for a merging limit.

[0283] In some implementations of example method 1500 or example method 1600, deactivating a particular token includes removing the particular token from a set of candidate tokens used by the tokenizer. In some implementations of example method 1500 or example method 1600, deactivating a particular token includes bypassing parsing logic in the tokenizer that causes the tokenizer to output the particular token.

[0284] In some implementations, example method 1500 or example method 1600 includes receiving runtime content for rendering using a user interface, wherein the runtime content is selectable at a first precision corresponding to individual content elements that compose the runtime content. In some implementations, example method 1500 or example method 1600 includes receiving data describing a runtime input that instructs selection of a portion of the runtime content. In some implementations, example method 1500 or example method 1600 includes tokenizing, using the updated tokenizer, the runtime content to obtain a tokenized representation of the runtime content. In some implementations, example method 1500 or example method 1600 includes processing, using a machine-learned model, the tokenized representation of the runtime content to predict a predicted selection boundary, wherein the predicted selection boundary defines a selection at a second precision corresponding to individual tokens that compose the tokenized representation of the runtime content. In some implementations, example method 1500 or example method 1600 includes outputting the predicted selection boundary.

[0285] In some implementations, example method 1500 or example method 1600 includes receiving data describing a correction input that indicates an adjustment to the predicted selection boundary. In some implementations, example method 1500 or example method 1600 includes using the adjusted predicted selection boundary as the ground truth selection boundary.

[0286] Figure 17 depicts a flowchart of a method 1700 for using a tokenizer according to aspects of the present disclosure.

[0287] One or more portion(s) of example method 1700 can be implemented by a computing system that includes one or more computing devices such as, for example, computing systems described with reference to the other figures. Each respective portion of example method 1700 can be performed by any (or any combination) of one or more computing devices. Moreover, one or more portion(s) of example method 1700 can be implemented on the hardware components of the device(s) described herein, for example, totrain one or more systems or models. Figure 17 depicts elements performed in a particular order for purposes of illustration and discussion. Those of ordinary skill in the art, using the disclosures provided herein, will understand that the elements of any of the methods discussed herein can be adapted, rearranged, expanded, omitted, combined, or modified in various ways without deviating from the scope of the present disclosure. Figure 17 is described with reference to elements / terms described with respect to other systems and figures for exemplary illustrated purposes and is not meant to be limiting. One or more portions of example method 1700 can be performed additionally, or alternatively, by other systems.

[0288] At 1702, example method 1700 can include receiving runtime content for rendering using a user interface (e.g., content data 400), wherein the runtime content is selectable at a first precision corresponding to individual content elements that compose the runtime content.

[0289] At 1704, example method 1700 can include receiving data describing a runtime input that instructs selection of a portion of the runtime content.

[0290] At 1706, example method 1700 can include tokenizing, using a tokenizer that was updated according to an implementation of example method 1500 or example method 1600, the runtime content to obtain a tokenized representation of the runtime content (e.g., tokenized content data 404).

[0291] At 1708, example method 1700 can include processing, using a machine- learned model (e.g., selection prediction model 406), the tokenized representation of the runtime content to predict a predicted selection boundary (e.g., predicted selection 408), wherein the predicted selection boundary defines a selection at a second precision corresponding to individual tokens that compose the tokenized representation of the runtime content.

[0292] At 1710, example method 1700 can include outputting the predicted selection boundary.

[0293] In some implementations, example method 1500, example method 1600, or example method 1700 includes rendering a suggested selection indicator based on the predicted selection boundary.

[0294] In some implementations of example method 1500, example method 1600, or example method 1700, updating the tokenizer includes resizing a layer of the machine- learned model to be compatible with a size of the updated set of candidate tokens.

[0295] In some implementations, example method 1500, example method 1600, or example method 1700 includes retraining the machine-learned model after updating the tokenizer.

[0296] In some implementations of example method 1500, example method 1600, or example method 1700, the machine-learned model was retrained using the plurality of selection records as training examples.

[0297] In some implementations of example method 1500, example method 1600, or example method 1700, a content element corresponds to a data object. In some implementations of example method 1500, example method 1600, or example method 1700, a content element corresponds to a textual character. In some implementations of example method 1500, example method 1600, or example method 1700, a content element corresponds to an image or an image patch. In some implementations of example method 1500, example method 1600, or example method 1700, a content element corresponds to a frame of a video. In some implementations of example method 1500, example method 1600, or example method 1700, a content element corresponds to a timestamp associated with timevarying content.

[0298] In some implementations of example method 1500, example method 1600, or example method 1700, the plurality of selection records are maintained in a rolling buffer.

[0299] In some implementations of example method 1500, example method 1600, or example method 1700, the tokenizer is updated responsive to detecting a trigger condition based on a number of selection records accumulated after a previous update.

[0300] In some implementations of example method 1500, example method 1600, or example method 1700, the content includes logographic text, and wherein the individual content elements correspond to logographs.

[0301] The technology discussed herein makes reference to servers, databases, software applications, and other computer-based systems, as well as actions taken and information sent to and from such systems. The inherent flexibility of computer-based systems allows for a great variety of possible configurations, combinations, and divisions of tasks and functionality between and among components. For instance, processes discussed herein can be implemented using a single device or component or multiple devices or components working in combination. Databases and applications can be implemented on a single system or distributed across multiple systems. Distributed components can operate sequentially or in parallel. In further examples, there is provided a computer program product including computer-executable instructions which, when executed by at least one computingapparatus (or one or more processors), cause the at least one computing apparatus (or one or more processors) to perform one or more of the method(s) or process(es) described with reference to the Figures and as described herein.

[0302] While the present subject matter has been described in detail with respect to various specific example embodiments or implementations thereof, each example is provided by way of explanation, not limitation of the disclosure. Those skilled in the art, upon attaining an understanding of the foregoing, can readily produce alterations to, variations of, and equivalents to such embodiments. Accordingly, the subject disclosure does not preclude inclusion of such modifications, variations or additions to the present subject matter as would be readily apparent to one of ordinary skill in the art. For instance, features illustrated or described as part of one embodiment can be used with another embodiment to yield a still further embodiment. Thus, it is intended that the present disclosure cover such alterations, variations, and equivalents.

[0303] Aspects of the disclosure have been described in terms of illustrative embodiments thereof. Any and all features in the following claims can be combined or rearranged in any way possible, including combinations of claims not explicitly enumerated in combination together, as the example claim dependencies listed herein should not be read as limiting the scope of possible combinations of features disclosed herein. Accordingly, the scope of the present disclosure is by way of example rather than by way of limitation, and the subject disclosure does not preclude inclusion of such modifications, variations or additions to the present subject matter as would be readily apparent to one of ordinary skill in the art. Moreover, terms are described herein using lists of example elements joined by conjunctions such as “and,” “or,” “but,” etc. It should be understood that such conjunctions are provided for explanatory purposes only. Clauses and other sequences of items joined by a particular conjunction such as “or,” for example, can refer to “and / or,” “at least one of’, “any combination of’ example elements listed therein, etc. Terms such as “based on” should be understood as “based at least in part on.”

[0304] The term “can” should be understood as referring to a possibility of a feature in various implementations and not as prescribing an ability that is necessarily present in every implementation. For example, the phrase “X can perform Y” should be understood as indicating that, in various implementations, X has the potential to be configured to perform Y, and not as indicating that in every instance X must always be able to perform Y. It should be understood that, in various implementations, X might be unable to perform Y and remain within the scope of the present disclosure.

[0305] The term “may” should be understood as referring to a possibility of a feature in various implementations and not as prescribing an ability that is necessarily present in every implementation. For example, the phrase “X may perform Y” should be understood as indicating that, in various implementations, X has the potential to be configured to perform Y, and not as indicating that in every instance X must always be able to perform Y. It should be understood that, in various implementations, X might be unable to perform Y and remain within the scope of the present disclosure.

Claims

WHAT IS CLAIMED IS:

1. A computer-implemented method for adaptive tokenization of data, the computer- implemented method comprising: obtaining plurality of selection records, wherein obtaining a respective selection record comprises: obtaining content rendered using a user interface, wherein the content is selectable at a precision corresponding to individual content elements that compose the content; tokenizing the content using a tokenizer to obtain a tokenized representation of the content, wherein one or more tokens of the tokenized representation each represent a plurality of the individual content elements; obtaining a ground truth selection boundary corresponding to a selection instruction that selects a portion of the content, wherein the ground truth selection boundary is between two adjacent content elements that were represented together by a respective token in the tokenized representation such that the ground truth selection boundary divides the content elements represented by the respective token into a respective first group on a first side of the boundary and a respective second group on a second side of the boundary; and storing, in the respective selection record, a subtoken value indicating the respective first group or the respective second group; processing the plurality of selection records to determine a recurrence measure associated with the subtoken value; and based on the recurrence measure, updating the tokenizer to use a token representing the sub token value.

2. The computer-implemented method of claim 1, wherein updating the tokenizer comprises: adding the token representing the subtoken value to a set of candidate tokens used by the tokenizer to tokenize content.

3. The computer-implemented method of claim 1, wherein: the token representing the subtoken value represents the respective first group, and the method comprises:updating the tokenizer to use an additional token representing the respective second group.

4. The computer-implemented method of claim 3, wherein updating the tokenizer comprises: deactivating the respective token.

5. The computer-implemented method of any of the preceding claims, wherein the sub token value comprises: the content elements of the respective first group or the respective second group; or a token identifier for the respective token and an index value corresponding to a boundary of the subtoken value within the content elements represented by the respective token.

6. The computer-implemented method of any of the preceding claims, comprising: storing, in the respective selection record, a representation of a contiguous sequence of two or more tokens within the selected portion, wherein the two or more tokens collectively represent a sequence of individual content elements; processing the plurality of selection records to determine a recurrence measure associated with the contiguous sequence; and based on the recurrence measure, updating the tokenizer to use a single token that represents the sequence of individual content elements.

7. A computer-implemented method for adaptive tokenization of data, the computer- implemented method comprising: obtaining plurality of selection records, wherein obtaining a respective selection record comprises: obtaining content rendered using a user interface, wherein the content is selectable at a precision corresponding to individual content elements that compose the content; tokenizing the content using a tokenizer to obtain a tokenized representation of the content, wherein tokens of the tokenized representation each represent one or more of the individual content elements;obtaining a ground truth selection boundary corresponding to a selection instruction that selects a portion of the content; and storing, in the respective selection record, a representation of a contiguous sequence of two or more tokens within the selected portion, wherein the two or more tokens collectively represent a sequence of individual content elements; processing the plurality of selection records to determine a recurrence measure associated with the contiguous sequence; and based on the recurrence measure, updating the tokenizer to use a single token that represents the sequence of individual content elements.

8. The computer-implemented method of any of claims 6 or 7, wherein updating the tokenizer comprises: adding the single token that represents the sequence of individual content elements to a set of candidate tokens used by the tokenizer to tokenize content.

9. The computer-implemented method of any of claims 6, 7, or 8, wherein updating the tokenizer comprises: deactivating the two or more tokens.

10. The computer-implemented method of claim 9, wherein deactivating the two or more tokens comprises: deactivating the two or more tokens based on determining that zero selection records of the plurality of selection records correspond to a ground truth selection boundary that divides the two or more tokens.

11. The computer-implemented method of claim 9, wherein deactivating the two or more tokens comprises: deactivating the two or more tokens based on determining that less than a threshold number of selection records of the plurality of selection records correspond to a ground truth selection boundary that divides the two or more tokens.

12. The computer-implemented method of any of claims 6 to 11, wherein the representation of the contiguous sequence comprises: token identifiers corresponding to the two or more tokens;element identifiers corresponding to the individual content elements in the sequence of individual content elements; or the individual content elements in the sequence of individual content elements.

13. The computer-implemented method of any of claims 6 to 12, wherein the contiguous sequence immediately precedes the selection boundary.

14. The computer-implemented method of any of the preceding claims, comprising: storing, in the respective selection record, a ground truth portion selected by the ground truth selection boundary.

15. The computer-implemented method of claim 14, wherein the ground truth portion comprises: the representation of the contiguous sequence; or the representation of the subtoken value.

16. The computer-implemented method of claim 14, wherein the ground truth portion comprises a predetermined number of tokens, wherein the predetermined number of tokens corresponds to a configurable hyperparameters for a merging limit.

17. The computer-implemented method of any of the preceding claims, wherein deactivating a particular token comprises: removing the particular token from a set of candidate tokens used by the tokenizer; or bypassing parsing logic in the tokenizer that causes the tokenizer to output the particular token.

18. The computer-implemented method of any of the preceding claims, comprising: receiving runtime content for rendering using a user interface, wherein the runtime content is selectable at a first precision corresponding to individual content elements that compose the runtime content; receiving data describing a runtime input that instructs selection of a portion of the runtime content; tokenizing, using the updated tokenizer, the runtime content to obtain a tokenized representation of the runtime content;processing, using a machine-learned model, the tokenized representation of the runtime content to predict a predicted selection boundary, wherein the predicted selection boundary defines a selection at a second precision corresponding to individual tokens that compose the tokenized representation of the runtime content; and outputting the predicted selection boundary.

19. The computer-implemented method of claim 18, comprising: receiving data describing a correction input that indicates an adjustment to the predicted selection boundary; and using the adjusted predicted selection boundary as the ground truth selection boundary.

20. A computer-implemented method for an improved content selection interface using adaptive tokenization of data, the computer-implemented method comprising: receiving runtime content for rendering using a user interface, wherein the runtime content is selectable at a first precision corresponding to individual content elements that compose the runtime content; receiving data describing a runtime input that instructs selection of a portion of the runtime content; tokenizing, using a tokenizer that was updated according to the method of any of the preceding claims, the runtime content to obtain a tokenized representation of the runtime content; processing, using a machine-learned model, the tokenized representation of the runtime content to predict a predicted selection boundary, wherein the predicted selection boundary defines a selection at a second precision corresponding to individual tokens that compose the tokenized representation of the runtime content; and outputting the predicted selection boundary.

21. The computer-implemented method of any of claims 18 to 20, comprising: rendering a suggested selection indicator based on the predicted selection boundary.

22. The computer-implemented method of any of claims 18 to 21, wherein updating the tokenizer comprises:resizing a layer of the machine-learned model to be compatible with a size of the updated set of candidate tokens.

23. The computer-implemented method of any of claims 18 to 22, comprising: retraining the machine-learned model after updating the tokenizer.

24. The computer-implemented method of claim 23, wherein the machine-learned model was retrained using the plurality of selection records as training examples.

25. The computer-implemented method of any of the preceding claims, wherein a content element corresponds to: a data object; a textual character; an image or an image patch; a frame of a video; or a timestamp associated with time-varying content.

26. The computer-implemented method of any of the preceding claims, wherein the plurality of selection records are maintained in a rolling buffer.

27. The computer-implemented method of any of the preceding claims, wherein the tokenizer is updated responsive to detecting a trigger condition based on a number of selection records accumulated after a previous update.

28. The computer-implemented method of any of the preceding claims, wherein the content comprises logographic text, and wherein the individual content elements correspond to logographs.

29. One or more non-transitory computer-readable media storing instructions that are executable by one or more processors to cause a computing system to perform operations, the operations comprising the method of any of the preceding claims.

30. A computing system, comprising: one or more processors; andone or more non-transitory computer-readable media storing instructions that are executable by the one or more processors to cause the computing system to perform operations, the operations comprising the method of any of the preceding claims.

31. A computer program product comprising instructions that are executable by one or more processors to cause a computing system to perform operations, the operations comprising the method of any of the preceding claims.

Images