Task augmentation and self-training for improved few-shot learning

Abstract

Task augmentation and self-training for improved few-shot learning: systems and methods that can leverage task-specific unlabeled data to improve downstream performance in data-constrained scenarios. Given a target task, the first technique, referred to as task augmentation, uses unlabeled text from the target domain to synthesize large amounts of in-domain training data for an auxiliary task. The second technique provides a self-training algorithm in which a model learns to improve itself using its predictions on unlabeled examples.

Description

[0001] Related applications

[0002] This application claims priority to and the benefit of U.S. Provisional Patent Application No. 63 / 194,474, filed on May 28, 2021. U.S. Provisional Patent Application No. 63 / 194,474 is hereby incorporated by reference in its entirety. Technical Field

[0003] The present disclosure relates generally to machine learning and, more particularly, to systems and methods for leveraging task-specific unlabeled data to improve downstream performance in data-constrained scenarios. Background Art

[0004] Recent advances in natural language processing (NLP) have demonstrated the effectiveness of applying large-scale Transformer language models to downstream tasks. While these models have achieved state-of-the-art results on many NLP benchmarks, they struggle when given limited training data for the downstream (or "target") task. For example, some studies have found that the BERT model is prone to performance degradation on small datasets. While large language models like GPT-3 have shown the ability to solve new tasks from just a few examples without any fine-tuning, their performance still lags far behind state-of-the-art fine-tuned results. Manually annotating large amounts of training data would improve performance, but again may be too expensive to obtain for many tasks and domains. Summary of the Invention

[0005] Various aspects and advantages of the embodiments of the present disclosure will be set forth in part in the following description, or can be learned from the description, or can be learned through practice of the embodiments.

[0006] One example aspect of the present disclosure relates to a computer-implemented method for achieving improved learning with fewer training examples. The method includes obtaining, by a computing system comprising one or more computing devices, an unlabeled training dataset associated with a target task, the unlabeled training dataset comprising a plurality of unlabeled training examples, wherein the plurality of unlabeled training examples are in-domain for the target task. The method includes accessing, by the computing system, a first machine learning model, the first machine learning model having been previously trained using a labeled training dataset associated with a pre-training task different from the target task, the labeled training dataset comprising a plurality of labeled training examples, wherein the plurality of labeled training examples are out-of-domain for the target task. The method includes processing, by the computing system, each unlabeled training example with the first machine learning model to generate a synthetic complement for each unlabeled training example, respectively, the plurality of training examples and the synthetic complement forming a synthetic training dataset. The method includes training, by the computing system, a second, different machine learning model using the synthetic training dataset.

[0007] In some implementations, the labeled training dataset includes a plurality of labeled natural language inference training examples, each labeled natural language inference training example including a first word-gram, a second word-gram, and a tag describing a relationship between the first word-gram and the second word-gram. In some implementations, the first machine learning model includes a generative language model that has been trained to process the first word-gram and the tag to predict the second word-gram.

[0008] In some implementations, each unlabeled training example in the unlabeled training dataset includes an unlabeled word-gram string. In some implementations, processing each unlabeled training example with the first machine learning model by the computing system to generate a synthetic supplement for each unlabeled training example includes processing each unlabeled word-gram string and the supplied label by the computing system to generate a synthetic word-gram string.

[0009] In some implementations, processing each unlabeled word string and the supplied token by the computing system to generate the composite word string includes processing each unlabeled word string and a plurality of different supplied tokens by the computing system to generate a plurality of different composite word strings for each unlabeled word string.

[0010] In some implementations, the method further includes filtering, by the computing system, the plurality of different synthetic word strings using a third machine learning model.

[0011] In some implementations, filtering the multiple different synthetic word strings using the third machine learning model by the computing system includes: for each pair of an unlabeled word string and a synthetic word string: processing the pair of unlabeled word string and synthetic word string using the third machine learning model by the computing system to generate a predicted tag; and determining, by the computing system, whether the predicted tag matches a supplied tag, wherein the supplied tag is supplied to generate the synthetic word string.

[0012] In some implementations, filtering the multiple different synthetic word strings using the third machine learning model by the computing system further includes, for each pair of unlabeled word strings and synthetic word strings and when the predicted tag matches the supplied tag: determining by the computing system whether a confidence value output by the third machine learning model for the predicted tag satisfies a threshold; when the confidence value output by the third machine learning model for the predicted tag satisfies the threshold: maintaining by the computing system the pair of unlabeled word strings and synthetic word strings in the synthetic training dataset; and when the confidence value output by the third machine learning model for the predicted tag does not satisfy the threshold: discarding by the computing system the pair of unlabeled word strings and synthetic word strings from the synthetic training dataset.

[0013] In some implementations, the method further includes, after the computing system uses the synthetic training dataset to train the second machine learning model: training the second machine learning model using a second labeled training dataset associated with the target task by the computing system, the second labeled training dataset comprising a second plurality of labeled training examples, wherein the second plurality of labeled training examples are in-domain for the target task.

[0014] Another example aspect relates to a computing system configured to perform improved learning with few training examples, the computing system comprising: one or more processors; and one or more non-transitory computer-readable media collectively storing instructions that, when executed by the one or more processors, cause the computing system to perform operations comprising: for each of a plurality of training iterations: accessing a current labeled training dataset associated with a target task, the current labeled training dataset comprising labeled training examples, wherein the labeled training examples are in-domain for the target task; training a base model using the current labeled training dataset; to generate a current student model; access an unlabeled training dataset associated with the target task, the unlabeled training dataset comprising unlabeled training examples, wherein the unlabeled training examples are in-domain for the target task; process each unlabeled training data with the current student model to generate a synthetic label for each unlabeled training example, respectively, the unlabeled training examples and the synthetic label forming a self-labeled training dataset; and combine some or all of the self-labeled training dataset with the original labeled training dataset to generate the current labeled training dataset for the next training iteration in the multiple training iterations; and output the current student model as an output model after the multiple training iterations.

[0015] In some implementations, the same base model is used in each of the multiple training iterations. In some implementations, combining some or all of the self-labeled training dataset with the original labeled training dataset to generate the current labeled training dataset for the next training iteration includes combining all of the self-labeled training dataset with the original labeled training dataset to generate the current labeled training dataset for the next training iteration. In some implementations, the base model includes a base language model; and the target task includes a natural language processing task.

[0016] Other aspects of the disclosure relate to various systems, apparatuses, non-transitory computer-readable media, user interfaces, and electronic devices.

[0017] These and other features, aspects and advantages of various embodiments of the present disclosure will become better understood with reference to the following description.The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate example embodiments of the present disclosure and, together with the description, serve to explain the relevant principles. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] A detailed discussion of embodiments for those of ordinary skill in the art is set forth in the specification with reference to the accompanying drawings, in which:

[0019] Figure 1A Example experimental results are provided showing that an example implementation of the proposed STraTA method significantly improves sample efficiency across different tasks.

[0020] Figure 1B An illustration of an example implementation of the proposed Self-Training with Task Augmentation (STraTA) method according to an example embodiment of the present disclosure is provided.

[0021] Figures 2A to 2D Depicted is a block diagram of an example task augmentation method according to an example embodiment of the present disclosure.

[0022] Figures 3A to 3E Depicted is a block diagram of an example task augmentation method applied in a natural language processing context according to an example embodiment of the present disclosure.

[0023] Figures 4A to 4B Depicted is a block diagram of an example self-learning method according to an example embodiment of the present disclosure.

[0024] Figure 5A Depicted is a block diagram of an example computing system according to an example embodiment of the present disclosure.

[0025] Figure 5B A block diagram of an example computing device is depicted, according to example embodiments of the present disclosure.

[0026] Figure 5C A block diagram of an example computing device is depicted, according to example embodiments of the present disclosure.

[0027] In various implementations, reference numbers that are repeated across multiple figures are intended to identify the same features. DETAILED DESCRIPTION

[0028] Overview

[0029] Generally, the present disclosure relates to systems and methods for leveraging task-specific unlabeled data to improve downstream performance in data-constrained scenarios. Given a target task, the first technique presented herein, which can be referred to as task augmentation, uses unlabeled text from the target domain to synthesize a large amount of in-domain training data for an auxiliary task. Task augmentation provides significant performance gains across diverse tasks, often outperforming competing fine-tuning methods. The second technique provides a self-training algorithm, in which a model learns to improve itself using its predictions on unlabeled examples. Example experiments included in the appendix reveal the importance of using a strong base model and training on a broad distribution of tasks for successful self-training. These techniques can significantly improve sample efficiency across a variety of tasks, as demonstrated across 12 NLP benchmark datasets. For example, with only 8 training examples per class from the SST-2 sentiment dataset, an example implementation of the present disclosure achieves results comparable to standard fine-tuning using 67K training examples. Therefore, the present disclosure provides two complementary approaches, task augmentation and self-training, to alleviate the need for labeled data by leveraging task-specific unlabeled data, which is relatively inexpensive to obtain.

[0030] More specifically, task augmentation uses unlabeled text from the domain of a given target task to simulate a large amount of in-domain training data for the auxiliary task of natural language inference (NLI). An example method can first train an NLI data generator by fine-tuning a pre-trained generative language model on the MNLI dataset in a text-to-text format. Then, given a target task (e.g., sentiment analysis) with unlabeled text (e.g., his acting was really awful), the generative language model can be used to generate NLI examples such as [his acting was really awful, he gave an incredible performance, contradiction]. Finally, before fine-tuning an additional model (e.g., a BERT model) on the target task, the additional model can be fine-tuned on the newly created auxiliary NLI dataset. Compared to other fine-tuning methods, task augmentation significantly improves downstream performance across different tasks.

[0031] The second approach (i.e., self-training) uses the predictions of the model (e.g., the predictions of the BERT model for unlabeled examples from the target task as pseudo-labels to augment the original labeled dataset). In some implementations, self-training starts with a base model (e.g., an auxiliary task model generated from task augmentation), where the base model is fine-tuned at each iteration using a concatenation of the labeled dataset and pseudo-labeled examples created from the previous training iteration. This process can be repeated for many iterations until a stopping criterion is reached. Example experiments reveal that using a strong base model and training on a wide distribution of tasks is important for successful deployment of NLP.

[0032] Furthermore, by combining task augmentation with self-training, example implementations of the present disclosure can significantly improve sample efficiency in terms of both performance and magnitude of variation, as empirically demonstrated across 12 NLP benchmark datasets. For example, on the sentiment analysis SST-2 dataset, which has only 8 training examples per class, comparable results are achieved relative to standard fine-tuning with 67K training examples.

[0033] The systems and methods of the present disclosure provide many technical effects and benefits. As an example technical effect, the systems and methods described herein can achieve improved performance with fewer labeled training examples. This means that the effort (e.g., use of computing resources) required to create additional labeled training examples is less. In addition, the technology described herein enables the model to adapt quickly to new tasks and domains. This makes it possible to fine-tune the model from a starting point rather than from scratch, which is equivalent to fewer training iterations overall (e.g., no new model needs to be generated for each task). Fewer training iterations overall corresponds to reduced use of computing resources, such as reduced processor usage, memory usage, and network bandwidth usage.

[0034] As another example improvement in computer functionality, example implementations of the present disclosure can significantly improve sample efficiency in both performance and magnitude of variation. Thus, the performance of computers including machine learning models can be improved.

[0035] Referring now to the drawings, example embodiments of the present disclosure will be discussed in greater detail.

[0036] Learning by Example

[0037] Despite their recent success in tackling many NLP tasks, large-scale pre-trained language models perform poorly in few-shot settings where only a few training examples are available. To address this shortcoming, example aspects of the present disclosure relate to STraTA, which represents self-training with task augmentation, an approach built on two key concepts for effectively utilizing unlabeled data. First, STraTA uses task augmentation, a novel technique for synthesizing large amounts of data from unlabeled text for the target task to assist in task fine-tuning. Second, STraTA performs self-training by further fine-tuning the strong base model created through task augmentation on a wide distribution of pseudo-labeled data. Example experiments included in U.S. Provisional Patent Application No. 63 / 194,474 demonstrate that STraTA is able to significantly improve sample efficiency across 12 few-shot benchmarks. Remarkably, on the SST-2 sentiment dataset, STraTA achieves results comparable to standard fine-tuning with 67K training examples with only 8 training examples per class. Our analysis reveals that task augmentation and self-training are both complementary and independently effective.

[0038] Figure 1A We provide example experimental results demonstrating that an example implementation of the proposed method, STraTA, significantly improves sample efficiency across different tasks. For example, when given only 8 labeled examples per class from the SST-2 sentiment dataset, the example implementation of STraTA is competitive with standard fine-tuning on 67K examples; on the SciTail entailment dataset, which has 512 labeled examples per class, the example implementation of STraTA outperforms standard fine-tuning on 27K examples.

[0039] Figure 1B An example of the proposed self-training with task augmentation (STraTA) method is provided. In task augmentation, a computing system can train an NLI data generation model and use it to synthesize a large amount of in-domain NLI training data for each given target task, which is then used to assist in (intermediate) fine-tuning. Some implementations of the self-training algorithm use a cascade of labeled and pseudo-labeled examples to iteratively learn a better model. At each iteration, an example implementation can start from the auxiliary task model generated by task augmentation and train on a wide distribution of pseudo-labeled data.

[0040] More generally, at a high level, task augmentation utilizes unlabeled text from the domain of a given target task to simulate a large amount of in-domain training data for the auxiliary task of natural language inference (NLI), which is then used to train a given model before applying it to the target task. To achieve this, an example implementation can first build an NLI data generator by fine-tuning a pre-trained generative language model on the MNLI dataset in a text-to-text format. Then, given a target task (e.g., sentiment analysis) with unlabeled text (e.g., his acting was really awful), the example implementation can use the NLI data generator to generate NLI examples (e.g., [his acting was really awful, he gave an incredible performance, contradiction]). Task augmentation alone can significantly improve downstream performance across different tasks, typically outperforming other fine-tuning methods, such as target task language model fine-tuning and intermediate task fine-tuning on MNLI, in both high- and low-data regimes.

[0041] After a strong auxiliary task model has been obtained using task augmentation, STraTA can use this model as a base model for self-training. Specifically, at each iteration, the base model can be fine-tuned using available labeled data for the target task. The resulting model's predictions for unlabeled examples are then used as pseudo-labels to augment the original labeled dataset. The term unlabeled text can refer to text snippets (e.g., sentences) from the target domain, and the term unlabeled examples can refer to examples that can be annotated using a class label set for the target task.

[0042] The newly formed labeled dataset can then be used to learn a better model in the next iteration, and this process can be repeated for many iterations until a stopping criterion is reached. While self-training has been extensively studied, example experiments reveal that using a strong base model and training on a wide distribution of pseudo-labeled data are key factors for successful deployment of NLP.

[0043] Using the proposed STraTA method, the example implementation is able to significantly improve sample efficiency in terms of both performance and inconsistency across 12 NLP benchmark datasets. For example, on the SST-2 sentiment dataset with only 8 training examples per class, the example implementation achieves results comparable to standard fine-tuning with 67K training examples (see Figure 1A ).

[0044] Example contributions provided by this disclosure include: proposing task augmentation, a novel data augmentation-based fine-tuning method, and demonstrating its effectiveness compared with other competing fine-tuning methods; proposing a simple but effective self-training algorithm and highlighting the important components of successful self-training, which will enable wider adoption of self-training in NLP; and using STraTA, an example implementation demonstrates the effectiveness of combining task augmentation and self-training in improving sample efficiency across NLP benchmarks.

[0045] Example Task Enhancement

[0046] Labeled data is often expensive and time-consuming to obtain, which motivates methods that learn from both labeled and unlabeled data. More formally, assume that we have a labeled dataset for our example implementation and unlabeled datasets Objectives and tasks You can get it from Remove the true value label y to artificially create unlabeled data (as in the main experiments), or the unlabeled data can be sourced from additional unlabeled text from the target domain or from related datasets / domains.

[0047] Some example implementations of the proposed method, task augmentation, and self-training using unlabeled data To make the target task The performance on is maximized, even when the number of labeled examples M is small (e.g., M = 16). In this section, example implementations first present a framework and implementation for task augmentation, which uses natural language inference (NLI) as an auxiliary (intermediate) training task to improve downstream performance.

[0048] Example framework for task augmentation: Task augmentation builds on the recent body of NLP research on intermediate task training, in which a pre-trained language model such as BERT is fine-tuned on an auxiliary task before the target task. This process is different from traditional data augmentation methods (e.g., vocabulary permutation or back-translation), which produce negligible improvements when combined with large-scale pre-trained language models. In prior work on intermediate fine-tuning, the auxiliary dataset used was a fixed dataset that was independent of the target task, such as MNLI or SQuAD. A limitation of this choice is the domain mismatch between the auxiliary task and the target task, which the task augmentation method proposed in the example addresses.

[0049] More specifically, the example implementation fine-tunes a pre-trained generative language model and uses it to train the auxiliary task from Synthesize a large amount of in-domain training data, which is then used to improve the model on the target task Performance on ( Figure 1B , left). In this work, the example implementation uses NLI as an auxiliary task for two main reasons: (1) NLI has been shown to be an effective auxiliary task for various target tasks, and (2) existing NLI datasets contain large training sets, which allows training reliable data generators.

[0050] Generating Synthetic NLI Data: To obtain the NLI data generator, the example implementation fine-tunes a pre-trained T5-3B model on MNLI, which contains 393K sentence pairs labeled {entailment, contradiction, neutral}. The T5-3B model is described in Raffel et al., Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research (JMLR 2020), 21(140):1-67.

[0051] Some example implementations convert each MNLI training example (sent A , sent B )→label is projected into text format (label, sent A )→sent B To obtain fine-tuned examples that look like [entailment, the fact can be known by you → you can know the fact]. Some example implementations fine-tune separate T5 models by class label. To overcome the bias in MNLI where hypotheses are often shorter than the premises, example implementations also include the opposite example: (opposite labeling, sent_B) → sent_A.

[0052] Some example implementations use the Adafactor optimizer to 16 = 65,536 steps with a constant learning rate of 0.001 on this dataset. The fine-tuned T5 data generator produces augmented examples for all target datasets. Specifically, at inference time, the example implementation feeds the model with NLI tags (e.g., entailment) and unlabeled sentences x from the target domain. j To generate some output sentence x k :(entailment, x j )→x k . Then you can create a j , xk )→entailment to form the data for intermediate fine-tuning. This approach has several advantages: (1) training labels are free, and (2) by using overgeneration, the example implementation can generate a large amount of in-domain NLI training data even for target tasks with small datasets.

[0053] Overgeneration and filtering: Some example implementations perform overgeneration and filtering to increase the amount and quality of synthetic NLI training data. Specifically, the example implementation generates 100 output samples per input (with duplicates removed) using the top k (k=40) samples, and uses the BERT model fine-tuned on MNLI (in its original form) as the NLI classifier to filter the synthetic training examples. Some example implementations retain the synthetic examples if the NLI classifier produces the same token as the token fed to the NLI data generator and is also confident in its prediction.

[0054] Some example implementations use examples when their predicted probability exceeds a certain threshold τ. Some example implementations assign τ a value in [0.3, 0.4, ..., 0.9] for each target task based on performance on the original MNLI development set. Intermediate fine-tuning can be performed on examples from both the original MNLI dataset and the final filtered task augmentation dataset. Some example implementations use a two-stage intermediate fine-tuning process, where the model is first trained on synthetic data and then fine-tuned on the original data. This often works well and is used in our experiments.

[0055] Example self-training

[0056] While task augmentation uses unlabeled text to generate synthetic data for an intermediate task, self-training is a complementary approach to improve a model by training it directly on the target task using pseudo-labeled examples. Some example implementations use a simple self-training algorithm where the model learns to improve itself using its predictions for unlabeled examples from a given target task. Some example implementations of the proposed method differ from traditional self-training methods in that they exploit a strong base model and allow it to learn from all available pseudo-labeled examples at each iteration, regardless of model confidence. Formally, given a target task With a small labeled dataset and unlabeled datasets Where M<<N.

[0057] Algorithm 1: Example self-training algorithm

[0058] initialization

[0059] t=0

[0060] A base model f0 is formed, which is initialized with pre-trained parameters from the pre-training / intermediate fine-tuning stage, and then a teacher model f1 is learned by training f0 on the original labeled dataset L.

[0061] repeat

[0062] t=t+1

[0063] Use the current teacher model f t To annotate (for t=1) or re-annotate (for t>1) all examples in U to obtain a pseudo-labeled example set

[0064] The entire set of pseudo-band labeled examples is added to the original labeled dataset L to form a new labeled dataset.

[0065] The student model f is learned by training the base model f0 on the current labeled dataset and optionally fine-tuning it on L t+1 The resulting student model f t+1 To be used as a teacher for the next iteration.

[0066] Until convergence or the maximum number of iterations is reached

[0067] Starting with a strong base model: An important component of a self-training algorithm is the base model f0. Successful self-training generally requires a good base model that can provide a large proportion of "correct" predictions or pseudo-labels on unlabeled examples; otherwise, errors may propagate or amplify in later stages of self-training. At each self-training iteration, some example implementations always start from the same base model f0, which is initialized with pre-trained parameters from a pre-training / intermediate fine-tuning stage (e.g., the auxiliary task training stage in task augmentation), and then fine-tunes all its parameters using available labeled and pseudo-labeled data.

[0068] Self-training on a wide distribution of pseudo-labeled data: Another important factor is the selection of pseudo-labeled examples at each self-training iteration. Traditional self-training methods usually select the current teacher model f t A small set of highly confident (e.g., predicted class label probability above a threshold) examples is added to the labeled dataset at each iteration until the pool of unlabeled data is This can be problematic because state-of-the-art language models like BERT are overconfident and poorly calibrated.

[0069] To address these issues, some implementations encourage the use of the entire pseudo-labeled example set at each self-training iteration. Add to the original labeled dataset to learn from the "natural" wide distribution of pseudo-labeled data. Removing the examples with the lowest confidence pseudo-labels may be helpful for some tasks. A development set can be used to evaluate whether this filtering is necessary, depending on availability. The example implementation also uses f t Re-annotate the original unlabeled data pool All examples in , because the example implementations expect f t Better than f t-1 .

[0070] Figures 2A to 2D An example task enhancement process according to an example implementation of the present disclosure is shown. In particular, Figure 2A , a first machine learning model is shown being trained using a labeled training dataset associated with a pre-training task. The labeled training dataset is typically out-of-distribution with respect to the downstream target task.

[0071] exist Figure 2B , the first machine learning model (e.g., from Figure 2A ) is shown as a method for generating a synthetic complement of unlabeled training examples from an unlabeled training dataset, wherein the unlabeled training dataset is in-domain with respect to a target task that is different from the pre-training task. Each synthetic complement of unlabeled training examples can be combined or associated with the unlabeled training examples to generate a synthetic training dataset.

[0072] exist Figure 2C , a second machine learning model is shown using a synthetic training dataset (e.g., from Figure 2B ) for training.

[0073] exist Figure 2D , a second machine learning model is shown being further trained on a labeled training dataset, where the labeled training dataset is in-domain with respect to the target task.

[0074] Figures 3A to 3E An example task enhancement process for natural language processing according to an example implementation of the present disclosure is shown. Figure 3A , a first machine learning generative language model is shown to be trained using a labeled natural language inference training dataset associated with a pre-training task. The labeled training dataset is typically out-of-distribution with respect to the downstream target task.

[0075] In particular, Figure 3A As shown, each training example from the labeled natural language inference set may include a first token string, a second token string, and a tag describing the relationship between the first token string and the second token string. Figure 3A In the illustrated scheme, a first machine learning generated language model processes a first string and a token to generate a predicted second string. The predicted second string can be compared with the true value second string to train the first machine learning generated language model.

[0076] exist Figure 3B , the first machine learning generates a language model (e.g., from Figure 3A ) is shown for generating a synthetic complement of unlabeled training examples from an unlabeled training dataset, wherein the unlabeled training dataset is in-domain with respect to a target task different from the pre-training task. In particular, given an unlabeled string and a supplied label, a first machine learning generative language model can generate a synthetic string that has a supplied label relationship with the first string. The synthetic string of each unlabeled string can be combined or associated with the unlabeled string to generate a synthetic training dataset. In some implementations, a plurality of different supplied labels can be provided (e.g., sequentially over different inference iterations) to generate a plurality of synthetic strings having different relationships with the unlabeled string, thereby generating a large number of synthetic training examples.

[0077] exist Figure 3C , a third machine learning language model is shown for filtering a synthetic training data set. For example, using the third machine learning model to filter a plurality of different synthetic word strings can include the following items for each pair of an unlabeled word string and a synthetic word string: processing the pair of the unlabeled word string and the synthetic word string with the third machine learning model to generate a predicted tag; and determining whether the predicted tag matches the supplied tag, wherein the supplied tag is supplied to generate the synthetic word string. In some implementations, if the predicted tag matches the supplied tag, the pair can be retained. Conversely, if the predicted tag does not match the supplied tag, the pair can be discarded.

[0078] In some implementations, if the predicted tag matches the supplied tag, further evaluation steps may be performed. For example, when the predicted tag matches the supplied tag, the computing system may determine whether a confidence value output by the third machine learning model for the predicted tag satisfies a threshold. When the confidence value output by the third machine learning model for the predicted tag satisfies the threshold, the pair of unlabeled word strings and synthetic word strings may be maintained in the synthetic training dataset. Conversely, when the confidence value output by the third machine learning model for the predicted tag does not satisfy the threshold, the pair of unlabeled word strings and synthetic word strings may be discarded from the synthetic training dataset.

[0079] exist Figure 3D , the second machine learning model is shown using a synthetic training dataset (e.g., from Figure 3B Data from Figure 3C In particular, the second machine-learned language model can process the unlabeled string and the synthesized string to generate an output. The output can be compared with the label (e.g., the supplied label or the predicted label) to train the second machine-learned language model. Figure 3E , a second machine learning model is shown as being further trained on a labeled training dataset, where the labeled training dataset is in-domain with respect to the target task.

[0080] Figures 4A to 4B A block diagram of an example self-learning method according to an example embodiment of the present disclosure is depicted. Figures 4A to 4B The operation shown.

[0081] In particular, Figure 4A The base model can be trained using a current labeled training dataset associated with the target task. The current labeled training dataset can include labeled training examples that are in-domain for the target task. The labeled training examples can be actual labels or pseudo-labels generated from a previous training iteration.

[0082] The base model can be trained using the current labeled training dataset to generate the current student model. In some implementations, the same base model (e.g., with the same parameter values) can be used as the starting point for each iteration. In some implementations, the base model can be Figure 2D or Figure 3E The second machine learning model after training is shown.

[0083] Steering Figure 4B The operation may include accessing an unlabeled training dataset associated with the target task. The unlabeled training dataset may include unlabeled training examples, wherein the unlabeled training examples are in-domain for the target task. The operation may include processing each unlabeled training data with the current student model to generate a synthetic label for each unlabeled training data. In particular, the unlabeled training examples and the synthetic label may be combined or associated to form a self-labeled training dataset.

[0084] Finally, if Figure 4B As shown, the self-learning technique may include inserting some or all of the self-labeled training dataset into the original labeled training dataset to create a current in-domain labeled training dataset for the next training iteration in a plurality of training iterations (e.g., back to Figure 4A ). After a number of training iterations (e.g., when a stopping condition is met), the technique may include outputting the current student model as the output model.

[0085] Example devices and systems

[0086] Figure 5A A block diagram of an example computing system 100 is depicted, according to an example embodiment of the present disclosure. System 100 includes a user computing device 102, a server computing system 130, and a training computing system 150 communicatively coupled via a network 180.

[0087] The user computing device 102 can be any type of computing device, such as, for example, a personal computing device (e.g., a laptop or desktop), a mobile computing device (e.g., a smartphone or tablet), a gaming console or controller, a wearable computing device, an embedded computing device, or any other type of computing device.

[0088] The user computing device 102 includes one or more processors 112 and memory 114. The one or more processors 112 can be any suitable processing device (e.g., a processor core, a microprocessor, an ASIC, an FPGA, a controller, a microcontroller, etc.) and can be a single processor or multiple processors operatively connected. The memory 114 can include one or more non-transitory computer-readable storage media, such as RAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, etc., and combinations thereof. The memory 114 can store data 116 and instructions 118, which are executed by the processor 112 to cause the user computing device 102 to perform operations.

[0089] In some implementations, the user computing device 102 may store or include one or more machine learning models 120. For example, the machine learning model 120 may be or may otherwise include various machine learning models, such as a neural network (e.g., a deep neural network) or other types of machine learning models, including nonlinear models and / or linear models. The neural network may include a feedforward neural network, a recurrent neural network (e.g., a long short-term memory recurrent neural network), a convolutional neural network, or other forms of neural networks. Some example machine learning models may utilize an attention mechanism such as self-attention. For example, some example machine learning models may include a multi-head self-attention model (e.g., a transformer model). Reference Figures 2A to 4B An example machine learning model 120 is discussed.

[0090] In some implementations, the one or more machine learning models 120 can be received from the server computing system 130 via the network 180, stored in the user computing device memory 114, and then used or otherwise implemented by the one or more processors 112. In some implementations, the user computing device 102 can implement multiple parallel instances of a single machine learning model 120 (e.g., thereby performing parallel tasks across multiple instances of the model).

[0091] Additionally or alternatively, one or more machine learning models 140 may be included in or otherwise stored and implemented by a server computing system 130 that communicates with the user computing device 102 according to a client-server relationship. For example, the machine learning models 140 may be implemented by the server computing system 140 as part of a web service (e.g., an NLP service). Thus, one or more models 120 may be stored and implemented at the user computing device 102, and / or one or more models 140 may be stored and implemented at the server computing system 130.

[0092] The user computing device 102 may also include one or more user input components 122 for receiving user input. For example, the user input component 122 may be a touch-sensitive component (e.g., a touch-sensitive display or touchpad) that is sensitive to the touch of a user input object (e.g., a finger or a stylus). The touch-sensitive component may be used to implement a virtual keyboard. Other example user input components include a microphone, a traditional keyboard, or other means by which a user can provide user input.

[0093] The server computing system 130 includes one or more processors 132 and memory 134. The one or more processors 132 can be any suitable processing device (e.g., a processor core, a microprocessor, an ASIC, an FPGA, a controller, a microcontroller, etc.) and can be a single processor or a plurality of processors operatively connected. The memory 134 can include one or more non-transitory computer-readable storage media, such as RAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, etc., and combinations thereof. The memory 134 can store data 136 and instructions 138 that are executed by the processor 132 to cause the server computing system 130 to perform operations.

[0094] In some implementations, server computing system 130 includes or is otherwise implemented by one or more server computing devices. In instances where server computing system 130 includes multiple server computing devices, such server computing devices may operate according to a sequential computing architecture, a parallel computing architecture, or some combination thereof.

[0095] As described above, the server computing system 130 may store or otherwise include one or more machine learning models 140. For example, the model 140 may be or may otherwise include various machine learning models. Example machine learning models include neural networks or other multi-layer nonlinear models. Example neural networks include feedforward neural networks, deep neural networks, recurrent neural networks, and convolutional neural networks. Some example machine learning models may utilize attention mechanisms such as self-attention. For example, some example machine learning models may include a multi-head self-attention model (e.g., a transformer model). Figures 2A to 4B An example model 140 is discussed.

[0096] User computing device 102 and / or server computing system 130 may train models 120 and / or 140 via interaction with training computing system 150 communicatively coupled via network 180. Training computing system 150 may be separate from server computing system 130 or may be part of server computing system 130.

[0097] The training computing system 150 includes one or more processors 152 and a memory 154. The one or more processors 152 can be any suitable processing device (e.g., a processor core, a microprocessor, an ASIC, an FPGA, a controller, a microcontroller, etc.) and can be a single processor or a plurality of processors operatively connected. The memory 154 can include one or more non-transitory computer-readable storage media, such as RAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, etc., and combinations thereof. The memory 154 can store data 156 and instructions 158 that are executed by the processor 152 to cause the training computing system 150 to perform operations. In some implementations, the training computing system 150 includes one or more server computing devices or is otherwise implemented by one or more server computing devices.

[0098] The training computing system 150 may include a model trainer 160 that trains the machine learning models 120 and / or 140 stored at the user computing device 102 and / or the server computing system 130 using various training or learning techniques such as, for example, backpropagation of errors. For example, a loss function may be backpropagated through the model to update one or more parameters of the model (e.g., based on the gradient of the loss function). Various loss functions may be used, such as mean squared error, likelihood loss, cross entropy loss, hinge loss, and / or various other loss functions. Gradient descent techniques may be used to iteratively update the parameters over many training iterations.

[0099] In some implementations, performing backpropagation of the error may include performing truncated backpropagation through time.The model trainer 160 may perform a number of generalization techniques (eg, weight decay, temporal backoff, etc.) to improve the generalization ability of the trained model.

[0100] In particular, model trainer 160 can train machine learning models 120 and / or 140 based on training dataset 162. In some implementations, if the user has provided consent, the training examples can be provided by user computing device 102. Thus, in such implementations, model 120 provided to user computing device 102 can be trained by training computing system 150 on user-specific data received from user computing device 102. In some instances, this process can be referred to as personalizing the model.

[0101] The model trainer 160 includes computer logic for providing the desired functionality. The model trainer 160 can be implemented in hardware, firmware, and / or software that controls a general-purpose processor. For example, in some implementations, the model trainer 160 includes a program file stored on a storage device, loaded into memory, and executed by one or more processors. In other implementations, the model trainer 160 includes one or more sets of computer-executable instructions, wherein the one or more sets of computer-executable instructions are stored in a tangible computer-readable storage medium such as RAM, a hard disk, or optical or magnetic media.

[0102] The network 180 may be any type of communication network, such as a local area network (e.g., an intranet), a wide area network (e.g., the Internet), or some combination thereof, and may include any number of wired or wireless links. In general, communications on the network 180 may be carried via any type of wired and / or wireless connection using a wide variety of communication protocols (e.g., TCP / IP, HTTP, SMTP, FTP), encodings or formats (e.g., HTML, XML), and / or protection schemes (e.g., VPN, secure HTTP, SSL).

[0103] In some implementations, the input of the machine learning model of the present disclosure may be image data. The machine learning model is capable of processing the image data to generate an output. As an example, the machine learning model is capable of processing the image data to generate an image recognition output (e.g., recognition of the image data, potential embedding of the image data, an encoded representation of the image data, a hash value of the image data, etc.). As another example, the machine learning model is capable of processing the image data to generate an image segmentation output. As another example, the machine learning model is capable of processing the image data to generate an image classification output. As another example, the machine learning model is capable of processing the image data to generate an image data modification output (e.g., a change to the image data, etc.). As another example, the machine learning model is capable of processing the image data to generate an encoded image data output (e.g., an encoded representation and / or compressed representation of the image data, etc.). As another example, the machine learning model is capable of processing the image data to generate an upscaled image data output. As another example, the machine learning model is capable of processing the image data to generate a prediction output.

[0104] In some implementations, the input of the machine learning model of the present disclosure may be text or natural language data. The machine learning model is capable of processing text or natural language data to generate an output. As an example, the machine learning model is capable of processing natural language data to generate a language encoding output. As another example, the machine learning model is capable of processing text or natural language data to generate a potential text embedding output. As another example, the machine learning model is capable of processing text or natural language data to generate a translation output. As another example, the machine learning model is capable of processing text or natural language data to generate a classification output. As another example, the machine learning model is capable of processing text or natural language data to generate a text segmentation output. As another example, the machine learning model is capable of processing text or natural language data to generate a semantic intent output. As another example, the machine learning model is capable of processing text or natural language data to generate an upscaled text or natural language output (e.g., text or natural language data of higher quality than the input text or natural language, etc.). As another example, the machine learning model is capable of processing text or natural language data to generate a prediction output.

[0105] In some implementations, the input of the machine learning model of the present disclosure may be speech data. The machine learning model is capable of processing speech data to generate an output. As an example, the machine learning model is capable of processing speech data to generate a speech recognition output. As another example, the machine learning model is capable of processing speech data to generate a speech translation output. As another example, the machine learning model is capable of processing speech data to generate a potential embedding output. As another example, the machine learning model is capable of processing speech data to generate an encoded speech output (e.g., an encoded representation and / or compressed representation of speech data, etc.). As another example, the machine learning model is capable of processing speech data to generate an upscaled speech output (e.g., speech data of a higher quality than the input speech data, etc.). As another example, the machine learning model is capable of processing speech data to generate a text representation output (e.g., a text representation of the input speech data, etc.). As another example, the machine learning model is capable of processing speech data to generate a prediction output.

[0106] In some implementations, the input to the machine learning model of the present disclosure may be latently coded data (e.g., a latent space representation of the input, etc.). The machine learning model can process the latently coded data to generate an output. As an example, the machine learning model can process the latently coded data to generate a recognition output. As another example, the machine learning model can process the latently coded data to generate a reconstruction output. As another example, the machine learning model can process the latently coded data to generate a search output. As another example, the machine learning model can process the latently coded data to generate a re-clustering output. As another example, the machine learning model can process the latently coded data to generate a prediction output.

[0107] In some implementations, the input to the machine learning model of the present disclosure may be statistical data. The statistical data may be, represent, or otherwise include data calculated and / or computed from some other data source. The machine learning model may process the statistical data to generate an output. As an example, the machine learning model may process the statistical data to generate a recognition output. As another example, the machine learning model may process the statistical data to generate a prediction output. As another example, the machine learning model may process the statistical data to generate a classification output. As another example, the machine learning model may process the statistical data to generate a segmentation output. As another example, the machine learning model may process the statistical data to generate a visualization output. As another example, the machine learning model may process the statistical data to generate a diagnostic output.

[0108] In some implementations, the input to a machine learning model of the present disclosure may be sensor data. The machine learning model can process the sensor data to generate an output. As an example, the machine learning model can process the sensor data to generate a recognition output. As another example, the machine learning model can process the sensor data to generate a prediction output. As another example, the machine learning model can process the sensor data to generate a classification output. As another example, the machine learning model can process the sensor data to generate a segmentation output. As another example, the machine learning model can process the sensor data to generate a visualization output. As another example, the machine learning model can process the sensor data to generate a diagnostic output. As another example, the machine learning model can process the sensor data to generate a detection output.

[0109] In some cases, the machine learning model can be configured to perform a task that includes encoding input data for reliable and / or efficient transmission or storage (and / or corresponding decoding). For example, the task can be an audio compression task. The input can include audio data and the output can include compressed audio data. In another example, the input includes visual data (e.g., one or more images or videos), the output includes compressed visual data, and the task is a visual data compression task. In another example, the task can include generating an embedding for the input data (e.g., input audio or video data).

[0110] In some cases, the input includes visual data and the task is a computer vision task. In some cases, the input includes pixel data from one or more images and the task is an image processing task. For example, the image processing task may be image classification, where the output is a set of scores, each score corresponding to a different object class and representing the likelihood that the one or more images depict an object belonging to that object class. The image processing task may be object detection, where the image processing output identifies one or more regions in the one or more images and, for each region, identifies the likelihood that the region depicts an object of interest. As another example, the image processing task may be image segmentation, where the image processing output defines, for each pixel in the one or more images, the corresponding likelihood of belonging to each class in a predetermined set of classes. For example, the class set may be foreground and background. As another example, the class set may be object classes. As another example, the image processing task may be depth estimation, where the image processing output defines a corresponding depth value for each pixel in the one or more images. As another example, the image processing task may be motion estimation, where the network input includes multiple images, and the image processing output defines, for each pixel in one of the input images, the motion of the scene depicted at that pixel between the images in the network input.

[0111] In some cases, the input includes audio data representing spoken utterances and the task is a speech recognition task. The output may include text output mapped to the spoken utterances. In some cases, the task includes encrypting or decrypting the input data. In some cases, the task includes a microprocessor execution task such as branch prediction or memory address translation.

[0112] Figure 5A An example computing system that can be used to implement the present disclosure is illustrated. Other computing systems may also be used. For example, in some implementations, the user computing device 102 may include a model trainer 160 and a training dataset 162. In such implementations, the model 120 may be both trained and used locally at the user computing device 102. In some such implementations, the user computing device 102 may implement the model trainer 160 to personalize the model 120 based on user-specific data.

[0113] Figure 5B Depicted is a block diagram of an example computing device 10 performing in accordance with an example embodiment of the present disclosure. Computing device 10 may be a user computing device or a server computing device.

[0114] Computing device 10 includes a number of applications (e.g., applications 1 through N). Each application includes its own machine learning library and machine learning model. For example, each application may include a machine learning model. Example applications include a text messaging application, an email application, a dictation application, a virtual keyboard application, a browser application, and the like.

[0115] like Figure 5B As illustrated, each application can communicate with many other components of the computing device, such as, for example, one or more sensors, a context manager, a device state component, and / or additional components. In some implementations, each application can use an API (e.g., a public API) to communicate with each device component. In some implementations, the API used by each application is specific to that application.

[0116] Figure 5C Depicted is a block diagram of an example computing device 50 performing in accordance with an example embodiment of the present disclosure. Computing device 50 may be a user computing device or a server computing device.

[0117] The computing device 50 includes a number of applications (e.g., applications 1 through N). Each application communicates with a central intelligence layer. Example applications include a text messaging application, an email application, a dictation application, a virtual keyboard application, a browser application, and the like. In some implementations, each application can use an API (e.g., a public API across all applications) to communicate with the central intelligence layer (and the models stored therein).

[0118] The central intelligence layer includes many machine learning models. For example, Figure 5C As shown, a corresponding machine learning model can be provided for each application and managed by the central intelligence layer. In other implementations, two or more applications can share a single machine learning model. For example, in some implementations, the central intelligence layer can provide a single model for all applications. In some implementations, the central intelligence layer is included in the operating system of the computing device 50 or is otherwise implemented by the operating system of the computing device 50.

[0119] The central intelligence layer can communicate with the central device data layer. The central device data layer can be a centralized repository for data of the computing devices 50. Figure 5C As illustrated, the central device data layer can communicate with many other components of the computing device, such as, for example, one or more sensors, a context manager, a device state component, and / or additional components. In some implementations, the central device data layer can use an API (e.g., a dedicated API) to communicate with each device component.

[0120] Additional public content

[0121] The techniques discussed herein refer to servers, databases, software applications, and other computer-based systems, as well as the actions taken and the information sent to and from such systems. The inherent flexibility of computer-based systems allows for a variety of possible configurations, combinations, and partitioning of tasks and functionality between and among components. For example, the processes discussed herein can be implemented using a single device or component or multiple devices or components working in combination. Databases and applications can be implemented on a single system or distributed across multiple systems. Distributed components can operate sequentially or in parallel.

[0122] Although the subject matter has been described in detail with respect to various specific example embodiments of the subject matter, each example is provided as an illustration of the present disclosure, not as a limitation. Those skilled in the art, after obtaining an understanding of the foregoing, can easily produce changes, variations, and equivalents of such embodiments. Therefore, the subject disclosure does not exclude the inclusion of such modifications, variations, and / or additions to the subject matter as would be readily apparent to one of ordinary skill in the art. For example, a feature illustrated or described as part of one embodiment can be used together with another embodiment to produce yet another embodiment. Therefore, the present disclosure is intended to cover such changes, variations, and equivalents.

Claims

1. A computer-implemented method for achieving improved learning with few training examples, the method comprising: Obtaining, by a computing system comprising one or more computing devices, an unlabeled training dataset associated with a target task, the unlabeled training dataset comprising a plurality of unlabeled training examples, wherein the plurality of unlabeled training examples are in-domain for the target task; accessing, by the computing system, a first machine learning model, the first machine learning model having been previously trained using a labeled training dataset associated with a pre-training task different from the target task, the labeled training dataset comprising a plurality of labeled training examples, wherein the plurality of labeled training examples are out-of-domain for the target task, wherein an input to the first machine learning model comprises at least one of: image data, video data, text or natural language data, speech data, latent coding data, statistical data, and sensor data; processing, by the computing system, each unlabeled training example with the first machine learning model to generate a synthetic complement for each unlabeled training example, the plurality of training examples and the synthetic complement forming a synthetic training dataset; and The synthetic training dataset is used by the computing system to train a second machine learning model that is different from the first machine learning model.

2. The computer-implemented method of claim 1 , wherein: The labeled training dataset includes a plurality of labeled natural language inference training examples, each labeled natural language inference training example including a first word string, a second word string, and a tag describing a relationship between the first word string and the second word string; and The first machine learning model includes a generative language model that has been trained to process the first word string and the token to predict the second word string.

3. The computer-implemented method of claim 1 , wherein: Each unlabeled training example in the unlabeled training dataset includes an unlabeled word string; as well as Processing, by the computing system, each unlabeled training example with the first machine learning model to generate a synthetic supplement for each unlabeled training example, respectively, includes processing, by the computing system, each unlabeled word string and the supplied tags to generate a synthetic word string.

4. The computer-implemented method of claim 3, wherein: Processing, by the computing system, each unlabeled word string and the supplied token to generate the composite word string includes processing, by the computing system, each unlabeled word string and a plurality of different supplied tokens to generate a plurality of different composite word strings for each unlabeled word string.

5. The computer-implemented method of claim 4 , further comprising: The computing system uses a third machine learning model to filter the plurality of different synthetic word strings.

6. The computer-implemented method of claim 5, wherein: Using the third machine learning model to filter the plurality of different synthetic word strings by the computing system includes: For each pair of untokenized word string and synthesized word string: processing, by the computing system, the pair of unlabeled word-grams and synthetic word-grams using the third machine learning model to generate predicted labels; and A determination is made by the computing system as to whether the predicted token matches a supplied token, wherein the supplied token is supplied to generate the composite word string.

7. The computer-implemented method of claim 6, wherein: Filtering, by the computing system, the plurality of different synthetic word-gram strings using the third machine learning model further includes, for each pair of an unlabeled word-gram string and a synthetic word-gram string and when the predicted tag matches the supplied tag: Determining, by the computing system, whether a confidence value output by the third machine learning model for the predicted label satisfies a threshold; When the confidence value output by the third machine learning model for the predicted label meets the threshold: maintaining, by the computing system, the pair of unlabeled word-grams and synthetic word-grams in the synthetic training dataset; as well as When the confidence value output by the third machine learning model for the predicted label does not meet the threshold: the computing system discards the pair of unlabeled word strings and synthetic word strings from the synthetic training dataset.

8. The computer-implemented method of any one of claims 1 to 7, further comprising, after training the second machine learning model using the synthetic training dataset by the computing system: The second machine learning model is trained by the computing system using a second labeled training dataset associated with the target task, the second labeled training dataset comprising a second plurality of labeled training examples, wherein the second plurality of labeled training examples are in-domain for the target task.

9. A computing system configured to perform improved learning with few training examples, the computing system comprising: one or more processors; as well as one or more non-transitory computer-readable media that collectively store instructions that, when executed by the one or more processors, cause the computing system to perform operations comprising: For each of multiple training iterations: Accessing a current labeled training dataset associated with a target task, the current labeled training dataset comprising labeled training examples, wherein the labeled training examples are in-domain for the target task; Using the current labeled training dataset to train a base model to generate a current student model, wherein the input of the base model includes at least one of the following: image data, video data, text or natural language data, speech data, latent coding data, statistical data, and sensor data; accessing an unlabeled training dataset associated with the target task, the unlabeled training dataset comprising unlabeled training examples, wherein the unlabeled training examples are in-domain for the target task; Processing each unlabeled training data with the current student model to generate a synthetic label for each unlabeled training example, the unlabeled training examples and the synthetic label forming a self-labeled training dataset; and combining some or all of the self-labeled training dataset with an original labeled training dataset to generate the current labeled training dataset for a next training iteration in the plurality of training iterations; and After the multiple training iterations, the current student model is output as the output model.

10. The computing system of claim 9, wherein: The base model includes the second machine learning model trained according to the method of any one of claims 1-8.

11. The computing system of claim 9, wherein: The same base model is used in each of the multiple training iterations.

12. The computing system of claim 9, wherein: Combining some or all of the self-labeled training dataset with the original labeled training dataset to generate the current labeled training dataset for the next training iteration includes combining all of the self-labeled training dataset with the original labeled training dataset to generate the current labeled training dataset for the next training iteration.

13. The computing system of any one of claims 9 to 12, wherein: The base model includes a base language model; and The target task includes a natural language processing task.

14. One or more non-transitory computer-readable media that collectively store a machine learning model that has been trained according to the method of any one of claims 1-8.

15. One or more non-transitory computer-readable media that collectively store a machine learning model that has been trained by executing the operations of a computing system according to any one of claims 9-13.

Images