Decoupling optimization of models during pre-training

Abstract

The disclosed concept relates to pre-training of a machine learning model. One example method involves performing a separate optimization of a first machine learning model and a second machine learning model. The first machine learning model may be optimized based at least on the first prediction, and the second machine learning model may be optimized based at least on the second prediction. The first prediction may represent a prediction of a masked value in the first sequence of values, and the second prediction may represent whether the first value is replaced by a different value predicted by the first machine learning model.

Description

Background Technology

[0001] Machine learning can be used to perform a wide range of tasks, such as natural language processing, financial analysis, and image processing. Machine learning models can be trained using several methods, such as supervised learning, semi-supervised learning, unsupervised learning, and reinforcement learning. In supervised learning, labeled training examples are used to train the model to map inputs to outputs. In unsupervised learning, the model can learn from patterns present in an unlabeled dataset. Summary of the Invention

[0002] This summary is provided to introduce the chosen concepts in a simplified form, which will be further described in the detailed description below. This summary is not intended to identify key or essential features of the claimed subject matter, nor is it intended to limit the scope of the claimed subject matter.

[0003] This specification generally relates to techniques for pre-training machine learning models. One example includes a method or technique that can be executed on a computing device. The method or technique may include obtaining a first sequence of first values, and masking one or more of the first values ​​in the first sequence to obtain a masked first sequence having one or more of the first values ​​and one or more masked values. The method or technique may also include using a first machine learning model to determine a first prediction of the one or more masked values ​​in the masked first sequence. The method or technique may also include replacing the one or more masked values ​​with the first prediction to obtain a second sequence of second values. The method or technique may also include using a second machine learning model to determine a second prediction regarding whether the second value is present in the first sequence or replaced by a different value predicted by the first machine learning model. The method or technique may also include performing a decoupling optimization of the first machine learning model and the second machine learning model, wherein the first machine learning model is optimized based at least on the first prediction, and the second machine learning model is optimized based at least on the second prediction.

[0004] Another example includes a system having a hardware processing unit and a storage resource storing computer-readable instructions. When executed by the hardware processing unit, the computer-readable instructions can cause the system to obtain a pre-trained machine learning model, which has been pre-trained to predict whether a second value in a second sequence exists in a first sequence of first values ​​or is replaced by a different value predicted by another machine learning model, wherein the pre-trained machine learning model and the other machine learning model have been separately optimized. The computer-readable instructions can also cause the system to use task-specific training data to adjust the pre-trained machine learning model for a specific task to obtain a adjusted machine learning model.

[0005] Another example includes a computer-readable storage medium storing computer-readable instructions. When executed by a processing unit, the computer-readable instructions can cause the processing unit to perform actions. The actions may include obtaining a first sequence of first values ​​and masking one or more of the first values ​​in the first sequence to obtain a masked first sequence having one or more of the first values ​​and one or more masked values. The actions may also include using a first machine learning model to determine a first prediction of the one or more masked values ​​in the masked first sequence. The actions may further include replacing the one or more masked values ​​with the first prediction to obtain a second sequence of second values. The actions may further include using a second machine learning model to determine a second prediction regarding whether the second value is present in the first sequence or replaced by a different value predicted by the first machine learning model. The actions may further include performing a decoupling optimization of the first machine learning model and the second machine learning model, wherein the first machine learning model is optimized based at least on the first prediction, and the second machine learning model is optimized based at least on the second prediction.

[0006] The examples listed above are intended to provide a quick reference to help the reader, and are not intended to define the scope of the concepts described in this article. Attached Figure Description

[0007] Detailed description of the embodiments is given with reference to the accompanying drawings. In the drawings, the leftmost numeral(s) of the reference numeral(s) identifies the drawing in which that reference numeral(s) first appears. The use of similar reference numerals in different instances in the specification and drawings may indicate similar or identical items.

[0008] Figure 1 Examples of natural language processing models that can be used as primary or secondary models are shown, consistent with some implementations of this concept.

[0009] Figure 2 An example of an alternative token detection consistent with some implementations of this concept is shown.

[0010] Figure 3 An example training workflow consistent with some implementations of this concept is shown.

[0011] Figure 4 Example methods or techniques for pre-training machine learning models that are consistent with some implementations of this concept are shown.

[0012] Figure 5 Example methods or techniques for conditioning and adopting machine learning models that are consistent with some implementations of this concept are shown.

[0013] Figure 6 Example systems consistent with some implementations of this concept are shown.

[0014] Figure 7 Examples of labeled, task-specific training data consistent with some implementations of this concept are shown.

[0015] Figure 8 and Figure 9 An example graphical user interface that can be adopted is shown, which is consistent with some implementations of this concept.

[0016] Figures 10 to 16 Experimental results obtained using some embodiments of this concept, consistent with some implementations of this concept, are shown. Detailed Implementation Machine Learning Overview

[0017] There are various types of machine learning frameworks that can be trained to perform a given task. Support vector machines, decision trees, and neural networks are just a few examples of machine learning frameworks used in a wide range of applications, such as image processing and natural language processing. Some machine learning frameworks (such as neural network execution) use layers of nodes that perform specific operations.

[0018] In a neural network, nodes are connected to each other via one or more edges. A neural network may include an input layer, an output layer, and one or more intermediate layers. Each node can process its corresponding input according to a predefined function and provide output to subsequent layers, or in some cases, to previous layers. The input to a given node can be multiplied by the corresponding weight value of the edge between that input and the node. Furthermore, nodes may have their own bias values, which are also used to produce the output. Various training procedures can be applied to learn the edge weights and / or bias values. The term "parameter" as used herein, when used without modification, refers to learnable values, such as edge weights and bias values ​​that can be learned by training machine learning models (such as neural networks).

[0019] Neural network architectures can have different layers that perform different specific functions. For example, one or more layers of nodes can work together to perform specific operations such as pooling, encoding, or convolution. For the purposes of this document, the term "layer" refers to a set of nodes that share inputs and outputs, such as those inputs and outputs to or from external sources or other layers in the network. The term "operation" refers to a function that can be performed by one or more node layers. The term "model architecture" refers to the overall architecture of a layered model, including the number of layers, the connectivity of the layers, and the types of operations performed by individual layers. The term "neural network architecture" refers to the model structure of a neural network. The terms "trained model" and / or "tuned model" refer to the model structure along with the parameters for a model structure that has been trained or tuned. Note, for example, that if two models are trained on different training data, or if there is an underlying stochastic process during training, the two trained models can share the same model structure but have different values ​​for the parameters.

[0020] There are many machine learning tasks where training data is relatively scarce. A widely used approach to train models using limited task-specific training data involves "transfer learning." In transfer learning, a model is first pre-trained on another task with a large amount of training data available, and then the model is fitted to that specific task using task-specific training data.

[0021] As used herein, the term "pre-training" refers to training a model on a set of pre-training data in a manner that allows subsequent tuning of those model parameters for use in one or more specific tasks. In some cases, pre-training can involve a self-supervised learning process on unlabeled pre-training data, where the "self-supervised" learning process involves potentially learning the structure from pre-training examples without explicit (e.g., manually provided) labels. Subsequent modifications to the model parameters obtained through pre-training are referred to herein as "tuning." Tuning can be performed using supervised learning from explicitly labeled training data for one or more tasks, and in some cases, different tasks are used for tuning instead of pre-training.

[0022] A common approach to model pre-training involves the use of an auxiliary model pre-trained alongside the main model. For example, in a natural language context, an auxiliary model (text generator) can be trained to determine predicted values ​​for masked tokens from a first sequence of tokens. These predicted values ​​can then be used to replace the masked tokens to obtain a second sequence of tokens. The main model (discriminator) can be trained to predict whether a token in the second sequence is present in the first sequence or replaced by a different value predicted by the auxiliary model. The pre-trained main model can then be applied to various natural language processing tasks, such as sentiment detection, sentence classification, etc. However, traditional methods tend to jointly optimize the auxiliary and main models (including shared parameters) during pre-training. This can degrade the performance of the main model, especially when the auxiliary model is large.

[0023] The disclosed implementations provide several mechanisms for addressing these issues. First, the disclosed implementations can perform separate optimization of the auxiliary model and the main model. Second, in some implementations, parameters are not shared between the auxiliary model and the main model. Therefore, the optimization of the main model can be performed independently, and the resulting pre-trained main model will tend to learn pre-trained parameters that allow for very accurate performance when the pre-trained main model is subsequently tuned to a specific task.

[0024] While the examples below are taken from natural language processing, the disclosed techniques can be used to train (e.g., pre-train, conditioning, etc.) models for a wide range of problem domains. For example, optimized machine learning models as described herein can be used to perform image processing, computer vision, radar, sonar, or other sensing tasks. Furthermore, the disclosed implementations can be employed in medical scenarios (e.g., regression learning for predicting drug efficacy), genetic marker analysis, protein analysis, and the like. Example Natural Language Processing Model

[0025] Figure 1An exemplary natural language processing model 100 is shown, which can be trained using the disclosed implementation. As discussed in more detail below, the natural language processing model 100 is an example of a machine learning model that can be used to perform one or more natural language processing tasks. For the purposes of this document, the term "natural language" refers to language commonly used by humans for writing or conversation.

[0026] Natural Language Processing Model 100 may receive language input 102, which may include words, tokens, sentences, phrases, or other language representations. This language input may be processed by an embedding layer 104, which includes a dictionary encoder 104 (1) and a transformer encoder 104 (2). Both the dictionary encoder and the transformer encoder operate to produce vectors representing individual words, tokens, sentences, or phrases in a vector space, where semantically and / or syntactically similar words, tokens, sentences, or phrases are relatively close to each other, and words, sentences, tokens, or phrases with lower semantic or syntactic similarity are relatively more separated. These vectors are also referred to herein as “embedded”.

[0027] The dictionary encoder 104(1) can generate a sequence of input embedding vectors 106 for each word or token in the language input 102. The input of the dictionary encoder can be a sequence of tokens of length m, X = {x1, ..., x...} m Specific tokens can be used to define the beginning of each sequence and separate individual sentences within a given sequence. A dictionary encoder maps X to a sequence of input embedding vectors, with each token corresponding to a vector. In some implementations, the input embedding vectors are constructed by summing the corresponding word, fragment, and positional embedding for each word.

[0028] The transformer encoder 104(2) can obtain contextual information for each word (e.g., via self-attention) and generate a sequence of context embedding vectors 108. Self-attention is a mechanism for calculating the similarity between tokens based on their different positions within a relevant sentence. In some implementations, the transformer encoder is a multi-layer bidirectional transformer encoder configured to map the input embedding vector 106 to the context embedding vector. As discussed in more detail below, the context embedding vector can be used as a shared representation of input phrases or sentences across different tasks. The context embedding vector represents a word or token along with the context in which each word or token appears in the underlying document, query, or other input. Note that both the dictionary encoder 104(1) and the transformer encoder 104(2) can be initialized using unsupervised techniques and subsequently updated by conditioning with labeled training data specific to the task.

[0029] As discussed in more detail below, the context embedding vector 108 can be input to the task-specific layer 110 to perform task-specific processing. The task-specific layer 110 can evaluate the context embedding vector 108 to produce task-specific output. Note that the following examples of the task-specific layer 110 are just a few examples of the types of task-specific layers that can be adopted.

[0030] In some implementations, the task-specific layer can be a sentence classification layer that uses category labels to tag sentences. As a concrete example, a sentence classification layer can predict whether an English sentence is grammatically correct. Another example classification task is determining whether the sentiment of a sentence extracted from a movie review is positive or negative. As discussed in more detail below, the output of a task-specific layer for a classification task can include a corresponding probability for each potential category.

[0031] In other implementations, the task-specific layer 110 may be a pairwise text similarity layer that performs a regression task on a pair of input sentences. This regression task may involve outputting a real-valued similarity score indicating the semantic similarity between the two sentences.

[0032] In other implementations, the task-specific layer 110 may be a pairwise text classification layer that predicts a relationship (e.g., implication) between a pair of input sentences, wherein the relationship is selected from a set of predefined labels. For example, the labels may indicate whether one input sentence has an implication relationship, a contradictory relationship, or a neutral relationship with respect to another input sentence.

[0033] In other implementations, the task-specific layer 110 may be a relevance ranking layer that outputs a relevance score that conveys the relevance of two input items (e.g., the relevance of a document to a query). This relevance score can be used in subsequent natural language processing operations to rank candidate answers according to their relevance to the query. Replaced token detection

[0034] Figure 2An example of a replaced token detection 200 that can be used for pre-training is shown. The input sequence 202 is selectively (e.g., randomly) masked to obtain a masked input sequence 204. An auxiliary model 206 predicts the values ​​of the masked tokens “the” and “ate”. The predicted sequence 208 includes the unmasked tokens of the input sequence, which have predicted values ​​for the masked tokens “the” and “chewed”, respectively. The main model 210 outputs a matching prediction 212 regarding whether each token in the predicted sequence exists in the original input sequence or is replaced by a different value predicted by the auxiliary model. Here, the matching prediction is correct; for example, the main model correctly predicts that “the” predicted by the auxiliary model exists in the original input sequence, and that “chewed” predicted by the auxiliary model replaces the actual value “ate” from the input sequence. However, in other cases, the matching prediction may be incorrect, and the parameters of the main model can be adjusted based on the error in the matching prediction. Similarly, the parameters of the auxiliary model can be adjusted based on the error in the predicted sequence. In some implementations, both the auxiliary model and the main model adopt... Figure 1 The encoder architecture shown has corresponding task-specific layers. The task-specific layers of the auxiliary model can be trained to output the predicted values ​​of the masked tokens, and the task-specific layers of the main model can be trained to predict whether the tokens in the predicted sequence are present in the original input sequence or replaced by tokens predicted by the auxiliary model. Example training workflow

[0035] Figure 3 The illustration depicts an example training workflow 300 for training a machine learning model, consistent with some implementation methods. This training workflow... Figure 1 The natural language processing model 100 shown is used as an example model to describe this training workflow, but this training workflow can be performed on many different types of machine learning models.

[0036] The training workflow 300 may include a pre-training phase 302 and a tuning phase 304. As discussed further below, the pre-training phase may employ replaced token detection to determine pre-trained parameters for one or more layers of the machine learning model, and the tuning phase may be used to adapt those pre-trained parameters to a specific task. Also described further below, in some cases, the tuning phase is applied to one or more additional layers that were not initialized during the pre-training phase.

[0037] In some implementations, the pre-training phase 302 may utilize unlabeled pre-training data 306 to perform auxiliary model optimization 308. As described above, this may involve masked token prediction by the auxiliary model using unsupervised learning from the unlabeled pre-training data. For example, the unlabeled pre-training data may provide an unlabeled corpus of documents in a given natural language. Auxiliary model optimization may involve masking tokens and training the embedding layers of the auxiliary model to predict masked tokens. Additional details can be found in Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” 2018, arXiv preprint arXiv:1810.04805. Auxiliary model optimization may involve adaptive optimization, for example, using a first instance of the Adam optimizer, which may adapt the learning rate of the auxiliary model's parameters as pre-training progresses.

[0038] The pre-training phase can also involve main model optimization.310 As mentioned earlier, main model optimization can involve token replacement detection, such as predicting whether a token in the predicted sequence is present in the original input sequence or replaced by a token predicted by the auxiliary model. Main model optimization can also involve adaptive optimization, such as using a second instance of the Adam optimizer, which can adapt the learning rate of the main model's parameters as pre-training progresses. Note that the learning rates for auxiliary model optimization and main model optimization can be different because the model is optimized separately during pre-training. Note that auxiliary model optimization and main model optimization are not necessarily performed sequentially, but can be performed together as described in more detail below.

[0039] The main model optimization 310 produces pre-trained main model parameters 312, which can be output to the model history 314. As described in more detail below, the tuning phase 304 can involve tuning the pre-trained model parameters and the initialized (e.g., randomized) parameters of the task-specific layers. Together, the pre-trained parameters of the embedding layers and the initialized parameters of the task-specific layers form the first iteration of the task-specific machine learning model. As discussed in more detail below, subsequent tuning iterations can be used to modify these initial parameters to obtain subsequent iterations of the task-specific machine learning model.

[0040] One specific tuning method is to use stochastic gradient descent to tune the pre-trained parameters and the parameters of the task-specific layer. In each training iteration or each round, the batch selector 316 selects a batch of labeled task-specific data from the labeled task-specific examples 318 and outputs the selected batch as labeled training samples 320.

[0041] Labeled training samples 320 are fed into parameter tuning process 322, where model parameters are tuned to obtain an updated model 324, which is then output to model history 314. The next tuning iteration can be performed by retrieving the previous model 326 from model history 314, and tuning iterations continue until a stopping condition is met (e.g., model convergence, reaching a threshold accuracy on the test dataset, training budget being exhausted, and / or all labeled task-specific examples 318 have been exhausted). First example method

[0042] Figure 4 An example method 400 consistent with this concept is shown. As discussed in more detail below, method 400 can be implemented on many different types of devices, such as via one or more cloud servers, via client devices such as laptops, tablets, or smartphones, or via a combination of one or more servers and client devices.

[0043] Method 400 begins at box 402, where a first sequence of first values ​​is obtained. In a natural language implementation, the first sequence of first values ​​may be a sequence of tokens from text, such as extracted corpus of pre-trained data from human-written documents.

[0044] Method 400 continues at box 404, where one or more first values ​​in the first sequence are masked. For example, in a natural language implementation, one or more tokens may be masked and replaced using the [masked] tokens. In some cases, the tokens are masked randomly.

[0045] At box 406, the first machine learning model determines a first prediction of the masked value. For example, the first machine learning model could be a generator that outputs a probability distribution of the masked tokens based on surrounding tokens (e.g., previous tokens and / or subsequent tokens).

[0046] Method 400 continues at box 408, where the masked values ​​are replaced using the first prediction to obtain a second sequence of second values. For example, the probability distribution output by the generator can be randomly sampled to select a specific token, which is used to replace a given masked specific token. The remaining tokens in the second sequence can be the same tokens in the same positions as in the first sequence.

[0047] Method 400 continues at box 410, where the second machine learning model determines a second prediction regarding whether the second value is present in the first sequence or replaced by a different value predicted by the first machine learning model. For example, the second machine learning model could be a discriminator that predicts whether a given token is initially present in the same position in the first sequence from the text corpus or replaced by a different token output by the text generator.

[0048] Method 400 continues at box 412, where separate optimization of the first and second machine learning models is performed. For example, in some cases, a first instance of Adam optimization (or another adaptive optimizer) based on a first prediction is applied to the generator, and a second instance of Adam optimization (or another adaptive optimizer) based on a second prediction is applied to the discriminator. In some cases, both the generator and the discriminator include encoders that do not share parameters with each other (e.g., do not share token embeddings).

[0049] Method 400 continues at box 414, where the pre-trained model is output. For example, the pre-trained model could be a second model with parameters (weights, biases, embeddings, etc.) learned via the separation optimization at box 412. As described above, the pre-trained model can include at least an encoder for a discriminator. For example, the pre-trained model can be output to permanent memory, output from one computational module in memory to another, or sent via a network to another device for conditioning, etc. Second example method

[0050] Figure 5 The figure illustrates an example method 500 consistent with this concept. As discussed further below, method 500 can be implemented on many different types of devices, such as via one or more cloud servers, via client devices such as laptops, tablets, or smartphones, or via a combination of one or more servers and client devices.

[0051] Method 500 begins at box 502, where a pre-trained model is obtained. For example, the pre-trained model can be a main model with parameters (weights, biases, embeddings, etc.) that are learned via segregated optimization with an auxiliary model as described above. As mentioned above, the pre-trained model can include an encoder from a pre-trained discriminator. For example, the pre-trained model can be obtained from permanent storage, from memory, or received from another device via a network.

[0052] Method 500 continues at box 504, where the pre-trained model is tuned for a specific task. For example, one or more task-specific layers can be added to the pre-trained encoder of the discriminator, and the entire model can be tuned using labeled training data for the specific task, resulting in a tuned machine learning model for that task. Thus, tuning can involve adjusting both the pre-trained parameters of the encoder and the parameters of one or more task-specific layers.

[0053] Method 500 continues at block 506, where a regulated task-specific model is output. For example, the regulated task-specific model may be output to permanent storage, from one computing module in storage to another computing module, or sent over a network to another device for performing a specific task, etc.

[0054] Method 500 continues at block 508, wherein the task is performed using a tuned task-specific model. For example, input data may be provided to the tuned task-specific model, which may process the input data to output a prediction. In some implementations, the input data may include a query, and the tuned task-specific model may predict the intent of the query result. The predicted intent may be used to determine the query result, which may be sent in a response to the query. Example System

[0055] This implementation can be executed in various scenarios and on various devices. As discussed in more detail below, Figure 6 An example system 600 in which this embodiment can be adopted is shown.

[0056] like Figure 6 As shown, system 600 includes client device 610, server 620, server 630, and client device 640 connected via one or more networks 650. Note that client devices can be mobile devices such as smartphones or tablets, or fixed devices such as desktop computers, server devices, etc. Similarly, servers can be implemented using various types of computing devices. In some cases, Figure 6 Any of the devices shown, especially servers, can be implemented in data centers, server clusters, etc.

[0057] Figure 6Certain components of the devices shown may be referred to herein by reference numerals in parentheses. For the purposes described below, parentheses (1) indicate the presence of a given component on client device 610, (2) indicate the presence of a given component on server 620, (3) indicate the presence on server 630, and (4) indicate the presence on client device 640. Parentheses are generally not used to refer to components in this document unless a specific instance of a given component is identified.

[0058] Typically, devices 610, 620, 630, and / or 640 may have corresponding processing resources 601 and storage resources 602, which will be discussed in more detail below. The devices may also have various modules that utilize the processing and storage resources to perform the techniques discussed herein. Storage resources may include persistent storage resources, such as magnetic drives or solid-state drives, and volatile memory, such as one or more random access memory devices. In some cases, modules are provided as executable instructions that are stored on a persistent storage device, loaded into a random access memory device, and read from the random access memory by the processing resources for execution.

[0059] The client device 610 may include a configuration module 611 that can interact with the model pre-training module 621 on the server 620. Generally, the configuration module can provide certain configuration parameters to the model pre-training module. These configuration parameters may include architecture parameters and pre-training parameters (e.g., hyperparameters). Architecture parameters can specify the structure of the machine learning model, such as the number of nodes, the arrangement of nodes in layers, and the connections between nodes and / or layers. Architecture parameters can also specify input and output data for each node and / or layer.

[0060] Configuration parameters may also include pre-training parameters that specify information such as learning rate or other hyperparameters (e.g., learning rate), unsupervised learning parameters or self-supervised learning parameters and / or data source, and / or supervised learning parameters and / or data source. The model pre-training module 621 uses these training configuration parameters to perform model pre-training functions on the model specified by the architecture parameters. Specifically, the model training module may execute method 400 based on the pre-training configuration parameters. As just one example, the unsupervised learning data source may include a repository of one or more sentences.

[0061] The model pre-training module 621 can output a pre-trained model to the server 630. The model tuning module 631 can tune the pre-trained model (e.g., using a supervised learning data source such as task-specific labeled training examples, e.g., by executing boxes 502, 504, and 506 of method 500). The model execution module 632 can execute the final model in response to received input, e.g., executing box 508 of method 500. For example, the interface module 641 on the client device 640 can provide input data to the model execution module for evaluating any task provided in the task-specific layer. The model execution module can use the final model to process the uploaded input data and provide model output as a response by sending the model output to the client device 640 via network(s) 650. Example labeled training data

[0062] Figure 7 An example of labeled sentiment data 700 is shown, which can be used to train a task-specific layer to perform sentence classification for positive or negative sentiment. As mentioned earlier, labeled sentiment data can be provided in task-specific example 318 for training workflow 300.

[0063] The labeled sentiment data includes positive example 710 and negative example 720. Each positive example 710 includes a label indicating that a human or automated agent has manually labeled the sentence as indicating positive sentiment, and each negative example 720 includes a label indicating that a human or automated agent has manually labeled the sentence as indicating negative sentiment. Output Example

[0064] Figure 8 Example review page 800 for laptop 802 is shown. Here, the user has selected to filter reviews for negative comments via dropdown menu 804, and in response, the review page displays three reviews 806, 808, and 810. Reviews 806 and 810 are clearly negative reviews and are therefore appropriate to show to the user in response to their request for negative reviews. However, review 808 is a very positive review of the laptop. A discriminator-based model pre-trained using the coupled optimization with the generator as described above may misclassify this review as negative because the embeddings learned during pre-training may not accurately reflect the meaning of terms such as "stupendous" and "discriminating."

[0065] Figure 9An example review page 900 for a laptop 802, generated using the disclosed techniques, is shown. Here, review 808 has been replaced by review 908, which is actually a negative review and is therefore correctly displayed to users who wish to see negative reviews of the laptop. Because the disclosed implementation can learn discriminator embeddings that more accurately reflect the meaning of certain items (e.g., “amazing” and “unique” in this example), the sentiment of reviews with these items can be predicted more accurately from a task-specific model tuned to the discriminator. Specific algorithms and experimental results

[0066] The following sections describe the specific algorithms used for decoupling optimization of the model and also present experimental results obtained by adjusting a model pre-trained using decoupling optimization. In the discussion below, MLNI refers to Multi-genre Natural Language Inference, a dataset of labeled sentence pairs with textual entailment relations. QQP refers to Quora Question Pairs, a dataset of labeled question pairs indicating whether questions are semantically repetitive. RTE refers to Recognizing Textual Entailment, a dataset of labeled sentences indicating whether the meaning of one sentence can be inferred from another. QNLI refers to a question-answering dataset containing questions and their corresponding answers, which are identified in the provided text segments. MPRC (Microsoft Research Paraphrase Corpus) refers to the Microsoft Research Paraphrase Corpus, a dataset of labeled sentences with paraphrasing relationships indicating whether one sentence accurately paraphrases another. CoLA refers to the Corpus of Linguistic Acceptability, a dataset of labeled sentences focused on grammatical acceptability. SST refers to the Stanford Sentiment Treebank, a labeled dataset containing examples expressing positive or negative emotions. STS-B refers to the Semantic Textual Similarity Benchmark, a dataset with labels indicating the similarity between two sentences. Coupling optimization

[0067] While pre-training using coupled optimization can be effective, the performance of the resulting model depends on careful selection of the model size for the auxiliary generator, leading to high trial-and-error costs. The importance of controlling generator capacity during coupled optimization training is highlighted below. Adaptive optimizers like Adam can cause the weighting of individual losses to become ineffective in coupled optimization, and therefore coupled optimization may fail to effectively control generator training. To regain control of the generator, the disclosed implementation decouples the generator optimizer instance and the discriminator optimizer instance, rather than relying on a weighted combination of objectives for the generator and discriminator. Compared to the original design, this technique significantly reduces the sensitivity of coupled optimization pre-training and achieves considerable performance gains.

[0068] An example of coupling optimization is ELECTRA-style pre-training, such as Clark et al. (2020), Pre-training text encoders as discriminators rather than generators, preprinted in https: / / arxiv.org / abs / 2003.10555. ELECTRA has shown significant potential in enhancing the effectiveness and efficiency of training large language models (LLMs). ELECTRA trains a discriminator model (the main model used in downstream tasks) to detect which tokens in the input sequence are replaced by a generator model (an auxiliary model not used in downstream tasks).

[0069] Despite its effectiveness, ELECTRA's performance is highly sensitive to the choice of generator size. For example... Figure 10 As depicted in Figure 1000 of MNLI via generator depth, variations in generator size can lead to a significant performance degradation when fine-tuning the discriminator in downstream tasks. This sensitivity can be addressed by carefully selecting the generator size in practical applications, but this can be time-consuming and extremely resource-intensive.

[0070] The following evaluation of the discriminator's ability to detect replaced tokens confirms that a large generator capacity may impair the effectiveness of pre-training. This evaluation also confirms that this performance degradation occurs during the pre-training phase, rather than during the fine-tuning phase. Coupled optimization techniques such as ELETRCA may not be able to effectively control the generator capacity during pre-training by relying on a weight ratio that combines the training objectives of the generator and discriminator to predict a balanced optimization of both. However, this approach has been largely observed to be ineffective because constant scaling of the loss does not affect the adaptive optimizer, as seen in Kingma et al., (2014), Adam: A method for stochastic optimization, arXiv preprint arXiv:1412.6980.

[0071] To regain control over generator training, the disclosed implementation performs generator optimization separately from the discriminator optimizer. This technique, hereinafter referred to as Decoupled Optim, effectively mitigates the generator's sensitivity to ELECTRA-style pre-training and recovers the performance loss caused by large generators. Furthermore, the disclosed technique enhances the flexibility of accelerating discriminator optimization without being hindered by the instability of generator training, thereby delivering significant performance gains on strong baselines. The following experiments utilize standard BERT. base and BERT large (Devlin et al., (2018), Bert: Pre-training of deep bidirectional transformers for language understanding, arXiv preprint arXiv: 1810.04805) conducted a pre-training setup on GLUE (Wang et al., (2018), GLUE: A multi-task benchmark and analysis platform for natural language understanding, arXiv preprint arXiv: 1804.07461), and the disclosed technique consistently outperformed traditional pre-training techniques. Language modeling pre-training with masking

[0072] Masked Language Modeling (MLM). MLM methods (such as BERT) pre-train language models to predict randomly masked tokens in a sequence. Specifically, given an input sequence... MLM through location Randomly select several tokens and replace them with [mask] tokens to generate a masked sequence. The model is then trained to predict a given masked sequence. The original token. The training objective can be expressed by the formula: in Represents model parameters, and Given a masked sequence Below, the model is in the token The first The predicted probability at each location. Replaced token detection pre-training

[0073] Unlike MLM, ELECTRA constructs a pre-trained task called Replaced Token Detection (RTD), which involves two deep neural models (generator and generator). (Auxiliary model) and discriminator Joint training of the (main model). Here, the generator is typically pre-trained using MLM, while the discriminator is pre-trained to detect tokens in the sequence replaced by the generator.

[0074] Specifically, given a masked sequence constructed for MLM By Each [mask] token in the [mask] is replaced with [from] The generator at the token position predicts the distribution of the sampled tokens to generate a corrupted sequence. Right now and The sampled tokens will be referred to as replaced tokens hereafter, because these tokens will differ from the original tokens at the corresponding positions should the generator fail to correctly predict the masked tokens using the one-hot probability distribution. The discriminator is then trained to predict... The training objective can be defined as follows: Does the replaced token match the original token? in Quantizing the first through a discriminator The output scalar score is the probability that a token is replaced, where l is the loss function, which is typically binary cross-entropy (BCE). It is an indicator function, that is: Note that in ELECTRA, the training objective of the discriminator is defined over all input tokens, not over a subset that is randomly masked (as is the case in MLM). Impact of generator capacity

[0075] In ELECTRA-style pre-training, optimal discriminator performance can only be achieved by a generator that is neither too large nor too small. For example... Figure 10 As shown, generators with more layers consistently impair discriminator performance for downstream tasks. This is partly because large generator capacities result in slower pre-training of the discriminator. As discussed in more detail below,

[0076] The following experiments evaluate the performance of pre-trained models in a two-stage setup (i.e., pre-training and fine-tuning) to determine whether performance degradation occurs during the pre-training phase (i.e., the discriminator is not properly trained) or the fine-tuning phase (i.e., the discriminator is not properly fine-tuned). The results show that performance degradation occurs during the pre-training phase. For example, as... Figure 11 As shown in Figure 1100, the discriminator trained with a deeper generator consistently achieves worse RTD performance, which is related to... Figure 10 This corresponds to their poor performance on downstream tasks.

[0077] It has been observed that, to achieve better RTD performance for deep generators, training the discriminator on a shallow generator may be more effective than training it on the deep generator itself. For example... Figure 11 As shown, the discriminator trained with a -1 layer or -2 layer generator performs the worst on replaced tokens sampled from the very same generator compared to other discriminators trained with -1 or -2 layer generators. This observation implies that discriminators trained with deep generators are not fully optimized for their pre-training objectives, and the performance degradation is likely due to slow convergence during the pre-training process.

[0078] As mentioned above, controlling the generator capacity is important for optimizing ELECTRA. However, the coupled optimization involved in ELECTRA may have shortcomings in controlling the generator capacity. In ELECTRA, loss weights are initially introduced... To balance generator and discriminator optimization. Specifically, the generator and discriminator are jointly optimized using the following combined training loss: However, change The value of may not work as expected. First, constant scaling of the loss will not affect adaptive optimizers such as Adam, which can be used in pre-trained algorithms such as ELECTRA to ensure training stability. Specifically, Adam updates the model parameters by the ratio between the first and second impulses of the gradient of the model parameters, i.e.: in It is the learning rate, and Model parameters Compared to The gradient.

[0079] Therefore, for all generator parameters not shared with the discriminator (denoted as...) ) and all discriminator parameters not shared with the generator It is important to note the loss weights. This does not affect the update rule for these parameters, because any constant scaling of the gradient will be canceled out in equation (1). Therefore, these parameters will be trained using the same learning rate, regardless of... What is the value of ?

[0080] Loss weights in ELECTRA The only parameter affecting this is the embedding shared between the generator and the discriminator. The gradient will be: This means the update rule will become: Therefore, if If the value is large, then the updates of these embeddings will be contributed more by gradients from the discriminator.

[0081] Due to loss weights It cannot balance the updates of most model parameters, therefore it cannot effectively control generator learning. For example... Figure 12 Figure 1200 shows the accuracy of the generator after coupling pre-training, with increased loss weights. This has a minimal impact on the generator performance in the original ELECTRA design. Another way to control generator learning is to change the learning rate. However, this will also change the learning rate for the discriminator, ultimately leading to worse pre-training performance. Another approach is to reduce the generator model size, similarly... Figure 12 As shown. While effective, this approach introduces a dependence on careful selection of the generator size for pre-training performance, which can be time-consuming and extremely resource-intensive in practice. Decoupling optimization

[0082] To control generator capacity, the disclosed implementation separates the generator optimizer and the discriminator optimizer. Specifically, the generator parameters and discriminator parameters are now updated using rules based on this separation, i.e.: To control generator capacity, the disclosed implementation adjusts the optimizer instance for the generator (e.g., adjusts...). (Generator learning rate). Figure 13Figure 1300 shows the accuracy of the decoupled pre-trained generator, illustrating the reduction for large generators. The generator capacity can be effectively controlled during pre-training.

[0083] This means that decoupling optimization can handle large generators and reduce the sensitivity of ELECTRA-style pre-training to generator size selection. As further described below, this method outperforms the original ELECTRA design and its variants.

[0084] In some implementations, unlike ELECTRA which shares embeddings, the embeddings are shared between the generator and the discriminator. The following results show that, utilizing decoupling optimization, the discriminator learned from randomly initialized embeddings performs exceptionally well.

[0085] Furthermore, utilizing a decoupled optimizer design not only makes it easier to control the generator capacity but also allows for better pre-training performance by increasing the discriminator learning rate. In the original ELECTRA design, the same learning rate is allocated to both the generator and the discriminator. Therefore, attempts to accelerate discriminator optimization by increasing the learning rate result in a larger generator capacity, leading to worse pre-training performance. Moreover, an increased learning rate can even cause training failure, as MLM-style generator training can become unstable with larger learning rates. The original ELECTRA design diverges within K training steps, even with conservative choices of loss weights (e.g., ) and generator size (e.g., layers).

[0086] However, by using an optimizer that decouples the optimization, the discriminator optimization can be accelerated without being hindered by the generator learning. Figure 14 The empirical experiments are shown in Figures 1410, 1420, and 1430, illustrating the decoupled MNLI accuracy. The pre-training performance tends to improve as the discriminator's learning rate increases. Note that the discriminator learning rate can typically be correlated with... -3 It is also large, being several times the learning rate suitable for generator training. Furthermore, it was observed that with improved discriminator optimization, the best generator shifts to a generator with a larger capacity, even as large as the discriminator itself (layer). analyze

[0087] Consider a simplified RTD task where only one token in the input sequence is replaced; the remaining unchanged tokens in that sequence are called the context. Let... Let be a word in the sentence, and let... These are the remaining context words in the same sentence. Given the context, the generator is trained to predict the original token, i.e.: For discriminator training, consider the detection of this single replaced token. Discriminator The optimization objective can therefore be described as: Ideally, the discriminator optimization objective should be aligned with the discriminator performance for downstream tasks. However, the discriminator performance evaluated for a given distribution of replaced tokens may not always be indicative of downstream performance. Ideally, the discriminator should be able to detect any possible token that has been replaced in the sequence, regardless of the specific distribution from which this replaced token is sampled. To this end, the ideal optimization objective of the discriminator is defined as the highest possible discriminator loss achievable by any distribution of replaced tokens (i.e., the probability distribution from which the replaced tokens are sampled).

[0088] Definition 1 (The ideal optimization objective of the discriminator) Let It is a series of replaced token distributions. The ideal optimization objective of the discriminator is... It can be defined as:

[0089] In practice, such an ideal optimization objective may be difficult to use as a loss function for training the discriminator, since enumerating all possible distributions of replaced tokens is not necessarily feasible. However, by making an assumption about the “difficulty” of the replaced token distributions, the highest discriminator loss can be constrained over all possible distributions of replaced tokens. This upper bound can be further approximated by the discriminator loss over the generator’s predicted distribution (as the replaced token distribution). Therefore, the optimization objective defined by the generator (i.e., equation (3)) can serve as a reasonable alternative to the ideal objective.

[0090] Suppose that if the sampled, replaced tokens more frequently revert to the original tokens, the discriminator is more likely to make a detection error, i.e.: in It depends on the discriminator. The monotonically increasing concave function. Note that the ideal discriminator objective is constrained by the alternative objective defined by the generator. The intuitive understanding here is that, given Hypothesis 1, the generator's predicted distribution should approximate the most difficult alternative token distribution, and this approximation error is precisely constrained by the generator's performance.

[0091] Lemma 1 (The discriminator objective defined by the generator is a substitute for the ideal objective). Let For any context-token pair, the upper bound of the discriminator loss is:

[0092] Lemma 1 implies that the direction is towards the generator Training with the defined substitution target does indeed optimize the ideal discriminator target, proving the validity of the basic ELECTRA design of using a generator to sample the substituted tokens for discriminator training. However, a well-performing generator may also be less effective at optimizing the ideal target. This is because a well-performing generator will approach the most difficult distribution of substituted tokens based on Hypothesis 1, resulting in a significantly higher discriminator loss in Equation (6). As proven by Lemma 2 (the dependence of the discriminator loss on the generator performance). Let The variance of the generator loss is given by: Lemma 2 shows that the discriminator loss of a given generator is inversely correlated with the generator loss. This means that a strong generator can produce a significantly higher discriminator loss. If this high discriminator loss cannot be sufficiently reduced through the optimization process (which is possible since the training budget is always limited), then the ideal objective cannot be adequately optimized.

[0093] To further illustrate the effects of generator optimization and discriminator optimization, a simplified model of the optimization process is considered, based on trajectory analysis of gradient descent for deep linear neural networks.

[0094] Proposition 1 (Gradient Descent Trajectory of Deep Linear Neural Networks (Informal)). In gradient descent, let... for The updated model parameters. To satisfy certain regular learning rates, and Given the number of layers in the model, we get: in It is a normal number.

[0095] Along with Lemmas 1 and 2, the following holds. Theorem 1 (Optimization of the Ideal Discriminator Objective). Consider the optimization of the discriminator after training the generator with several updates. Let... The learning rate of the generator, and The generator depth. The generator loss after the next update is: in Subsequently, It is the learning rate of the discriminator. It's the discriminator depth. Then... After the discriminator is updated: Note that for simplicity, some constants in (7) have been omitted.

[0096] Therefore, in terms of generator optimization, for a given discriminator setting, increasing the generator learning rate initially improves discriminator performance but then degrades it, and similarly... Figure 15 The performance of the generator and discriminator learning rates is shown in Figure 1500. This yields the range in which the generator learning rate is optimal for discriminator performance. Similarly, due to Increasing the generator depth will initially help the discriminator performance but then degrade it.

[0097] In discriminator optimization, increasing the discriminator learning rate or depth almost always helps improve discriminator performance. Ideally, if the discriminator optimization is sufficient (e.g., by making in equation (10)...), or If the discriminator performance increases monotonically with the generator performance, then the discriminator performance will increase monotonically. Figure 15 As shown, the discriminator learning rate is sufficiently large. Unfortunately, given training instability and / or a limited training budget, it is not always possible to fully optimize the discriminator in practice.

[0098] Interestingly, equation (10) shows that as the discriminator becomes more fully optimized, the optimal generator should be moved to a generator with a larger capacity, as... Figure 15 As shown. This is consistent with... Figure 14 This corresponds to the empirical observations in the text.

[0099] Finally, equation (10) also reflects the limitations of the original ELECTRA design mentioned in Section 3.2. Furthermore... Figure 15 As shown, since the generator and discriminator are assigned almost the same learning rate, the original design can only reach the line (-D subspace) across the entire optimization space. Furthermore, this line is truncated due to the generator's training instability, where the discriminator's learning rate remains small, and therefore the discriminator's performance remains suboptimal. In contrast, decoupling optimization can increase the discriminator's learning rate without being affected by the generator's learning, thus achieving better performance by exploring the entire optimization space. experiment

[0100] The experiments were conducted using two standard settings, basic and large, following BERT and Meng et al. (2021), Coco-lm: Coco-lm: Correcting and contrasting text sequences for language model pretraining, Advances in Neural Information Processing Systems, 34, 23102-23114, and Bajaj et al. (2022), Metro: Efficient denoising pretraining of large scale autoencoding language models with model generated signals, arXiv preprint arXiv: 2204.06644. Specifically, Wikipedia and BookCorpus, Zhu et al. (2015), Aligning book and movies: Towards story-like visual explanations by watching movies and readingbook, In Proceedings of the IEEE International conference on computer vision (pp. 19-27), (16GB of text, 256M samples) were used for pretraining with a sequence length of 512. A case-sensitive sentence piece BPE vocabulary of K tokens was used because the larger vocabulary size improved LLM without significant additional training and inference costs.

[0101] Pre-training was performed using a batch size of 2048, with updates of 125K. For decoupling optimization, the same hyperparameter combination was used in both the basic and large settings, i.e., the generator learning rate was set to... -4 And the discriminator learning rate is set to -3 The primary model (discriminator) in the basic settings follows BERT. baseThe architecture is a 12-layer transformer with 768 hidden dimensions plus 32-bit T5 relative positional encoding. Admin (adaptive model initialization) is used for model initialization to stabilize training. See Liu et al., (2020), Understanding the difficulty of training transformers, arXiv preprint arXiv:2004.08249 and Liu et al., (2021), Multi-head or single-head? An empirical comparison for transformer training, arXiv preprint arXiv:2106.09650. The main model in the large settings follows BERT. Large This refers to a 24-layer transformer with 1024 hidden dimensions and 128 relative position encoded binary bits. The auxiliary model (generator) in the base has the same architecture as the main model.

[0102] Following the settings in Meng et al., (2021), Coco-lm: Correcting and contrasting text sequence for language model pretraining, Advances in Neural Information Processing Systems, 34, 23102-23114; and Bajaj et al., (2022), Metro: Efficientdenoising pretraining of large autoencoding language model with model-generated signal, arXiv preprint arXiv: 2204.06644, downstream task evaluation was performed. Specifically, the tuned model was evaluated on GLUE (Wang et al., (2018), GLUE: A multi-task benchmark and analysis platform for natural language understanding, arXiv preprint arXiv: 1804.07461), a language understanding benchmark, where the single-task, single-model fine-tuning settings followed previous work. Hyperparameters were trained using optimizers such as AdaMax (Kingma et al., (2014), Adam: A method for stochastic optimization, arXiv preprint arXiv:1412.6980), from Liu et al., (2019), Multi-task deep neural networks for natural language understanding, arXiv preprint arXiv:1901.11504 and Liu et al., (2020), The Microsoft toolkit of multi-task deep neural networks for natural language understanding, arXiv preprint arXiv:2002.07972.

[0103] The following reports the Spearman correlation for STS-B, the Matthews correlation for CoLA, and the accuracy for the remaining datasets:

[0104] Table 1: Downstream evaluation results for decoupling optimization and competing baselines are presented under basic and large settings. Decoupling optimization outperforms previous state-of-the-art methods in both overall GLUE scores and specific results on large datasets, and is considered more reliable.

[0105] Experiments were conducted using various pre-trained hyperparameters in the basic setup to verify the robustness of decoupling optimization relative to changes in generator capacity. For example... Figure 16 As shown in Figure 1600, the decoupling optimization yields more stable downstream performance as generator capacity / hyperparameters change. Furthermore, decoupling optimization consistently achieves better performance when using generators with larger sizes.

[0106] In each pre-training step, decoupling optimization does not introduce additional model parameters or computational costs compared to the original ELECTRA design. Note that the separate generator and discriminator embeddings in decoupling optimization involve the same amount of operations as the shared embeddings in the original ELECTRA design, because the gradients of the embeddings must be backpropagated from the generator loss and discriminator loss separately in both scenarios.

[0107] As mentioned above, using a large auxiliary generator typically leads to a decline in downstream performance of the main discriminator model. This performance degradation is due to insufficient control over the generator capacity during pre-training, highlighting long-standing neglects in ELECTRA-style training. The results above demonstrate that decoupling optimization significantly improves training robustness and downstream performance. Additional details

[0108] As mentioned, the machine learning model employed for decoupling optimization can be a neural network. In a natural language processing scenario, one or more mapping layers can perform certain encoding functions. For example, one or more mapping layers can include a dictionary encoder that maps a sequence of tokens to a sequence of input embedding vectors. One or more mapping layers can also include a transformer encoder that maps input embedding vectors to a sequence of context embedding vectors.

[0109] In general, the techniques described above can be used to pre-train an auxiliary (e.g., discriminator) model, which can then be tuned to perform one or more tasks. In natural language processing scenarios, such tuned models can be employed in search engines, for example, to match queries with documents or determine the sentiment of a given query. As another example, tuned models can be used in digital assistants or robots to rank candidate responses to user utterances. In the context of image processing, the tuned model can have task-specific layers (indicating the presence of certain categories of objects in a given image, semantically labeled images, etc.) and one or more shared convolutional layers. Note that one or more task-specific layers can then be ported to the discriminator's encoder for further tuning. Technical effect

[0110] As mentioned above, pre-training of machine learning models can involve the use of an auxiliary model alongside the main model. However, conventional approaches to this tend to (1) share parameters between the main and auxiliary models during pre-training, and (2) jointly optimize the auxiliary and main models together. Therefore, it is difficult to control the capacity of the auxiliary model during pre-training. Furthermore, as mentioned above, an auxiliary model with a very large capacity can negatively impact the performance of the main model.

[0111] By separating the optimization of the auxiliary model and the main model during pre-training, the disclosed technique allows for accelerated optimization of the main model. By increasing the learning rate of the main model, it can learn pre-trained parameters that allow for accurate tuning at a later time. Therefore, the resulting final tuned model can exhibit higher accuracy than when using joint optimization techniques.

[0112] Furthermore, since the main model can employ a higher learning rate, fewer pre-training examples can be used to pre-train the model. Therefore, processing and / or storage resources that might otherwise be used for pre-training with more samples are preserved. Additionally, pre-trained parameters can be tuned for faster convergence, and thus, satisfactory accuracy can be achieved using relatively fewer labeled training examples during tuning. Therefore, processing and / or storage resources can also be preserved during tuning. Equipment implementation

[0113] As mentioned above Figure 6 The system 600 includes several devices, including client device 610, server 620, server 630, and client device 640. It should also be noted that not all device implementations are shown, and other device implementations should be obvious to those skilled in the art from the above and below description.

[0114] As used herein, the terms “device,” “computer,” “computing device,” “client device,” and / or “server device” can refer to any type of device having a certain amount of hardware processing power and / or hardware storage / retention capacity. Processing power can be provided by one or more hardware processors (e.g., hardware processing units / cores) that can execute data in the form of computer-readable instructions to provide functionality. Computer-readable instructions and / or data can be stored on memory, such as memory / retention devices and / or data storage. The term “system” as used herein can refer to a single device, multiple devices, etc.

[0115] Storage resources can be internal or external to the corresponding device associated with them. Storage resources can include any one or more of volatile or non-volatile memory, hard disk drives, flash memory devices, and / or optical storage devices (e.g., CDs, DVDs, etc.). As used herein, the term "computer-readable medium" can include signals. Conversely, the term "computer-readable storage medium" does not include signals. Computer-readable storage media includes "computer-readable storage devices." Examples of computer-readable storage devices include volatile storage media such as RAM and non-volatile storage media such as hard disk drives, optical disks, and flash memory.

[0116] In some cases, the device is configured with general-purpose hardware processors and storage resources. In others, the device may include a System-on-a-Chip (SOC) type design. In an SOC design implementation, the functionality provided by the device can be integrated onto a single SOC or multiple coupled SOCs. One or more associated processors may be configured to coordinate with shared resources (such as memory, storage, etc.) and / or one or more dedicated resources (such as hardware blocks configured to perform certain specific functions). Therefore, the terms “processor,” “hardware processor,” or “hardware processing unit” as used herein may also refer to a central processing unit (CPU), a graphics processing unit (GPU), a controller, a microcontroller, a processor core, or other types of processing devices suitable for implementation in both traditional computing architectures and SOC designs.

[0117] Alternatively or additionally, the functions described herein may be performed at least in part by one or more hardware logic components. For example, but not limited to, illustrative types of hardware logic components that may be used include field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-chips (SoCs), complex programmable logic devices (CPLDs), etc.

[0118] In some configurations, any modules / code discussed herein may be implemented in software, hardware, and / or firmware. In any case, modules / code may be provided during the manufacture of the device or by an intermediary preparing to sell the device to the end user. In other cases, the end user may install these modules / code later, such as by downloading executable code and installing it on the corresponding device.

[0119] It's also important to note that devices typically have input and / or output capabilities. For example, computing devices can have various input mechanisms, such as keyboards, mice, touchpads, voice recognition, gesture recognition (e.g., using depth cameras, such as stereo or time-of-flight camera systems, infrared camera systems, RGB camera systems, or using accelerometers / gyroscopes, facial recognition, etc.). Devices can also have various output mechanisms, such as printers, monitors, etc.

[0120] It should also be noted that the devices described herein can function independently or collaboratively to implement the described techniques. For example, the methods and functions described herein can be executed on a single computing device and / or distributed across multiple computing devices communicating via multiple networks 650. Without limitation, the multiple networks 650 may include one or more local area networks (LANs), wide area networks (WANs), the Internet, etc. Additional examples

[0121] Various examples have been described above. Additional examples are described below. One example includes a method performed on a computing device, the method comprising: obtaining a first sequence of first values; masking one or more first values ​​in the first sequence to obtain a masked first sequence having one or more first values ​​and one or more masked values; using a first machine learning model to determine a first prediction of the one or more masked values ​​in the masked first sequence; replacing the one or more masked values ​​with the first prediction to obtain a second sequence of second values; using a second machine learning model to determine a second prediction regarding whether the second value is present in the first sequence or replaced by a different value predicted by the first machine learning model; and performing a decoupling optimization of the first machine learning model and the second machine learning model, the first machine learning model being optimized at least based on the first prediction, and the second machine learning model being optimized at least based on the second prediction.

[0122] Another example may include any of the above and / or the following examples, wherein the first prediction represents a probability distribution over the predicted values ​​for the masked values.

[0123] Another example may include any of the above and / or the following examples, wherein the substitution includes sampling from the probability distribution.

[0124] Another example may include any of the above and / or the following examples, wherein the first sequence of the first value includes a token obtained from the text.

[0125] Another example may include any of the above and / or the following examples, wherein the first machine learning model is a text generator and the second machine learning model is a discriminator.

[0126] Another example may include any of the above and / or the following examples, wherein the first machine learning model and the second machine learning model use embeddings to represent the token.

[0127] Another example may include any of the above and / or the following examples, wherein the first machine learning model includes a first encoder, the second machine learning model includes a second encoder, and the first encoder does not share an embedding with the second encoder.

[0128] Another example may include any of the above and / or the following examples, wherein the first sequence of the first values ​​comprises unlabeled pre-training data, the separation optimization is performed during pre-training, and the separation optimization involves a first adaptive optimization of the first parameters of the first machine learning model and a second adaptive optimization of the second parameters of the second machine learning model.

[0129] Another example may include any of the above and / or the following examples, wherein the first adaptive optimization and the second adaptive optimization are performed using the Adam optimizer.

[0130] Another example may include any of the above and / or the following examples, wherein the method further includes adjusting the pre-trained second machine learning model for a specific task after the optimization of the second machine learning model yields a pre-trained second machine learning model.

[0131] Another example may include any of the examples above and / or below, wherein the adjustment is based on labeled training data for the specific task.

[0132] Another example may include any of the above and / or the following examples, where the specific task includes one or more of the following: predicting textual implications, predicting answers to questions, predicting paraphrasing relationships, predicting grammatical acceptability, predicting sentiment, or predicting sentence similarity.

[0133] Another example includes a system comprising a hardware processing unit and a storage resource storing computer-readable instructions that, when executed by the hardware processing unit, cause the hardware processing unit to obtain a pre-trained machine learning model. The pre-trained machine learning model has been pre-trained to predict whether a second value in a second sequence is present in a first sequence of first values ​​or is replaced by a different value predicted by another machine learning model. The pre-trained machine learning model and the other machine learning model have been separately optimized, and the pre-trained machine learning model has been adjusted for a specific task using task-specific training data to obtain a adjusted machine learning model.

[0134] Another example may include any of the above and / or the following examples, wherein the pre-trained machine learning model is pre-trained using unlabeled pre-training data.

[0135] Another example may include any of the examples above and / or below, wherein the task-specific training data includes labeled training data.

[0136] Another example may include any of the above and / or the following examples, wherein the labeled training data includes labeled examples of text.

[0137] Another example may include any of the above and / or the following examples, wherein the pre-trained machine learning model utilizes an embedding representation token, the embedding being not shared with the other machine learning model.

[0138] Another example may include any of the above and / or the following examples, wherein the computer-readable instructions, when executed by the hardware processing unit, cause the hardware processing unit to receive input data and process the input data using the adjusted machine learning model.

[0139] Another example may include any of the above and / or the following examples, wherein the input data includes a query, the processing includes using the adjusted machine learning model to predict the intent of the query, determining query results based at least on the predicted intent, and responding to the query using the query results.

[0140] Another example may include any of the above and / or the following examples, wherein a computer-readable storage medium stores computer-readable instructions that, when executed by a processing unit, cause the processing unit to perform actions, the actions including obtaining a first sequence of first values, masking one or more first values ​​in the first sequence to obtain a masked first sequence having one or more first values ​​and one or more masked values, using a first machine learning model to determine a first prediction of the one or more masked values ​​in the masked first sequence, replacing the one or more masked values ​​with the first prediction to obtain a second sequence of second values, using a second machine learning model to determine a second prediction regarding whether the second value exists in the first sequence or is replaced by a different value predicted by the first machine learning model, and performing a separation optimization of the first machine learning model and the second machine learning model, wherein the first machine learning model is optimized at least based on the first prediction and the second machine learning model is optimized at least based on the second prediction. in conclusion

[0141] Although the subject matter has been described in language specific to structural features and / or methodological actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are disclosed as exemplary forms of implementing the claims, and other features and actions that a person skilled in the art will recognize are within the scope of the claims.

Claims

1. A method executed on a computing device, the method comprising: The first sequence that obtains the first value; Mask one or more first values ​​among the first values ​​in the first sequence to obtain a masked first sequence, the masked first sequence having one or more first values ​​among the first values ​​and one or more masked values; Using a first machine learning model, determine a first prediction of the value of one or more masks in the first masked sequence; The first prediction is used to replace one or more masked values ​​to obtain a second sequence of second values; Using a second machine learning model, a second prediction is made regarding whether the second value exists in the first sequence or is replaced by a different value predicted by the first machine learning model; as well as Perform separate optimization of the first machine learning model and the second machine learning model, wherein the first machine learning model is optimized based at least on the first prediction, and the second machine learning model is optimized based at least on the second prediction.

2. The method of claim 1, wherein the first prediction represents a probability distribution over predicted values ​​for the masked values.

3. The method of claim 2, wherein the replacement comprises sampling from the probability distribution.

4. The method of claim 3, wherein the first sequence of the first values ​​comprises a token obtained from the text.

5. The method of claim 4, wherein the first machine learning model is a text generator and the second machine learning model is a discriminator.

6. The method of claim 5, wherein the first machine learning model and the second machine learning model use embeddings to represent the token.

7. The method of claim 6, wherein the first machine learning model includes a first encoder, the second machine learning model includes a second encoder, and the first encoder does not share an embedding with the second encoder.

8. The method of claim 7, wherein the first sequence of the first values ​​comprises unlabeled pre-trained data, the separation optimization is performed during pre-training, and the separation optimization involves a first adaptive optimization of the first parameters of the first machine learning model and a second adaptive optimization of the second parameters of the second machine learning model.

9. The method of claim 8, wherein the first adaptive optimization and the second adaptive optimization are performed using the Adam optimizer.

10. The method according to any one of claims 1 to 9, further comprising: After optimizing the second machine learning model to obtain a pre-trained second machine learning model, the pre-trained second machine learning model is adjusted for a specific task.

11. The method of claim 10, wherein the adjustment is based on labeled training data for the specific task.

12. The method of claim 11, wherein the specific task includes one or more of the following: predicting textual entailment, predicting answers to questions, predicting paraphrasing relationships, predicting grammatical acceptability, predicting sentiment, or predicting sentence similarity.

13. A system comprising: Hardware processing unit; as well as Storage resources that store computer-readable instructions, which, when executed by the hardware processing unit, cause the hardware processing unit to: A pre-trained machine learning model is obtained, which has been pre-trained to predict whether a second value in a second sequence exists in a first sequence of a first value or is replaced by a different value predicted by another machine learning model. The pre-trained machine learning model and the other machine learning model have been separately optimized. as well as Using task-specific training data, the pre-trained machine learning model is tuned for a specific task to obtain a tuned machine learning model.

14. The system of claim 13, wherein the pre-trained machine learning model is pre-trained using unlabeled pre-training data.

15. The system of claim 14, wherein the task-specific training data includes labeled training data.

16. The system of claim 15, wherein the labeled training data includes labeled examples of text.

17. The system of claim 16, wherein the pre-trained machine learning model utilizes an embedding representation token, the embedding being not shared with the other machine learning model.

18. The system according to any one of claims 13 to 16, wherein the computer-readable instructions, when executed by the hardware processing unit, cause the hardware processing unit to: Receive input data; and The input data is processed using the adjusted machine learning model.

19. The system of claim 18, wherein the input data includes a query, and the processing includes: The intent of the query is predicted using the adjusted machine learning model. The query results are determined at least based on the predicted intent. as well as Reply to the query using the query results.

20. A computer-readable storage medium storing computer-readable instructions that, when executed by a processing unit, cause the processing unit to perform an action, the action comprising: The first sequence that obtains the first value; Mask one or more first values ​​among the first values ​​in the first sequence to obtain a masked first sequence, the masked first sequence having one or more first values ​​among the first values ​​and one or more masked values; Using a first machine learning model, determine a first prediction of the value of one or more masks in the first masked sequence; The first prediction is used to replace one or more masked values ​​to obtain a second sequence of second values; Using a second machine learning model, a second prediction is made regarding whether the second value exists in the first sequence or is replaced by a different value predicted by the first machine learning model; as well as Perform separate optimization of the first machine learning model and the second machine learning model, wherein the first machine learning model is optimized based at least on the first prediction, and the second machine learning model is optimized based at least on the second prediction.

Images