Adversarial training of machine learning models

Abstract

This document relates to training of machine learning models such as neural networks. An example method includes providing a machine learning model having one or more layers and associated parameters, and performing a pre-training phase on the parameters of the machine learning model to obtain pre-trained parameters. The example method also includes performing an adjustment phase on the machine learning model using labeled training samples to adjust the pre-trained parameters. The adjustment phase can include performing noise adjustment on the labeled training examples to obtain noise-adjusted training samples. The adjustment phase can also include adjusting the pre-trained parameters based at least on the labeled training examples and the noise-adjusted training examples to obtain adapted parameters. The example method can also include outputting an adjusted machine learning model having the adapted parameters.

Description

Background Technology

[0001] Machine learning can be used to perform a wide range of tasks, such as natural language processing, financial analysis, and image processing. Machine learning models can be trained using several methods, such as supervised learning, semi-supervised learning, unsupervised learning, and reinforcement learning. In methods such as supervised or semi-supervised learning, labeled training examples are used to train the model to map inputs to outputs. However, for many machine learning tasks, the amount of labeled training data is limited. Summary of the Invention

[0002] This summary is provided to introduce, in a simplified form, some concepts that will be further described in the following detailed description. This summary is not intended to identify key or essential features of the claimed subject matter, nor is it intended to limit the scope of the claimed subject matter.

[0003] This description generally relates to techniques for training machine learning models. One example includes a method or technique that can be performed on a computing device. The method or technique may include providing a machine learning model having one or more layers and associated parameters. The method or technique may also include performing a pre-training phase on the parameters of the machine learning model to obtain pre-trained parameters. The method or technique may further include performing a conditioning phase on the machine learning model using labeled training samples to condition the pre-trained parameters. The conditioning phase may include performing noise conditioning on the labeled training samples to obtain noise-conditioned training samples, and conditioning the pre-trained parameters based at least on the labeled training samples and the noise-conditioned training samples to obtain adapted parameters. The method or technique may also include outputting a conditional machine learning model with adapted parameters.

[0004] Another example includes a system having a hardware processing unit and storage resources for storing computer-readable instructions. When executed by the hardware processing unit, the computer-readable instructions can cause the hardware processing unit to: receive input data, process the input data using a machine learning model having a first layer and a second layer to obtain a result, and output the result. The first layer can be pre-trained during a pre-training phase, and the first and second layers can be tuned together using virtual adversarial regularization.

[0005] Another example includes a system having a hardware processing unit and storage resources for storing computer-readable instructions. When executed by the hardware processing unit, the computer-readable instructions can cause the hardware processing unit to: obtain a machine learning model; and perform a supervised learning process on the machine learning model. The supervised learning process may include adjusting the parameters of the machine learning model based at least on a training loss on labeled training samples including model inputs and corresponding labels. The supervised learning process may also include adjusting the parameters of the machine learning model based at least on biases in the model output caused by noise added to the model inputs.

[0006] The examples listed above are intended to provide a quick reference to help the reader, rather than to limit the scope of the ideas described in this article. Attached Figure Description

[0007] The accompanying drawings are provided for detailed description. In the drawings, the leftmost numeral(s) of the reference numeral(s) identifies the drawing in which that reference numeral(s) first appears. The use of similar reference numerals in different instances of the description and drawings may indicate similar or identical items.

[0008] Figure 1 An example machine learning model consistent with some implementations of this concept is shown.

[0009] Figure 2 An example training workflow for training machine learning models is shown, which is consistent with some implementations of this concept.

[0010] Figure 3A and Figure 3B A graphical representation of the model output distribution, consistent with some implementations of this concept, is provided.

[0011] Figures 4-7 An example multi-task machine learning model consistent with some implementations of this concept is shown.

[0012] Figure 8 Examples of labeled, task-specific training data consistent with some implementations of this concept are shown.

[0013] Figure 9 and Figure 10 Example graphical user interfaces that can be adopted are shown, which are consistent with some implementations of this concept.

[0014] Figure 11 Example systems consistent with some implementations of this concept are shown.

[0015] Figure 12 Example methods or techniques for training and utilizing machine learning models that are consistent with some implementations of this concept are shown.

[0016] Figure 13 Example methods or techniques for training machine learning models that are consistent with some implementations of this concept are shown.

[0017] Specific implementation method

[0018] There are various types of machine learning frameworks that can be trained to perform a given task. Support vector machines, decision trees, and neural networks are just a few examples of machine learning frameworks that have been widely used in various applications, such as image processing and natural language processing. Some machine learning frameworks, such as neural networks, use layers of nodes that perform specific operations.

[0019] In a neural network, nodes are interconnected via one or more edges. A neural network may include an input layer, an output layer, and one or more intermediate layers. Individual nodes can process their corresponding inputs according to a predefined function and provide outputs to the next layer (or, in some cases, the previous layer). The input to a given node can be multiplied by the corresponding weight value of the edge between the input and the node. Additionally, nodes may have individual bias values, which are also used to produce the output. Various training procedures can be applied to learn the edge weights and / or bias values. When used without modification, the term "parameter" in this document refers to learnable values, such as edge weights and bias values, that can be learned by training machine learning models (such as neural networks).

[0020] Neural network architectures can have different layers that perform different specific functions. For example, one or more layers of nodes can work together to perform specific operations such as pooling, encoding, or convolution. For the purposes of this document, the term "layer" refers to a group of nodes that share inputs and outputs (e.g., inputs and outputs going to or from external sources or other layers in the network). The term "operation" refers to a function that can be performed by one or more layers of nodes. The term "model architecture" refers to the overall architecture of a layered model, including the number of layers, the connections between layers, and the types of operations performed by each layer. The term "neural network architecture" refers to the model structure of a neural network. The terms "trained model" and / or "tuned model" refer to the model structure and the parameters of that model structure that have been trained or tuned. Note that, for example, two trained models can share the same model structure but have different parameter values ​​if they are trained on different training data or if there are underlying stochastic processes during training.

[0021] As mentioned earlier, many machine learning tasks suffer from a relative lack of training data. A widely used approach to train a model with limited, task-specific training data for a particular task involves "transfer learning." In transfer learning, the model is first pre-trained on another task with a substantial amount of training data available, and then tuned to the specific task using task-specific training data. However, conventional techniques used to do this tend to employ aggressive tuning updates, which can lead to overfitting and / or knowledge loss in the final model, as discussed in more detail below. The term "training" as used in this paper encompasses both the pre-training of the model and subsequent tuning updates; that is, the term "training" covers any process of updating the model's parameters using labeled or unlabeled training examples.

[0022] As mentioned above, aggressively updating a pre-trained model to a finite, task-specific training dataset can cause the model to overfit the training data. When a model is overfitted to a given training dataset, it cannot generalize well to new examples. Furthermore, aggressive updates can lead to knowledge forgetting, where knowledge learned during the pre-training process is lost during the tuning process. Various methods have been used to address these tuning problems, such as using heuristic learning rates, freezing certain model layers for a portion of the tuning process, and so on. These specialized methods have achieved some success, but often require a significant amount of tuning work from highly skilled machine learning experts.

[0023] The disclosed implementation provides several mechanisms to address the potential for model overfitting and knowledge forgetting. For example, the disclosed implementation provides an adversarial regularization mechanism that can help mitigate overfitting of the model during training or tuning. As discussed in more detail below, the adversarial regularization mechanism encourages the model to produce a smooth output function, e.g., an output function that does not change drastically for small perturbations in the input.

[0024] The disclosed implementation also provides a nearest neighbor mechanism, which helps prevent knowledge forgetting during model training or tuning. As discussed in more detail below, the nearest neighbor mechanism encourages the model to produce outputs similar to those of previous iterations. In other words, for each training iteration, the nearest neighbor update mechanism prevents large changes in model parameters relative to one or more previous instances of the model from previous training iterations.

[0025] In summary, the disclosed mechanisms can stabilize the training or tuning of machine learning models by mitigating overfitting and knowledge forgetting. The discussion below provides some concrete examples in the context of natural language processing. However, as discussed further below, the disclosed techniques can also be used for tasks other than natural language processing, such as image recognition and financial analysis.

[0026] Example Natural Language Processing Model

[0027] Figure 1 An exemplary natural language processing model 100 that can be trained using the disclosed implementation is shown. The natural language processing model 100 is an example of a machine learning model that can be used to perform one or more natural language processing tasks, as discussed in more detail below. For the purposes of this document, the term "natural language" refers to language that is typically used by humans for writing or conversation.

[0028] Natural Language Processing Model 100 can receive language input 102, which may include words, tokens, sentences, phrases, or other language representations. The language input can be processed by layer 104, which includes a dictionary encoder 104 (1) and a transformer encoder 104 (2). Typically, both the dictionary encoder and the transformer encoder operate to produce vectors representing individual words, tokens, sentences, or phrases in a vector space where words, tokens, sentences, or phrases with semantic and / or syntactic similarity are relatively close to each other, while those with lower semantic or syntactic similarity are relatively far apart. These vectors are also referred to as “embeddings” in this paper.

[0029] The dictionary encoder 104(1) can generate a sequence of input embedding vectors 106 for each word or symbol in the language input 102. The input to the dictionary encoder can be a sequence of symbols X = {x1, ..., x} of length m. m}. Specific symbols can be used to describe the beginning of each sequence and to separate sentences within a given sequence. A dictionary encoder maps X to a sequence of input embedding vectors, one for each symbol. In some implementations, the input embedding vectors are constructed by summing the word, fragment, and position embeddings for each word.

[0030] Transformer encoder 104(2) can obtain contextual information for each word, for example, via self-attention, and generate a sequence of context embedding vectors 108. Self-attention is a mechanism that associates symbols at different positions within a sentence to compute the similarity between these symbols. In some implementations, the transformer encoder is a multi-layer bidirectional transformer encoder configured to map the input embedding vector 106 to the context embedding vector. As discussed in more detail below, the context embedding vector can be used as a shared representation of input phrases or sentences across different tasks. The context embedding vector represents a word or symbol and the context in which each word or symbol appears in the underlying document, query, or other input. Note that both the dictionary encoder and the transformer encoder can be initialized using unsupervised techniques and then updated by conditioning with labeled data for a specific task.

[0031] The context embedding vector 108 can be input to the task-specific layer 110 to perform task-specific processing, as discussed in more detail below. The task-specific layer 110 can evaluate the context embedding vector 108 to produce a task-specific output. Note that the following examples of the task-specific layer 110 are just a few example types of task-specific layers that can be adopted.

[0032] In some implementations, the task-specific layer can be a single-sentence classification layer, which uses category labels to annotate sentences. As a concrete example, a single-sentence classification layer can predict whether an English sentence is grammatically correct. Another example classification task is determining whether a sentence extracted from a movie review is in a positive or negative mood. As discussed in more detail below, the output of a task-specific layer for a classification task can include the corresponding probability for each potential category.

[0033] In other implementations, the task-specific layer 110 can be a pairwise text similarity layer that performs a regression task on a pair of input sentences. The regression task can include outputting a real-valued similarity score indicating the semantic similarity between the two sentences.

[0034] In other implementations, the task-specific layer 110 can be a pairwise text classification layer that determines the relationship between a pair of input sentences, where the relationship is selected from a predefined set of labels. For example, the labels can indicate whether one input sentence has an entailment, contradiction, or neutral relationship with respect to another input sentence.

[0035] In other implementations, the task-specific layer 110 can be a relevance ranking layer whose output conveys a relevance score that evaluates the relevance of two inputs (e.g., the relevance of a document to a query). The relevance score can be used in subsequent natural language processing operations to rank candidate answers according to their relevance to the query.

[0036] Training Workflow Example

[0037] Figure 2 An example training workflow 200 for training machine learning models, consistent with some implementations, is shown. Using... Figure 1 The natural language processing model 100 shown is used as an example model to describe this training workflow, but this training workflow can be performed on many different types of machine learning models.

[0038] Training workflow 200 may include a pre-training phase 202 and a tuning phase 204. As discussed in more detail below, the pre-training phase may be used to determine the pre-trained parameters of one or more layers of a machine learning model, and the tuning phase may be used to adapt these pre-trained parameters to a specific task. As described below, in some cases, the tuning phase is applied to one or more additional layers that were not initialized during the pre-training phase.

[0039] In some implementations, the pre-training phase 202 can utilize unlabeled training data 206 to perform unsupervised learning 208. Unsupervised learning can output pre-trained parameters 210. For example, unlabeled training data can provide an unlabeled corpus of documents in a given natural language. An embedding layer 104 can be pre-trained via unsupervised learning to predict symbols in the corpus. One approach involves masking symbols and training the embedding layer to predict the masked symbols bidirectionally (e.g., from left to right and from right to left). Another approach involves training the embedding layer to predict whether a given sentence is the next sentence in the corpus given prior context. Additional details can be found in the following literature: Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” 2018, arXiv preprint arXiv: 1810.04805.

[0040] After the embedding layer 104 is pre-trained, the embedding layer and the task-specific layer 110 can be tuned together in the tuning phase 204. Initially, the pre-trained parameters of the embedding layer and the initialized (e.g., random) parameters of the task-specific layer 110 can be provided in the model history 212. In summary, the pre-trained parameters of the embedding layer and the initialized parameters of the task-specific layer form the first iteration of the machine learning model. Subsequent tuning iterations can be used to modify these initial parameters to obtain subsequent iterations of the machine learning model, as discussed in more detail below.

[0041] One specific tuning method is to use stochastic gradient descent to train the parameters of the embedding layer 104 and the parameters of the task-specific layer 110. In each training iteration or epoch, the batch selector 214 selects a batch of labeled task-specific data from the labeled task-specific examples 216 and outputs these as labeled training samples 218.

[0042] Annotated training samples 218 are input to a noise adjustment process 220, which outputs noise-adjusted training samples 222. For example, noise-adjusted training samples can add noise to the embedding vector representation of the labeled training samples, where the embedding vector representation is generated using one or two layers of embedding layers 104. The noise-adjusted training samples and the labeled training samples are input to a parameter adjustment process 224, where model parameters are adjusted to obtain an updated model 226, which is then output to the model history 212. The next adjustment iteration can be performed by retrieving the previous model 228 from the model history 212 and continuing the adjustment iteration until a stopping condition is met, such as model convergence, reaching a threshold accuracy on the test dataset, exhausting the training budget, and / or exhausting all labeled task-specific examples 216.

[0043] Algorithm details for parameter adjustment

[0044] In some implementations, the parameter tuning process 224 in the training workflow 200 utilizes adversarial regularization and nearest-neighbor update mechanisms. Adversarial regularization can effectively control model complexity using explicit regularization. Specifically, given a given... The model f(·θ) represents the target task (e.g., a batch of labeled task-specific examples) and n data points, where x i Let y represent the embedding of the input sentence obtained from the first embedding layer of the language model, and y i These are related tags, and the disclosed implementation can be optimized and fine-tuned as follows:

[0045]

[0046] Where (θ) is the training loss term, defined as:

[0047]

[0048] and It depends on the loss function of the target task, λ s >0 is an adjustment parameter, and s (θ) is a smoothness-inducing adversarial regularizer. s (θ) can be defined as:

[0049]

[0050] Where ∈>0 is the adjustment parameter. Note that for classification tasks, f(·;θ) outputs a probability simplex, and It can be chosen as the symmetric KL-divergence, i.e.,

[0051]

[0052] For regression tasks, f(·θ) outputs a scalar, and It can be chosen as the squared loss, i.e. Notice, s The calculation of (θ) can include a maximization problem, and can be solved efficiently by projective gradient ascent.

[0053] Smoothing-induced adversarial regularizers are essentially measuring... The local Lipschitz continuity of the measurement f is measured below. More precisely, when the noise adjustment process 220 is directed towards x... i Inject a very small perturbation (norm bounded by ∈). When ), the output of f will not change much. A perturbation can be injected on the "adversarial direction" estimated above, where the adversarial direction is the direction in which adding random noise to a given input causes the maximum deviation in the model output. By reducing and / or minimizing the objective in (1), training with noise-adjusted training samples 222 can encourage f to be optimized across all x. i The neighborhood is smooth. This smoothing-induced property can help prevent overfitting and improve generalization to low-resource target domains for a given task.

[0054] Note that the training loss (θ) is proportional to the difference between the model's prediction in the current iteration and the labels in the training data. For example, consider a model classifying the sentiment of a sentence as positive or negative. Here, the (θ) term is proportional to the number of labels the model incorrectly predicts in a given batch. Therefore, consider, for example, a single training example labeled as having positive sentiment. If the model output f for that training example is 0.6 negative 0.4 positive, this means the model estimates that the training example has a 60% probability of being negative and a 40% probability of being positive. Since the model predicted a negative sentiment label for that example, there will be a training loss as a result of this incorrect prediction. This will also be true, for example, if the model output f is 0.7 negative 0.3 positive, this is still counted as a single missed label for the purpose of calculating the training loss (θ). Therefore, (θ) is proportional to the total number of missed labels in a given batch, and independent of the underlying probabilities output by the model.

[0055] In comparison, s The term s(θ) is proportional to the difference between the model output for a given labeled training sample and its noise-adjusted counterpart. Thus, for example, for the sentiment of a given training sample, suppose the model output is 0.6 negative and 0.4 positive, and the noise-adjusted counterpart is also estimated to be 0.6 negative and 0.4 positive. In this case, s(θ) is zero because the model replicates its output for both samples, and this is true regardless of the label of the training sample. In contrast, if the model outputs different values ​​(e.g., 0.55, 0.45) for the noise-adjusted sample compared to the retrieved training sample, then... s (θ) will be non-zero. Furthermore, s (θ) is proportional to this difference; for example, as the model output for noise-adjusted samples deviates further from the model output for the actual training samples within a given batch, s (θ) increases. The term "virtual adversarial regularization" is used in this paper to refer to various mechanisms that adjust model parameters by considering how the model output changes when noise is added to the model input. The term "virtual" applies when the model output, rather than the labels of the training samples, is used for training purposes.

[0056] There are many different methods to solve equation (1), i.e., to compute the next set of model parameters using a given batch of training data. In some cases, aggressively updating the model parameters may lead to knowledge forgetting; for example, the model parameters for a given training batch may deviate from the previous model parameters in a way that is detrimental to the utility of the model in the next iteration.

[0057] The nearest neighbor update mechanism for solving (1) is described below. This mechanism can be used to mitigate the consequences of aggressive model updates. The nearest neighbor update mechanism can impose a penalty in each adjustment iteration. Starting from the first iteration of the model denoted by f(·θ0), for the (t+1)th training iteration:

[0058] θ t+1 = θ (θ)+μ Breg (θ,θ t (2)

[0059] Where μ > 0 is the adjustment parameter, and Breg (·,·) is the Bregman divergence, defined as

[0060]

[0061] in The definition is as above. It can be seen that when μ is large, adjusting the Bregman divergence in each iteration can act as a powerful regularizer and prevent the model parameters θ from being affected. t+1 The next set with the parameter θ from the previous iteration t Too much deviation. As a result, the nearest neighbor update mechanism can effectively preserve knowledge of out-of-domain data (e.g., unlabeled training data 206) obtained by pre-training the model. Note that equation (2) can be solved using a stochastic gradient descent algorithm, such as the one in the following literature: Diederik Kingma and Jimmy Ba, “ADAM: A Method for Stochastic Optimization”, arXiv preprint arXiv: 1412.6980v9.

[0062] The nearest neighbor update mechanism can be accelerated by introducing additional momentum into the update. Specifically, in the (t+1)th iteration, the Momentum Bregman Nearest Neighbor (MBPP) method employs:

[0063]

[0064] in And β∈(0,1) is the momentum parameter.

[0065] As described above, the disclosed nearest neighbor mechanism can prevent the introduction of new model parameters θ. t+1 With model parameters θ from previous iterations tToo much deviation. In other words, the adjustment of model parameters is constrained based on the difference between the output of the current model iteration and the output of at least one previous iteration of the model. In some cases, the nearest neighbor mechanism can maintain the overall average of previous model instances by averaging the individual parameters in each iteration, and use the average of previous models to constrain parameter updates in the next adjustment iteration.

[0066] Algorithm pseudocode

[0067] The pseudocode for a specific algorithm is provided below, which can be used to implement the parameter tuning process using a smooth induced adversarial regularizer with p=∞ and the momentum Bregman nearest neighbor method.224 For simplicity of comments, use... And AdamUpdate B Let represent the update rules for the ADAM method (3) used to optimize using mini-batch B; use Π A This represents the projection onto A.

[0068] Input: T: total number of iterations, X: dataset, θ0: parameters of the pre-trained model, S: total number of iterations used to solve (2), σ 2 : The variance of the random initialization, Used for updating The number of iterations, η: used for updating The learning rate, β: momentum parameter.

[0069]

[0070]

[0071] Example model output characteristics

[0072] One potential benefit of the disclosed implementation includes the disclosed adversarial regularization mechanism. In adversarial training, the model is encouraged to replicate predictions (e.g., labels) for noisy-adjusted inputs. In contrast, the disclosed technique can provide a virtual adversarial regularization training mechanism that encourages the model to replicate its output for noisy-adjusted samples. Specifically, the regularization term... Encourage model training to favor model parameters that tend to produce a smooth model output distribution. This can help avoid model overfitting, which can occur if the model is trained using only the training loss, for example, by omitting the regularization term from equation (1).

[0073] Another potential benefit of the disclosed implementation is the use of a nearest neighbor mechanism to update model parameters. The nearest neighbor mechanism can help avoid knowledge forgetting that might occur with more aggressive model updates (e.g., by favoring the creation of new models that are relatively similar to those from previous training iterations). However, it should be noted that the disclosed regularization technique can also be employed without a nearest neighbor update mechanism (e.g., using more aggressive and / or conventional model update techniques).

[0074] Figure 3A and Figure 3B A visual illustration is provided of how the disclosed mechanism can produce a relatively smooth output distribution for a trained model. Figure 3A A visualization of the output function learned using training loss without the disclosed adversarial regularization and nearest neighbor mechanisms is shown 300, and Figure 3B A visualization of another output function learned using training loss is shown in the case of the disclosed adversarial regularization mechanism and the nearest neighbor mechanism.350

[0075] exist Figure 3A and Figure 3B In this context, each training data point is represented by a corresponding point 302. Figure 3A The decision boundary 304 is shown as learned by training a machine learning model via a training loss on labeled training data. Figure 3B The diagram shows the loss used on labeled training data, along with another decision boundary 306 learned through the disclosed adversarial regularization and nearest neighbor mechanisms. For comparative purposes, Figure 3B The decision boundary 304 is shown as a dashed line.

[0076] Comparing the two figures reveals that decision boundary 304 is relatively clear near training data point 308, while decision boundary 306 gradually thins around that training data point. Therefore, decision boundary 306 encompasses a larger range of neighboring data points around training data point 308. Intuitively, this is based on the fact that the model has been trained in a way that encourages similar model outputs for labeled inputs and corresponding noise-adjusted inputs near the training examples.

[0077] Example Multitasking Architecture

[0078] In some implementations, the exposed adversarial regularization and nearest neighbor mechanisms can be used to train models with multi-task architectures. Figure 4 An example multi-task model 400 that can be trained using the disclosed techniques is shown.

[0079] The multi-task model 400 may share certain components of the natural language processing model 100, such as the dictionary encoder 104(1) and the transformer encoder 104(2), as well as their inputs and outputs. (See above references.) Figure 1 Descriptions of these components are already provided, so they will not be repeated here. Instead... Figure 1 The task-specific layer 110 shown can be used in a multi-task model, which may have multiple task-specific layers 402, including a single-sentence classification layer 402(1), a pairwise text similarity layer 402(2), a pairwise text classification layer 402(3), and / or a relevance ranking layer 402(4). These task-specific layers may output single-sentence classification output 404(1), pairwise text similarity output 404(2), pairwise text classification output 404(3), and / or relevance score output 404(4), respectively.

[0080] The training of the multi-task model 400 can be modified as described above, similar to the modifications mentioned above. Figure 2 The training workflow 200 shown is conducted in the manner discussed. The labeled task-specific examples 216 may include a collection of different labeled training examples used to train each task-specific layer in the task-specific layers. In other words, the task-specific examples of the layers may include multiple batches of labeled training data for the single sentence classification layer 402(1), multiple batches of labeled training data for the pairwise text similarity layer 402(2), multiple batches of labeled training data for the pairwise text classification layer 402(3), and / or multiple batches of labeled training data for the relevance ranking layer 402(4).

[0081] During the adjustment phase, batch selector 214 may first select a batch of labeled single-sentence classification training data and perform parameter tuning process 224 by propagating the loss calculated for the output of the single-sentence classification layer 402(1) for that batch. Next, batch selector 214 may select a batch of labeled pairwise text similarity training data and perform parameter tuning process 224 by propagating the loss calculated for the output of the pairwise text similarity layer 402(2). Next, batch selector 214 may select a batch of labeled pairwise text classification training data and perform parameter tuning process 224 by propagating the loss calculated for the output of the pairwise text classification layer 402(3). Next, batch selector 214 may select a batch of labeled pairwise correlation ranking training data and perform parameter tuning process 224 by propagating the loss calculated for the output of the pairwise correlation ranking layer 402(4).

[0082] At this point, the conditioning phase 204 can be performed again sequentially on each task-specific layer using more batches of appropriate training data. This process can continue for several iterations until the conditioning phase is complete.

[0083] Figures 4-7 The diagram illustrates how the different components of a multi-task natural language processing model 400 operate during training, depending on which task-specific layer 402 is being trained. Assume there are four labeled training datasets: a first training dataset with single sentences labeled with correct classification, a second training dataset with paired sentences labeled to reflect their semantic similarity, a third training dataset with paired sentences labeled to reflect entailment, neutral, and / or contradictory relationships, and a fourth training dataset with sentences labeled to indicate whether they contain the correct answer to a given query.

[0084] Figure 4 This demonstrates how to train using a first training dataset that can be used to train the single-sentence classification layer 402(1). Figure 4 The components of the multi-task natural language processing model 400 during training using the first training dataset are shown in bold. The first training dataset is fed into the embedding layer 104, and the context embedding vectors 108 generated from the first training dataset are used by the single-sentence classification layer 402(1) to produce the single-sentence classification output 404(1). The parameters of the single-sentence classification layer 402(1), the dictionary encoder 104(1), and the transformer encoder 104(2) can be updated using the disclosed adversarial regularization and neighbor-to-neighbor mechanisms. Note that the parameters of other task-specific layers (pairwise text similarity layer 402(2), pairwise text classification layer 402(3), and relevance ranking layer 402(4)) are not used.

[0085] Figure 5 The second training dataset, which can be used to train the pairwise text similarity layer 402(2), is shown. Similarly, components of the multi-task natural language processing model 400 during training with the second training dataset are shown in bold. The second training dataset is fed into the embedding layer 104, and the context embedding vectors 108 generated from the second training dataset are used by the pairwise text similarity layer 402(2) to produce the pairwise text similarity layer output 404(2). Typically, the parameters of the pairwise text similarity layer 402(2), the dictionary encoder 104(1), and the transformer encoder 104(2) can be updated using the disclosed adversarial regularization and neighbor-to-neighbor mechanisms. Note that the second training dataset is not used to update other task-specific layers (single-sentence classification layer 402(1), pairwise text classification layer (3), and relevance ranking layer 402(4)).

[0086] Figure 6 The third training dataset, which can be used to train the pairwise text classification layer 402(3), is shown. Similarly, components of the multi-task natural language processing model 400 during training with the third training dataset are shown in bold. The third training dataset is fed into the embedding layer 104, and the context embedding vectors 108 generated from the third training dataset are used by the pairwise text classification layer 402(3) to produce the pairwise text classification layer output 404(3). Typically, the parameters of the pairwise text classification layer 402(3), the dictionary encoder 104(1), and the transformer encoder 104(2) can be updated using the disclosed adversarial regularization and neighbor-to-neighbor mechanisms. Note that the third training dataset is not used to update other task-specific layers (single-sentence classification layer 402(1), pairwise text similarity layer (2), and relevance ranking layer 402(4)).

[0087] Figure 7 The fourth training dataset, which can be used to train the relevance ranking layer 402(4), is shown. Similarly, components of the multi-task natural language processing model 400 during training with the fourth training dataset are shown in bold. The fourth training dataset is fed into the embedding layer 104, and the context embedding vectors 108 generated from the fourth training dataset are used by the relevance ranking layer 402(3) to produce relevance score outputs 404(3). Typically, the parameters of the relevance ranking layer 402(4), the dictionary encoder 104(1), and the transformer encoder 104(2) can be updated using the disclosed adversarial regularization and neighbor-to-neighbor mechanisms. Note that the fourth training dataset is not used to update other task-specific layers (single-sentence classification layer 402(1), pairwise text similarity layer (2), and pairwise text classification layer 402(3)).

[0088] Multi-task training typically helps prevent overfitting for several reasons. First, shared layers can be trained using loss functions tailored to different tasks. Second, training datasets for each task can have different characteristics, such as words or symbols used in different contexts, varying frequencies of use, etc. By using publicly available adversarial regularization mechanisms and nearest neighbor mechanisms with task-specific training data for different tasks, multi-task models that generalize well to a wide range of applications can be generated.

[0089] Examples of labeled sentiment data and user experience

[0090] The following description provides example scenarios to demonstrate how the disclosed implementations can be used to deliver an improved user experience compared to traditional model training techniques. Specifically, the description below shows how training examples can lead to model overfitting, resulting in incorrect application behavior. Furthermore, the description shows how models that do not overfit to the training dataset can generalize more effectively and avoid the drawbacks associated with overfitting.

[0091] Figure 8 An example of labeled sentiment data 800 is shown that can be used to train a task-specific layer to perform the classification of positive or negative sentiment in sentences. As previously described, labeled sentiment data can be provided in labeled task-specific example 216 for use in training workflow 200.

[0092] The labeled sentiment data includes positive example 810 and negative example 820. Each positive example 810 includes a label indicating that a human or automated agent has manually labeled the sentence as indicating positive sentiment, and each negative example 820 includes a label indicating that a human or automated agent has manually labeled the sentence as indicating negative sentiment.

[0093] For illustrative purposes, it is assumed that conventional techniques, which may tend to cause the model to overfit the underlying training data, are used. Figure 8 The model was trained on the labeled sentiment data 800 shown. In particular, note the negative example 820(2). This example describes some undesirable features of the car in a sarcastic tone—readers know that 137 horsepower, excessive engine revs on hills, and 14 mpg are not ideal features for a car. However, the commentator uses wording such as “stupendous,” which is a typical positive term used negatively in this review. The commentator also includes the term “discriminating” to describe the type of driver who would choose this car, but this same term usually implies a positive aspect of the car, yet it is used negatively.

[0094] Furthermore, note that the terms "amazing" and "tasteful" may be used with relatively low frequency. Therefore, there may not be many corresponding examples of these terms in the labeled sentiment data set of 800. As a result, conventional training techniques may overfit the model to the dataset by predicting that reviews with one or both of these terms are negative, when in fact these terms are commonly used by humans to describe positive characteristics of products or services.

[0095] Figure 9Example review page 900 for laptop 902 is shown. Here, the user has selected to filter negative reviews via drop-down menu 904, and in response, the review page displays three reviews: 906, 908, and 910. Reviews 906 and 910 are clearly negative reviews and are therefore appropriate to be displayed to the user in response to a request for a negative review. However, review 908 is a very positive review of the laptop, but includes the wording "amazing" and "tasteful." If, as described above, the model overfits the meaning of these words to the negative example 820(2), the model may incorrectly characterize the review as negative.

[0096] Figure 10 An example review page 1000 for a laptop 902, generated using the disclosed techniques, is shown. Here, review 908 has been replaced by review 1002, which is actually a negative review and is therefore correctly displayed to users who wish to see a negative review of the laptop. Because the disclosed implementation avoids overfitting to examples such as negative example 820(2), the disclosed implementation correctly recognizes that the wording “amazing” and “tasteful” are not strongly associated with a negative review and accordingly does not misclassify review 908 as a negative review.

[0097] application

[0098] Typically, trained models can be used to perform one or more tasks. In natural language processing scenarios, trained models can be employed in search engines, for example, to match queries with documents or determine the sentiment of a given query. As another example, trained models can be used in digital assistants or robots to rank candidate responses to user utterances. In the context of image processing, trained models may have task-specific layers to indicate the presence of certain classifications of objects in a given image, semantically labeled image, etc., as well as one or more shared convolutional layers.

[0099] In other applications, machine learning models trained or conditioned as described herein can be used to perform computer vision, radar, sonar, or other sensing tasks. Furthermore, the disclosed implementations can be employed in medical scenarios (e.g., regression learning for predicting drug efficacy), genetic marker analysis, and more.

[0100] Furthermore, while the previous examples introduced the idea of ​​pre-training certain model layers via unsupervised learning, this is not an inherent limitation of the disclosed technique. For example, consider a scenario where there is a large amount of manually labeled training data for a specific task, such as human-annotated animal images where the labels correctly identify the animal species in the images, such as dog, cat, tiger, chicken, etc. Some or all layers of the model (such as convolutional or pooling layers) can be pre-trained on this dataset using supervised learning.

[0101] Subsequently, the disclosed techniques can be used to adapt the model to different image processing tasks. For example, suppose limited labeled training data is available for images of dogs, where the labels are semantic labels identifying a specific breed of dog. The disclosed implementation can be adopted by using a supervised learning process to pre-train the model using a species-labeled dataset, and by using the disclosed adversarial regularization mechanism and nearest neighbor update mechanism as another supervised learning process to tune the model using a breed-labeled dataset. Note that animal and dog breeds are just a few examples of different types of objects that can be classified by processing images using a model trained according to the disclosed techniques.

[0102] Example System

[0103] This implementation can be executed in various scenarios and on various devices. Figure 11 An example system 1100 in which this implementation can be adopted is shown, as discussed in more detail below.

[0104] like Figure 11 As shown, system 1100 includes client device 1110, server 1120, server 1130, and client device 1140 connected by one or more networks 1150. Note that client devices can be mobile devices such as smartphones and / or tablets, or fixed devices such as desktop computers, server devices, etc. Similarly, servers can be implemented using various types of computing devices. In some cases, Figure 11 Any of the devices shown (especially servers) can be implemented in data centers, server clusters, etc.

[0105] Figure 11 Certain components of the devices shown may be referred to herein by reference numerals in parentheses. For the purposes of the following description, parentheses (1) indicate the presence of a given component on client device 1110, (2) indicate the presence of a given component on server 1120, (3) indicate the presence of a given component on server 1130, and (4) indicate the presence of a given component on client device 1140. Unless a specific instance of a given component is identified, this document will refer to components without parentheses in general.

[0106] Typically, devices 1110, 1120, 1130, and / or 1140 may have corresponding processing resources 1101 and storage resources 1102, which will be discussed in more detail below. These devices may also have various modules that utilize the processing and storage resources to execute the techniques discussed herein. Storage resources may include permanent storage resources (such as magnetic or solid-state drives) and volatile storage (such as one or more random access memory devices). In some cases, modules are provided as executable instructions that are stored on permanent storage devices, loaded into random access memory devices, and read from random access memory by processing resources for execution.

[0107] Client device 1110 may include configuration module 1111, which can interact with model pre-training module 1121 and model tuning module 1122 on server 1120. The model tuning module can provide adversarial regularization mechanism 1123 and nearest neighbor update mechanism 1124. Generally, the configuration module can provide certain configuration parameters to the model pre-training module and model tuning module. Configuration parameters may include architecture parameters and training parameters. Architecture parameters can specify the structure of the machine learning model, such as the number of nodes, the arrangement of nodes in layers, and the connections between nodes and / or layers. Architecture parameters can also specify the input and output data for each node and / or layer.

[0108] Configuration parameters may also include training parameters specifying information such as learning rate, unsupervised learning parameters, unlabeled data sources, supervised learning parameters, labeled data sources, adjustment parameters for adversarial regularization, and / or adjustment parameters for neighbor updates. The model pre-training module 1121 and / or the model tuning module 1122 can use these training configuration parameters to perform model training functions on the model specified by the architecture parameters. As just one example, configuration parameters may identify one or more unlabeled training data sources, such as one or more sentence repositories that can be used to pre-train a language model using masked language prediction and / or next-sentence prediction. As another example, configuration parameters may identify one or more labeled training data sources, which may include task-specific training examples labeled by humans or individual machine learning models.

[0109] Model tuning module 1122 can output the trained final model to server 1130. Model execution module 1131 can execute the final model in response to received input. For example, interface module 1141 on client device 1140 can provide input data to model execution module for evaluation of any task provided in a task-specific layer. Model execution module can use the final model to process the uploaded input data and provide model output as a response by sending the model output to client device 1140 via network(s)1150.

[0110] In some instances, server 1130 also includes application module 1132 that interacts with model execution module 1131. For example, the application module may provide a search engine that, for instance, uses relevance scores as described above to rank documents using a trained model in response to queries provided by client device 1140. As another example, the application module may provide a shopping service that allows users of client device 1140 to filter negative or positive reviews based on review sentiment determined by a trained model, as described above.

[0111] As another example, application module 1132 can provide a crowdsourced image repository that allows users to search for images of specific dog breeds. As previously mentioned, the application module can invoke model execution module 1131 to classify the individual images based on the breed of dog depicted therein. In a further implementation, the application module can provide financial services, social media services, digital assistant services, etc., by invoking a machine learning model that has been trained using the disclosed techniques.

[0112] First example method

[0113] Figure 12 An example method 1200 consistent with this concept is shown. As discussed in more detail below, method 1200 can be implemented on many different types of devices (e.g., via one or more cloud servers, via client devices such as laptops, tablets, or smartphones, or via a combination of one or more servers, client devices, etc.).

[0114] Method 1200 begins at box 1202, in which a machine learning model is provided. For example, the machine learning model may be a neural network as described above. Note that box 1202 may include the architecture of the machine learning model (e.g., node structure, connections between nodes, etc.) generated manually or using automated techniques. Alternatively, box 1202 may include receiving the machine learning architecture from a remote device or via user input.

[0115] Method 1200 continues in box 1204, where a pre-training phase is performed on one or more first layers of the machine learning model, also as described above. In some implementations, boxes 1202 and 1204 can be combined by receiving a model with one or more layers that have already been pre-trained.

[0116] Method 1200 continues at box 1206, where a conditioning phase is performed on the machine learning model, as discussed in the appendix. In a multi-task scenario, the machine learning model may include one or more shared layers and multiple task-specific layers.

[0117] Method 1200 continues at box 1208, where the adjusted machine learning model is output. The adjusted machine learning model may have parameters that were adapted for one or more tasks during the adjustment phase.

[0118] Method 1200 continues at box 1210, where a tuned machine learning model is used to perform a specific task, as described above.

[0119] Second example method

[0120] Figure 13 An example method 1300 consistent with this concept is shown. As discussed in more detail below, method 1300 can be implemented on many different types of devices (e.g., via one or more cloud servers, via client devices such as laptops, tablets, or smartphones, or via a combination of one or more servers, client devices, etc.). In general, method 1300 can be used to tune pre-trained machine learning models (e.g., in box 1206 of method 1200) and / or train machine learning models from scratch.

[0121] Method 1300 begins at box 1302, where a training objective with a regularization term is defined, such as a symmetric KL divergence as described above. A symmetric model output divergence like this measures the divergence in a first and a second output distribution generated by the model, where the second output distribution is computed by adding noise to the input used to generate the first output distribution. In other words, the symmetric model output divergence metric reflects the bias in the model output of the machine learning model caused by adding noise to the model input.

[0122] Method 1300 continues in box 1304, where the nearest neighbor technique is used to solve for the training objective, as described above. In some implementations, the nearest neighbor technique uses a model parameter divergence metric that reflects the divergence between the model parameters (e.g., neural network weights) of the updated model and the model parameters from previous iterations. More specifically, the model parameter divergence metric can be proportional to the difference between the output of the model in the current iteration on a batch of training inputs and the output of the model in at least one previous iteration. By updating the model parameters in each conditioning iteration using the nearest neighbor technique, each model update is disadvantageous to the new model with a larger deviation in model parameters relative to previous iterations.

[0123] Method 1300 continues in box 1306, where the model parameters are updated based on the solution generated in box 1304. In some cases, method 1300 is executed iteratively; for example, boxes 1304 and 1306 may be executed multiple times (e.g., on different batches of training data) until a stopping condition is met, at which point the final trained model can be output.

[0124] Experimental results

[0125] The disclosed technology is used in applications similar to Figure 1 The architecture shown is used to train several single-task models, and they are evaluated using the GLUE benchmark available at gluebenchmark.com. The table below provides the SMART... BERT SMART RoBERTa The results, along with those of several other recently reported models used for comparison purposes, SMART BERT It is a BERT-based model tuned using the disclosed techniques, SMART. RoBERTa It is a RoBERTa-based model tuned using the disclosed techniques:

[0126]

[0127] BERT BASE

[0128]

[0129]

[0130] RoBERTa LARGE

[0131]

[0132] The disclosed technology has also been used in applications similar to Figures 4-7The architecture shown is used to train several multi-task models, and evaluation is also performed using the GLUE benchmark. The table below provides examples of SMART-MT-DNN trained using the disclosed techniques. v0 The results for SMART-MT-DNN, along with several other recently reported results for other models used for comparison purposes. SMART-MT-DNN v0 It is a multi-task model that is pre-trained using labeled training data for different tasks and subsequently modulated using the disclosed adversarial regularization and nearest neighbor mechanisms. SMART-MT-DNN is a multi-task model in which multi-task learning is performed together with the disclosed adversarial regularization and nearest neighbor mechanisms.

[0133]

[0134] In the table above, MLNI refers to Multi-genre Natural Language Inference, a dataset of sentence pairs annotated with textual entailment relations. QQP refers to Quora Question Pairs, a dataset of question pairs annotated to indicate whether the questions repeat each other's meanings. RTE refers to Recognizing Textual Entailment, a dataset of sentences annotated to indicate whether the meaning of one sentence can be inferred from another. QNLI is a question-and-answer dataset containing questions and their corresponding answers identified within the span of text provided in the dataset. MPRC refers to the Microsoft Research Paraphrase Corpus, a dataset of sentences annotated with paraphrasing relations indicating whether one sentence accurately paraphrases another. CoLA refers to the Corpus of Linguistic Acceptability, a dataset of sentences annotated with grammatical acceptability. SST refers to the Stanford Sentiment Treebank, a dataset of example sentences labeled with positive or negative emotions expressed by examples. STS-B refers to the Semantic Textual Similarity Benchmark, a dataset with labels indicating the similarity between two sentences.

[0135] Device implementation method

[0136] As per the above reference Figure 11The system 1100 includes several devices, including client device 1110, server 1120, server 1130, and client device 1140. It should also be noted that not all device implementations can be shown, and other device implementations should be obvious to those skilled in the art based on the above and below descriptions.

[0137] As used herein, the terms “device,” “computer,” “computing device,” “client device,” and / or “server device” can refer to any type of device having a certain amount of hardware processing power and / or hardware storage / memory capacity. Processing power can be provided by one or more hardware processors (e.g., hardware processing units / cores) that can execute data in the form of computer-readable instructions to provide functionality. Computer-readable instructions and / or data can be stored on memory, such as storage devices / RAM and / or data storage devices. The term “system” as used herein can refer to a single device, multiple devices, etc.

[0138] Storage resources can be internal or external to their associated corresponding devices. Storage resources can include any one or more of volatile or non-volatile memory, hard disk drives, flash memory devices, and / or optical storage devices (e.g., CDs, DVDs, etc.). As used herein, the term "computer-readable medium" can include signals. In contrast, the term "computer-readable storage medium" does not include signals. Computer-readable storage media includes "computer-readable storage devices." Examples of computer-readable storage devices include volatile storage media (such as RAM) and non-volatile storage media (such as hard disk drives, optical discs, and flash memory, etc.).

[0139] In some cases, the device is configured with general-purpose hardware processors and storage resources. In others, the device may include a System-on-a-Chip (SOC) type design. In an SOC design implementation, the functionality provided by the device can be integrated onto a single SOC or multiple coupled SOCs. One or more associated processors can be configured to cooperate with shared resources (such as memory, storage devices, etc.) and / or one or more dedicated resources (such as hardware blocks configured to perform certain specific functions). Therefore, the terms “processor,” “hardware processor,” or “hardware processing unit” used herein may also refer to a central processing unit (CPU), graphics processing unit (GPU), controller, microcontroller, processor core, or other types of processing devices suitable for implementation in traditional computing architectures and SOC designs.

[0140] Alternatively or additionally, the functions described herein may be performed at least in part by one or more hardware logic components. For example, but not limited to, the types of hardware logic components shown include field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), etc.

[0141] In some configurations, any modules / code discussed herein may be implemented using software, hardware, and / or firmware. In any case, the modules / code may be provided during device manufacturing or by an intermediary preparing to sell the device to the end user. In other cases, the end user may install these modules / code later, such as by downloading and installing the executable code on the corresponding device.

[0142] It is also noted that devices can typically have input and / or output functions. For example, computing devices can have various input mechanisms, such as keyboards, mice, touchpads, voice recognition, gesture recognition (e.g., using depth cameras (such as stereo or time-of-flight camera systems, infrared camera systems, RGB camera systems) or using accelerometers / gyroscopes, facial recognition, etc.). Devices can also have various output mechanisms, such as printers, monitors, etc.

[0143] It should also be noted that the devices described herein can implement the described techniques independently or collaboratively. For example, the methods and functions described herein can be executed on a single computing device and / or distributed across multiple computing devices communicating via multiple networks 1150. Without limitation, the multiple networks 1150 may include one or more local area networks (LANs), wide area networks (WANs), the Internet, etc.

[0144] Various device examples have been described above. Additional examples are described below. One example includes a method performed on a computing device, the method comprising: providing a machine learning model having one or more layers and associated parameters; performing a pre-training phase on the parameters of the machine learning model to obtain pre-trained parameters; and performing a conditioning phase on the machine learning model using labeled training samples to condition the pre-trained parameters. The conditioning phase includes: performing noise conditioning on the labeled training samples to obtain noise-adjusted training samples; and conditioning the pre-trained parameters based at least on the labeled training samples and the noise-adjusted training samples to obtain adapted parameters. The method includes outputting a conditioned machine learning model with adapted parameters.

[0145] Another example may include any of the examples above and / or the examples below, wherein the adjustment includes calculating a loss function comprising: a first term proportional to the difference between the prediction of the machine learning model and the labels of the labeled training samples; and a second term proportional to the difference between the output of the machine learning model for the labeled training samples and the output of the machine learning model for the noise-adjusted training samples.

[0146] Another example may include any of the examples above and / or the examples below, wherein the tuning phase includes multiple tuning iterations, and the method further includes: determining the difference between the output of the current iteration of the machine learning model and the output of at least one previous iteration of the machine learning model, and constraining the tuning of parameters based at least on the difference.

[0147] Another example may include any of the examples above and / or the examples below, wherein the adjustments include: performing adversarial regularization based at least on noise-adjusted training samples; and performing neighbor-to-neighbor updates of parameters based at least on differences.

[0148] Another example may include any of the examples above and / or the examples below, wherein the method further includes: performing a specific task on the input data using the adjusted machine learning model after the adjustment phase.

[0149] Another example may include any of the examples above and / or the examples below, where the machine learning model includes one or more embedding layers and at least one task-specific layer.

[0150] Another example may include any of the examples above and / or the examples below, where one or more embedding layers include a dictionary encoder or a transformer encoder.

[0151] Another example may include any of the examples above and / or the examples below, where the pre-training phase includes unsupervised learning of the parameters of one or more embedding layers.

[0152] Another example may include any of the examples above and / or the examples below, where the tuning phase adjusts the parameters of one or more embedded layers and the parameters of a task-specific layer.

[0153] Another example may include any of the above examples and / or the following examples, wherein the task-specific layer is selected from the group that includes: single sentence classification layer, pairwise text similarity layer, pairwise text classification layer and pairwise ranking layer.

[0154] Another example includes a system comprising a hardware processing unit and a storage resource storing computer-readable instructions that, when executed by the hardware processing unit, cause the hardware processing unit to: receive input data; process the input data using a machine learning model having a first layer and a second layer to obtain a result, the first layer having been pre-trained in a pre-training phase and the first and second layers having been tuned together using virtual adversarial regularization; and output the result.

[0155] Another example may include any of the examples above and / or the examples below, where the input data includes queries and documents, and the results characterize the similarity between the queries and documents.

[0156] Another example may include any of the examples above and / or the examples below, where the input data consists of sentences and the results characterize the sentiment of the sentences.

[0157] Another example may include any of the examples above and / or the examples below, where the input data includes an image and the result represents an object present in the image.

[0158] Another example may include any of the examples above and / or the examples below, wherein computer-readable instructions, when executed by the hardware processing unit, cause the hardware processing unit to: pre-train the first layer using unsupervised learning; and use virtual adversarial regularization to modulate the first and second layers.

[0159] Another example may include any of the examples above and / or the examples below, wherein computer-readable instructions, when executed by the hardware processing unit, cause the hardware processing unit to use a proximity mechanism to regulate the first and second layers.

[0160] Another example includes a system comprising a hardware processing unit and a storage resource storing computer-readable instructions that, when executed by the hardware processing unit, cause the hardware processing unit to: obtain a machine learning model; and perform a supervised learning process on the machine learning model, the supervised learning process including adjusting the parameters of the machine learning model based at least on a training loss on labeled training samples, the labeled training samples including model inputs and corresponding labels, and biases in the model output of the machine learning model due to noise added to the model inputs.

[0161] Another example may include any of the examples above and / or the examples below, wherein computer-readable instructions, when executed by a hardware processing unit, cause the hardware processing unit to estimate the adversarial direction of the added noise, and wherein the adversarial direction for a particular input is the direction in which adding noise to the particular input results in the maximum deviation in the model output.

[0162] Another example may include any of the examples above and / or the examples below, wherein the machine learning model includes layers that output word or symbol embeddings, and computer-readable instructions, when executed by the hardware processing unit, cause the hardware processing unit to add noise to the word or symbol embeddings.

[0163] Another example may include any of the examples above and / or the examples below, wherein the supervised learning process further includes adjusting parameters based at least on the deviation of the model output of the current iteration of the machine learning model from the model output of at least one previous iteration of the machine learning model.

[0164] in conclusion

[0165] Although the subject matter has been described in language specifically used for structural features and / or methodological actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are disclosed as examples of implementing the claims, and other features and actions that a person skilled in the art will recognize are intended to fall within the scope of the claims.

Claims

1. A method executed on a computing device, the method comprising: Provide a machine learning model having one or more layers and associated parameters, wherein the input data of the machine learning model includes one of the following: queries and documents, sentences, and images; A pre-training phase is performed on the parameters of the machine learning model to obtain pre-trained parameters; A tuning phase is performed on the machine learning model using labeled training samples to tune the pre-trained parameters, the tuning phase including: Perform noise adjustment on the labeled training samples to obtain noise-adjusted training samples, and The pre-trained parameters are adjusted to obtain adapted parameters, the adjustment being based at least on the difference between a first output of the machine learning model for the labeled training samples and a second output of the machine learning model for the noise-adjusted training samples, wherein the first output and the second output are output by the same machine learning model and include different values ​​corresponding to different probabilities of a particular labeled training sample having a particular label. as well as The output is a tuned machine learning model with the adapted parameters, the output of which represents the similarity between the query and the document, or the sentiment of the sentence, or the object presented in the image.

2. The method of claim 1, wherein the adjustment includes calculating a loss function, the loss function comprising: The first term is proportional to the difference between the prediction of the machine learning model and the label of the labeled training sample; as well as The second term is proportional to the difference between the first output of the machine learning model for the labeled training samples and the second output of the machine learning model for the noise-adjusted training samples.

3. The method according to claim 1, wherein the adjustment phase includes multiple adjustment iterations, and the method further includes: Determine the difference between the output of the current iteration of the machine learning model and the output of at least one previous iteration of the machine learning model; as well as The adjustment of the parameters is constrained at least based on the differences.

4. The method of claim 3, wherein the adjustment includes: Adversarial regularization is performed based at least on the noise-adjusted training samples, and neighbor-to-neighbor updates of the parameters are performed based at least on the differences.

5. The method according to claim 1, further comprising: Following the conditioning phase, the conditioned machine learning model is used to perform a specific task on the input data.

6. The method of claim 1, wherein the machine learning model comprises one or more embedding layers and at least one task-specific layer.

7. The method of claim 6, wherein the one or more embedding layers comprise a dictionary encoder or a transformer encoder.

8. The method of claim 6, wherein the pre-training phase includes unsupervised learning of the parameters of the one or more embedding layers.

9. The method of claim 8, wherein the adjustment phase adjusts the parameters of the one or more embedded layers and the parameters of the task-specific layer.

10. The method of claim 9, wherein the task-specific layer is selected from the group consisting of: a single-sentence classification layer, a pairwise text similarity layer, a pairwise text classification layer, and a pairwise ranking layer.

11. A system comprising: Hardware processing unit; as well as Storage resources that store computer-readable instructions, which, when executed by the hardware processing unit, cause the hardware processing unit to: Receive input data, which includes one of the following: queries and documents, sentences, and images; The input data is processed using a machine learning model with a first layer and a second layer to obtain a result. The first layer has been pre-trained in a pre-training phase. The first layer and the second layer have been adjusted together based at least on the difference between a first output of the machine learning model for labeled training samples and a second output of the machine learning model for noise-adjusted training samples obtained by adding noise to the labeled training samples. The first output and the second output are output by the same machine learning model in the adjustment phase and include different values ​​corresponding to different probabilities of a particular labeled training sample having a particular label. as well as Output the result, which represents the similarity between the query and the document, the sentiment of the sentence, or the object presented in the image.

12. The system of claim 11, wherein the computer-readable instructions, when executed by the hardware processing unit, cause the hardware processing unit to: The first layer is pre-trained using unsupervised learning; and Virtual adversarial regularization is used to regulate the first and second layers.

13. The system of claim 11, wherein the computer-readable instructions, when executed by the hardware processing unit, cause the hardware processing unit to: The nearest neighbor mechanism is used to adjust the first layer and the second layer.

14. A system comprising: Hardware processing unit; as well as Storage resources that store computer-readable instructions, which, when executed by the hardware processing unit, cause the hardware processing unit to: Obtain a machine learning model, wherein the input data of the machine learning model includes one of the following: queries and documents, sentences, and images; as well as A supervised learning process is performed on the machine learning model, the supervised learning process including adjusting the parameters of the machine learning model based on at least the following: The training loss on labeled training samples, wherein the labeled training samples include model input and corresponding labels; and The difference between the first output of the machine learning model for the labeled training samples and the second output of the machine learning model for noise-adjusted training samples obtained by adding noise to the labeled training samples, wherein the first output and the second output are output by the same machine learning model and include different values ​​corresponding to different probabilities of a particular labeled training sample having a particular label. and The output of the machine learning model represents the similarity between the query and the document, the sentiment of the sentence, or the objects presented in the image.

15. The system of claim 14, wherein the computer-readable instructions, when executed by the hardware processing unit, cause the hardware processing unit to: Estimate the counter-direction of the added noise. The adversarial direction for a specific input is the direction in which adding noise to the specific input causes the maximum deviation in the model output.

16. The system of claim 14, wherein the machine learning model includes a layer that outputs word or symbol embeddings, and the computer-readable instructions, when executed by the hardware processing unit, cause the hardware processing unit to: The noise is added to the word or the symbol embedding.

17. The system of claim 14, wherein the supervised learning process further includes adjusting the parameters based at least on the deviation of the model output of the current iteration of the machine learning model relative to the model output of at least one previous iteration of the machine learning model.

Images