Training machine learning models using teacher annealing

Abstract

The invention discloses training machine learning models using teacher annealing. Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training machine learning models using teacher annealing.

Description

[0001] Description of the case

[0002] This application is a divisional application of Chinese invention patent application No. 202010404322.8, with a filing date of May 13, 2020. Technical Field

[0003] This specification relates to training a machine learning model. Background Art

[0004] A machine learning model receives input and generates an output, e.g., a predicted output, based on the received input and on parameter values ​​based on the model.

[0005] A neural network is a machine learning model that uses one or more layers of nonlinear units to predict an output for a received input. Some neural networks include one or more hidden layers in addition to an output layer. The output of each hidden layer is used as the input for the next layer in the network, which is the next hidden layer or output layer. Each layer of the network generates an output based on the received input according to the current values ​​of the corresponding set of parameters. Summary of the invention

[0006] This specification broadly describes a system implemented as one or more computer programs on one or more computers in one or more locations or multiple locations that trains a machine learning model using teacher annealing.

[0007] The machine learning model being trained will be referred to as the "student machine learning model" in this specification, and the parameters of the student machine learning model, i.e., the parameters updated through training, will be referred to as "student parameters".

[0008] In particular, during training, the system uses both the true output and the teacher output generated by one or more trained teacher machine learning models. By performing teacher annealing, the system repeatedly adjusts weight values ​​during training that define the weights between the teacher output and the true output used in calculating the objective function for training.

[0009] Particular embodiments of the subject matter described in this specification can be implemented to realize one or more of the following advantages.

[0010] This specification describes the use of teacher annealing to improve the training of a student machine learning model when the output generated by an already trained teacher model is available. By using teacher annealing early in the training, the student model mainly refines, that is, learns mainly from the output of the teacher model to obtain as useful training signal as possible. Towards the end of training, the student model relies mainly on the true output. This progress allows the student machine learning model to achieve performance that exceeds the teacher machine learning model used for the task on any given task, even if the student machine learning model is a multi-task machine learning model and the teacher model is a single-task machine learning model specific to the task. In particular, using these techniques, the student model may exceed the performance of the teacher without training on any more training data used to train the teacher. In addition, in a multi-task setting, the described techniques allow the student machine learning model to achieve robust multi-task gains across multiple tasks at once, that is, relative to conventional training techniques for those tasks.

[0011] In addition, by utilizing the teacher model during training as described in this specification, the student model can be trained to perform equally well or even better than the teacher model even when the student model consumes fewer computing resources than the teacher model to generate outputs. For example, in the case where both the teacher model and the student model are single-task models, the student model can have fewer parameters than the teacher model, or generate student outputs with fewer iterations than the iterations required for the teacher model. As a specific example, the student model and the teacher model can have similar architectures, but the student model has fewer neural network layers and therefore has fewer parameters. As another specific example, the teacher model can be an autoregressive model that generates outputs through multiple iterations, while the student model is a feedforward model that generates student outputs through a single forward pass of the student model. As another example, when the teacher model is a single-task model and the student model is a multi-task model, for example, due to having certain parameters shared between all multi-tasks, the student model can have much fewer parameters than the total number of parameter combinations in the teacher model.

[0012] The details of one or more embodiments of the subject matter described in this specification are set forth in the drawings and description below. Other features, aspects, and advantages of the subject matter will become apparent from the description, drawings, and claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 An example machine learning model training system is shown.

[0014] Figure 2 is a flowchart of an example process for training a student machine learning model.

[0015] Figure 3is a flow chart of another example process for determining updates to student parameters using current weight values.

[0016] Like reference numbers and numerals in the various drawings indicate like elements. DETAILED DESCRIPTION

[0017] Figure 1 An example machine learning model training system 100 is shown. The machine learning model training system 100 is an example of a system implemented as a computer program on one or more computers in one or more locations in which the systems, components, and techniques described below can be implemented.

[0018] This training system 100 trains a machine learning model 110. The machine learning model 110 being trained will be referred to in this specification as the "student machine learning model" and the parameters 118 of the student machine learning model, i.e., the parameters updated by the training, will be referred to as "student parameters."

[0019] In some embodiments, the student machine learning model 110 is a single-task machine learning model that is configured only to perform a single machine learning task, and the system 100 trains the student machine learning model 110 to perform the single machine learning task.

[0020] For example, a single task may be a natural language processing task, such as an entailment task, a paraphrase task, a text similarity task, a sentiment task, a grammar task, etc.

[0021] As another example, the single task may be an image processing task, such as image classification, object detection, semantic segmentation, image enhancement, domain transfer, etc.

[0022] As another example, a single task may be a health prediction task, where the input to the machine learning model is the patient's electronic health record data, and the model output for a given patient may be a probability distribution over patient health-related categories (e.g., a possible diagnosis for the patient, possible future health events associated with the patient, etc.).

[0023] As another example, the single task may be an audio processing task, such as speech recognition, language recognition, hot word detection, etc.

[0024] Thus, in these cases, the student machine learning model 110 receives task-specific input and generates a student output for that task.

[0025] In some other embodiments, the student machine learning model 110 is a multi-task machine learning model configured to perform multiple machine learning tasks, and the system 100 trains the student machine learning model 110 to perform all of the multiple machine learning tasks.

[0026] In other words, in these embodiments, the student machine learning model 110 receives inputs of a type common to all machine learning tasks and generates a model output comprising multiple student outputs, one for each of the machine learning tasks.

[0027] For example, multi-tasks can be multiple different natural language processing tasks that can be performed on the same input text sequence.

[0028] As another example, multitasking may be multiple different image processing tasks that may be performed on the same input image.

[0029] The machine learning model 110 can have any architecture suitable for the type of model input processed by the machine learning model 110. For example, when the model input is image or audio data, the machine learning model 110 can be a convolutional neural network. When the model input is a sequence of text or other features such as electronic medical record features or audio features, the machine learning model 110 can be based on a self-attention neural network (e.g., a transformer) or a recurrent neural network (e.g., a long short-term memory (LSTM) neural network). When the model input includes inputs of multiple modalities such as images and text, the model 110 can include different types of neural networks, such as convolutional layers and self-attention or recurrent layers.

[0030] When model 110 is a multi-task model, model 110 may include an initial layer shared among all tasks and a corresponding set of task-specific output layers for each of the multiple tasks. As another example, all layers of model 110 may be shared between tasks, and model inputs processed by model 110 may each include an identifier or other data identifying the task to be performed by model 110 for the model input.

[0031] The system 100 receives training data 140 for training the student machine learning model 110. More specifically, for a given task on which the system 100 is training the student machine learning model 110, the training data 140 includes training inputs 142 and, for each training input 142, a true output 144 for the given task. The true output 144 is the output that should be generated by the student machine learning model 110 for the given task by processing the training input 142. In other words, the true output 144 is the known (assumed) accurate output for the given task.

[0032] The system 100 then uses the true value output 144 and one or more teacher machine learning models 120 to train the student machine learning model 110.

[0033] Typically, each teacher machine learning model 120 is a machine learning model that has been trained to perform one or more tasks that the student machine learning model 110 is being trained to perform.

[0034] In particular, when the student machine learning model 100 is a single-task model being trained for a single task, the system 100 uses a single teacher model 120 that has been trained to perform the single task (or equivalently, a collection of single-task teacher models whose outputs are combined to generate a teacher output).

[0035] In some of these cases, the student machine learning model 110 can be a smaller model than the teacher model 120, i.e., can have fewer parameters and a lower computational footprint than the teacher model 120, and the system 100 can use a training process to generate a trained model that is more computationally efficient than a single teacher model 120 while having comparable or higher accuracy than the teacher model 120.

[0036] For example, the teacher model 120 may have a similar architecture as the student model 110, but the student model 110 may have fewer neural network layers and, therefore, fewer parameters.

[0037] In other cases of these situations, the teacher machine learning model 120 can be an autoregressive model that generates outputs over many time steps, while the student machine learning model 110 is a feed-forward model that only requires a single time step to generate outputs. For example, the teacher model 120 can be an autoregressive convolutional, self-attention, or recurrent neural network, while the student model 110 is a feed-forward convolutional neural network.

[0038] In other of these cases, the teacher model 120 and the student model 110 may have the same architecture, and the system may use a training process to generate a trained model that performs better than the teacher model 120.

[0039] In some cases, when the student model 110 is a multi-task model, the system 100 uses a single teacher model 120 that is also a multi-task model, and the system 100 uses the same teacher model 120 for all tasks. That is, in these cases, the system may be training a model that is more computationally efficient than the teacher model 120, or training a model that has the same architecture but improves the performance of the teacher model 120, as described above.

[0040] In other cases, when the student model 110 is a multi-task model, the system 100 uses multiple different teacher models 120 for different tasks on which the multi-task student model 110 is trained. For example, the system 100 can use a corresponding single-task teacher model 120 for each of the multiple tasks that the student model 110 is being trained to perform. Thus, in these cases, the system 100 is using a training process to train a single model 110 that has far fewer parameters and less computation than the combined parameters and computation of all the teacher models 120.

[0041] Typically, during training, a training engine 150 in the system 100 repeatedly uses the true output 144 and the teacher output 124 generated by one or more teacher models 120 to determine errors in the student output 114 generated by the student machine learning model 110. The training engine 150 then uses the error to update the values ​​of the model parameters 118.

[0042] More specifically, the training engine 150 uses a technique that will be referred to as teacher annealing to repeatedly adjust the weight values ​​that define the weighting between the teacher output 124 (i.e., the output generated by one of the teacher models 120) and the true value output 144 used in calculating the objective function for training. Figures 2 to 3 Training the model 110 using teacher annealing is described in more detail.

[0043] In some embodiments, the training engine 150 pre-trains the student machine learning model 110 on an unsupervised task before training the student model 110 using teacher annealing. In some cases, pre-training the student model 110 can improve the performance of the final trained model without requiring additional labeled data. For example, when the task is a natural language processing task, the unsupervised task can be an unsupervised language modeling task, for example, as described in Devlin, Jacob; Chang, Ming-Wei; Lee, Kenton; Toutanova, Kristina (October 11, 2018). "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding". arXiv:1810.04805v2.

[0044] Once the model 110 is trained, the system 100 can provide data specifying the trained model for use in processing new network inputs. That is, the system 100 can output the training values ​​of the model parameters for later use in processing inputs using the trained model, such as by outputting to a user device or by storing in a memory accessible to the system 100.

[0045] Alternatively or in addition to outputting trained model data, system 100 can instantiate an instance of a machine learning model with trained values ​​for model parameters, e.g., by receiving input to be processed through an application program interface (API) provided by the system, processing the received input using the trained model to generate model output, and then providing the generated model output, classification output, or both in response to the received input.

[0046] Figure 2 is a flow chart of an example process 200 for training a student machine learning model. For convenience, process 200 is described as being performed by a system of one or more computers located in one or more locations. For example, a machine learning model training system, such as a suitably programmed Figure 1 The machine learning model training system 100 can execute process 200.

[0047] The system initializes the weight values ​​defining the weighting between the teacher output and the true value output (step 202). In other words, the system sets the weight values ​​to fixed initial values ​​before starting training.

[0048] When computing the weighted combination of the teacher outputs, the weight values ​​determine the weights assigned to the teacher outputs for a given training input and the true outputs for a given training input.

[0049] In some cases, the weight value λ can be the weight assigned to the true value output in the weighted combination, and then the weight assigned to the teacher output is equal to 1 - λ. In these cases, the system initializes the weight value to a value equal to 0 or within the range of a threshold 0, i.e., such that the teacher output is initially strongly favored in the weighted combination relative to the true value output.

[0050] In other cases, the weight value λ can be a weight assigned to the teacher output in the weighted combination, and then a weight assigned to the true value output equal to 1 - λ. In these cases, the system initializes the weight value to a value equal to 1 or within the range of a threshold of 1, i.e., such that the teacher output is initially strongly favored in the weighted combination relative to the true value output.

[0051] The system trains the student machine learning model on the training data until criteria for updating weight values ​​are met (step 204).

[0052] In particular, the system repeatedly performs training iterations on mini-batches of training iterations to optimize an objective function that measures, for any given training input, (i) a weighted combination of the teacher output and the true output for the given training input and (ii) the error between the student output generated by the student machine learning model for the training input.

[0053] For example, when λ is the weight assigned to the true value output, the objective function l for task T can be expressed as:

[0054]

[0055] in, is the training input for task T The true value output, is the teacher model trained for the task T according to the parameters θ of the teacher model T The training value of The generated teacher output, and It is through the student model according to the current value of the student parameter θ for the training input Generated student output.

[0056] The objective function measures the error in a way that is appropriate for a given task T.

[0057] For example, when the task T is a classification task and the student output, teacher output, and true output are probability distributions, the objective function can be a cross-entropy loss that measures the cross-entropy between the weighted combination and the student output.

[0058] As another example, when task T is a regression task and the student output, teacher output, and true output are all ordered sets of one or more regression values, the objective function can be a distance loss that measures the distance between the weighted combination and the student output, e.g., L2 distance loss.

[0059] In each training iteration, the system uses a small batch of training data to compute an update to the current values ​​of the student model parameters at the end of the iteration, and then applies the update to the current values ​​of the student parameters. Figure 3 Description Updates the current value.

[0060] The system can determine that the criteria for updating the weight values ​​are met at any appropriate point during training. For example, the system can determine that the criteria are met every N training iterations, i.e., after every N updates applied to the student model. As another example, the system can determine that the criteria are met whenever a certain amount of time elapses during training. As yet another example, the system can maintain a set of performance benchmarks, and can determine that the criteria are met whenever the performance of the student model, for example as measured on a validation dataset, reaches one of the performance benchmarks.

[0061] Once the criteria are met during training, the system updates the weight values ​​to gradually favor true value outputs in the weighting (step 206).

[0062] For example, when a true value output is assigned a weight equal to a weight value, the system may linearly increase the weight at certain points during training, i.e., such that the weight value moves from a starting point at or near zero to approach 1 each time a criterion is met.

[0063] That is, the system updates the weight values ​​according to a linear schedule to gradually favor truth outputs, e.g., linearly increasing the weights after every N updates are applied to the student parameters (when the truth output is assigned a weight equal to the weight value) or linearly decreasing the weights after every N updates have been applied (when the teacher output is assigned a weight equal to the weight value).

[0064] As another example, the system can update weight values ​​according to an exponential schedule to gradually favor truth outputs, e.g., exponentially increasing weights after every N updates have been applied to the student parameters (when the truth output is assigned a weight equal to the weight value), or exponentially decreasing weights after every N updates have been applied (when the teacher output is assigned a weight equal to the weight value).

[0065] The process 200 then returns to step 204, i.e., the system continues to train the model until the criteria for updating the weight values ​​are met again. The system can continue to repeat steps 204 and 206 until the termination criteria for training are met, e.g., a specified number of iterations of step 204 have been performed, a specified amount of time has elapsed, or the student model parameters have converged.

[0066] Therefore, by repeatedly performing steps 204 and 206 during training, the system repeatedly updates the weight values ​​to gradually favor the true value output in the weighting. That is, as training progresses, the system will continue to adjust the weight values ​​so that the true value output is given an increasingly greater weight in the weighted combination relative to the teacher output.

[0067] Figure 3 is a flow chart of an example process 300 for determining updates to current values ​​of parameters of a student machine learning model. For convenience, process 300 is described as being performed by a system of one or more computers located in one or more locations. For example, a machine learning model training system, such as a suitably programmed Figure 1 The machine learning model training system 100 can perform process 300 by adjusting the weight values ​​as described in this specification.

[0068] The system can perform process 300 for each training input in the mini-batch of training inputs to determine a corresponding update for each training input. The system can then combine the updates, for example by summing or averaging the updates, and apply the combined updates to the current values ​​of the student parameters, for example by adding the combined updates to the current values ​​or subtracting the combined updates from the current values.

[0069] When the student model is a single-task model, the training inputs in the mini-batch will all be training inputs of the single task.

[0070] When the student model is a multi-task model, the mini-batch may include training inputs for different tasks in the multi-task. For example, the system may sample each training input in the mini-batch from the overall training dataset including training data for all tasks, or may sample a specified number of training inputs from each of the multi-tasks to generate the mini-batch.

[0071] The system obtains a training input for a first machine learning task (step 302) and a true output for the training input (step 304).

[0072] The system uses the trained first teacher machine learning model to process the training inputs to generate teacher outputs for the first machine learning task (step 306). As described above, the trained first teacher machine learning model has been trained to perform the first machine learning task. In some cases, before starting training, the system pre-processes all training data for the first task using the trained first teacher machine learning model to generate teacher outputs for the training inputs in the training data. In other cases, the system uses the trained first machine learning model to process the training inputs online, i.e., when the teacher outputs are needed for training during training.

[0073] The system determines a weighted combination of the teacher output for the first machine learning task and the true output for the first training input based on the weight values ​​(step 308). In other words, the system calculates the weighted combination of the teacher output and the true output by weighting the true output and the teacher output based on the current weights as of the current iteration of process 300.

[0074] The system processes the training input using the student machine learning model and according to the student parameters to generate a student output for a first machine learning task (step 310).

[0075] The system then determines the gradient of the student parameters with respect to an objective function that measures the error between the weighted combination and the student output by backpropagation (step 312), and determines an update to the student parameters based on the gradient (step 314). For example, the system can determine the update based on the update rule of the optimizer used to train the machine learning model (e.g., stochastic gradient descent, rmsProp, Adam, etc.), i.e., by applying the optimizer's update rule to the gradient.

[0076] This specification uses the term "configured" in conjunction with system and computer program components. For a system of one or more computers to be configured to perform a particular operation or action, it is meant that the system has installed thereon software, firmware, hardware, or a combination of software, firmware, hardware that, in operation, causes the system to perform those operations or actions. For one or more computer programs to be configured to perform a particular operation or action, it is meant that the one or more programs include instructions that, when executed by a data processing device, cause the device to perform the operation or action.

[0077] Embodiments of the subject matter and functional operations described in this specification may be implemented with digital electronic circuits, with tangibly implemented computer software or firmware, with computer hardware including the structures disclosed in this specification and their structural equivalents, or with a combination of one or more of them. Embodiments of the subject matter described in this specification may be implemented as one or more computer programs, i.e., one or more modules of computer program instructions encoded on a tangible non-transitory storage medium for execution by a data processing device or for controlling the operation of a data processing device. The computer storage medium may be a machine-readable storage device, a machine-readable storage substrate, a random or serial access storage device, or a combination of one or more of them. Alternatively or in addition, the program instructions may be encoded on an artificially generated propagation signal, such as a machine-generated electrical, optical, or electromagnetic signal, which is generated to encode information for transmission to a suitable receiver device for execution by a data processing device.

[0078] The term "data processing apparatus" refers to data processing hardware and includes all kinds of apparatus, devices and machines for processing data, including by way of example a programmable processor, a computer or a plurality of processors or computers. An apparatus may also be or further include special-purpose logic circuitry, for example, an FPGA (field programmable gate array) or an ASIC (application-specific integrated circuit). An apparatus may optionally include, in addition to hardware, code that creates an execution environment for a computer program, for example, code constituting processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them.

[0079] A computer program, which may also be referred to or described as a program, software, software application, app, module, software module, script or code, may be written in any form of programming language including compiled or interpreted languages ​​or declarative or procedural languages; and it may be deployed in any form, including as a stand-alone program or as a module, component, subroutine or other unit suitable for use in a computing environment. A program may, but need not, correspond to a file in a file system. A program may be stored in a portion of a file that holds other programs or data, such as one or more scripts stored in a markup language document; in a single file dedicated to the program or in multiple coordinated files, such as files storing one or more modules, subroutines or portions of code. A computer program may be deployed to execute on one computer or on multiple computers located at one site or distributed over multiple sites and interconnected by a data communications network.

[0080] In this specification, the term "database" is used broadly to refer to any data set: the data need not be structured in any particular way, or at all, and it may be stored on storage devices in one or more locations. Thus, for example, an index database may include multiple data sets, each of which may be organized and accessed differently.

[0081] Similarly, the term "engine" is used broadly in this specification to refer to a software-based system, subsystem, or process that is programmed to perform one or more specific functions. Typically, an engine will be implemented as one or more software modules or components installed on one or more computers in one or more locations. In some cases, one or more computers will be dedicated to a specific engine; in other cases, multiple engines may be installed and run on the same computer or multiple computers.

[0082] The processes and logic flows described in this specification can be performed by one or more programmable computers executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows can also be performed by a special purpose logic circuit such as an FPGA or ASIC, or by a combination of a special purpose logic circuit and one or more programmed computers.

[0083] A computer suitable for executing a computer program may be based on a general purpose microprocessor or a special purpose microprocessor or both, or any other type of central processing unit. Typically, a central processing unit will receive instructions and data from a read-only memory or a random access memory or both. The essential elements of a computer are a central processing unit for executing or implementing instructions and one or more storage devices for storing instructions and data. The central processing unit and the memory may be supplemented by a dedicated logic circuit or incorporated in the dedicated logic circuit. Typically, a computer will also include one or more large capacity storage devices for storing data, such as a disk, a magneto-optical disk or an optical disk, or be operationally coupled to receive data from the one or more large capacity storage devices or to transfer data to the one or more large capacity storage devices or both for storing data. However, a computer need not have such a device. In addition, a computer may be embedded in another device, such as a mobile phone, a personal digital assistant (PDA), a mobile audio or video player, a game controller, a global positioning system (GPS) receiver or a portable storage device, such as a universal serial bus (USB) flash drive, etc.

[0084] Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and storage devices, including by way of example semiconductor storage devices, such as EPROM, EEPROM, and flash memory devices; magnetic disks, such as internal hard disks or removable disks; magneto-optical disks; and CD ROM and DVD-ROM disks.

[0085] To provide interaction with a user, embodiments of the subject matter described in this specification may be implemented on a computer having a display device, such as a CRT (cathode ray tube) or LCD (liquid crystal display) monitor, for displaying information to the user, and a keyboard and pointing device, such as a mouse or trackball, that the user can use to provide input to the computer. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback, such as visual feedback, auditory feedback, or tactile feedback; and input from the user may be received in any form, including acoustic, voice, or tactile input. In addition, the computer may interact with the user by sending documents to and receiving documents from a device used by the user; for example, by sending a web page to a web browser on the user's device in response to receiving a request from the web browser. In addition, the computer may interact with the user by sending a text message or other form of message to a personal device, such as a smart phone running a messaging application, and then receiving a response message from the user.

[0086] The data processing apparatus for implementing the machine learning model may also include, for example, dedicated hardware accelerator units for processing common and computationally intensive parts of machine learning training or production, i.e., inference, workloads.

[0087] Machine learning models can be implemented and deployed using a machine learning framework, such as the TensorFlow framework, the Microsoft Cognitive Toolkit framework, the Apache Singa framework, or the Apache MXNet framework.

[0088] Embodiments of the subject matter described in this specification may be implemented in a computing system that includes a back-end component, such as a data server; or includes a middleware component, such as an application server; or includes a front-end component, such as a client computer with a graphical user interface, a web browser, or an app that a user can use to interact with an implementation of the subject matter described in this specification; or includes any combination of one or more such back-end, middleware, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication, such as a communication network. Examples of communication networks include local area networks (LANs) and wide area networks (WANs), such as the Internet.

[0089] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by means of computer programs running on respective computers and having a client-server relationship with each other. In some embodiments, the server transmits data such as HTML pages to the user device, for example, for the purpose of displaying data to a user interacting with the device as a client and receiving user input from the user. Data generated at the user device, for example, the result of the user interaction, may be received from the device at the server.

[0090] Although this specification contains many specific implementation details, these should not be interpreted as limitations on the scope of any invention or that may be claimed, but rather as descriptions of features that may be specific to a particular embodiment of a particular invention. Certain features described in this specification in the context of separate embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented in multiple embodiments individually or in any suitable subcombination. In addition, although features may be described above as functioning in certain combinations and even initially claimed as such, one or more features from a claimed combination may be removed from the combination in some cases, and a claimed combination may be directed to a subcombination or variations of a subcombination.

[0091] Similarly, although operations are depicted in the drawings and described in the claims in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in a sequential order, or that all illustrated operations be performed to achieve the desired results. In some cases, multitasking and parallel processing can be advantageous. In addition, the separation of various system modules and components in the above-described embodiments should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0092] Specific embodiments of the subject matter have been described. Other embodiments are within the scope of the appended claims. For example, the actions recited in the claims may be performed in a different order and still achieve the desired results. As an example, the processes depicted in the accompanying drawings do not necessarily require the specific order or sequential order shown to achieve the desired results. In some cases, multitasking and parallel processing may be advantageous.

Claims

1. A method for training a student machine learning model having multiple student parameters to perform multiple machine learning tasks, wherein: The student machine learning model is configured to receive a model input and process the model input according to the student parameters to generate an output, the output comprising a corresponding student output for each of the plurality of machine learning tasks, the method comprising: The student machine learning model is trained on training data, the training data comprising a plurality of first training inputs and, for each first training input, a corresponding true value output of the first training input for each of the plurality of machine learning tasks, wherein for each first training input, the training comprises: For each machine learning task in the plurality of machine learning tasks, processing the training input using a trained teacher machine learning model for the machine learning task to generate a teacher output for the machine learning task, wherein the trained teacher machine learning model for the machine learning task is a single-task model that has been trained to perform the machine learning task; For each of the plurality of machine learning tasks, determining a weighted combination of the teacher output of the machine learning task and the true output of the first training input according to a weight value defining a weight between the teacher output and the true output; processing the training input using the student machine learning model and according to the student parameters to generate a corresponding student output for each of the plurality of machine learning tasks; determining a gradient of the student parameters with respect to an objective function, the objective function measuring, for each machine learning task in the plurality of machine learning tasks, an error between the weighted combination of the machine learning task and the student output of the machine learning task; and An update to the student parameter is determined based on the gradient.

2. The method according to claim 1, further comprising: Before training, the weight values ​​are initialized.

3. The method according to claim 2, further comprising: During the training, the weight values ​​are repeatedly updated to gradually favor the true value output in the weighting.

4. The method according to claim 3, wherein: Repeatedly updating the weight value includes: The weight values ​​are repeatedly linearly increased during the training period.

5. The method according to claim 1, wherein: For a first machine learning task among the multiple machine learning tasks, the student output, the teacher output, and the true output are probability distributions, and wherein the objective function includes a cross-entropy loss that measures the cross-entropy between the weighted combination of the first machine learning task and the student output of the first machine learning task.

6. The method according to claim 1, wherein: A first machine learning task among the multiple machine learning tasks is a regression task, wherein the student output, the teacher output, and the true value output are each an ordered set of one or more regression values, and wherein the objective function includes a distance loss that measures the distance between the weighted combination of the first machine learning task and the student output of the first machine learning task.

7. The method according to claim 1, wherein: The student machine learning model is a neural network comprising an encoder neural network shared among all of the multiple tasks and a corresponding output neural network for each of the multiple machine learning tasks.

8. The method according to claim 1, wherein: The model input to the student machine learning model is a text sequence in natural language, and wherein the multiple machine learning tasks are different natural language processing tasks that can be performed on the text sequence.

9. The method according to claim 1, further comprising: Prior to the training, the student machine learning model is pre-trained on an unsupervised task.

10. The method according to claim 9, wherein: The plurality of machine learning tasks are natural language processing tasks, and the unsupervised task is an unsupervised language modeling task.

11. The method according to claim 1, further comprising: After training the student machine learning model to perform the plurality of machine learning tasks: Receive new network input; as well as The new network input is processed using the trained student machine learning model to generate a corresponding new network output for each of the plurality of machine learning tasks.

12. A system comprising one or more computers and one or more storage devices storing instructions, wherein the instructions, when executed by the one or more computers, cause the one or more computers to perform operations for training a student machine learning model having a plurality of student parameters to perform a plurality of machine learning tasks, wherein: The student machine learning model is configured to receive a model input and process the model input according to the student parameters to generate an output, the output comprising a corresponding student output for each of the plurality of machine learning tasks, the operations comprising: The student machine learning model is trained on training data, the training data comprising a plurality of first training inputs and, for each first training input, a corresponding true value output of the first training input for each of the plurality of machine learning tasks, wherein for each first training input, the training comprises: For each machine learning task in the plurality of machine learning tasks, processing the training input using a trained teacher machine learning model for the machine learning task to generate a teacher output for the machine learning task, wherein the trained teacher machine learning model for the machine learning task is a single-task model that has been trained to perform the machine learning task; For each of the plurality of machine learning tasks, determining a weighted combination of the teacher output of the machine learning task and the true output of the first training input according to a weight value defining a weight between the teacher output and the true output; processing the training input using the student machine learning model and according to the student parameters to generate a corresponding student output for each of the plurality of machine learning tasks; determining a gradient of the student parameters with respect to an objective function, the objective function measuring, for each machine learning task in the plurality of machine learning tasks, an error between the weighted combination of the machine learning task and the student output of the machine learning task; and An update to the student parameter is determined based on the gradient.

13. The system of claim 12, the operations further comprising: Before training, the weight values ​​are initialized.

14. The system of claim 13, the operations further comprising: During the training, the weight values ​​are repeatedly updated to gradually favor the true value output in the weighting.

15. The system of claim 14, wherein: Repeatedly updating the weight value includes: The weight values ​​are repeatedly linearly increased during the training period.

16. The system of claim 12, wherein: For a first machine learning task among the multiple machine learning tasks, the student output, the teacher output, and the true output are probability distributions, and wherein the objective function includes a cross-entropy loss that measures the cross-entropy between the weighted combination of the first machine learning task and the student output of the first machine learning task.

17. The system of claim 12, wherein: A first machine learning task among the multiple machine learning tasks is a regression task, wherein the student output, the teacher output, and the true value output are each an ordered set of one or more regression values, and wherein the objective function includes a distance loss that measures the distance between the weighted combination of the first machine learning task and the student output of the first machine learning task.

18. The system of claim 12, wherein: The student machine learning model is a neural network comprising an encoder neural network shared among all of the multiple tasks and a corresponding output neural network for each of the multiple machine learning tasks.

19. The system of claim 12, wherein: The model input to the student machine learning model is a text sequence in natural language, and wherein the multiple machine learning tasks are different natural language processing tasks that can be performed on the text sequence.

20. One or more non-transitory computer-readable storage media storing instructions that, when executed by one or more computers, cause the one or more computers to perform operations for training a student machine learning model having multiple student parameters to perform multiple machine learning tasks, wherein: The student machine learning model is configured to receive a model input and process the model input according to the student parameters to generate an output, the output comprising a corresponding student output for each of the plurality of machine learning tasks, the operations comprising: The student machine learning model is trained on training data, the training data comprising a plurality of first training inputs and, for each first training input, a corresponding true value output of the first training input for each of the plurality of machine learning tasks, wherein for each first training input, the training comprises: For each machine learning task in the plurality of machine learning tasks, processing the training input using a trained teacher machine learning model for the machine learning task to generate a teacher output for the machine learning task, wherein the trained teacher machine learning model for the machine learning task is a single-task model that has been trained to perform the machine learning task; For each of the plurality of machine learning tasks, determining a weighted combination of the teacher output of the machine learning task and the true output of the first training input according to a weight value defining a weight between the teacher output and the true output; processing the training input using the student machine learning model and according to the student parameters to generate a corresponding student output for each of the plurality of machine learning tasks; determining a gradient of the student parameters with respect to an objective function, the objective function measuring, for each machine learning task in the plurality of machine learning tasks, an error between the weighted combination of the machine learning task and the student output of the machine learning task; and An update to the student parameter is determined based on the gradient.

Images