Framework for learning transfer learning

Abstract

A method (400) includes receiving a source dataset (104) and a target dataset (106), and identifying a loss function (201) for a deep learning model (150) based on the source dataset and the target dataset. The loss function (201) includes encoder weights (210), source classifier layer weights (202), target classifier layer weights (204), a coefficient (206), and policy weights (208). During a first phase of each of a plurality of learning iterations of a learning-to-learn (L2TL) architecture (200), the method further includes applying gradient descent-based optimization to learn the encoder weights, the source classifier layer weights, and the target classifier weights that minimize the loss function, and determining the coefficient by sampling actions of a policy model (109). During a second phase of each of the plurality of learning iterations, the policy weights that maximize an evaluation metric (220) are determined.

Description

Technical Field

[0001] This disclosure relates to a framework for learning transfer learning. Background Art

[0002] A machine learning model receives input and produces an output, such as a predicted output, based on the received input. Some machine learning models are parametric models and produce an output based on the received input and the values ​​of the model's parameters.

[0003] Some machine learning models are deep models that use multiple layers to produce outputs for received inputs. For example, a deep neural network is a deep machine learning model that includes an output layer and one or more hidden layers, each of which applies a nonlinear transformation to the received input to produce an output. Summary of the Invention

[0004] One aspect of the present disclosure provides a method for adaptively learning the weight distribution of a deep learning model, wherein the deep learning model is jointly optimized for a source dataset and a target dataset. The method includes receiving a source dataset and a target dataset at data processing hardware, and identifying, by the data processing hardware, a loss function for the deep learning model based on the source dataset and the target dataset. The loss function includes encoder weights, source classifier layer weights, target classifier layer weights, coefficients, and policy weights. During a first phase of each of a plurality of learning iterations of a learning-to-transfer learning (L2TL) architecture configured to learn the weight distribution of the deep learning model, the method further includes: applying, by the data processing hardware, a gradient descent-based optimization to learn the encoder weights, source classifier layer weights, and target classifier layer weights that minimize the loss function; the data processing hardware determines the coefficients by sampling actions of the policy model. During a second phase of each of a plurality of learning iterations for the L2TL architecture, the method further includes determining, by the data processing hardware, a policy weight that maximizes an evaluation metric of the loss function.

[0005] Implementations of the present disclosure may include one or more of the following optional features. In some implementations, determining policy weights that maximize an evaluation metric for a loss function includes using encoder weights and target classifier layer weights learned during a first phase. The policy model may be fixed when performing the first phase of a learning iteration. The policy model may include a reinforcement learning-based policy model. The evaluation metric for the loss function may quantify the performance of the deep learning model in a target evaluation dataset that includes a subset of data samples in the target dataset that have not been previously seen by the deep learning model.

[0006] In some examples, during a first phase of each of a plurality of learning iterations, the method further comprises: sampling, by data processing hardware, a training batch of source data samples from a source dataset having a specific size; and selecting, by the data processing hardware, source data samples from the training batch of source data samples, the source data samples having the N best confidence scores for training the deep learning model to learn the encoder weights, source classifier layer weights, and target classifier layer weights that minimize the loss function. In another example, during a second phase of each of a plurality of learning iterations, the method further comprises, by the data processing hardware, training the policy model using a policy gradient on a target evaluation dataset to calculate a reward that maximizes the evaluation metric. In these additional examples, determining the policy weights that maximize the evaluation metric of the loss function is based on the calculated reward.

[0007] In some embodiments, a source dataset includes a first plurality of images, and a target dataset includes a second plurality of images. Here, the number of images in the first plurality of images of the source dataset is greater than the number of images in the second plurality of images of the target dataset. The L2TL architecture may include an encoder network layer, a source classifier layer, and a target classifier layer.

[0008] Another aspect of the present disclosure provides a system for adaptively learning weight distribution for a deep learning model that is jointly optimized for a source dataset and a target dataset. The system includes data processing hardware and memory hardware in communication with the data processing hardware of a user device. The memory hardware stores instructions that, when executed on the data processing hardware, cause the data processing hardware to: receive a source dataset and a target dataset, and identify a loss function for a deep learning model based on the source dataset and the target dataset. The loss function includes encoder weights, source classifier layer weights, target classifier layer weights, coefficients, and a policy weight. During a first phase of each of a plurality of learning iterations of a learning-to-transfer learning (L2TL) architecture configured to learn the weight distribution for the deep learning model, the operations further include: applying a gradient descent-based optimization to learn the encoder weights, the source classifier layer weights, and the target classifier layer weights that minimize the loss function; and determining the coefficients by sampling actions of a policy model. During a second phase of each of a plurality of learning iterations for the L2TL architecture, the operations further include determining a policy weight that maximizes an evaluation metric of the loss function.

[0009] This aspect may include one or more of the following optional features. In some embodiments, determining the policy weights that maximize the evaluation metric of the loss function includes using encoder weights and target classifier layer weights learned during the first phase. When performing the first phase of the learning iteration, the policy model can be fixed. The policy model may include a reinforcement learning-based policy model. The evaluation metric of the loss function can quantify the performance of the deep learning model in a target evaluation dataset, wherein the target evaluation dataset includes a subset of data samples in the target dataset that have not been previously seen by the deep learning model.

[0010] In some examples, during a first phase of each of a plurality of learning iterations, the operations further include: sampling a training batch of source data samples from a source dataset having a specific size; selecting the source data samples from the training batch of source data samples, the source data samples having the N best confidence scores for training the deep learning model to learn the encoder weights, the source classifier layer weights, and the target classifier layer weights that minimize the loss function. In other examples, during a second phase of each of a plurality of learning iterations, the operations further include training the policy model using a policy gradient on a target evaluation dataset to calculate a reward that maximizes an evaluation metric. In these additional examples, determining the policy weights that maximize the evaluation metric of the loss function is based on the calculated reward.

[0011] In some embodiments, a source dataset includes a first plurality of images, and a target dataset includes a second plurality of images. Here, the number of images in the first plurality of images of the source dataset is greater than the number of images in the second plurality of images of the target dataset. The L2TL architecture may include an encoder network layer, a source classifier layer, and a target classifier layer.

[0012] The details of one or more embodiments of the disclosure are set forth in the accompanying drawings and the description below. Other aspects, features, and advantages will be apparent from the description and drawings, and from the claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 Schematic diagram of a system including a learning-to-transfer learning (L2TL) architecture and a deep learning model jointly optimized for a source dataset and a training dataset.

[0014] Figure 2 is a diagram of an example of a transfer learning architecture that improves transfer learning on a target dataset by judiciously extracting information from a source dataset.

[0015] Figure 3is a diagram illustrating an example of a transfer learning algorithm describing the first and second stages of each of multiple learning iterations for updating the weights of a deep learning model.

[0016] Figure 4 is an exemplary arrangement of operations for a method for adaptively learning weight distribution of a deep learning model that is jointly optimized for a source dataset and a target dataset.

[0017] Figure 5 is a schematic diagram of an exemplary computing device that can be used to implement the systems and methods described herein.

[0018] Like reference numbers in the various drawings represent like elements. DETAILED DESCRIPTION

[0019] Deep neural networks are superior at understanding images, text, and audio. The performance of deep neural networks improves significantly with more training data. Because applications are diverse and span use cases with small training datasets, traditional training methods are often insufficient to produce high performance. Leveraging additional source datasets and "migrating" relevant information to the target dataset has become very beneficial. Transfer learning, which usually takes the form of obtaining a pre-trained model on a large-scale source dataset and then further training it on the target dataset (called fine-tuning), has become the standard approach for most real-world artificial intelligence applications. Compared to training from random initialization, fine-tuning produces considerable performance improvements and convergence acceleration, as demonstrated for object recognition, semantic segmentation, language understanding, speech synthesis, audio-visual recognition, and language translation.

[0020] Embodiments herein are directed to a learning-to-transfer learning (L2TL) architecture executed on data processing hardware for improving transfer learning on a target dataset by intelligently extracting information from a source dataset. In contrast to transfer learning techniques that use a fixed weighted allocation function to measure the relevance between the source and target domains, the L2TL architecture can adaptively learn an allocation for a deep learning model that is jointly optimized for the source and target datasets. As will become apparent, the learning of adaptive weights for the deep learning model can be guided by the performance of the deep learning model on a target evaluation dataset. Specifically, a target performance metric can be used to quantify performance, whereby the L2TL architecture can directly improve the relevance of the target evaluation performance.

[0021] While optimizing the importance weighting function, one option for choosing the scaling coefficients is to alternate them between (1,0) and (0,1). This will train the source dataset until the policy weights are optimized. Convergence is achieved, and then the target dataset is trained until convergence is achieved using pre-trained weights from the source dataset. In some embodiments, the framework optimizes a single coefficient as a scale-invariant optimization. The policy optimization step is decoupled from the gradient descent-based trainable parameter optimization, and signals are shared between them.

[0022] Embodiments employ two phases in each of multiple learning iterations for an L2TL architecture. During the first phase of each of the multiple learning iterations, the L2TL architecture applies gradient descent-based optimization to learn encoder weights, source classifier layer weights, and target classifier weights that minimize the L2TL architecture's loss function, and also determines coefficients by sampling actions of a policy model. During the first phase, the policy model may be fixed. In some cases, the loss may be skewed when most source dataset samples in a batch are irrelevant, while some batches contain more relevant examples. To address this issue, the L2TL architecture may sample larger batches and dynamically select more relevant examples. At each iteration, the L2TL architecture may sample training batches of a specific size and train using the top samples with the highest confidence scores. This approach may yield computational benefits because gradients are not computed for most source dataset samples until convergence.

[0023] During the second phase for each of multiple learning iterations, the L2TL architecture uses the encoder weights learned during the first phase to optimize the policy weights that maximize the evaluation metric on the target evaluation dataset. . This can be viewed as an RL problem, such that the policy model outputs a single coefficient for optimizing the action and reward of the importance-weighted function. When training the policy model, the L2TL architecture can use policy gradients to maximize the reward for the target dataset using batches of a specific size. The L2TL architecture can use a moving average baseline to reduce variance. The L2TL architecture can use reinforcement learning to compute policy gradients. The L2TL architecture can also optimize policy gradients using gradient-based optimization of stochastic objectives (e.g., the Adam optimizer).

[0024] Reference Figure 1In some embodiments, system 100 includes a computing environment 130 that includes resources 132, such as data processing hardware 134 (e.g., a server or CPU) and / or remote memory hardware 136 that stores instructions that, when executed on data processing hardware 102, cause data processing hardware 102 to perform operations. A learning-to-transfer learning (L2TL) architecture 200 and a deep learning model 150 can reside on resources 132. Typically, deep learning model 150 comprises a deep learning network, i.e., a machine learning model that includes multiple model layers to produce outputs for received inputs. In the illustrated example, deep learning model 150 is jointly optimized for source dataset 104 and target dataset 106, and L2TL architecture 200 is configured to adaptively learn the weight distribution of deep learning model 150.

[0025] Figure 2 An example of an L2TL architecture 200 is shown, executing on data processing hardware 134 and receiving a source dataset 104 and a target dataset 106. Source dataset 104 includes a first plurality of images, while target dataset 106 includes a second plurality of images. The first plurality of images in source dataset 104 includes a greater number of images than the second plurality of images in target dataset 106. In other embodiments, source dataset 104 and target dataset 106 include data samples other than images, such as, but not limited to, audio samples or text samples. Target dataset 106 includes a specialized training sample set, e.g., images of dogs, for training deep learning model 150 to perform a specialized target task, such as identifying images containing dogs. However, source dataset 104 may comprise a very large public dataset of images spanning various classes that may or may not include dogs. Notably, source dataset 104 and target dataset 106 may each be labeled. There may be situations where source samples in source dataset 104 may have features that are implicitly correlated with target samples in target dataset 106 and would benefit the learning process, but they may belong to different classes. For example, consider an example in which a deep learning model 150 is trained to perform classification of bird images. The source dataset 104 may not contain bird images, but may have images of airplanes with similar visual patterns, which would be helpful in training the bird classifier because they share similar visual patterns to learn valuable representations of the original data. The L2TL architecture 200 is designed to automatically handle such cases using its policy learning and can further push performance in ways that manual source dataset selection or fixed domain similarity methods may not be able to handle. The L2TL architecture 200 considers the co-optimization of the deep learning model 150 for the source and target tasks while scaling the constituent loss terms using adaptive weights 202, 204, 208, 210. Figure 2 The dashed boxes in the L2TL structure 200 correspond to trainable functions.

[0026] The L2TL architecture 200 is configured to improve transfer learning to the target dataset 106 by intelligently extracting information from the source dataset 104. The L2TL architecture 200 is configured to identify a loss function 201 for the deep learning network 150 based on the source dataset 104 and the target dataset 106. The loss function 201 may include encoder weights 210 associated with the encoder neural network 152 of the deep learning model 150, source classifier layer weights 202 associated with the source classifier layer 154 of the deep learning model 150, target classifier layer weights 204 associated with the target classifier layer 156 of the deep learning model 150, coefficients 206 for the source dataset 104 and the target dataset 106, and policy weights 208 associated with the policy model 109 of the deep learning model 150. The loss function 201 may be represented as follows.

[0027]

[0028] Where (x, y) is the input and output pair, B S and B T are the source batch size and target batch size, α s [i] and α t [i] is the scaling factor 206 at the i-th iteration, λ is the importance weighting function 212, f S (·; Ω, ζ S ) and f T (·;Ω, ζ T ) is for a model with trainable parameters Ω (encoding weights 210), ζ S (source classifier layer weights 202) and ζ T The encoding function of the source dataset 104 and the target dataset 106 (target classifier layer weights 204). As will become apparent, the learning objective of the loss function 201 represented by equation (1) generalizes to unseen target evaluation datasets 106' via maximization of the performance metric R, 220.

[0029] During the first phase of each of a plurality of learning iterations for the L2TL architecture 200, the L2TL architecture 200 applies gradient descent-based optimization to learn encoder weights Ω 210 that minimize the loss function 201, source classifier layer weights ζ S 202 and target classifier layer weight ζ T 204, and determine the coefficients 206 by sampling the actions of the policy model 109. For example, gradient descent-based optimization is applied to learn the adaptive weights Ω 210,ζ that minimize the loss function 201S 202 and ζ T 204 can be expressed by the following equation.

[0030] (2)

[0031] During the first phase, the policy model 109 remains fixed. Although most batches will contain relevant samples from the source dataset 104, if most of the source dataset samples in the batch are irrelevant (and would ideally receive lower weights), the loss function 201 may be skewed. To mitigate this problem, the first phase can use a larger batch size at each iteration and dynamically select the most relevant examples from it. At each iteration, the L2TL architecture performs a batch size M S · B S The training batches are sampled and the highest of those with the highest weights is used during that iteration B S This approach also yields computational benefits because the gradients are not computed for most of the source dataset samples until convergence.

[0032] Figure 2 An L2TL architecture 200 is shown having a training objective for each of the source dataset 104 and the target dataset 106 during a first phase of a learning iteration, which includes minimizing the gradient between the respective inputs x and their corresponding labels / outputs y of the source dataset 104 and the target dataset 106. Here, the encoder network 152 encodes input features (e.g., images) from the source dataset 104, and the source classifier layer 154 (also referred to as the "source decision layer") uses the encoded input features to output confidence scores, whereby the training objective determines the source dataset classification loss (which corresponds to the first term in equation (1) of the loss function 201) between the confidence scores of the input features and the corresponding labels / outputs y. ). The source dataset classification loss is multiplied by the importance weight 212, which corresponds to the importance weight 212 in equation (1) of the loss function 201. , then the summation function 203 converts the target dataset classification loss (which corresponds to the second term in equation (1) of the loss function 201) ) is aggregated with the product of the source dataset classification loss and the importance weight 212 to determine the aggregated gradient of the loss function 201. Based on the aggregated gradient of the loss function at each iteration, the L2TL architecture 200 determines / updates the encoder weights Ω 210, the source classifier layer weights ζ S 202 and target classifier layer weight ζ T 204, until the source data set 104 converges.

[0033] The encoder network 152 also encodes input features (e.g., images) from the target dataset 106, and the target classifier layer 156 (also called the "target decision layer") uses the encoded features to output confidence scores, whereby the training objective determines the target dataset classification loss between the confidence scores of the input features of the target dataset and the corresponding labels / outputs y. As described above, the sum function 203 aggregates the target dataset classification loss with the product of the source dataset classification loss and the importance weight 208 to determine the aggregated gradient of the loss function 201 .

[0034] During the second phase of each of the multiple learning iterations for the L2TL architecture 200, the L2TL architecture 200 determines the policy weights Φ 208 that maximize the evaluation metric R, 220 of the loss function 201. Here, the L2TL architecture 200 uses the encoder weights Ω 210 and the target classifier layer weights ζ learned during the first phase. T 204 to determine a policy weight Φ 208 that maximizes an evaluation metric R, 220. As used herein, an evaluation metric for a loss function (also referred to as an "evaluation metric") 220 quantifies the performance of the deep learning model 150 (i.e., the encoder network 152 and the target classification layer 156) on the target evaluation dataset 106'. In some examples, the target evaluation dataset 106' includes a subset of data samples (e.g., images) in the target dataset 106 that have not been previously seen by the deep learning model 150 during the first phase of training. Therefore, determining the policy weight Φ 208 includes optimizing the policy weight 208 so that the evaluation metric R on the target evaluation dataset 106' is D’T 220 maximization can be expressed as follows.

[0035] (3)

[0036] where D' Tis an unseen target evaluation dataset 106′, which may include a subset of data samples (input features x, output labels y) in the target dataset 106 that were not previously seen by the deep learning model 150 during the first phase. When the policy model 109 is fixed during the first phase, during the second phase, the L2TL architecture 200 trains the policy model 109 using policy gradients on the target evaluation dataset 106′ to calculate a reward that maximizes the evaluation metric 220 using equation (3). Therefore, the L2TL architecture 200 determines / updates the policy weights Φ 208 that maximize the evaluation metric R 220 based on the calculated reward. Based on the updated policy weights Φ 208, the importance weights 212 that will be applied as multipliers to the source dataset classification loss to be determined during the first phase of the subsequent learning iteration are updated by sampling the actions of the policy model 109. Therefore, the L2TL architecture 200 uses the policy model 109 to learn the encoder weights Ω 210, the source classifier layer weights ζ S 202 and target classifier layer weight ζ T 204 and determines / updates the policy weight Φ 208 that maximizes the evaluation metric R 220 on the target evaluation dataset 106'. As a result, the L2TL architecture 200 outperforms general relevance by directly targeting specific relevance that improves the performance of the target evaluation dataset 106'. Figure 3 Provides a description for updating the weights Ω 210, ζ S 202, ζ T 204 and Φ 208 of an example L2TL algorithm for the first and second phases of each of a plurality of learning iterations.

[0037] Effectively, the policy model 109 adaptively outputs importance weights 212 for each of the multiple classes in the source dataset 104, thereby enabling the L2TL architecture 200 to learn the source dataset class weights to directly optimize the performance of the target dataset 106. Utilizing the adaptive assignment of importance weights 212, the L2TL architecture 200 assigns higher weights to more relevant data samples in the source dataset 104, thereby extracting relevant information from the source dataset 104 to train the deep learning model 150 to learn the object target. The policy model 109 may include a reinforcement learning-based policy model. Furthermore, in some examples, the L2TL architecture 200 generates a ranking of the samples in the source dataset 104 based on their contribution to the target task.

[0038] Figure 4 A flow chart of an example arrangement of operations of a method 400 for adaptively learning a weight distribution of a deep learning model 150 that is jointly optimized for a source dataset 104 and a target dataset 106 is provided. Figure 1 The operations of the method 400 can be performed by executing instructions stored on the memory hardware 136 Figure 1 At operation 402, the method 400 includes receiving the source dataset 104 and the target dataset 106. At operation 404, the method 400 includes identifying the loss function 201 of the deep learning model 150 based on the source dataset 104 and the target dataset 106. The loss function 201 includes the encoder weights 210, the source classifier layer weights 202, the target classifier layer weights 204, the coefficients 206, and the policy weights 208.

[0039] During a first phase of each of a plurality of learning iterations of a learning-to- transfer-learning (L2TL) architecture 200 configured to learn the weight assignments of the deep learning model 150, the method 400 further includes, at operation 406, applying gradient descent-based optimization to learn the encoder weights, the source classifier layer weights, and the target classifier weights that minimize the loss function, and at operation 408, determining the coefficients 206 by sampling actions of the policy model 109. During a second phase of each of the plurality of learning iterations of the L2TL architecture 200, the method 400 further includes, at operation 410, determining the policy weights 208 that maximize the evaluation metric 220 of the loss function 201.

[0040] A software application (i.e., a software resource) can refer to computer software that causes a computing device to perform a task. In some examples, a software application can be referred to as an “application,” an “app,” or a “program.” Example applications include, but are not limited to, system diagnostic applications, system management applications, system maintenance applications, word processing applications, spreadsheet applications, messaging applications, media streaming applications, social networking applications, and gaming applications.

[0041] A non-transitory memory can be a physical device that is used to temporarily or permanently store programs (e.g., sequences of instructions) or data (e.g., program state information) used by a computing device. A non-transitory memory can be a volatile and / or non-volatile addressable semiconductor memory. Examples of non-volatile memory include, but are not limited to, flash memory and read-only memory (ROM) / programmable read-only memory (PROM) / erasable programmable read-only memory (EPROM) / electrically erasable programmable read-only memory (EEPROM) (e.g., commonly used for firmware such as boot programs). Examples of volatile memory include, but are not limited to, random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), phase change memory (PCM), and disks or tapes.

[0042] Figure 5is a schematic diagram of an exemplary computing device 500 that can be used to implement the systems and methods described in this document. Computing device 500 is intended to represent various forms of digital computers, such as laptops, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The components shown here, their connections and relationships, and their functions are merely exemplary and are not intended to limit the embodiments of the invention described and / or claimed in this document.

[0043] Computing device 500 includes a processor 510 (e.g., data processing hardware), memory 520 (e.g., memory hardware), storage device (e.g., memory hardware) 530, a high-speed interface / controller 540 connected to memory 520 and a high-speed expansion port 550, and a low-speed interface / controller 560 connected to a low-speed bus 570 and storage device 530. Each of components 510, 520, 530, 540, 550, and 560 is interconnected using various buses and can be mounted on a common motherboard or otherwise as appropriate. Processor 510 can process instructions for execution within computing device 500, including instructions stored in memory 520 or on storage device 530, to display graphical information of a graphical user interface (GUI) on an external input / output device (e.g., a display 580 coupled to high-speed interface 540). In other embodiments, multiple processors and / or multiple buses, as well as multiple memories and memory types, can be used as appropriate. Furthermore, multiple computing devices 500 may be connected, with each device providing portions of the necessary operations (e.g., as a server bank, a group of blade servers, or a multi-processor system). Processor 510 may correspond to Figure 1 data processing hardware 134 and executes the L2TL architecture 200 and the deep learning model 150.

[0044] Memory 520 stores information non-transitorily within computing device 500. Memory 520 can be a computer-readable medium, a volatile memory unit, or a non-volatile memory unit. Non-transitory memory 520 can be a physical device used to temporarily or permanently store programs (e.g., instruction sequences) or data (e.g., program state information) used by computing device 500. Examples of non-volatile memory include (but are not limited to) flash memory and read-only memory (ROM) / programmable read-only memory (PROM) / erasable programmable read-only memory (EPROM) / electrically erasable programmable read-only memory (EEPROM) (e.g., commonly used for firmware, such as boot programs). Examples of volatile memory include, but are not limited to, random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), phase change memory (PCM), and disk or tape.

[0045] The storage device 530 can provide mass storage for the computing device 500. In some embodiments, the storage device 530 is a computer-readable medium. In various embodiments, the storage device 530 can be a floppy disk device, a hard disk device, an optical disk device, or a magnetic tape device, a flash memory or other similar solid-state memory device, or an array of devices, including devices in a storage area network or other configuration. In another embodiment, a computer program product is tangibly embodied in an information carrier. The computer program product includes instructions that, when executed, perform one or more methods, such as those described above. The information carrier is a computer or machine-readable medium, such as the memory 520, the storage device 530, or a memory on the processor 510.

[0046] The high-speed controller 540 manages bandwidth-intensive operations of the computing device 500, while the low-speed controller 560 manages less bandwidth-intensive operations. This allocation of responsibilities is exemplary only. In some embodiments, the high-speed controller 540 is coupled to the memory 520, the display 580 (e.g., via a graphics processor or accelerator), and the high-speed expansion port 550, which can accept various expansion cards (not shown). In some embodiments, the low-speed controller 560 is coupled to the storage device 530 and the low-speed expansion port 590. The low-speed expansion port 590, which can include various communication ports (e.g., USB, Bluetooth, Ethernet, wireless Ethernet), can be coupled to one or more input / output devices, such as a keyboard, a pointing device, a scanner, or a networking device, such as a switch or router, for example, via a network adapter.

[0047] The computing device 500 can be implemented in a variety of different forms, as shown. For example, it can be implemented as a standard server 500a, or multiple times as a laptop computer 500b in a group of such servers 500a, or as part of a rack server system 500c.

[0048] Various embodiments of the systems and techniques described herein can be implemented in digital electronic and / or optical circuitry, integrated circuits, specially designed ASICs (application-specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include implementation in one or more computer programs executable and / or interpretable on a programmable system comprising at least one programmable processor, which can be either special purpose or general purpose, coupled to receive data and instructions from and send data and instructions to a storage system, at least one input device, and at least one output device.

[0049] These computer programs (also referred to as programs, software, software applications, or code) include machine instructions for a programmable processor and may be implemented in high-level procedural and / or object-oriented programming languages ​​and / or in assembly / machine language. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, non-transitory computer-readable medium, device, and / or apparatus (e.g., disk, optical disk, memory, programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including machine-readable media that receive machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal used to provide machine instructions and / or data to a programmable processor.

[0050] The processes and logic flows described in this specification can be performed by one or more programmable processors (also referred to as data processing hardware), executing one or more computer programs to perform functions by operating on input data and generating outputs. The processes and logic flows can also be performed by dedicated logic circuits, such as FPGAs (field programmable gate arrays) or ASICs (application-specific integrated circuits). For example, processors suitable for executing computer programs include general-purpose and special-purpose microprocessors, and any one or more processors of any type of digital computer. Typically, a processor will receive instructions and data from a read-only memory or a random access memory or both. The basic elements of a computer are a processor for executing instructions and one or more memory devices for storing instructions and data. Typically, a computer will also include or be operably coupled to receive data from or transfer data to one or more mass storage devices, which are used to store data, such as magnetic disks, magneto-optical disks, or optical disks. However, a computer does not necessarily have such devices. Computer-readable media suitable for storing computer program instructions and data include all forms of nonvolatile memory, media, and storage devices, including, for example, semiconductor memory devices, such as EPROM, EEPROM, and flash memory devices; magnetic disks, such as internal hard disks or removable disks; magneto-optical disks; CD ROMs and DVD-ROM disks. The processor and memory can be supplemented by, or incorporated in, special purpose logic circuitry.

[0051] To provide for user interaction, one or more aspects of the present disclosure may be implemented on a computer having a display device, such as a CRT (cathode ray tube), LCD (liquid crystal display) monitor, or touch screen for displaying information to the user, and optionally a keyboard and a pointing device, such as a mouse or trackball, through which the user can provide input to the computer. Other types of devices may also be used to provide for user interaction; for example, feedback provided to the user may be any form of sensory feedback, such as visual feedback, auditory feedback, or tactile feedback; and input from the user may be received in any form, including sound, voice, or tactile input. Furthermore, a computer may interact with a user by sending documents to and receiving documents from a device used by the user; for example, by sending a web page to a web browser on a user's client device in response to a request received from the web browser.

[0052] A number of embodiments have been described. However, it will be appreciated that various modifications may be made without departing from the spirit and scope of the present disclosure. Accordingly, other embodiments are within the scope of the appended claims.

Claims

1. A method (400) for learning transfer learning, characterized in that include: receiving, at data processing hardware (134), a source dataset (104) of source data samples and a target dataset (106) of target data samples, the source dataset (104) and the target dataset (106) comprising images, text, or audio; The data processing hardware (134) trains a deep learning model (150) based on the source data set (104) and the target data set (106) using a loss function (201) for transfer learning, wherein the loss function (201) includes: a source dataset classification loss based on source classifier layer weights (202) of a source classifier layer (154) of the deep learning model and encoder weights (210) of an encoder network layer (152) of the deep learning model, the encoder network layer being configured to encode the source data samples of the source dataset and the target data samples of the target dataset, the source classifier layer being configured to generate a source confidence score based on the encoded source data samples of the source dataset encoded by the encoder network layer; a target dataset classification loss based on the encoder weights (210) and a target classifier layer weights (204) of a target classifier layer (156) of the deep learning model, the target classifier layer weights being different from the source classifier layer weights, the target classifier layer being different from the source classifier layer and being configured to generate a target confidence score based on an encoded target data sample of the target dataset encoded by the encoder network layer; one or more scaling factors (206); and Strategy Weight (208); During a first phase of each of a plurality of learning iterations of a learning-to-transfer-learning (L2TL) architecture (200) configured to learn a weight distribution of the deep learning model (150): Sampling the source data samples of the training batch; selecting the source data samples from the training batch, the source data samples having N best source confidence scores; Sampling the target data samples of the training batch; determining, by the data processing hardware (134), the one or more scaling factors (206) by sampling actions of the policy model (109); and applying, by the data processing hardware (134), a gradient descent-based optimization to learn the encoder weights (210), the source classifier layer weights (202), and the target classifier layer weights (204) that minimize the loss function (201) based on the selected source data samples, the target data samples of the training batch, and the one or more scaling factors (206); and During a second phase of each of the plurality of learning iterations of the L2TL architecture (200), the policy weights (208) that maximize an evaluation metric (220) are determined by the data processing hardware (134), the evaluation metric (220) representing the performance of the deep learning model (150) on a target evaluation dataset (106').

2. The method (400) according to claim 1, characterized in that While performing the first phase of the learning iteration, the policy model (109) is fixed.

3. The method (400) according to claim 2, characterized in that The policy model (109) includes a policy model based on reinforcement learning.

4. The method (400) according to claim 3, characterized in that Determining the policy weights (208) that maximize the evaluation metric (220) of the loss function (201) includes using the encoder weights (210) and the target classifier layer weights (204) learned during the first phase.

5. The method (400) according to claim 4, characterized in that The evaluation metric (220) of the loss function (201) quantifies the performance of the deep learning model (150) on the target evaluation dataset (106'), wherein the target evaluation dataset (106') includes a subset of data samples in the target dataset (106) that have not been previously seen by the deep learning model (150).

6. The method (400) according to claim 1, characterized in that Also included is, during the second phase of each of the plurality of learning iterations: training the policy model (109) by the data processing hardware using policy gradients on a target evaluation dataset (106') to compute a reward that maximizes the evaluation metric (220), Wherein determining the policy weight (208) that maximizes the evaluation metric (220) of the loss function (201) is based on the calculated reward.

7. The method (400) according to claim 6, characterized in that in: The source dataset (104) includes a first plurality of images; and The target dataset (106) includes a second plurality of images.

8. The method (400) according to claim 7, characterized in that The number of images in the first plurality of images of the source dataset (104) is greater than the number of images in the second plurality of images of the target dataset (106).

9. The method (400) according to any one of claims 1 to 8, characterized in that The L2TL architecture (200) includes the encoder network layer (152), the source classifier layer (154) and the target classifier layer (156).

10. A system (100) for learning transfer learning, characterized in that include: Data processing hardware (134); and Memory hardware (136) in communication with the data processing hardware (134) and storing instructions that, when executed on the data processing hardware (134), cause the data processing hardware (134) to: receiving a source dataset (104) of source data samples and a target dataset (106) of target data samples, the source dataset (104) and the target dataset (106) comprising images, text, or audio; Based on the source dataset (104) and the target dataset (106), a deep learning model (150) is trained using a loss function (201) for transfer learning, wherein the loss function (201) includes: a source dataset classification loss based on source classifier layer weights (202) of a source classifier layer (154) of the deep learning model and encoder weights (210) of an encoder network layer (152) of the deep learning model, the encoder network layer being configured to encode the source data samples of the source dataset and the target data samples of the target dataset, the source classifier layer being configured to generate a source confidence score based on the encoded source data samples of the source dataset encoded by the encoder network layer; a target dataset classification loss based on the encoder weights (210) and a target classifier layer weights (204) of a target classifier layer (156) of the deep learning model, the target classifier layer weights being different from the source classifier layer weights, the target classifier layer being different from the source classifier layer and being configured to generate a target confidence score based on an encoded target data sample of the target dataset encoded by the encoder network layer; one or more scaling factors (206); and Strategy Weight (208); During a first phase of each of a plurality of learning iterations of a learning-to-transfer-learning (L2TL) architecture (200) configured to learn a weight distribution of the deep learning model (150): Sampling the source data samples of the training batch; selecting the source data samples from the training batch, the source data samples having N best source confidence scores; Sampling the target data samples of the training batch; determining the one or more scaling factors (206) by sampling actions of the policy model (109); and applying gradient descent-based optimization to learn the encoder weights (210), the source classifier layer weights (202), and the target classifier layer weights (204) that minimize the loss function (201) based on the selected source data samples, the target data samples of the training batch, and the one or more scaling factors (206); and During a second phase of each of the plurality of learning iterations of the L2TL architecture (200), the policy weights (208) are determined that maximize an evaluation metric (220), the evaluation metric (220) representing the performance of the deep learning model (150) on a target evaluation dataset (106').

11. The system (100) according to claim 10, characterized in that While performing the first phase of the learning iteration, the policy model (109) is fixed.

12. The system (100) according to claim 11, characterized in that The policy model (109) includes a policy model based on reinforcement learning.

13. The system (100) according to claim 12, characterized in that To determine the policy weights (208) that maximize the evaluation metric (220) of the loss function (201), the instructions further cause the data processing hardware to use the encoder weights (210) and the target classifier layer weights (204) learned during the first phase.

14. The system (100) according to claim 13, characterized in that The evaluation metric (220) of the loss function (201) quantifies the performance of the deep learning model (150) on the target evaluation dataset (106'), wherein the target evaluation dataset (106') includes a subset of data samples in the target dataset (106) that have not been previously seen by the deep learning model (150).

15. The system (100) according to claim 10, characterized in that During the second phase of each of the plurality of learning iterations, the instructions further cause the data processing hardware to further include: training the policy model (109) using policy gradients on a target evaluation dataset (106') to compute a reward that maximizes the evaluation metric (220), Wherein determining the policy weight (208) that maximizes the evaluation metric (220) of the loss function (201) is based on the calculated reward.

16. The system (100) according to claim 15, characterized in that in: The source dataset (104) includes a first plurality of images; and The target dataset (106) includes a second plurality of images.

17. The system (100) according to claim 16, characterized in that The number of images in the first plurality of images of the source dataset (104) is greater than the number of images in the second plurality of images of the target dataset (106).

18. The system (100) according to any one of claims 10-17, characterized in that The L2TL architecture (200) includes the encoder network layer (152), the source classifier layer (154), and the target classifier layer (156).

Images