Learning data enhancement policies

Abstract

The invention relates to learning data enhancement policies. Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for learning data enhancement policies for training machine learning models. In one aspect, a method includes receiving training data for training a machine learning model to perform a particular machine learning task; determining a plurality of data enhancement policies, including at each of the plurality of time steps: generating a current data enhancement policy based on a quality metric of the data enhancement policy generated at a previous time step; training a machine learning model on the training data by using the current data enhancement strategy; and, after the machine learning model has been trained using the current data enhancement policy, determining a quality metric for the current data enhancement policy using the machine learning model; and selecting a final data enhancement policy based on the determined quality metric for the data enhancement policy.

Description

Description of the case

[0001] This application is a divisional application of Chinese invention patent application No. 201980014733.1, filed on May 20, 2019. Technical Field

[0002] This specification relates to processing data using machine learning models. Background Art

[0003] A machine learning model receives an input and generates an output, e.g., a predicted output, based on the received input. Some machine learning models are parametric models and generate an output based on the received input and the values ​​of the model's parameters.

[0004] Some machine learning models are deep models that use a multi-layer model to generate an output for a received input. For example, a deep neural network is a deep machine learning model that includes an output layer and one or more hidden layers, each of which applies a nonlinear transformation to a received input to generate an output. Summary of the invention

[0005] This specification describes a training system implemented as a computer program on one or more computers in one or more locations.

[0006] According to a first aspect, a method is provided, comprising receiving training data for training a machine learning model to perform a specific machine learning task. The training data comprises a plurality of training inputs. A plurality of data augmentation strategies are determined, wherein each data augmentation strategy has a plurality of data augmentation strategy parameters, and the plurality of data augmentation strategy parameters define a process for transforming the training inputs before the training inputs are used to train the machine learning model.

[0007] At each of a plurality of time steps, a current data augmentation policy is generated based on a quality metric of the data augmentation policy generated at a previous time step. The quality metric of the data augmentation policy represents the performance of the machine learning model for a particular machine learning task as a result of training the machine learning model using the data augmentation policy. Using the current data augmentation policy, the machine learning model is trained on the training data. Training the machine learning model using the data augmentation policy includes: selecting a batch of training data; determining an enhanced batch of training data by transforming the training inputs in the batch of training data according to the data augmentation policy; and adjusting current values ​​of machine learning model parameters based on the enhanced batch of training data. After training the machine learning model using the current data augmentation policy, the quality metric of the current data augmentation policy is determined using the machine learning model.

[0008] Based on the quality metric of the determined data augmentation strategy, a final data augmentation strategy is selected, and a final trained machine learning model is generated by training the final machine learning model using the final data augmentation strategy.

[0009] In some embodiments, the specific machine learning task is an image processing task including classification or regression.

[0010] In some embodiments, each data augmentation strategy includes one or more sub-strategies. Each sub-strategy includes a sequence of one or more transformation tuples, wherein, for each transformation tuple, a data augmentation strategy parameter defines: (i) a transformation operation, and (ii) a size of the transformation operation. According to the data augmentation strategy, transforming training inputs in a batch of training data includes, for each training input: identifying a sub-strategy included in the data augmentation strategy; and transforming the training input according to the identified sub-strategy by sequentially applying each transformation tuple included in the identified sub-strategy to the training input.

[0011] In some implementations, identifying a sub-strategy included in the data augmentation strategy for the training input includes randomly sampling a sub-strategy included in the data augmentation strategy.

[0012] In some implementations, applying the transform tuple to the training input includes applying the transform operations in the transform tuple to the training input at a transform operation size in the transform tuple.

[0013] In some embodiments, for each transformation tuple, the data augmentation strategy parameters further define a probability of applying a transformation operation; and applying the transformation tuple to the training input includes: applying the transformation operations in the transformation tuple to the training input with the transformation probability in the transformation tuple and the transformation operation size in the transformation tuple.

[0014] In some embodiments, the machine learning model is a neural network, and adjusting current values ​​of machine learning model parameters based on an enhanced batch of training data includes: determining the gradient of a loss function using the enhanced batch of training data; and using the gradient to adjust current values ​​of the machine learning model parameters.

[0015] In some embodiments, generating a current data augmentation policy based on a quality metric of a data augmentation policy generated at a previous time step includes generating the current data augmentation policy using a policy neural network according to current values ​​of policy neural network parameters; and training the policy neural network through reinforcement learning techniques, and at each time step, the reinforcement learning reward signal is based on the quality metric of the current data augmentation policy at the time step.

[0016] In some implementations, for each data augmentation policy parameter, the policy network outputs a score distribution that defines possible values ​​for the data augmentation policy parameter.

[0017] In some implementations, determining the current data augmentation strategy from the policy network output includes sampling a value for each data augmentation strategy parameter using a score distribution for the data augmentation strategy parameter.

[0018] In some embodiments, the policy neural network is a recurrent neural network.

[0019] In some implementations, generating the current data augmentation strategy based on a quality metric of a data augmentation strategy generated at a previous time step includes generating the current data augmentation strategy using a genetic programming process.

[0020] In some embodiments, after a machine learning model has been trained using a current data augmentation strategy, using the machine learning model to determine a quality metric of the current data augmentation strategy includes: determining a performance metric of the machine learning model for a specific machine learning task using validation data comprising multiple training inputs; and determining a quality metric based on the performance metric.

[0021] In some implementations, training inputs included in the validation data are not included in the training data.

[0022] In some implementations, selecting the final data augmentation strategy based on the quality metric of the determined data augmentation strategies includes selecting the determined data augmentation strategy with the highest quality score.

[0023] In some implementations, the training input is an image.

[0024] According to a second aspect, there is provided a system comprising one or more computers and one or more storage devices storing instructions which, when executed by the one or more computers, cause the one or more computers to perform operations including the operations of the first aspect.

[0025] According to a third aspect, one or more computer storage media storing instructions are provided that, when executed by one or more computers, cause the one or more computers to perform operations including the operations of the first aspect.

[0026] Particular embodiments of the subject matter described in this specification can be implemented to realize one or more of the following advantages.

[0027] The training system described in this specification can use a "learned" data enhancement strategy instead of a manually designed data enhancement strategy to train a machine learning model. The training system can identify a specific data enhancement strategy by automatically searching the space of possible data enhancement strategies to learn a data enhancement strategy, which enables a machine learning model trained using the specific data enhancement strategy to effectively perform a machine learning task.

[0028] A machine learning model trained using a learned data augmentation policy may perform better (e.g., by achieving higher prediction accuracy) than a machine learning model trained using a manually designed data augmentation policy. Furthermore, using a learned data augmentation policy may enable a machine learning model to be trained to achieve an acceptable level of performance using less training data than would otherwise be required, thereby enabling more efficient use of computing resources (e.g., memory).

[0029] The data enhancement strategies learned by the training system described in this specification can be transferred between training data sets in some cases. That is, the data enhancement strategies learned with reference to the first training data set can be used to effectively train the machine learning model on the second training data set (i.e., even if the data enhancement strategies are not learned with reference to the second training data set). The transferability of the data enhancement strategies learned by the training system can reduce the consumption of computing resources, for example, by enabling the learned data enhancement strategies to be reused on new training data sets, rather than learning new data enhancement strategies for new training data sets.

[0030] The details of one or more embodiments of the subject matter of this specification are set forth in the following drawings and description. Other features, aspects, and advantages of the subject matter will become apparent from the description, drawings, and claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 An exemplary training system is shown.

[0032] Figure 2 An exemplary policy generation engine is shown.

[0033] Figure 3 is a diagram of an exemplary data augmentation strategy.

[0034] Figure 4 is an illustration of the effect of applying different sub-strategies of the data augmentation strategy to the original image.

[0035] Figure 5 Examples of performance gains that can be achieved by using a training system are shown.

[0036] Figure 6 Describes the different types of transformation operations that can be applied to images.

[0037] Figure 7 is a flow chart of an exemplary process for automatically selecting a data augmentation strategy and using the data augmentation strategy to train a machine learning model.

[0038] Like reference numbers and designations throughout the various drawings represent like elements. DETAILED DESCRIPTION

[0039] This specification describes a training system implemented as a computer program on one or more computers in one or more locations that trains a machine learning model to perform a machine learning task using a data augmentation strategy. The data augmentation strategy can be used to increase the number and diversity of training inputs used in training the machine learning model, thereby resulting in a trained machine learning model that performs the machine learning task more efficiently (e.g., through higher prediction accuracy). The training system automatically searches the space of possible data augmentation strategies to identify a specific data augmentation strategy, and then trains the machine learning model using the specific data augmentation strategy.

[0040] Figure 1 An exemplary training system 100 is shown. Training system 100 is an example of a system implemented as a computer program on one or more computers in one or more locations in which the systems, components, and techniques described below may be implemented.

[0041] The training system 100 is configured to generate a trained machine learning model 102 that is trained to perform a machine learning task using the following training machine learning model 104: (i) a training data set 106, and (ii) a "final" data augmentation strategy 108. As described in more detail below, the training system 100 identifies the final data augmentation strategy 108 by searching a space of possible data augmentation strategies.

[0042] The machine learning task may involve processing an image to generate a prediction based on the image. That is, the machine learning model 104 may be configured to process an input including an image to generate a corresponding output, such as a classification output, a regression output, or a combination thereof. Below are several examples of machine learning tasks.

[0043] In one example, the machine learning model is configured to process an image to generate a classification output that includes a corresponding score for each of a plurality of categories. The score for a category indicates the likelihood that the image belongs to the category. In some cases, the category may be an object class (e.g., dog, cat, person, etc.), and if the image depicts an object included in the object class corresponding to the category, the image may belong to the category. In some cases, the category may represent a global image attribute (e.g., an image depicting a daytime or nighttime scene, or an image depicting a summer or winter scene), and if the image has a global attribute corresponding to the category, it may belong to the category.

[0044] In another example, the machine learning model is configured to process an image to generate a pixel-level classification output that includes, for each pixel, a corresponding score for each of a plurality of categories. For a given pixel, the score for the category indicates the likelihood that the pixel belongs to that category. In some cases, the category may be an object class, and if the pixel belongs to a part of an object included in the object class corresponding to the category, the pixel may belong to the category. That is, the pixel-level classification output may be a semantic segmentation output.

[0045] In another example, the machine learning model is configured to process an image to generate a regression output that estimates one or more continuous variables (i.e., that can assume an infinite number of possible values) that characterize the image. In a specific example, the regression output can estimate the coordinates of a bounding box that encloses each object depicted in the image. The coordinates of the bounding box can be defined by the xy coordinates of the vertices of the bounding box.

[0046] For convenience, this specification mainly describes machine learning tasks as processing images to generate predictions based on the images. More generally, a machine learning task can be any task that involves processing inputs to generate predictions based on the inputs. Some other examples of machine learning tasks are described below.

[0047] In one example, the machine learning task can be a speech recognition task, wherein the machine learning model is configured to process a representation of an audio waveform to generate an output representing a sequence of phonemes, characters, or words corresponding to the audio waveform.

[0048] In another example, the machine learning task may be a video analysis task, wherein a machine learning model is configured to process a sequence of video frames to generate an output that characterizes the video frames, for example, by characterizing whether the video frames depict a person performing a particular action.

[0049] In another example, the machine learning task can be a natural language processing task, wherein a machine learning model is configured to process a portion of text to generate an output that represents the portion of text, for example, by representing a translation of the portion of text into another natural language.

[0050] The training data 106 consists of a plurality of training examples, where each training example specifies a training input and a corresponding target output. The training input includes an image. The target output represents the output that should be generated by the machine learning model by processing the training input. For example, the target output can be a classification output that specifies a category (e.g., an object class) corresponding to the input image, or a regression output that specifies one or more continuous variables (e.g., object bounding box coordinates) corresponding to the input image.

[0051] The machine learning model 104 may have any suitable machine learning model architecture. For example, the machine learning model may be a neural network model, a random forest model, a support vector machine (SVM) model, a linear model, or a combination thereof.

[0052] The system 100 may receive the training data 106 and the data defining the machine learning model 104 in any of a variety of ways. For example, the system 100 may receive the training data 106 and the data defining the machine learning model 104 as an upload from a remote user of the system 100, such as over a data communications network, e.g., using an application programming interface (API) that makes the system 100 available.

[0053] A data augmentation strategy is defined by a set of parameters (referred to in this document as "data augmentation strategy parameters") that specify the process of transforming (i.e., including in the training examples) the training inputs before using them to train a machine learning model. Figure 3 , describing examples of data augmentation strategies and corresponding data augmentation strategy parameters in more detail.

[0054] The process for transforming the training inputs typically includes applying one or more operations (referred to in this document as "transformation operations") to images included in the training inputs. These operations can be any suitable kind of image processing operations, such as translation operations, rotation operations, shearing operations, color inversion operations, or combinations thereof. The data augmentation strategy parameters can specify which transformation operations should be applied, at what magnitude, and with what probability. Reference Figure 6 Examples of possible transformation operations are described in more detail.

[0055] The system 100 implements a search algorithm to identify a data augmentation strategy with a high "quality metric" from the space of possible data augmentation strategies. The quality metric of a data augmentation strategy characterizes the performance (e.g., prediction accuracy) of a machine learning model trained using the data augmentation strategy. For convenience, a higher quality metric is understood herein to imply better performance (e.g., higher prediction accuracy).

[0056] The system 100 can determine a quality metric for a data augmentation strategy by evaluating the performance of a machine learning model trained using the data augmentation strategy on a “validation set.” The validation set consists of one of more training examples that were not used in training the machine learning model. For example, the system 100 can determine the quality metric 110 based on any appropriate performance metric of the machine learning model trained on the validation set, such as (in the case of a classification task) an FI score or Matthews correlation coefficient, or (in the case of a regression task) a squared error or absolute error. For a performance metric (e.g., a squared error performance metric) where lower values ​​indicate better performance of the trained machine learning model, the quality metric for the data augmentation strategy may be inversely proportional to the performance metric (i.e., so that a higher quality metric still implies better performance).

[0057] The system 100 may obtain a validation set, for example, by randomly dividing a larger training data set to generate training data 106 (ie, for training a machine learning model) and a validation set (ie, for evaluating the performance of the trained machine learning model).

[0058] As used throughout this document, the space of possible data augmentation strategies refers to the space parameterized by possible values ​​of the data augmentation strategy parameters.

[0059] The training system 100 includes a training engine 112 and a policy generation engine 114 .

[0060] In each of a plurality of iterations, referred to in this specification as "time steps," the policy generation engine 114 generates one or more "current" data augmentation policies 116. For each current data augmentation policy 116, the system 100 uses the training engine 112 to train the machine learning model 104 using the current data augmentation policy, and thereafter, determines a quality metric 110 for the current data augmentation policy. The policy generation engine 114 uses the quality metric 110 of the current data augmentation policy 116 to improve the expected quality metric of the data augmentation policy generated for the next time step.

[0061] Training a machine learning model refers to determining the training values ​​of the parameters of the machine learning model from the initial values ​​of the parameters of the machine learning model. The training engine 112 can start training the machine learning model 104 from, for example, randomly selected or default initial values ​​of the machine learning model parameters.

[0062] Typically, a data augmentation policy can be used to train a machine learning model by transforming the training inputs of existing training examples to generate "new" training examples, and then training the machine learning model using the new training examples (either in place of or in addition to the existing training examples). For example, an image included in the training inputs of a training example can be transformed by applying one or more image processing operations specified by a data augmentation policy to the image.

[0063] In some cases, the training inputs of a training example may be transformed (e.g., according to a data augmentation strategy) while maintaining the same corresponding target outputs. For example, for an image classification task where the target output specifies the type of object depicted in the training input, applying image processing operations (e.g., rotation, shearing, etc.) to an image included in the training input will not affect the type of object depicted in the image. Thus, in this example, the transformed training input will correspond to the same target output as the original training input.

[0064] However, in some cases, transforming the training input of a training example may also require changing the target output of the training example. In one example, the target output corresponding to the training input may specify the coordinates of a bounding box that encloses an object depicted in an image of the training input. In this example, applying a translation operation to the image of the training input would require applying the same translation operation to the bounding box coordinates specified by the target output.

[0065] The specific operations performed by the training engine 112 to train the machine learning model 104 using the data augmentation strategy depend on the architecture of the machine learning model 104, for example, whether the machine learning model 104 is a neural network model or a random forest model. Figure 7 , describes in more detail an example of training a neural network model using data augmentation strategies.

[0066] The policy generation engine 114 can use any of a variety of techniques to search the space of possible data augmentation policies. In one embodiment, the policy generation engine 114 uses a neural network, referred to herein as a "policy" network, to generate data augmentation policies. The policy network can be trained using reinforcement learning techniques, where a reward is provided by a quality metric corresponding to the data augmentation policy generated by the policy neural network (i.e., the policy network is trained to maximize). Figure 2 An exemplary implementation of the policy generation engine 114 using a policy network trained using reinforcement learning techniques is described in more detail.

[0067] In other embodiments, the strategy generation engine 114 can use any of a variety of other techniques to search the space of possible data enhancement strategies. For example, the strategy generation engine 114 can use, for example, the random search technique described in reference to H. Mania, A. Guy, and B. Recht, "Simple random search provides a competitive approach to reinforcement learning", arXiv:1803.07055v1, 2018. As another example, the strategy generation engine 114 can use, for example, the evolutionary search technique described in reference to E. Real, A. Aggarwal, Y. Huang, and QV Le: "Regularized evolution for image classifier architecture search", arXiv:1802.01548, 2018 (i.e., the "genetic programming process").

[0068] The system 100 may continue to generate data augmentation policies until the search termination criteria are met. For example, if data augmentation policies have been generated for a predetermined number of time steps, the training system 100 may determine that the search termination criteria are met. As another example, if a quality metric of the generated data augmentation policies meets a predetermined threshold, the training system 100 may determine that the search termination criteria are met.

[0069] After determining that the search termination criteria are met, the system 100 selects a final data augmentation strategy based on the corresponding quality metrics 110 of the generated data augmentation strategies. For example, the system 100 may select the data augmentation strategy with the highest quality score generated by the training system 100 as the final data augmentation strategy. Figure 3 Described in more detail, the system 100 may combine a predetermined number (eg, 5) of data augmentation strategies generated by the training system with the highest quality scores to generate a final data augmentation strategy 108 .

[0070] The system 100 may generate a trained machine learning model 102 by training the machine learning model 104 on the training data 106 using a final data augmentation strategy 108 , and then outputting data defining the trained machine learning model 102 .

[0071] In some embodiments, instead of outputting the trained machine learning model 102 or in addition to outputting the trained machine learning model 102, the system 100 uses the trained machine learning model 102 to process a request received by a user, for example, through an API provided by the system 100. That is, the system 100 can receive an input to be processed, process the input using the trained machine learning model 102, and, in response to the received input, provide an output generated by the trained machine learning model 102 or data derived from the generated output.

[0072] Although the training system 100 determines the final data augmentation strategy 108 with respect to a particular training data set 106, the final data augmentation strategy 108 (in some cases) can be transferred to other training data sets. That is, the final data augmentation strategy 108 determined with respect to the training data 106 can be used to effectively train other machine learning models on different training data sets.

[0073] Figure 2 An exemplary policy generation engine 114 is shown. The policy generation engine 114 includes: a policy neural network 202, which is configured to generate a policy network output according to the parameters of the policy network 202; and a parameter update engine 204, which is configured to adjust the policy network parameter values. Each policy network output 206 generated by the policy network 202 defines a corresponding data enhancement strategy 116.

[0074] In particular, each policy network output 206 includes a corresponding output at each of a plurality of output positions, and each output position corresponds to a different data enhancement policy parameter. Thus, each policy network output includes a corresponding value of a corresponding data enhancement policy parameter at each output position. In general, the values ​​of the data enhancement policy parameters specified by a given policy network output define a data enhancement policy.

[0075] At each time step, the policy generation engine 114 uses the policy network 202 to generate one or more policy network outputs 206 based on the current values ​​of the policy network parameters, each policy network output 206 defining a corresponding data augmentation policy 116 .

[0076] The policy network 202 is a recurrent neural network that includes one or more recurrent neural network layers, such as a long short-term memory (LSTM) layer or a gated recurrent unit (GRU) layer. The policy network 202 generates the policy network output 206 by sequentially generating a corresponding data augmentation policy parameter corresponding to each output position in the sequence of output positions of the policy network output 206. In particular, for each output position, the policy network 202 receives as input the value of the data augmentation policy parameter corresponding to the previous output position in the policy network output 206, and processes the input to update the current hidden state of the policy network 202. For the first output position in the policy network output, since there is no previous output position, the policy network 202 can process a predetermined placeholder input.

[0077] Policy network 202 also includes a corresponding output layer for each output position in policy network output 206. The output layer corresponding to the output position processes the updated hidden state of policy network 202 corresponding to the output position to generate an output that defines a score distribution for the possible values ​​of the data augmentation policy parameter corresponding to the output position. For example, the output layer can first project the updated hidden state of policy network 202 into an appropriate dimension for the number of possible values ​​of the corresponding data augmentation policy parameter. The output layer can then apply softmax to the projected hidden state to generate a corresponding score for each of the multiple possible values ​​for the corresponding data augmentation policy parameter.

[0078] For each output position, the policy network 202 generates a value for the corresponding data augmentation policy parameter by sampling from the score distribution of possible values ​​of the data augmentation policy parameter (i.e., generated by the corresponding output layer). The possible values ​​that a given data augmentation policy parameter can take are fixed before training, and the number of possible values ​​can be different for different data augmentation policy parameters.

[0079] Typically, the values ​​of the policy network parameters do not change within a time step. However, by sampling values ​​of the data augmentation policy parameters from a score distribution of possible values, the policy network 202 can generate multiple different policy network outputs 206 at each time step (i.e., each policy network output 206 corresponds to a different data augmentation strategy 116).

[0080] For each data augmentation strategy 116 generated at a time step, the training system 100 trains the machine learning model using the data augmentation strategy 116 and then determines a corresponding quality metric 110 of the trained machine learning model (as described above).

[0081] Then, the parameter update engine 204 uses the quality metric 110 as a reward signal to update the current value of the policy network parameter using reinforcement learning techniques. That is, the parameter update engine 204 adjusts the current value of the policy network parameter by training the policy network using reinforcement learning techniques to generate a policy network output, which results in improving the quality metric of the corresponding data enhancement strategy. More specifically, the parameter update engine 204 trains the policy network to generate a policy network output that maximizes the received reward determined by the quality metric of the corresponding data enhancement strategy. In particular, the reward for a given policy network output is a function of the quality metric of the data enhancement strategy defined by the given policy network output. For example, the reward can be one of the following or directly proportional to it: a quality metric, a square of a quality metric, a cube of a quality metric, a square root of a quality metric, and the like.

[0082] In some cases, parameter update engine 204 uses a policy gradient technique to train policy network 202 to maximize the expected reward. For example, the policy gradient technique can be a REINFORCE technique or a proximal policy optimization (PPO) technique. For example, at a given time step, parameter update engine 204 updates the current value of the policy network parameters using a gradient combination of the policy network parameters given by the following expression:

[0083] Where k indexes the policy network output generated at that time step, K is the total number of policy network outputs generated at that time step, t indexes the output position in the policy network output, T is the total number of output positions in the policy network output, is the gradient operator with respect to the policy network parameters, is the score corresponding to the data augmentation policy parameter generated at output position t in the policy network output k, R k is the reward of the policy network output k, and b is a baseline function, such as an exponential moving average of previous rewards.

[0084] By repeatedly updating the values ​​of the policy network parameters, parameter update engine 204 can train policy network 202 to generate a policy network output that defines a data enhancement strategy that results in improved performance of the trained machine learning model on the machine learning task. That is, updating the values ​​of the policy network parameters in this manner can improve the expected accuracy of the validation set of the machine learning model trained using the data enhancement strategy proposed by the policy network.

[0085] Figure 3300 is a diagram of an exemplary data augmentation strategy 300. The data augmentation strategy 300 is composed of one or more "sub-strategies" 302-A to 302-N. Each sub-strategy specifies a series of transformation operations (e.g., 304-A to 304-M), such as image processing operations, such as translation, rotation, or shear operations. Each transformation operation has an associated magnitude (e.g., 306-A to 306-M) and an associated probability (e.g., 308-A to 308-M).

[0086] The size of a transformation operation is an ordered set of one or more values ​​that specify how the transformation operation is applied to the training input. For example, the size of a translation operation can specify the number of pixels that the image should be translated in the x and y directions. As another example, the size of a rotation operation can specify the number of radians that the image should be rotated.

[0087] To transform a training input using a data augmentation strategy 300, a sub-strategy is selected from a set of sub-strategies included in the data augmentation strategy, such as by randomly sampling sub-strategies. Then, for each transformation operation in a sequence of transformation operations specified by the selected sub-strategy, the transformation operation is applied to the training input with a probability associated with the transformation operation. If a transformation operation is applied to the training input, it is applied with a magnitude associated with the transformation operation. The transformation operations applied to the training input are applied according to the order of the transformation operations specified by the sub-strategy. Figure 4 A diagram of the effect of applying different sub-strategies to an image is shown in FIG.

[0088] The parameters of the exemplary data augmentation strategy 300 include, for each transformation operation of each sub-strategy: (i) the type of transformation operation (e.g., translation, rotation, shear, etc.), (ii) the magnitude of the transformation operation, and (iii) the probability of applying the transformation operation.

[0089] Each data augmentation strategy parameter can have a predetermined set of possible values. For example, the type of transformation operation can be selected from a predetermined set of possible transformation types. The size of the transformation operation can have, for example, a predetermined number of possible values ​​that are evenly spaced throughout a continuous range of allowable values. For example, for a rotation operation, the continuous range of allowable values ​​can be [0, 2π] radians, and the possible values ​​of the size of the rotation operation can be: {0, π / 2, π, 3π / 2}. The probability of applying the transformation operation can have, for example, a predetermined number of possible values ​​that are evenly spaced throughout the range [0, 1]. In one example, the possible values ​​of the probability of applying the transformation operation can be: {0, 0.2, 0.4, 0.6, 0.8, 1}.

[0090] A data augmentation policy can have a predetermined number of sub-policies, and each sub-policy can specify a predetermined number of transformation operations. In some cases, the probability of applying a transformation operation may be 0, resulting in the transformation operation never being applied to the training input. Designing the space of data augmentation policies in this way can increase the diversity of data augmentation policies discovered.

[0091] In a specific example, a data augmentation policy may have 5 sub-policies, each specifying 2 transform operations. Each transform operation has its own scalar size and probability value, and can be any of 16 different transform types. In this example, the data augmentation policy will have a total of 5×2×2×16=320 parameters. If the size of the transform operation has 10 possible values, and the probability of applying the transform operation has 11 possible values, then the space of possible data augmentation policies will include approximately 29×10 32 data augmentation strategy.

[0092] As reference Figure 1 As described above, the training system 100 may combine a predetermined number of data augmentation strategies with the highest quality scores generated by the training system 100 to generate a final data augmentation strategy. Figure 3 The data augmentation policies of the form described can be combined by aggregating their respective sub-policies into a single combined data augmentation policy. To apply the combined data augmentation policy to the training input, one of the sub-policies can be randomly selected from the set of combined sub-policies, and the training input can be transformed according to the sequence of transformation operations specified by the randomly selected sub-policy (as described above).

[0093] Figure 4 4 is an illustration of the effects of applying different sub-strategies to the original image 400. For example, “Sub-strategy 1” specifies a series of transformation operations including: (i) an equalization operation with a corresponding probability of 0.4 and a size of 4, and (ii) a rotation operation with a corresponding probability of 0.8 and a size of 8 degrees. Figure 6 The equalization operation and the rotation operation are described in more detail. The effect of applying sub-strategy 1 to the original image 400 at three different times is illustrated by 402, 404 and 406. Figure 4 The effects of applying “Sub-Strategy 2 ,” “Sub-Strategy 3 ,” “Sub-Strategy 4 ,” and “Sub-Strategy 5 ” to the original image 400 are also illustrated.

[0094] Figure 5An example of the performance gain that can be achieved by using the training system described in this specification is shown. For each of multiple training datasets (i.e., "CIFAR-10", "CIFAR-100", "SVHN", "StanfordCars", and "ImageNet"), the error rate of a neural network model trained using a data augmentation strategy selected by the training system 100 (i.e., evaluated on the corresponding validation set) exceeds some of the best published results for that training dataset.

[0095] Figure 6 Describes the different types of transformation operations that can be applied to an image. The value of one of the parameters of any given augmentation strategy can define Figure 6 Which transform types in the set of transform types are shown to be included in a given enhancement policy.

[0096] Figure 7 700 is a flow chart of an exemplary process 700 for automatically selecting a data augmentation strategy and using the data augmentation strategy to train a machine learning model. For convenience, process 700 is described as being performed by a system of one or more computers located in one or more locations. For example, a system appropriately programmed with reference to this specification, such as Figure 1 The training system 100 of the training system can perform process 700.

[0097] The system receives training data (702) for training a machine learning model to perform a specific machine learning task. For example, the system can receive the training data through an API available to the system. The training data includes a plurality of training examples, each example specifying a training input and a corresponding target output.

[0098] The system performs steps 704 through 708 of process 700 at each of a plurality of time steps. For convenience, each of steps 704-708 is described as being performed at a "current" time step.

[0099] The system generates one or more current data augmentation policies based on quality metrics of the data augmentation policies generated at previous time steps (704). If the current time step is a first time step, the system can generate the data augmentation policies in any of a variety of ways, for example, by randomly selecting parameter values ​​that define each data augmentation policy. In one embodiment, the system can use a policy network to generate a current data augmentation policy reward (e.g., as described in reference to FIG. 1 ) based on a reward defined by the quality metrics of the data augmentation policies generated at previous time steps. Figure 2 As described above), the policy network is trained using reinforcement learning techniques. In other embodiments, the system can use any of a variety of appropriate search techniques, such as random search techniques or evolutionary search techniques to generate the current data enhancement strategy.

[0100] For each current data augmentation strategy, the system trains a machine learning model on the training data using the current data augmentation strategy (706). In one example, the machine learning model is a neural network model, and the system trains the neural network model in multiple training iterations. At each training iteration, the system selects a current "batch" (i.e., collection) of one or more training examples, and then determines an "augmented" batch of training examples by transforming the training inputs in the current batch of training examples using the current data augmentation strategy. Optionally, the system can adjust the target outputs in the current batch of training examples to account for the transformations applied to the training inputs (as described above). The system processes the transformed training inputs according to the current parameter values ​​of the machine learning model to generate corresponding outputs. The system then determines the gradient of the objective function, which can measure the similarity between: (i) the output generated by the machine learning model, and (ii) the target output specified by the training examples, and uses the gradient to adjust the current values ​​of the machine learning model parameters. The system can determine the gradient using, for example, a backpropagation process, and the system can use the gradient to adjust the current values ​​of the machine learning model parameters using any suitable gradient descent optimization process, such as an RMSprop or Adam process.

[0101] For each current data augmentation strategy, after the machine learning model has been trained using the current data augmentation strategy, the system determines a quality metric for the current data augmentation strategy using the machine learning model (708). For example, the system can determine the quality metric as a performance metric (e.g., FI score or mean squared error) of the trained machine learning model on a validation dataset consisting of a plurality of training examples that were not used to train the machine learning model.

[0102] The system may repeatedly perform steps 704 to 708 until a search termination criterion is met (eg, if steps 704 to 708 have been performed a predetermined number of times).

[0103] After the system determines that the search termination criteria are met, the system selects a final data augmentation strategy based on the quality metrics of the data augmentation strategies generated in steps 704-708 (710). For example, the system can generate the final data augmentation strategy by combining a predetermined number of data augmentation strategies generated during steps 704 to 708 that have the highest quality scores.

[0104] The system generates a final trained machine learning model by training the machine learning model on the training data using the final data augmentation strategy (712). The system can provide the final trained machine learning model to a user of the system. Alternatively or in combination, the system can use the trained machine learning model to process a request received from the user.

[0105] In some cases, the system may train the final machine learning model using the final data augmentation strategy for more training iterations than when the system trained the machine learning model using the “current” data augmentation strategy generated in step 706. For example, the system may train the final machine learning model using the final data augmentation strategy until a convergence criterion is met, e.g., until the prediction accuracy of the final machine learning model evaluated on the validation set reaches a minimum. On the other hand, the system may train each of the current data augmentation strategies generated in step 706 for a smaller (e.g., fixed) number of training iterations.

[0106] This specification uses the term "configured" in conjunction with system and computer program components. For a system of one or more computers to be configured to perform a particular operation or action, it is meant that the system has installed thereon software, firmware, hardware, or a combination of software, firmware, hardware that, in operation, causes the system to perform those operations or actions. For one or more computer programs to be configured to perform a particular operation or action, it is meant that the one or more programs include instructions that, when executed by a data processing device, cause the device to perform the operation or action.

[0107] Embodiments of the subject matter and functional operations described in this specification may be implemented with digital electronic circuits, with tangibly implemented computer software or firmware, with computer hardware including the structures disclosed in this specification and their structural equivalents, or with a combination of one or more of them. Embodiments of the subject matter described in this specification may be implemented as one or more computer programs, i.e., one or more modules of computer program instructions encoded on a tangible non-transitory storage medium for execution by a data processing device or for controlling the operation of a data processing device. The computer storage medium may be a machine-readable storage device, a machine-readable storage substrate, a random or serial access storage device, or a combination of one or more of them. Alternatively or additionally, the program instructions may be encoded on an artificially generated propagation signal, such as a machine-generated electrical, optical, or electromagnetic signal, which is generated to encode information for transmission to a suitable receiver device for execution by a data processing device.

[0108] The term "data processing apparatus" refers to data processing hardware and encompasses all kinds of apparatus, devices and machines for processing data, including by way of example a programmable processor, a computer or a plurality of processors or computers. An apparatus may also be or further include special-purpose logic circuitry, for example an FPGA (field programmable gate array) or an ASIC (application-specific integrated circuit). An apparatus may optionally include, in addition to hardware, code that creates an execution environment for a computer program, for example, code constituting processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them.

[0109] A computer program, which may also be referred to or described as a program, software, software application, app, module, software module, script or code, may be written in any form of programming language including compiled or interpreted languages ​​or declarative or procedural languages; and it may be deployed in any form, including as a stand-alone program or as a module, component, subroutine or other unit suitable for use in a computing environment. A program may, but need not, correspond to a file in a file system. A program may be stored in a portion of a file that holds other programs or data, such as one or more scripts stored in a markup language document; in a single file dedicated to the program in question or in multiple coordinated files, such as files storing one or more modules, subroutines or portions of code. A computer program may be deployed to execute on one computer or on multiple computers located at one site or distributed over multiple sites and interconnected by a data communications network.

[0110] In this specification, the term "engine" is used broadly to refer to a software-based system, subsystem, or process that is programmed to perform one or more specific functions. Typically, an engine will be implemented as one or more software modules or components installed on one or more computers in one or more locations. In some cases, one or more computers will be dedicated to a specific engine; in other cases, multiple engines may be installed and run on the same computer or multiple computers.

[0111] The processes and logic flows described in this specification can be performed by one or more programmable computers executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows can also be performed by a special purpose logic circuit such as an FPGA or ASIC, or by a combination of a special purpose logic circuit and one or more programmed computers.

[0112] A computer suitable for executing a computer program may be based on a general-purpose microprocessor or a special-purpose microprocessor or both, or any other kind of central processing unit. Typically, a central processing unit will receive instructions and data from a read-only memory or a random access memory or both. The essential elements of a computer are a central processing unit for executing or implementing instructions and one or more storage devices for storing instructions and data. The central processing unit and the memory may be supplemented by or incorporated into a special logic circuit. Typically, a computer will also include one or more mass storage devices for storing data, such as a disk, a magneto-optical disk, or an optical disk, or be operationally coupled to receive data from or transfer data to the one or more mass storage devices, or both for storing data. However, a computer does not necessarily have such a device. In addition, a computer may be embedded in another device, such as a mobile phone, a personal digital assistant (PDA), a mobile audio or video player, a game controller, a global positioning system (GPS) receiver, or a portable storage device, such as a universal serial bus (USB) flash drive, etc.

[0113] Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and storage devices, including by way of example semiconductor storage devices, such as EPROM, EEPROM, and flash memory devices; magnetic disks, such as internal hard disks or removable disks; magneto-optical disks; and CD ROM and DVD-ROM disks.

[0114] To provide interaction with a user, embodiments of the subject matter described in this specification may be implemented on a computer having a display device for displaying information to the user, such as a CRT (cathode ray tube) or LCD (liquid crystal display) monitor, and a keyboard and pointing device that the user can use to provide input to the computer, such as a mouse or trackball. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback, such as visual feedback, auditory feedback, or tactile feedback; and input from the user may be received in any form, including acoustic, voice, or tactile input. In addition, the computer may interact with the user by sending documents to and receiving documents from a device used by the user; for example, by sending a web page to a web browser on the user's device in response to receiving a request from the web browser. In addition, the computer may interact with the user by sending a text message or other form of message to a personal device and then receiving a response message from the user, such as a smart phone running a messaging application.

[0115] The data processing apparatus for implementing a machine learning model may also include, for example, dedicated hardware accelerator units for processing common and computationally intensive parts of machine learning training or production, i.e., inference, workloads.

[0116] Machine learning models can be implemented and deployed using a machine learning framework, such as the TensorFlow framework, the Microsoft Cognitive Toolkit framework, the Apache Singa framework, or the Apache MXNet framework.

[0117] Embodiments of the subject matter described in this specification may be implemented in a computing system that includes a back-end component, such as a data server; or includes a middleware component, such as an application server; or includes a front-end component, such as a client computer with a graphical user interface, a web browser, or an app that a user can use to interact with an implementation of the subject matter described in this specification; or includes any combination of one or more such back-end, middleware, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication, such as a communication network. Examples of communication networks include local area networks (LANs) and wide area networks (WANs), such as the Internet.

[0118] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by means of computer programs running on respective computers and having a client-server relationship with each other. In some embodiments, the server transmits data such as HTML pages to the user device, for example, for the purpose of displaying data to a user interacting with the device as a client and receiving user input from the user. Data generated at the user device, for example, the result of the user interaction, may be received from the device at the server.

[0119] Although this specification contains many specific implementation details, these should not be interpreted as limitations on the scope of any invention or that may be claimed, but rather as descriptions of features that may be specific to a particular embodiment of a particular invention. Certain features described in this specification in the context of separate embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented in multiple embodiments individually or in any suitable subcombination. In addition, although features may be described above as functioning in certain combinations and even initially claimed as such, one or more features from a claimed combination may be removed from the combination in some cases, and a claimed combination may be directed to a subcombination or variations of a subcombination.

[0120] Similarly, although operations are depicted in the drawings and described in the claims in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in a sequential order, or that all illustrated operations be performed to achieve the desired results. In some cases, multitasking and parallel processing can be advantageous. In addition, the separation of various system modules and components in the above-described embodiments should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0121] Specific embodiments of the subject matter have been described. Other embodiments are within the scope of the appended claims. For example, the actions recited in the claims may be performed in a different order and still achieve the desired results. As an example, the processes depicted in the accompanying drawings do not necessarily require the specific order or sequential order shown to achieve the desired results. In some cases, multitasking and parallel processing may be advantageous.

Claims

1. A method for determining a data augmentation strategy, the data augmentation strategy defining a process for transforming a training input before the training input is used to train a machine learning model, the method comprising, at each of a plurality of time steps: For one or more previous time steps, generating a current data augmentation strategy based on a quality metric of a data augmentation strategy generated at the previous time step, wherein: for each previous time step, a quality metric of the data augmentation policy generated at the previous time step characterizing a performance metric of the machine learning model after the machine learning model has been trained using the data augmentation policy generated at the previous time step; as well as The current data enhancement strategy includes multiple sub-strategies, among which: Each sub-strategy in the current data augmentation strategy specifies: (i) a corresponding sequence of transformation operations, and (ii) a corresponding probability of applying each transformation operation in the sequence of transformation operations; Training the machine learning model using the current data augmentation strategy includes: Select a batch of training inputs; According to the current data enhancement strategy, determining an enhanced batch of training inputs by transforming the training inputs in the batch of training inputs includes, for each training input in the batch of training inputs: Selecting a sub-strategy from among the plurality of sub-strategies included in the current data enhancement strategy; and Transforming the training input according to the selected sub-strategy by applying a sequence of transformation operations specified by the selected sub-strategy to the training input, comprising: applying each transformation operation in the sequence of transformation operations with a corresponding probability specified by the selected sub-strategy; and adjusting current values ​​of parameters of the machine learning model based on the augmented batch of training inputs; and After the machine learning model has been trained using the current data augmentation strategy, a quality metric of the current data augmentation strategy is determined using the machine learning model.

2. The method according to claim 1, wherein: Selecting a sub-strategy from among the multiple sub-strategies included in the current data enhancement strategy includes: A sub-strategy among the plurality of sub-strategies included in the current data augmentation strategy is randomly sampled.

3. The method according to claim 1, wherein: The machine learning model is a neural network, and adjusting current values ​​of parameters of the machine learning model based on the enhanced batch of training inputs includes: Using the augmented batch of training data, determine the gradient of the loss function: and The gradients are used to adjust current values ​​of the machine learning model parameters.

4. The method according to claim 1, wherein: For each previous time step, the quality metric of the data augmentation strategy generated at the previous time step represents a performance metric of the machine learning model on a set of validation data after the machine learning model has been trained using the data augmentation strategy generated at the previous time step, wherein the machine learning model has not yet been trained on the validation data.

5. The method according to claim 1, further comprising: Selecting a final data augmentation strategy based on the quality metrics of the generated data augmentation strategies; as well as A final trained machine learning model is generated by training the final machine learning model using the final data augmentation strategy.

6. The method according to claim 1, wherein: Training the machine learning model using the current data augmentation strategy includes: training the machine learning model to perform a specific machine learning task.

7. The method according to claim 6, wherein: The specific machine learning task is an image processing task including classification or regression.

8. The method according to claim 1, wherein: Each sub-strategy in the current data augmentation strategy specifies: (i) a corresponding sequence of transform operations, and (ii) a corresponding size of each transform operation in the sequence of transform operations.

9. The method according to claim 1, wherein: Generating the current data augmentation strategy based on a quality metric of a data augmentation strategy generated at the previous time step for one or more previous time steps includes: generating the current data augmentation strategy using a policy neural network according to current values ​​of policy neural network parameters; and The policy neural network is trained by reinforcement learning techniques based on a reinforcement learning reward signal, and at each time step, the reinforcement learning reward signal is based on a quality metric of the current data augmentation policy generated at that time step.

10. The method according to claim 7, wherein: The current data enhancement strategy is defined by current values ​​of multiple data enhancement strategy parameters, and wherein, for each of the multiple data enhancement strategy parameters, the policy network output of the policy neural network defines a score distribution of possible values ​​of the data enhancement strategy parameter.

11. The method according to claim 10, wherein: Generating the current data augmentation strategy includes sampling a current value of each data augmentation strategy parameter using the score distribution of possible values ​​of the data augmentation strategy parameter.

12. The method according to claim 9, wherein: The policy neural network is a recurrent neural network.

13. The method according to claim 1, wherein: Generating the current data enhancement strategy includes: The current data augmentation strategy is generated using a genetic programming process.

14. A non-transitory computer storage medium storing instructions which, when executed by one or more computers, cause the one or more computers to perform the method according to any one of claims 1 to 13.

15. A system comprising: one or more computers; and One or more storage devices, the one or more storage devices being coupled to the one or more computers, wherein the one or more storage devices store instructions which, when executed by the one or more computers, cause the one or more computers to perform the method according to any one of claims 1 to 13.

16. A method comprising, at each of a plurality of time steps: Generating a current data augmentation strategy based on a quality metric of a data augmentation strategy generated at one or more previous time steps, the current data augmentation strategy defining a process for transforming a training input to a machine learning model, wherein: for each previous time step, a quality metric of the data augmentation policy generated at the previous time step characterizing a performance metric of the machine learning model after the machine learning model has been trained using the data augmentation policy generated at the previous time step; as well as The current data augmentation strategy defines a process of transforming a training input by defining the following for each transform operation in a sequence of transform operations defined by the current data augmentation strategy: (i) the corresponding probability distribution over the space of transformation types of the transformation operation, and (ii) a corresponding probability distribution over the space of transform sizes of the transform operation; Training the machine learning model using the current data augmentation strategy includes: Select a batch of training inputs; According to the current data enhancement strategy, determining an enhanced batch of training inputs by transforming the training inputs in the batch of training inputs includes, for each training input in the batch of training inputs: For each transformation operation in the transformation operation sequence of the current data augmentation strategy: selecting a transform type for the transform operation according to the probability distribution on the space of transform types for the transform operation; selecting a transform size of the transform operation according to the probability distribution over the space of transform sizes of the transform operation; transforming the training input by applying the transform operation with the selected transform type and transform size; and adjusting current values ​​of parameters of the machine learning model based on the augmented batch of training inputs; and After the machine learning model has been trained using the current data augmentation strategy, a quality metric of the current data augmentation strategy is determined using the machine learning model.

17. The method according to claim 16, wherein: The current data enhancement strategy is defined by current values ​​of a plurality of data enhancement strategy parameters; and The current values ​​of the plurality of data augmentation strategy parameters are defined for each transformation operation in the sequence of transformation operations of the current data augmentation strategy: (i) the corresponding probability distribution over the space of transformation types of the transformation operation, and (ii) The corresponding probability distribution over the space of transform sizes of the transform operations.

18. The method according to claim 16, wherein: The machine learning model is a neural network, and adjusting current values ​​of parameters of the machine learning model based on the enhanced batch of training inputs includes: Using the augmented batch of training data, determine the gradient of the loss function: and The gradients are used to adjust current values ​​of the machine learning model parameters.

19. The method according to claim 16, wherein: For each previous time step, the quality metric of the data augmentation strategy generated at the previous time step represents a performance metric of the machine learning model on a set of validation data after the machine learning model has been trained using the data augmentation strategy generated at the previous time step, wherein the machine learning model has not yet been trained on the validation data.

20. The method of claim 16, further comprising: Selecting a final data augmentation strategy based on the quality metrics of the generated data augmentation strategies; as well as A final trained machine learning model is generated by training the final machine learning model using the final data augmentation strategy.

21. The method according to claim 16, wherein: Training the machine learning model using the current data augmentation strategy includes: training the machine learning model to perform a specific machine learning task.

22. The method according to claim 21, wherein: The specific machine learning task is an image processing task including classification or regression.

23. The method of claim 17, wherein: Generating the current data augmentation strategy based on a quality metric of a data augmentation strategy generated at the previous time step for one or more previous time steps includes: generating the current data augmentation strategy using a policy neural network according to current values ​​of policy neural network parameters; and The policy neural network is trained by reinforcement learning techniques based on a reinforcement learning reward signal, and at each time step, the reinforcement learning reward signal is based on a quality metric of the current data augmentation policy generated at that time step.

24. The method of claim 23, wherein: For each data augmentation policy parameter of the plurality of data augmentation policy parameters, a policy network output of the policy neural network defines a score distribution of possible values ​​of the data augmentation policy parameter; and Generating the current data augmentation strategy includes sampling a current value of each data augmentation strategy parameter using the score distribution of possible values ​​of the data augmentation strategy parameter.

25. The method according to claim 23, wherein: The policy neural network is a recurrent neural network.

26. The method of claim 16, wherein: Generating the current data enhancement strategy includes: The current data augmentation strategy is generated using a genetic programming process.

27. The method of claim 16, wherein: Generating the current data enhancement strategy includes: A random search process is used to generate the current data augmentation strategy.

28. A non-transitory computer storage medium storing instructions which, when executed by one or more computers, cause the one or more computers to perform the method of any one of claims 16 to 27.

29. A system comprising: one or more computers; and One or more storage devices, the one or more storage devices being coupled to the one or more computers, wherein the one or more storage devices store instructions which, when executed by the one or more computers, cause the one or more computers to perform the method according to any one of claims 16 to 27.

Images