Method and apparatus for optimising training of a machine learning model

Abstract

A computer-implemented method for training a machine learning (ML) model to perform a specific task, the method comprising: obtaining a training dataset S100; and obtaining a validation dataset S102.

Description

Field

[001] The present application generally relates to a method and apparatus for optimising training of a machine learning, ML, model for a specific task. In particular, the present application provides a method for optimising the fine-tuning of an ML model in a resourceefficient manner, where the fine-tuning may be to personalise the ML model for a specific user and / or a specific task. Background

[002] As explained in more detail below, the present techniques are focussed on general differentiable meta learning, which includes bi-level optimisation (BLO) problems such as hyperparameter optimisation (HPO) and few-shot meta-learning. The problem, terminology and notation used herein is introduce via a representative BLO problem of learning neural network regulariser hyperparameters (aka weight decay, or L1 / 2-regularisation). Expressed as a BLO, this is min E(x,y)~vUv(^y;0*G))] (1) s. t. 9*(A) = argnnnlE^y)^[ZT(x,y; 0)] + R(A, 9), where 9 are neural network parameters to learn, A are all hyperparameters to learn, T and V indicate training and validation data sets, respectively, lT and lv indicate training and validation losses, and R is the regularisation or weight decay term. In the parameter-wise regularisation case, R(A, 9) = Yj ^-9^ or Yj Aj\9j\ in which A has the same structure as 9.

[003] The main challenge in differentiable BLO is to efficiently and accurately compute the hypergradient dlv / dA to update the hyperparameters. In the literature there exist some well-known approaches. Some aim to unroll the inner-loop optimisation (i.e., forward- or reversemode differentiation, denoted by FMD and RMD, respectively), while others aim to compute it via the implicit function theorem (IFT) to circumvent the infeasible memory cost of saving intermediate Hessian matrices (in FMD) or computation graphs (in RMD) from unrolled inner optimisation steps. Despite the theoretical promise of IFT-based meta-gradient for memoryefficiency, it still suffers from a difficult and potentially unstable Hessian inverse, and it is highly reliant on the gradient-equal-to-zero (i.e., perfectly converged) condition for the inner optimisation. In practice these drawbacks mean that it is unreliable and hard to tune.

[004] A fundamental issue for all these existing BLO solutions is that the inner optimisation is deterministic, and they are unable to account for the uncertainty in the inner problem and / or its solution. When applied to contemporary deep learning there are at least two highly practical examples of inner optimisation uncertainty: When implemented in practice by minibatch SGD with a finite number of steps, the inner loop is not perfectly converged (violating IFT’s assumption); and when applied to non-convex neural networks, the inner optimisation has many local minima due to overparametrisation, leading to uncertainty over which inner minima is returned.

[005] The present applicant has therefore identified the need for improved techniques for optimising training of a machine learning, ML, model for a specific task Summary

[006] Generally speaking, the present techniques address the above-mentioned issues by introducing a new stochastic gradient approach to hyper-gradient calculation for BLO. The standard deterministic BLO problem is generalised to a stochastic optimisation where the inner optimisation produces a smooth probability distribution, and the outer optimisation takes an expectation with respect to the inner posterior. This stochastic generalisation allows the uncertainty in the inner optimisation arising from noise, the use of minibatch SGD, or from multiple local minima to be dealt with. More specifically, SGLD is exploited to obtain posterior samples of the inner optimisation posterior, and a new efficient hyper-gradient algorithm is derived by taking derivatives of the SGLD step equations.

[007] In a first approach of the present techniques, there is provided a computer-implemented method for training a machine learning, ML, model to perform a specific task, the method comprising: obtaining a training dataset comprising a plurality of data items suitable for training the specific task to be performed by the ML model; obtaining a validation dataset comprising a plurality of data items suitable for validating the ML model; and training the ML model using the training dataset, by: initialising a value of each hyperparameter in a set of hyperparameters, wherein the hyperparameters are configurable variables of a process to train the ML model; choosing a value of each ML model parameter in a set of ML model parameters; iteratively learning over a first predetermined number M+B of iterations, using the initial value for each hyperparameter and using the training dataset, an updated set of ML model parameters that minimise a training loss, wherein the training loss assesses how well predictions of the ML model using the learned set of ML model parameters fit the training dataset; for each iteration m that is greater than a second predetermined number B of iterations and less than or equal to the first predetermined number M+B of iterations: calculating, using the validation dataset, a validation loss for each learned ML model parameter, wherein the validation loss assesses how well predictions of the ML model using the learned set of ML model parameters fit the validation dataset; averaging the validation loss for the learned ML model parameters calculated for each iteration; determining an adjustment to the value of each hyperparameter using the average validation loss; adjusting the value of each hyperparameter using the determined adjustment and repeating the choosing, learning, calculating, averaging, determining and adjusting steps; and setting a value of each hyperparameter (using the value of each hyperparameter after the optimisation process is complete).

[008] That is, when training the ML model using a fixed hyperparameter set, a total (M+B) iterations are performed, where for the first B (m=1...B) iterations a training loss is minimised, and once m>B (m=B+1...B+M), the validation loss is minimised. The latter involves accumulating the validation loss (hyper)gradients for these M iterations. In the end, the accumulated (hyper)gradients are used for updating the values of the hyperparameters.

[009] When training a machine learning, ML, model, it is necessary to set or decide the values of a set of hyperparameters. Hyperparameters are configuration variables that are used to manage ML model training. These are set prior to training the ML model and are different from model parameters that are learned during the learning process (e.g. the weights of neural network nodes). Selecting the right values for the hyperparameters (and the right hyperparameters to vary), is important because they impact model performance and accuracy. For example, the learning rate of the ML model is a hyperparameter. If the learning rate is set too high, the model may converge too quickly, and thereby have a lower accuracy; similarly, if the learning rate is set too low, training may take too long, which may be problematic when resources are limited (e.g. when training is occurring on a resource-constrained device like a smartphone). Setting or deciding the values of the hyperparameters is performed either manually (slow and tedious) or automatically, and is known as “hyperparameter tuning” or “hyperparameter learning”. Typically, in either case, the hyperparameter tuning process is iterative - different hyperparameters and values thereof are used and the impact on, for example, accuracy of the ML model’s predictions, is assessed. Advantageously, the present techniques are particularly useful for performing personalized hyperparameter fine-tuning of on-device machine learning, ML, models, in a way that is more resource-efficient and without compromising accuracy. Furthermore, the present techniques are suitable for performing on- device training and fine-tuning of ML models while maintaining user privacy. That is, it is desirable to perform fine-tuning and training on user devices and using user data items as the training data, because this ensures user privacy and data security. Otherwise, personal data items (e.g., photos, videos, audio, etc. that a user has captured need to be sent to a central server, which is less secure and opens-up the user to other issues such as ID fraud).

[010] Generally speaking, as noted above, hyperparameter tuning is a type of bi-level optimization, BLO, problem, because one problem is embedded or nested within the other. This is usually referred to as an “outer” problem or task, and an “inner” problem or task. In the present context of hyperparameter tuning, the inner problem is to train the model on training data so that the model can make accurate predictions for similar data, while the outer problem is ensuring that the model also works well on validation data. Solving a BLO problem is difficult, and often, the BLO problem is reformulated as a problem for which solutions are readily available. With this in mind, the present techniques turn the BLO problem into a novel stochastic optimization (SO) formulation, which takes into account uncertainty (in data or optimization processes). This treatment makes the method robust to suboptimal inner solution or non-unique multiple inner minima due to over-parametrization in deep models. This is advantageous compared to some existing techniques which tackle the BLO problem of hyperparameter optimization in deterministic ways, and as a result, are impacted by noise in data and imperfect inner solutions.

[011] Compared to existing techniques that often exhibit unstable behaviour and hyperparameter sensitivity in practice, the present techniques lead to considerably more reliable solutions. This benefit originates from the feasible extension of the forward-mode differentiation, which does not rely on the numerically unstable gradient-equal-to-0 conditions at all (explained below).

[012] Compared to widely-used “backprop-over-unrolled-inner-optimization” techniques (a.k.a., reverse-mode differentiation or MAML), the present techniques result in drastic memory storage saving, taking constant memory footprint with respect to number of iterations. This is particularly advantageous because the training or personalisation of ML models can be more easily performed on resource-constrained devices, such as smartphones.

[013] Another benefit of the present techniques is that the final solutions (the learned hyperparameters for deep models) are more stable and reliable than existing methods. Whereas existing methods often exhibit unreliable behaviours, being sensitive to the underlying choices of the learning options, the present techniques show strong robustness to these option choices. For consumer products such as smartphones, the reliability of the trained model is a very important metric, beyond the accuracy, since ill-chosen learning options can lead to serious malfunctioning (e.g., non-sensical reply text generation in LLM chatbots).

[014] As explained in more detail below, the present techniques have been tested on various hyperparameter optimization problems / benchmarks for deep learning: L1-regularization strength learning for deep networks -- Vision Transformers for image classification (87M hyperparameters); optimal loss function learning - WideResNet for image classification; fewshot meta learning - ConvNet for minilmagenet classification with many different classification episodes; meta learning of implicit neural representation - synthesizing novel views of ShapeNet scenes from a single view; and invariance learning - automatic learning of input image transformations / augmentations. It is shown that the present techniques achieve an improvement in test predictions compared to existing techniques.

[015] As noted above, the present techniques involve solving the inner and outer problems using a stochastic optimisation process. The inner problem involves learning a set of ML model parameters using training data items. Learning a set of ML model parameters may comprise learning any one or both of: weights of at least one neural network of the ML model; and biases of at least one neural network of the ML model. These are dependent on the training data, which is related to a specific task to be performed by the ML model. Thus, the present techniques may be used in two ways - to train an ML model to perform a specific task (e.g. image analysis, speech analysis, etc.), and to fine-tune an already pre-trained ML model on user-specific data or to perform a user-specific task (e.g. fine-tune an object detection model to recognise a user’s dog in pictures of dogs, or fine tune a generative model (e.g. LLM) to generate text in the user’s writing style). In each case, the set of ML model parameters are dependent on the data items used to perform the training / fine-tuning.

[016] The step of choosing a value of each ML model parameter in a set of ML model parameters may comprise randomly selecting a value for each ML model parameter. In this way, the initial value of each ML model parameter is independent of the hyperparameter(s) and choice of the initial value of the hyperparameter(s).

[017] Alternatively, the step of choosing a value of each ML model parameter in a set of ML model parameters may comprise selecting a value for each ML model parameter that is dependent on the initial value of the hyperparameter(s).

[018] Generally speaking, training an ML model to minimise a training loss involves finding a minimum of a training loss function (i.e. calculating a derivative of the training loss function). However, the training loss function does not necessarily have a single minimum because the training data may be noisy. All existing techniques assume that there is a single minimum, which means that other possible solutions are overlooked. The present techniques advantageously therefore assume that there may be multiple minima, i.e. that there may be more than one optimum solution to the training problem (the inner problem).

[019] The stochastic optimisation process involves two iterative processes. The first iterative process is to train the ML model using a given initial value of each hyperparameter. The second iterative process is to determine an update to the value of each hyperparameter as a result of the training of the ML model, and then use the updated value(s) to perform the first iterative process again. In other words, the stochastic optimisation involves initialising a value of each hyperparameter in a set of hyperparameters, and using this to perform the training of the ML model. The initial value of the hyperparameter is used to iteratively train the ML model using the training dataset, using any suitable training process. The goal of the training is to determine the values of the ML model parameters that minimise a training loss. That is, the ML model is trained over a predetermined number M+B of iterations to determine the best values of the ML model parameters given the fixed value of the hyperparameter(s). When the number of iterations that have been performed m is greater than B, where B <M, the training of the ML model continues but validation of the ML model parameters is now performed in parallel. That is, the validation loss is determined using each set of ML model parameters for each iteration of the ML model training between B and M+B. This means that the validation loss is considered over a number of iterations rather than simply the last iteration of the training, which is advantageous because doing so smooths the validation loss. Furthermore, this enables computation of the expected value that is required under the stochastic optimisation of the present techniques. The expected value in stochastic optimisation helps make the solution robust to noise.

[020] As noted above, the validation losses calculated for each set of model parameters are averaged. The average validation loss (also known as an expected value) is used to determine an adjustment to the value of each hyperparameter (i.e. the magnitude and direction of change). Each hyperparameter value is then adjusted using the determined adjustment and the steps to choose the ML model parameters, learn the ML model parameters using the training data, calculate the validation loss, average the validation losses, determine an adjustment, and adjust the value of each hyperparameter are repeated. The outer problem is solved by selecting the hyperparameter values that generate the lowest average validation loss, which involves repeating the steps (second iterative process).

[021] Iteratively learning a set of ML model parameters may comprise using gradient descent to determine how to adjust the value of each ML model parameter in each iteration (e.g. the magnitude and direction of change).

[022] The method may further comprise discarding the value of each ML model parameter that is iteratively learned up to the second predetermined number B of iterations. This is useful because in the initial stages of training, the training loss is likely to be very large, and as a result, the validation loss will likely be very large. Thus, the validation process is only performed when it is more likely that the ML model parameters are getting closer to the optimum values (given the fixed hyperparameter value).

[023] The step of initialising a value of each hyperparameter in a set of hyperparameters may comprise initialising a value of any one or more of the following hyperparameters: learning rate; learning rate decay; momentum; number of neural network nodes; number of neural network layers; mini-batch size; and number of epochs. It will be understood that these are just some non-limiting and non-exhaustive examples of hyperparameters that may be optimised.

[024] Initialising a value of each hyperparameter in a set of hyperparameters may comprise randomly selecting a value for each hyperparameter.

[025] Determining an adjustment to the value of each hyperparameter using the average validation loss may comprise: calculating a hypergradient using the average validation loss, wherein the calculated hypergradient indicates how to adjust the value of each hyperparameter (i.e. magnitude and direction of change).

[026] Repeating the choosing, learning, calculating, averaging, determining and adjusting steps may comprise repeating the steps for a further predetermined number of iterations. That is, the second iterative process may be repeated a predetermined number of times. Other ways (e.g. total time) to determine when to the end the second iterative process may equally be utilised.

[027] The step of setting a value of each hyperparameter may comprise setting a value of each hyperparameter when the further predetermined number of iterations are completed.

[028] Up to now, the method has related to optimising the value of each hyperparameter. The method may therefore further comprise training the ML model using the training dataset and the set value of each hyperparameter. In other words, once the value of each hyperparameter has been optimised, the model is trained using this set, optimised value.

[029] In some cases, the ML model may be a pre-trained ML model comprising at least one untrained adapter module. Training the ML model may comprise training the at least one untrained adapter module.

[030] Training the ML model may comprise training the at least one untrained adapter module only and freezing the pre-trained ML model. The pre-trained ML model may have been trained to perform a general task, such as object detection. The adapter module(s) is trained to perform a user-specific, but related task, such as detecting their specific objects. The pre-trained ML model is advantageously not re-trained because this may not be feasible on a user device, and may lead to other issues such as catastrophic forgetting because all the original training data is not available. Thus, it may be useful to simply train a few parameters of the ML model, e.g. the adapter module(s). Adapter modules are small neural network modules that are inserted into pre-trained models to adapt the pre-trained models for new tasks and / or domains. They are typically added between layers of the pre-trained models. Training only the adapter modules is also known as parameter-efficient training, and is particularly suitable for resource-constrained devices such as smartphones and smart electronic appliances (and where it is undesirable for user data to leave the device for security / privacy reasons).

[031] The pre-trained ML model may be a pre-trained vision ML model, and training the at least one untrained adapter module may comprise training the at least one adapter module to perform a user-specific vision task. The user-specific vision task may be, for instance, userspecific object detection or recognition.

[032] The pre-trained ML model may be a pre-trained large language model, LLM, and training the at least one untrained adapter module may comprise training the at least one adapter module to perform a user-specific language task.

[033] The method of the first approach may be performed by / on a constrained-resource electronic user device.

[034] One example use case of the present techniques is to perform dataset or datapoint selection for personalized training / finetuning). For example, end users may want to train or finetune large deep networks on their user devices with their own data, which may not be plentiful. Scarce or insufficient data can easily make the model overfit. One way to solve this problem is to select the data that is most useful for personalising the model: given a pool of training datasets and a target user-specific task, select which subset of training datasets / datapoints is the most effective to the user-specific tasks. In contrast, the present techniques solve the problem by framing the problem as a hyperparameter optimisation problem, i.e. it is possible to find the optimal subset from the pool of training data that is best aligned with the user-personal task, in a highly robust manner. This is also applicable to various on-device applications including: personalized chatbots, speech / image recognition, and personalized image generation.

[035] Another example use case of the present techniques is to perform optimised finetuning of LLM chatbots). For example, pre-trained large language models that form the basis of chatbots can be fine-tuned on user-specific data. Typically, such models are fine-tuned by training adapter modules on user-specific data. Thus, the problem is how to decide the hyperparameters for training the adapter modules, including where to put the adapters in the pre-trained model and how many adapter hyperparameters to fine-tune. The present techniques learn the optimal hyperparameters using a more principled data-driven framework, with reliable performance. Reliability is a very important metric, beyond the accuracy, since ill-chosen hyperparameters or learning options can lead to serious malfunctioning (e.g., nonsensical reply text generation in LLM chatbots).

[036] In a second approach of the present techniques, there is provided an electronic user device for training a machine learning, ML, model to perform a specific task, the user device comprising: at least one processor coupled to a memory, arranged for: obtaining a dataset comprising a plurality of data items suitable for training the specific task to be performed by the ML model; obtaining a validation dataset comprising a plurality of data items suitable for validating the ML model; and training the ML model using the training dataset, by: initialising a value of each hyperparameter in a set of hyperparameters, wherein the hyperparameters are configurable variables of a process to train the ML model; choosing a value of each ML model parameter in a set of ML model parameters; iteratively learning over a first predetermined number M+B of iterations, using the initial value for each hyperparameter and using the training dataset, an updated set of ML model parameters that minimise a training loss, wherein the training loss assesses how well predictions of the ML model using the learned set of ML model parameters fit the training dataset; for each iteration m that is greater than a second predetermined number B of iterations and less than or equal to the first predetermined number M+B of iterations: calculating, using the validation dataset, a validation loss for each learned ML model parameter, wherein the validation loss assesses how well predictions of the ML model using the learned set of ML model parameters fit the validation dataset; averaging the validation loss for the learned ML model parameters calculated for each iteration; determining an adjustment to the value of each hyperparameter using the average validation loss; adjusting the value of each hyperparameter using the determined adjustment and repeating the choosing, learning, calculating, averaging, determining and adjusting steps; and setting a value of each hyperparameter.

[037] The features described above with respect to the first approach apply equally to the second approach and therefore, for the sake of conciseness, are not repeated.

[038] As noted above, the user device comprises at least one processor and memory. The memory may store instructions that, when executed by the at least one processor individually or collectively, cause the at least one processor to perform the above-described steps.

[039] The user device may be a smart device. The user device may be a smartphone. A smartphone is an example of a smart device. The user device may be a smart appliance. A smart appliance is another example of a smart device. An example of a smart appliance is a smart television (TV), a smart fridge, a smart oven, a smart vacuum cleaner, a smart robotic device, and so on. More generally, the user device may be a constrained-resource device, but which has the minimum hardware capabilities to personalise a trained ML model, and to use the personalised ML model. The user device may be any one of: a smartphone, tablet, laptop, computer or computing device, virtual assistant device, a vehicle, an autonomous vehicle, a robot or robotic device, a robotic assistant, image capture system or device, an augmented reality system or device, a virtual reality system or device, a gaming system, an Internet of Things device, or a smart consumer device (such as a smart fridge, smart vacuum cleaner, smart lawn mower, smart oven, etc). It will be understood that this is a non-exhaustive and non-limiting list of example devices.

[040] The user device may further comprise an image capture device for capturing images, and a storage storing captured images; wherein obtaining a training dataset comprises obtaining at least one data item representative of the task to be performed by the ML model from the storage.

[041] The user device may be a smartphone or a tablet computer. The user device may be a smart home appliance, such as, for example, a robot vacuum cleaner, a television, TV, a fridge, or an oven.

[042] In a related approach of the present techniques, there is provided a computer-readable storage medium comprising instructions which, when executed by a processor, causes the processor to carry out any of the methods described herein.

[043] As will be appreciated by one skilled in the art, the present techniques may be embodied as a system, method or computer program product. Accordingly, present techniques may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects.

[044] Furthermore, the present techniques may take the form of a computer program product embodied in a computer readable medium having computer readable program code embodied thereon. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. A computer readable medium may be, for example, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing.

[045] Computer program code for carrying out operations of the present techniques may be written in any combination of one or more programming languages, including object oriented programming languages and conventional procedural programming languages. Code components may be embodied as procedures, methods or the like, and may comprise subcomponents which may take the form of instructions or sequences of instructions at any of the levels of abstraction, from the direct machine instructions of a native instruction set to high-level compiled or interpreted language constructs.

[046] Embodiments of the present techniques also provide a non-transitory data carrier carrying code which, when implemented on a processor, causes the processor to carry out any of the methods described herein.

[047] The techniques further provide processor control code to implement the above-described methods, for example on a general purpose computer system or on a digital signal processor (DSP). The techniques also provide a carrier carrying processor control code to, when running, implement any of the above methods, in particular on a non-transitory data carrier. The code may be provided on a carrier such as a disk, a microprocessor, CD- or DVD-ROM, programmed memory such as non-volatile memory (e.g. Flash) or read-only memory (firmware), or on a data carrier such as an optical or electrical signal carrier. Code (and / or data) to implement embodiments of the techniques described herein may comprise source, object or executable code in a conventional programming language (interpreted or compiled) such as Python, C, or assembly code, code for setting up or controlling an ASIC (Application Specific Integrated Circuit) or FPGA (Field Programmable Gate Array), or code for a hardware description language such as Verilog (RTM) or VHDL (Very high speed integrated circuit Hardware Description Language). As the skilled person will appreciate, such code and / or data may be distributed between a plurality of coupled components in communication with one another. The techniques may comprise a controller which includes a microprocessor, working memory and program memory coupled to one or more of the components of the system.

[048] It will also be clear to one of skill in the art that all or part of a logical method according to embodiments of the present techniques may suitably be embodied in a logic apparatus comprising logic elements to perform the steps of the above-described methods, and that such logic elements may comprise components such as logic gates in, for example a programmable logic array or application-specific integrated circuit. Such a logic arrangement may further be embodied in enabling elements for temporarily or permanently establishing logic structures in such an array or circuit using, for example, a virtual hardware descriptor language, which may be stored and transmitted using fixed or transmittable carrier media.

[049] In an embodiment, the present techniques may be realised in the form of a data carrier having functional data thereon, said functional data comprising functional computer data structures to, when loaded into a computer system or network and operated upon thereby, enable said computer system to perform all the steps of the above-described method.

[050] The method described above may be wholly or partly performed on an apparatus, i.e. an electronic device, using a machine learning or artificial intelligence model. The model may be processed by an artificial intelligence-dedicated processor designed in a hardware structure specified for artificial intelligence model processing. The artificial intelligence model may be obtained by training. Here, "obtained by training" means that a predefined operation rule or artificial intelligence model configured to perform a desired feature (or purpose) is obtained by training a basic artificial intelligence model with multiple pieces of training data by a training algorithm. The artificial intelligence model may include a plurality of neural network layers. Each of the plurality of neural network layers includes a plurality of weight values and performs neural network computation by computation between a result of computation by a previous layer and the plurality of weight values.

[051] As mentioned above, the present techniques may be implemented using an Al model. A function associated with Al may be performed through the non-volatile memory, the volatile memory, and the processor. The processor may include one or a plurality of processors. At this time, one or a plurality of processors may be a general purpose processor, such as a central processing unit (CPU), an application processor (AP), or the like, a graphics-only processing unit such as a graphics processing unit (GPU), a visual processing unit (VPU), and / or an Al-dedicated processor such as a neural processing unit (NPU). The one or a plurality of processors control the processing of the input data in accordance with a predefined operating rule or artificial intelligence (Al) model stored in the non-volatile memory and the volatile memory. The predefined operating rule or artificial intelligence model is provided through training or learning. Here, being provided through learning means that, by applying a learning algorithm to a plurality of learning data, a predefined operating rule or Al model of a desired characteristic is made. The learning may be performed in a device itself in which Al according to an embodiment is performed, and / or may be implemented through a separate server / system.

[052] The Al model may consist of a plurality of neural network layers. Each layer has a plurality of weight values, and performs a layer operation through calculation of a previous layer and an operation of a plurality of weights. Examples of neural networks include, but are not limited to, convolutional neural network (CNN), deep neural network (DNN), recurrent neural network (RNN), restricted Boltzmann Machine (RBM), deep belief network (DBN), bidirectional recurrent deep neural network (BRDNN), generative adversarial networks (GAN), and deep Q-networks.

[053] The learning algorithm is a method for training a predetermined target device (for example, a robot) using a plurality of learning data to cause, allow, or control the target device to make a determination or prediction. Examples of learning algorithms include, but are not limited to, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning. Brief description of the drawings

[054] Implementations of the present techniques will now be described, by way of example only, with reference to the accompanying drawings, in which:

[055] Figures 1(a) to (d) show data from the illustrative toy problem;

[056] Figure 2 shows the proposed method summarized as pseudocode;

[057] Figure 3 is a table showing the complexity of the competing algorithms;

[058] Figure 4 is a table showing results from the synthetic 1D BLO problem;

[059] Figure 5 is data showing the results from varying the number of inner loop iterations;

[060] Figure 6 is a table showing results from the Noisy synthetic 1D BLO problem;

[061] Figure 7 is data showing the results to test the goodness of first-order approximation;

[062] Figure 8 is a table showing L1Reg HPO results;

[063] Figure 9 is a table showing loss function learning results;

[064] Figure 10 is a table showing meta test accuracy (%) on few-shot learning with the Minilmagenet dataset;

[065] Figure 11 is a table showing meta learning of implicit neural representation on the ShapeNet datasets;

[066] Figure 12 is a flowchart of example steps to train an ML model using the present techniques; and

[067] Figure 13 is a block diagram of an example device for training the ML model. Detailed description of the drawings

[068] Broadly speaking, embodiments of the present techniques provide a method and apparatus for optimising training of a machine learning, ML, model for a specific task.

[069] Generally speaking, the present techniques tackle the general differentiable meta learning problem that is ubiquitous in modern deep learning, including hyperparameter optimisation, loss function learning, few-shot learning and more. These problems are often formalized as Bi-Level Optimisations (BLO). The present techniques turn a given BLO problem into a stochastic optimisation, where the inner loss function becomes a smooth probability distribution, and the outer loss becomes an expected loss over the inner distribution. To solve this stochastic optimisation, Stochastic Gradient Langevin Dynamics (SGLD) MCMC is adapted to sample inner distribution, and a recurrent algorithm is proposed to compute the MC-estimated hypergradient. The present derivation is similar to forward-mode differentiation, but a new first-order approximation is introduced that makes it feasible for large models without needing to store huge Jacobian matrices. The main benefits are two fold: i) the present stochastic formulation takes into account uncertainty, which makes the method robust to suboptimal inner optimisation or non-unique multiple inner minima due to overparametrisation; ii) Compared to existing methods that often exhibit unstable behavior and hyperparameter sensitivity in practice, the present method leads to considerably more reliable solutions. It is demonstrated that the new approach achieves promising results on diverse meta learning problems and easily scales to learning 87M hyper-parameters in the case of VIT-B.

[070] The present techniques are now described in more detail. First, an introduction to stochastic optimisation is provided, and it is shown that it generalises the BLO problem. It is argued that solving the stochastic optimisation is preferable to solving the deterministic BLO due to better handling of inner optimisation uncertainty, which is illustrated via a failure case of BLO. The proposed algorithm for solving the stochastic optimisation is described below, in which new recursions for meta gradients are derived from stochastic gradient MCMC (Langevin dynamics).

[071] Stochastic Optimisation (SO)

[072] The present techniques aim to solve the stochastic optimisation problem: mm Ep(0| / )[me)] s.t. p(0|2)=^^ (2) where / (2,0) and £(2,0) are the given scalar functions for the problem, and Z(2) is the normaliser that ensures p(0|2) to be a proper distribution. There are no restrictions or constraints on the problem instance functions f () and EQ except for the minimal assumption that evaluating these functions and taking their derivatives are straightforward and easy. It will be seen that this stochastic optimisation formulation is quite flexible and general enough to encompass the general Bi-Level Optimisation (BLO) problem, that often arises in deep learning, as a special case. This connection will be seen in detail below.

[073] BLO as a special case of Stochastic Optimisation’. The Bi-Level Optimisation (BLO) problem, despite its several different forms, can be formally and concisely expressed as: min£K(2,0*(2)) s.t. 0*(2) = argmin£r(2,0) (3) A ^0 or G

[074] Often, £VQ is called the outer objective and £TQ is called the inner objective. In typical hyperparameter optimisation (HPO) problems, £VQ and £TQ correspond to validation and training losses respectively. Meanwhile 2 is the set of hyperparameters to learn and 0 are the main parameters (weights of the neural network). It will be shown that this (deterministic) BLO problem in (3) is a special case of the stochastic optimisation in (2). First, the objective function f(A, 0) in (2) is made equal to £V(A, 9), that is, f(A,9): = £V(A,9), (4) and the distribution p(9^A) is defined as: p(0W- = "Ta?'’ zrW = f e~£^ / r d9, (5) and t >0 is a hyperparameter known as the temperature. In other words, the energy function of p(0|A) is set to be: E(A,9y. = £t(a,9) / t (6)

[075] Through the hyperparameter t it is possible to control how certain (or uncertain) the impact of the inner optimisation is on the outer optimisation. As an extreme case, it is obvious that t^O, p(9\A) is made to converge to the delta function 8(9 - 9*(A)), and thus the stochastic problem (2) coincides with (3).

[076] Why is SO better than deterministic BLO?

[077] BLO’s deterministic nature inner optimisation might be problematic in certain situations. In this section, this issue is highlighted via an illustrative example where SO succeeds and BLO fails.

[078] Mainstream BLO algorithms used in deep learning rely on one single optimal solution 0*00 for the inner optimisation given A, even though the inner optimisation problem can be noisy (e.g., noise in training data) and / or can have multiple different local / global optima. Note that the latter situation almost always arises for models with a high degree of redundancy (i.e., overparametrisation) such as deep neural networks. The worst scenario is that a (deterministic) BLO algorithm unfortunately selectings an optimum 0*(2) that leads to a poor (high) outer loss at each outer iteration, which can eventually lead us to select a poor outer variable A as the final solution of the BLO.

[079] On the other hand, the stochastic optimisation formulation in (2) essentially takes into account all 6 as possible solutions to the inner optimisation problem through a probability distribution p(6|A). It is also beneficial in that the solutions become more robust to potential noise that may reside in the inner problem such as noisy training data, or solution by minibatch- SGD, since the present techniques deal with the average outer loss over all possible 0’s instead of a single-outcome loss.

[080] To illustrate this benefit clearly and explicitly, a toy scenario is presented of the polynomial function regression hyperparameter learning problem. Figures 1A to 1D show this illustrative toy problem. Figure 1A shows training and validation data. Figure 1B shows two 0^) solutions at A = 0.1 - on the left is Good 0*(A) (val loss 0.0083), while on the right is Poor 0*00 (val loss 0.3833). Figure 1C shows £y (A, 0*(A)), where each row = A, each column = one of 6*(A). Blue (red) indicates low (high) loss value. Figure 1D shows row-wise average validation losses Ep^^^^A, 0)] employed in the proposed stochastic optimisation (SO).

[081] First, the quadratic ftrue(x) = - 0.4%2 - 0.2% - 0.3 is consdered as the true data generating function on %e[-l,l], from which four points are sampled: T = {(-0.75,-0.375),(0.75,-0.675)} constituting the training data, and V = {(-0.5, -0.3), (0.5, -0.5)} the validation data (see Figure 1 A).

[082] The present model is assumed to have a cubic form, f(x; A, 0) = Ax3 + 02x2 + &ix + 0o where A e [-1,1] is the hyperparameterthat decides the degree / smoothness of the polynomial, and 0 = [00,0n 02]t e [-1,1]3 are the main parameters. The losses are defined as: £K(A, 0) = (f(x;A,0) - y)2 and £r(A, 0) = E(x,y)er (f(x;A, 0) - y)2 for the validation and train dataset V and T, respectively.

[083] With the BLO formulation, it can be solve by an exhaustive tabular method. Namely, for each candidate A, all possible inner optimal solutions 0*(A) can be found, which are infinitely many, but can be enumerated for some grid of values. Then, for each candidate (A, 0*(A)) the outer loss value LV{A, 0*{A)) is evaluated. This is depicted in Figure 1C. It can be seen that for each row (given A), there are many different 0*(A) that yield different outer losses Lv (cooler colours for lower Lv than warmer). Note, however that they all attain perfect inner loss £r = 0 (i.e., all are global inner optima). Meanwhile, by inspection, it can be seen that the optimal A has to be 0 (thus quadratic), and 0*(O) (one of the fourth column in Figure 1C) identifies the true quadratic ftrue. Examples of good and poor 0 ' (A) at A = 0.1 are shown in Figure 1B.

[084] Now, consider typical deterministic BLO solutions, which select just one of the inner optima 0'(A) for each A. In Figure 1C, it is possible to think of them as randomly selecting a column for each row A. Then the final solution A is chosen as the one with the smallest validation loss among those randomly selected. So, there is a high chance that the final solution is not the optimal one A = 0 (e.g., when a good 0*(A)is chosen for some suboptimal A 0, but a poor 0*(A) is chosen for A = 0). On the other hand, the stochastic optimisation considers the average Ep^-J-C^A, 0)] for each A, and selects the A with the smallest average. This is visualised in Figure 1D, where each row (A) has an averaged validation loss. Clearly it is guaranteed to select the optimal A = 0 as the final solution.

[085] When these two approaches are run on the tabularised BLO problem with 21 (11) equally spaced A (0*(A)) grid point values, the deterministic BLO solution via the random column selection was able to find the optimal A = 0 for only three out of 20 different runs, whereas stochastic optimisation always found the optimal solution.

[086] Proposed Solution: SGLD Derivative Recursion

[087] The present techniques provide a novel method to solve the stochastic optimisation (2). Using the Monte Carlo approximation, ep(0A)[f^ 0)] « ^=i f(A,9m),9m ~ p(9\A), (7) where M is the number of MC samples. The (approximate) hypergradient of the objective can be written as: d~^, (8) and using the chain rule {d for total, d for partial gradients), df^,em) _ df^,0m) df(A,0m) d0m di - di df) dF ' '

[088] Note that each sample 9m is a function of A. However, in general it may be difficult to express 9m explicitly in terms of A.

[089] The present techniques instead adopt the stochastic-gradient Langevin dynamic SGLD, which repeats the recurrence: 9 _ (10) where z ~ J\T(O, k2 / ) and e is a small (ideally infinitesimal) step size. It is assumed that e1+“ is virtually 0 for a >0.5 (e.g., e2 0 and e15 0), which is reasonable as e is considered to be very small. A nice property is that (10) only requires the gradient of logp(0 |A), not the logdensity itself, so even though the normaliser Z(A) might be complicated, it possible to only deal with the energy function, that is, aiogp(e|A) = _ a£W) de de ' '

[090] It is known that after some burn-in period, the iterates of (10) converge to the samples of the stationary distribution p(9 |A). Also according to the Markov chain theorems, it is possible to start from any initial iterate 9 to make the chain converge to the target stationary distribution after a burn-in period. Let 9^ be the first iterate in the SGLD recurrence: 9^ can be either independent of A (e.g., chosen randomly and independently of A) or a function of A (e.g., A is the initial network parameters to be meta-learned in the MAML-type problems). In the former de^ de^> —— = 0, while for the latter —— is non-zero, but it is assumed that it is (sub)linearly sparse, dA dA de^ and can be efficiently computed (e.g., in the MAML-type problems 01°) = A, and —— = I). dA Also, let B be the the number of burn-in iterations before convergence. Therefore, the SGLD recurrence is run for (B + M) iterations in total, the first B iterates are thrown away, and the last M iterates are collected. Following the notation in (7), the following is set: 01: = 92 : = 9^+^,..., eM-. = e(B+M\

[091] Now, for the first recurrence O® 9m, that is, 0(1)= g(o)+^W(o^) + ^zOT (12) by taking the gradient of both sides with respect to A, de<v _ dew e d diogpce^w d r- m — ~ ft + 2F——+ TFez d9^ e 32logp(0^°)|A) 32logp(0^°)|A) d9^ = dA + dAd9 + 992 dF^ = (J + ^A&,0™»~ + ^B(A,0™), (14) Z CIA A where in (14) let (15) A(A,oy = B^,ey = 92wA). v ’ J de2 ’ v ’ J dAd0

[092] The derivatives for the next recurrence 0(1) -> 0(2) are derived similarly. Continuing this up to e^m\ gives: ( / +-(4(2,0^) + ••• + 4(2,0^))))-^ + -(5(2,0(°)) + ••• +5(2,0^1))). dA 2 dA 2 (16) The symbol originates from exploitation of e2 0. However, storing even a single 4(2,0) or 5(2,0) is computationally infeasible for large-scale scenarios due to its huge matrix dimension (dim(0) x dim(0)) or (dim(0) x dim(2)).

[093] Instead, a recursion is derived directly from (9). Specifically, consider the second term of (9), denoted as gm(Ay df^M) _ dfq,0^) d0^ di - di d0 ' M, ' 7 =-gmW

[094] Note that gm(2) is a dim(2)-dirnensional vector, thus easy to deal with computationally. Now the recursion for gm(2) is derived as follows: d0^m^ 9^ = —e---yr (18) = df(A,0W). ^0^ V + (2,0(^-1)) + 4(2,0^-1)) • d0 dA 2 dA _ df(A,0^ . ¢10^-^ £ + £A^ 0(m-l)) . d0 V dA 2 V 7 2 k 7 dA 7 (19) (20) , d0^ + edf^0^) , g(m_1)) + 00 dA 2 00 dA « Sm-lW + (0^)-0(^))32^^^ (21)

[095] To have (20) from (19), e2 0 is exploited using the expression (16). In the under-brace of (21), a first-order approximation is used. The quality of the present first-order approximation is also empirically demonstrated below.

[096] In summary, the recursion for gm(A) is: 9mW = 9m-iW + (e(m) - 9^) • . + . (6(2,0^1)) + 4(2,0^^)-^11). (22) dA

[097] In (22), it can be seen that it is not needed to store the large matrices A and B at all, since the vector-Hessian product forms are availale. They can be easily computed in most modern deep learning libraries with the auto-differentiation capability. In PyTorch for instance, • B in (22) can be obtained by calling autograd.grad() with = ^logp(0(m-1)|2) and t2 = 00 d0 as: grad(tv A, grad_outputs = t2), (23) recalling that B(A, 9y. = logp(0|2). It is possible to compute -A and (9^ - • oAd0 d0 ^2 f(A dg2—- in a similar manner. Overall the computation for (22) requires only linear time and memory complexity in dim(0) and dim(A). The initial g0(A) can be easily computed. Once gm(A) is computed, it can be plugged back into (17) and (8) to compute the ultimate hypergradient.

[098] The hypergradient h-. = ^Ep^em[f(A, 0)] can be approximately computed by the recursion: Initially g0(A) and h = 0, and for m = 1,...,B + M, • 0(^0 = 0^-^-)-^12^111---111-1-.^2^ (24) 2 d0 • Apply gm_A(A) -> gm(A) recursion step in (22) . Ifm> B, (25)

[099] The proposed method is summarized as pseudocode in Algorithm 1, shown in Figure 2. Unlike the I FT methods, the present techniques do not require Hessian inversion (e.g., the a2 inverse of—logp(0|2)). Furthermore, the IFT methods are all reliant to the zero inner gradient condition, thus can be sensitive (in an unpredictable way) to the quality of the inner optimisation.

[100] Besides, when the inner optimisation can only be performed with stochastic minibatchbased updates (true for most deep learning scenarios), then this further deteriorates IFT’s performance since the gradient vanishing condition would not hold for the minibatch version almost surely. On the other hand, in the present techniques, note that the gradient dO can be safely replaced by the minibatch stochastic gradient, as the stochastic-gradient MCMC. This can lead to the same guarantee of the convergence to the samples from p(B |A).

[101] Complexity Analysis

[102] The derivation in a deterministic setting, namely if all stochastic components are removed by dropping SGLD noise terms y / cz in (24), looks very similar to forward-mode differentiation (FMD). The main difference is that the recurrence is proposed on gm with the first-order approximation in (21), removing the need to save the Hessian matrices A and B. The FMD, on the other hand, without such approximation, needs to save these matrices, making it infeasible especially for high-dim A cases. This is the crucial point that enables the present algorithm to be practical in real-world deep learning situations.

[103] Computational Complexity. The time and space complexity of the proposed algorithm is analysed and compared with FMD algorithms as well as the reverse-mode differentiation (RMD). The analysis is based on the techniques from algorithmic differentiation. The complexity of the competing algorithms is summarised in the table in Figure 3.

[104] Related Work

[105] Existing methods for hypergradient estimation in BLO can fall into two categories: iFT-based and Unroll-based. The key ideas and known issues of these approaches are highlighted here. I FT hinges on the stationary condition of the inner optimal solution, and computes the hypergradient from this implicit definition. Two popular variants differ in how to approximate the Hessian inversion required in I FT - by Neumann series approximation and conjugate gradient respectively. Since both variants are reliant on the gradient-equal-to-0 stationarity condition and involve Hessian inversion, their overall performance is sensitive to the quality of the inner optimisation in an unpredictable, highly nonlinear way.

[106] Unroll-based approaches basically approximate the inner optimal solution by an SGD iterate at some final (finite) step. So there are a chain of dependency constraints, which can be used to compute the hypergradients by the chain rule. The two different ways of applying the chain rule correspond to FMD and RMD. Despite its simplicity, RMD suffers from excessive memory usage to maintain a large computation graph built for the unrolled inner optimisation, which in turn leads to truncated approximations that bring problems of their own. FMD also requires large computational resources to store huge Jacobian matrices and does not scale to high-dimensional A. Although the present derivation looks similar to FMD in some respects, the present new approximate recurrence circumvents FMD’s inherent overhead. Other gradient-free strategies for solving BLOs such as evolution also do not scale to highdimensional A.

[107] Experiments

[108] Synthetic 1D Problem-. A simple BLO problem is considered where, in (3), the following is defined: , 1 , 03 , Lv^,9} = (A - ey + (0 - -)2, Lt^,9} = — -(1-A2)0 in A, 9 e [0,1], The inner loss function admits a closed-form global minimum 0*(A) = V1 - A2. By plugging this in the outer loss function and with some 1D line search, the optimal solution is A* = 0.7487 at 9* = 0.6629. This problem is tackled using the hypergradient methods, and the results are summarized in Figure 4, which is a table showing results from the synthetic 1D BLO problem and the errors in each solution. For all competing methods, 100 inner iterations are run while for HPO-SGLD, the inner iterations are split into B = 50 burn-in steps and M = 50 MC sample accumulation steps, for fairness.

[109] It can be seen that both IFT-Neumann / CG and HPO-SGLD are equally good, accurately identifying the true optimal values. On the other hand, RMD and FMD require more inner iterations as they only reach comparable errors at 1000 inner iterations. The number of inner loop iterations are also varied, and the results are shown in Figure 5. Figure 5 shows, for the synthetic 1D problem, the number of inner iterations vs. the errors of the learned solutions, (Left) |A- A*| and (Right) |0-e*|.

[110] Noisy inner loss case. Now to make the problem more realistic and difficult, the problem is modified in a way that the inner loss function is randomly perturbed at every call. More specifically, the new inner loss function is defined as: LT(A, 9) = (1 / 3 + ej • 93 - (1 - A2 + e2) • 9 (26) where 61,e2 ~ Uniform(-0.3,0.3). This modification makes the problem more realistic by mimicking the real-world situations where noise is often observed from various sources in the problem data (e.g., stochastic minibatch formation or noise in inputs and / or labels) in the inner loss function. In this case, the present stochastic optimisation formulation is expected to be particularly useful. The results summarized in Figure 6 show that deterministic approaches like IFT-Neumann / CG are very sensitive to the noisy inner loss function, failing to attain the optimal values. This is mainly due to violation of the strict gradient-equal-to-0 condition for the implicit function theorem. On the other hand, the stochastic optimisation treatment, for different degrees of stochasticity considered (t = 10^-2-5-1°}), leads to more robust estimation of the optimal values. Considering the highly stochastic nature of the problem, it is seen that incorporating more stochasticity (i.e., larger t) in the present HPO-SGLD model, leads to more accurate estimation.

[111] Goodness of First-Order Approximation: To demonstrate the quality of the first-order approximation that was introduced in (21), that is, gm-iW + - dO dA. •d true product of gradient terms (left hand side) and the approximate values (right hand side) are recorded and compared. The results are visualised in Figure 7. Figure 7 shows (left) relative errors between the true products of gradients vs. the present first-order approximates, and (right) cumulative errors, i.e., differences between true hypergradients and the present estimates. The (red / dotted) vertical lines indicate the end of the burn-in period. It can be clearly seen from Figure 7 that the approximation quality is very good. The errors are measured from the beginning of the recurrences. Hence the final accumulated meta-gradient estimate would be even more accurate since the meta-gradient estimates are dropped at the early iterations (relatively larger errors) as burn-in steps.

[112] L1-Regulariser HPO in ERM Learning: The method is tested on the L1-regularised ERM training of a deep network as in (1). Individual weight L1 regularisation is considered, R(A, 0) = Xj in which A has the same structure as the main backbone 0. Note that this BLO is not feasible to solve by a grid / discrete search due to the large number of hyperparameters to be searched. With the Vision Transformer (ViT-B-16) main network, the competing methods are tested on the Oxford-Pets, DTD and Flowers datasets.

[113] The results are summarised in the table of Figure 8. It can be seen that I FT (both Neumann and CG Hessian inverse approximation schemes) exhibits high variances for some option choices: for Neumann (a, j), the Hessian impact and the length of the Neumann series, respectively; for CG (y, i), the Hessian regulariser and the number of CG iterations, 24 respectively. This may originate from the unknown and unstable behavior of the Hessian inverse approximation schemes especially when the gradient-equal-to-0 condition is violated. The present HPO-SGLD leads to relatively robust solutions for different choices of underlying options; t and the noise scale factors (t is the temperature parameter that turns the inner optimisation problem to a probability distribution, and k is used for noise scaling in SGLD). In other words, one of the benefits of the present techniques is more stable final solutions, being less sensitive to underlying parameter choices, which originates from the underlying forwardmode differentiation and the present stochastic treatment without relying on the gradient-equal-to-0 condition. Compared to RMD, the present HPO-SGLD has significantly lower test errors. A main drawback of the RMD is its large GPU memory footprint in order to hold the full unrolled computation graph, and this considerably limits the number of inner iterations to be applied, which in turn can degrade the performance. FMD is simply infeasible to run in this model scale.

[114] Learning an Optimal Loss Function: Although it is conventional practice in deep learning to adopt the cross-entropy (CE) loss function for classification problems to train a deep network, it can be argued that the CE loss is not necessarily optimal, and depending on the data distributions / characteristics there might exist a truly optimal training loss function that is highly different from the CE loss. The optimal loss function learning is considered as a meta learning or HPO. The third-order polynomial parameterised loss function / / (•<•) is used. The loss function optimisation is to find the best 1 e R8 where the model 6, when trained with the loss / ^(- / ), maximises its validation performance, can be formulated as BLO with: lt^,e): = ^x,y)~DtrainMf(%;0),y)L (27) 0): = E(Xiy>Dval[CE(J(%; 0),y)] (28)

[115] The results on CIFAR-10 with Alexnet and CIFAR-100 with Wide-ResNet-28-10 (WRN), are summarised in Figure 9. Compared to the evolutionary search method and RMD / FMD, the present techniques yield much higher test accuracy. FMD did not run for the large WRN network case due to its computational infeasibility. Whereas I FT methods are oftentimes sensitive to the approximation hyperparameters with large variances, the present techniques are less sensitive to the related options.

[116] Few-shot Meta Learning: The present method is tested on the few-shot learning (FSL) problem. In the episodic learning setup, there are many (classification) tasks of different input domains / distributions and class semantics, where the meta learner can observe each task one by one as a few labeled examples. The goal is to make a model adapt well to a novel unseen test task using only a few labeled representative samples from the test task. Despite abundance of existing FSL methods in the literature, here, the focus is on the MAML-type approaches where the initial model is meta-learned so that the learned initial model can adapt easily to a new task via a few gradient descent updates with the few-shot samples. That is, the hyperparameters A =

[117] The present techniques are compared with: MAML, its first-order approximation that detaches computation graphs for high order derivative terms (FO-MAML), and the IFT-based reformulation of MAML called the iMAML. The present techniques are also contrasted with Reptile, a smoothed first-order update method. For the backbone network the popular four-layer conv net is used, and the competing meta learning algorithms are tested on the Minilmagenet dataset. The results are shown in Figure 10. For fair comparison, similar settings are used for the competing methods. The present techniques, HPO-SGLD, achieve consistently higher test performance than competing methods, being robust to the choice of the temperature t.

[118] Meta Learning of Implicit Neural Representation’. Next, the present techniques are tested on the meta learning of the implicit neural representation (INR). The main goal of the INR is to mimic a 3D imaging function T:IR3 -> IR4 such that T(x,y,z) = (r,g,b,a) is a mapping of any 3D coordinate point of an object to its RGB color value and the depth o. Ideally, one would like to find a neural network model fe (called the implicit neural representation) that is closest to the true 3D imaging function T, namely fe(x,y,z) « T(x,y,z) for all (x,y,z). For a vanilla (non-meta learning) INR problem, this output matching is done with a random initial 6, which often requires long training time. The idea of meta learning of INR arose in Tancik et al, in which a good initial model 0° is meta-learned such that once the training starts from this 0°, it converges much more quickly and reliably than starting from a random initial. So it is similar to MAML in that what the meta model aims to learn is the initial model to start a regular training with.

[119] The present techniques were tested on the task of single-view synthesis for the ShapeNet datasets. The meta learning formulation for this task was previously introduced in Tancik et al, in which the Reptile meta-learned initial model outperformed standard random initialisation, as shown in Figure 11 for the single-view meta test scenarios. An experimental protocol similar to thatofTanciketal is followed. The present techniques, HPO-SGLD, achieve performance comparable to, or sometimes better than, the state-of-the-art techniques.

[120] In summary, the present techniques provide a new hypergradient estimation method for meta learning. The present stochastic optimisation formulation takes into account uncertainty in inner optimisation, rendering solutions to robust to noise and non-unique inner optima. The present forward recursion method enables computationally tractable solutions even in large scale scenarios (e.g. 87M parameters and 87M hyperparameters for VIT-B-16).

[121] Figure 12 is a flowchart of example steps to train an ML model using the present techniques. The method is a computer-implemented method, and may be used to train or fine-tune / personalise a machine learning, ML, model to perform a specific task. The method comprises: obtaining a training dataset comprising a plurality of data items suitable for training the specific task to be performed by the ML model (step S100); obtaining a validation dataset comprising a plurality of data items suitable for validating the ML model (step S102); and training the M L model using the training dataset, by: initialising a value of each hyperparameter in a set of hyperparameters, wherein the hyperparameters are configurable variables of a process to train the ML model (step S104); choosing a value of each ML model parameter in a set of ML model parameters (step S106); iteratively learning over a first predetermined number M+B of iterations, using the initial value for each hyperparameter and using the training dataset, an updated set of ML model parameters that minimise a training loss, wherein the training loss assesses how well predictions of the ML model using the learned set of ML model parameters fit the training dataset (step S108); for each iteration m that is greater than a second predetermined number B of iterations and less than or equal to the first predetermined number M+B of iterations (steps S110 and S112): calculating, using the validation dataset, a validation loss for each learned ML model parameter, wherein the validation loss assesses how well predictions of the ML model using the learned set of ML model parameters fit the validation dataset (step S114); averaging the validation loss for the learned ML model parameters calculated for each iteration (step S116); determining an adjustment to the value of each hyperparameter using the average validation loss (step S118); adjusting the value of each hyperparameter using the determined adjustment and repeating the choosing, learning, calculating, averaging, determining and adjusting steps (step S120); and setting a value of each hyperparameter (using the value of each hyperparameter after the optimisation process is complete, not shown in Figure 12).

[122] Figure 13 is a block diagram of an example device 100 for training or fine-tuning / personalising an ML model 110 to perform a specific task. The electronic user device 100 comprises: at least one processor 102 coupled to a memory 104, arranged for: obtaining a training dataset 106 comprising a plurality of data items suitable for training the specific task to be performed by the ML model; obtaining a validation dataset 108 comprising a plurality of 27 data items suitable for validating the ML model; and training the ML model using the training dataset, by: initialising a value of each hyperparameter in a set of hyperparameters, wherein the hyperparameters are configurable variables of a process to train the ML model; choosing a value of each ML model parameter in a set of ML model parameters; iteratively learning over a first predetermined number M+B of iterations, using the initial value for each hyperparameter and using the training dataset, an updated set of ML model parameters that minimise a training loss, wherein the training loss assesses how well predictions of the ML model using the learned set of ML model parameters fit the training dataset; for each iteration m that is greater than a second predetermined number B of iterations and less than or equal to the first predetermined number M+B of iterations: calculating, using the validation dataset, a validation loss for each learned ML model parameter, wherein the validation loss assesses how well predictions of the ML model using the learned set of ML model parameters fit the validation dataset; averaging the validation loss for the learned ML model parameters calculated for each iteration; determining an adjustment to the value of each hyperparameter using the average validation loss; adjusting the value of each hyperparameter using the determined adjustment and repeating the choosing, learning, calculating, averaging, determining and adjusting steps; and setting a value of each hyperparameter (using the value of each hyperparameter after the optimisation process is complete).

[123] The memory 104 may store instructions that, when executed by the at least one processor 102 individually or collectively, cause the at least one processor to perform the above-described steps.

[124] The user device 100 may further comprise at least one interface 114 for capturing data used to form the training dataset and validation dataset. (The validation dataset may be a portion of the training dataset that is not used for training, and is instead held-back for the validation process). For example, the interface 114 may be an image capture device for capturing images.

[125] The ML model 110 may be a pre-trained ML model and may be fine-tuned / personalised for a specific task. In some cases, because of the nature of the ML model, in order to finetune the ML model, one or more adapter modules 112 may be inserted into the ML model and the adapter module(s) may be trained using the training dataset. This is particularly useful if it is desirable to fine-tune a large model which could not be trained from scratch on-device.

[126] References: • Stochastic-gradient Langevin dynamic SGLD - Welling, M. and Teh, Y. W. Bayesian Learning via Stochastic Gradient Langevin Dynamics. In International Conference on Machine Learning, 2011. • I FT - Neumann series approximation - Lorraine, J., Vicol, P., and Duvenaud, D. Optimizing Millions of Hyperparameters by Implicit Differentiation, 2020. Al and Statistics (AISTATS). • IFT - conjugate gradient - Rajeswaran, A., Finn, C., Kakade, S., and Levine, S. MetaLearning with Implicit Gradients. In Advances in Neural Information Processing Systems, 2019. • FMD &RMD - Franceschi, L., Donini, M., Frasconi, P., and Pontil, M. Forward and Reverse Gradient-Based Hyperparameter Optimization. In International Conference on Machine Learning, 2017. • Vision Transformer (ViT-B-16) - Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Geliy, S., Uszkoreit, J., and Houlsby, N. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020. • MAML - Finn, C., Abbeel, P., and Levine, S. Model-agnostic meta-learning for fast adaptation of deep networks. In International Conference on Machine Learning, 2017. • iMAML- Rajeswaran, A., Finn, C., Kakade, S., and Levine, S. Meta-Learning with Implicit Gradients. In Advances in Neural Information Processing Systems, 2019. • Reptile - Nichol, A., Achiam, J., and Schulman, J. On First-Order Meta-Learning Algorithms. In arXiv preprint arXiv: 1803.02999, 2018. • Minilmagenet dataset - Vinyals, 0., Blundell, C., Lillicrap, T., Kavukcuoglu, K., and Wierstra, D. Matching networks for one shot learning. In Advances in Neural Information Processing Systems, 2016. • Tancik et al - Tancik, M., Mildenhall, B., Wang, T., Schmidt, D., Srinivasan, P. P., Barron, J. T., and Ng, R. Learned initializations for optimizing coordinate-based neural representations. In IEEE / CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021. • ShapeNet datasets - Chang, A. X., Funkhouser, T., Guibas, L., Hanrahan, P., Huang, Q., Li, Z., Savarese, S., Savva, M., Song, S., Su, H., Xiao, J., Yi, L., and Yu, F. ShapeNet: An Information-Rich 3D Model Repository. arXiv preprint arXiv: 1512.03012, 2015. • Wide-ResNet-28-10 (WRN) - Zagoruyko, S. and Komodakis, N. Wide residual networks. In BMVC, 2016. • Oxford-Pets - Parkhi, O. M., Vedaldi, A., Zisserman, A., and Jawahar, C. V. Cats and Dogs, 2012. IEEE Conference on Computer Vision and Pattern Recognition. • DTD - Cimpoi, M., Maji, S., Kokkinos, I., Mohamed, S., , and Vedaldi, A. Describing textures in the wild. In Proceedings of the IEEE Conf, on Computer Vision and Pattern Recognition (CVPR), 2014. • Flowers - Nilsback, M.-E. and Zisserman, A. Automated flower classification over a large 5 number of classes, 2008. Proceedings of the Indian Conference on Computer Vision, Graphics and Image Processing.

[127] Those skilled in the art will appreciate that while the foregoing has described what is 10 considered to be the best mode and where appropriate other modes of performing present techniques, the present techniques should not be limited to the specific configurations and methods disclosed in this description of the preferred embodiment. Those skilled in the art will recognise that present techniques have a broad range of applications, and that the embodiments may take a wide range of modifications without departing from any inventive 15 concept as defined in the appended claims.

Claims

1. A computer-implemented method for training a machine learning, ML, model to perform a specific task, the method comprising:obtaining a training dataset comprising a plurality of data items suitable for training the specific task to be performed by the ML model;obtaining a validation dataset comprising a plurality of data items suitable for validating the ML model; andtraining the ML model using the training dataset, by:initialising a value of each hyperparameter in a set of hyperparameters, wherein the hyperparameters are configurable variables of a process to train the ML model;choosing a value of each ML model parameter in a set of ML model parameters;iteratively learning over a first predetermined number M+B of iterations, using the initial value for each hyperparameter and using the training dataset, an updated set of ML model parameters that minimise a training loss, wherein the training loss assesses how well predictions of the ML model using the learned set of ML model parameters fit the training dataset;for each iteration m that is greater than a second predetermined number B of iterations and less than or equal to the first predetermined number M+B of iterations:calculating, using the validation dataset, a validation loss for each learned ML model parameter, wherein the validation loss assesses how well predictions of the ML model using the learned set of ML model parameters fit the validation dataset;averaging the validation loss for the learned ML model parameters calculated for each iteration;determining an adjustment to the value of each hyperparameter using the average validation loss;adjusting the value of each hyperparameter using the determined adjustment and repeating the choosing, learning, calculating, averaging, determining and adjusting steps; andsetting a value of each hyperparameter.

2. The method as claimed in claim 1 wherein iteratively learning a set of ML model parameters comprises learning any one or both of: weights of at least one neural network of the ML model; and biases of at least one neural network of the ML model.

3. The method as claimed in claim 1 or 2 wherein choosing a value of each ML modelparameter in a set of ML model parameters comprises randomly selecting a value for each ML model parameter.

4. The method as claimed in claim 1 or 2 wherein choosing a value of each ML model parameter in a set of ML model parameters comprises selecting a value for each ML model parameter that is dependent on the initial value of each hyperparameter.

5. The method as claimed in any preceding claim wherein iteratively learning a set of ML model parameters comprises using gradient descent to determine how to adjust the value of each ML model parameter in each iteration.

6. The method as claimed in any preceding claim further comprising discarding the value of each ML model parameter that is iteratively learned upto the second predetermined number B of iterations.

7. The method as claimed in any preceding claim wherein initialising a value of each hyperparameter in a set of hyperparameters comprises initialising a value of any one or more of the following hyperparameters: learning rate; learning rate decay; momentum; number of neural network nodes; number of neural network layers; mini-batch size; and number of epochs.

8. The method as claimed in any preceding claim wherein initialising a value of each hyperparameter in a set of hyperparameters comprises randomly selecting a value for each hyperparameter.

9. The method as claimed in any preceding claim wherein determining an adjustment to the value of each hyperparameter using the average validation loss comprises:calculating a hypergradient using the average validation loss, wherein the calculated hypergradient indicates how to adjust the value of each hyperparameter.

10. The method as claimed in any preceding claim wherein repeating the choosing, learning, calculating, averaging, determining and adjusting steps comprises repeating the steps for a further predetermined number of iterations.

11. The method as claimed in claim 10 wherein setting a value of each hyperparameter comprises setting a value of each hyperparameter when the further predetermined number of iterations are completed.

12. The method as claimed in any preceding claim further comprising training the ML model using the training dataset and the set value of each hyperparameter.

13. The method as claimed in any preceding claim wherein the ML model is a pre-trained ML model comprising at least one untrained adapter module, and wherein training the ML model comprises training the at least one untrained adapter module.

14. The method as claimed in claim 13 wherein training the M L model comprises training the at least one untrained adapter module only and freezing the pre-trained ML model.

15. The method as claimed in claim 13 or 14 wherein the pre-trained ML model is a pretrained vision ML model, and wherein training the at least one untrained adapter module comprises training the at least one adapter module to perform a user-specific vision task.

16. The method as claimed in claim 13 or 14 wherein the pre-trained ML model is a pretrained large language model, LLM, and wherein training the at least one untrained adapter module comprises training the at least one adapter module to perform a user-specific language task.

17. The method as claimed in any preceding claim wherein the method is performed by a constrained-resource electronic user device.

18. A computer-readable storage medium comprising instructions which, when executed by a processor, causes the processor to carry out the method of any one of claims 1 to 17.

19. An electronic user device for training a machine learning, M L, model to perform a specific task, the user device comprising:at least one processor coupled to a memory, arranged for:obtaining a dataset comprising a plurality of data items suitable for training the specific task to be performed by the ML model;obtaining a validation dataset comprising a plurality of data items suitable for validating the ML model; andtraining the ML model using the training dataset, by:initialising a value of each hyperparameter in a set of hyperparameters, wherein the hyperparameters are configurable variables of a process to train the ML model;choosing a value of each ML model parameter in a set of ML model parameters;iteratively learning over a first predetermined number M+B of iterations, using the initial value for each hyperparameter and using the training dataset, an updated set of ML model parameters that minimise a training loss, wherein the training loss assesses how well predictions of the ML model using the learned set of ML model parameters fit the training dataset;for each iteration m that is greater than a second predetermined number B of iterations and less than or equal to the first predetermined number M+B of iterations:calculating, using the validation dataset, a validation loss for each learned ML model parameter, wherein the validation loss assesses how well predictions of the ML model using the learned set of ML model parameters fit the validation dataset;averaging the validation loss for the learned ML model parameters calculated for each iteration;determining an adjustment to the value of each hyperparameter using the average validation loss;adjusting the value of each hyperparameter using the determined adjustment and repeating the choosing, learning, calculating, averaging, determining and adjusting steps; and setting a value of each hyperparameter.

20. The user device as claimed in claim 19 further comprising an image capture device for capturing images, and a storage storing captured images;wherein obtaining a training dataset comprises comprising at least one data item representative of the task to be performed by the ML model from the storage.

21. The user device as claimed in claim 19 or 20, wherein the electronic user device is a smartphone or a tablet computer.

22. The user device as claimed in claim 19 or 20, wherein the electronic user device is a smart home appliance.

23. The user device of claim 22, wherein the smart home appliance is:a robot vacuum cleaner; a television, TV; a fridge; or an oven.

Images