Method and system for adapting an ML model

Abstract

A computer-implemented method, performed by a server, for generating an adaptable machine learning (ML) model. A base ML model is obtained, along with a plurality of adapters each of which can be comb

Description

Field

[001] The present techniques generally relate to methods and systems in which machine learning (ML) models are adaptable for compositional tasks. In particular, the present techniques provide adaptable large language models (LLMs) which are suitable for use on resource-constrained devices. Background I

[002] Large language models (LLMs) have revolutionized the world by their remarkable ability to solve various text-based tasks with an excellent performance. LLMs typically work with natural language data (e.g. Korean or English). An LLM is a model that takes in contextual information (a.k.a. prompts, in natural language) and produces a response (also in natural language). LLMs are increasingly powerful and becoming an integral part of technology applications. They are now commonplace in server applications and their adoption for on-device applications is underway.

[003] As shown in the table below, LLMs often have billions of parameters, making it necessary to use multiple high-end GPUs for inference for the larger models. However, many of the use-cases where LLMs are valuable deal with private data that users have on their resource-constrained devices, making it desirable to avoid sending the data to remote servers. For example, users may want to use LLMs on their private messages, e.g. to provide summaries of long conversations or to propose suitable replies. To address such use-cases, smaller LLMs are being developed, typically of size 1B to 3B parameters, especially for on-device applications. The table below compares the number of parameters of standard LLMs (GPT-3 and LLaMA3) and smaller LLMs. Model G PT-3.5 LLaMA3-70B LLaMA3-8B Gemini namo Gemini pro Gemma-26 Phi-1 Phi-2 Phi-3 No. of parameters (billions) 175 70 7 1.8 3.25 2 1.3 2.7 3.8

[004] As foundational models, LLMs, can be fine-tuned for numerous downstream tasks including question answering, summarization, translation and many others. A common way to fine-tune or adapt LLMs is by inserting a small number of additional parameters, also known as adapters, resulting in parameter-efficient fine-tuning (PEFT). These techniques are described for example in “Parameter-efficient fine-tuning for large models: A comprehensive survey” by Han et al. published on the ArXiv in 2024 and “Delta tuning: A comprehensive study of parameter efficient methods for pre-trained language models” by Ding et al. published on the ArXiv in 2022. The most widely-used PEFT method is LoRA described in “Lora: Low-rank adaptation of large language models” by Hu et al. published on the arXiv in 2021. LoRA trains Low-Rank Adapters that are inserted into different layers of the LLM.

[005] LoRA injects a small number of additional parameters into the model that are then trained, and they take the form of low-rank factorized matrices to ensure compactness. A model is often fine-tuned on one task at a time, generating so-called task-specific or expert models that are then merged to obtain a final merged model solving multiple tasks. Merging is described for example in “Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time” by Wortsman et al. published in the International conference on machine learning 2022. A task may represent, for example: (i) different domain of expertise (e.g., math or biology Q&A); (ii) different downstream task (e.g., summarization or translation), (iii) different convergence points of an optimization (e.g., different runs of an LLM training experiment). There is typically no support for multi-tasking in on-device LLMs without such solutions.

[006] Multi-tasking via model merging in LLMs has attracted significant attention as described for example in “Model merging in Urns, mllms, and beyond: Methods, theories, applications and opportunities” by Yang et al. published on the arXiv in 2024. This is due to its simplicity and the ability to easily support multiple tasks with one model. In particular, model merging enables the use of an LLM for tasks on which it has not been trained before, avoiding the need for expensive retraining or even fine-tuning. The weights of the model are, for example, averaged, and then they can perform the additional tasks with strong performance. Various techniques have been developed to perform model merging more successfully, with the most famous ones being simple linear merging as described in the Wortsman paper above, TIES described in “Ties-merging: Resolving interference when merging models” by Yadav et al. published in Advances in Neural Information Processing Systems in 2024 and DARE described in “Language models are super mario: Absorbing abilities from homologous models as a free lunch” by Yu et al. published by Forty-first International Conference on Machine Learning in 2024.

[007] After merging the models, the resulting model can switch between tasks based on user instructions. For example, when translation and summarization models are merged, the final model can translate the input text or it can summarize it, when given the corresponding instruction. Existing merging methods are thus limited to scenarios where each test example involves only a single task. The limitation becomes apparent in applications where simultaneous execution of tasks is required, for example, summarizing the text and translating the summary to a different language. In such case the model is expected to perform both tasks for a given input.

[008] The present applicant has identified the need for an improved model which can solve multiple tasks, particularly LLM customization to different downstream tasks. Summary

[009] In a first approach of the present techniques, there is provided a computer-implemented method for generating an adaptable machine learning, ML, model. The method comprises: obtaining a base ML model; obtaining a plurality of adapters each of which is combinable with the base ML model to adapt the base ML model to perform a specific task and defining multiple compositional tasks each of which comprises a first task and a second task. Each compositional task is to be performed by the adaptable ML model. For each of the defined compositional tasks; the method comprises: initializing a set of calibration parameters; identifying a first adapter from the plurality of adapters, wherein the identified first adapter adapts the base ML model to perform the first task; identifying a second adapter from the plurality of adapters, wherein the identified second adapter adapts the base ML model to perform the second task; generating a calibrated adapter by combining the first adapter, the second adapter and the initialized set of calibration parameters which are associated with the defined compositional task; combining the calibrated adapter with the base ML model to form a calibrated ML model; and training the calibrated ML model by learning the set of calibration parameters which fine tunes the calibrated ML model to the defined compositional task. When learning the set of calibration parameters, the parameters of the base ML model and the adapters are preferably frozen. The method comprises outputting an adaptable ML model which comprises the base ML model, the plurality of adapters and the learnt calibration parameters. The adaptable ML model is usable by the user device to generate a calibrated ML model (i.e. base ML model combined with adapters and a set of calibration parameters) for performing one of the multiple compositional tasks. In other words, the adaptable ML model may be adapted to form multiple calibrated (or adapted) ML models; one for each compositional task. As explained below, when learning the set of calibration parameters, the parameters of the ML model and adapters are fixed because these have already been trained.

[010] In other words, we provide a method of adapting a machine learning, ML, model comprising a plurality of adapters each of which performs a separate task. The method comprises defining a compositional task which is a composition of a primary task (or first task) and a secondary task (or second task); identifying a first adapter from the plurality of adapters for the primary task; identifying a second adapter from the plurality of adapters for the secondary task; combining the first and second adapter, adding parameters to the combined first and second adapter, wherein the parameters are additional to the parameters of the primary and secondary adapters and training the added parameters to adapt the ML model to perform the compositional task.

[011] The method of generating the adaptable machine learning ML model may be done on a server or other similar computing device which has sufficient resource to carrying out the training. The adaptable ML model may be output from the server to a user device for inference. The user device may be a computationally restrained device, e.g. may have resource limits on the computing / processing power and / or memory. By training the adaptable ML model at the server and then outputting to the user device, the output ML model with its set of adapters and calibration parameters may be adapted or calibrated for use on the user device to perform well on compositional multi-tasking. As explained in more detail below, at inference, the method may be considered to combine existing adapters stored on the device and correct them via a small number of additional calibration parameters so that the calibrated ML model is fine-tuned (i.e. adapted) to the specific compositional task. The benefits include strong performance on compositional tasks, significantly better than that of existing methods for merging adapters. In particular, the method may be suitable for on-device frameworks where resources are constrained and where an efficient, yet well-performing solution is important.

[012] A compositional task 7^(%) may be defined as W = ^(T^y. x^yA^y2 where x is the input, y2\s the output of the compositional task, y± is the intermediate output after applying the primary task T1 (first task) to the input and T2 is the secondary task (second task). For example, we may want to provide a translated summary (i.e. two tasks - primary task translation, secondary task summarization - or vice versa) of a long text in a different language. More generally, N tasks can be used, for which the compositional task would be defined as 7[^](x)7w(...72(71(x))), where TN(-T2(T1(,x^y.x y± ^y2 -> ••• yN. Input x is first processed by first task Tr to produce yr, second task 72 subsequently transforms yr into y2, and so on until the Nth task has been completed. For example, defining at least one compositional task may comprise defining a compositional task which comprises at least two tasks, (i.e. first task translation, second task summarization). There may be a third task (i.e. for the example above, the third task: tone adjustment) It will be appreciated that the number of adapters which are identified corresponds to the number of tasks. In other words, for each defined compositional task comprising first, second and third tasks, the method may further comprise identifying a third adapter from the plurality of adapters, wherein the identified third adapter adapts the base ML model to perform the third task and generating the calibrated adapter may comprise combining the first adapter, the second adapter, the third adapter and the initialized set of calibration parameters. It will be appreciated that the method may be adapted for any number of tasks.

[013] Outputting the learnt calibration parameters as part of the adaptable ML model may comprise outputting an adaptable ML model comprising multiple sets of calibration parameters with each set of calibration parameters associated with a specific compositional task. In other words, there are multiple sets of calibration parameters which are output from the server to the user device to be stored on the user device for use at inference. Alternatively, prior to outputting the learnt calibration parameters, the method may further comprise combining the learnt sets of calibration parameters to form a shared set of calibration parameters; and outputting the adaptable ML model with a shared set of calibration parameters. By shared set, it is meant that there is a single set of calibration parameters for all compositional tasks. In this way, the number of parameters which is output and stored may be reduced and the reduction can be balanced against the decrease in performance for the compositional tasks.

[014] Each of the plurality of adapters may be a low-rank adapter (LoRA - for example as described in “Lora: Low-rank adaptation of large language models” by Hu et al. published in the ICLR in 2022). In other words, each adapter may be considered a neural network or ML model itself. Each adapter may be termed an adapter module, and the terms may be used interchangeably. The ML model may comprise a plurality of layers and each adapter may be inserted into a different layer of the ML model. Each adapter may comprise a set of weights / parameters which are added to the weights / parameter of the ML model. The weights / parameters of the adapters may be expressed as matrices. Prior to training the calibration parameters, the weights / parameters of the adapters may be trained separately to fine-tune the ML model. In other words, the weights / parameters of the ML model are frozen when training the weights / parameter (calibration parameters) of the adapters.

[015] The set of calibration parameters may be in the form of a vector or a matrix. It will be appreciated that a vector may be considered a matrix with a single column. Generating the calibrated adapter may comprise forming a merged adapter by merging the first adapter and the second adapter wherein the merged adapter comprises a merged weight / parameter matrix. Combining the merged adapter with the set of calibration parameters (either in the form of a vector or matrix of calibration parameters) may be done by matrix multiplication. It will be appreciated that where there are more than two adapters, the merged adapter may be formed by merging all the necessary adapters.

[016] Each of the adapters may comprise a first weight / parameter matrix and a second weight / parameter matrix, e.g. B e ]Rdxr,X e ]Rrxk. More generally, there are adapter matrices for all N tasks Combining (or merging - the terms can be used interchangeably) the first and second adapter may comprise generating a first combined weight / parameter matrix by combining the first weight / parameter matrix of the first adapter and the first weight / parameter matrix of the second adapter and generating a second combined weight / parameter matrix by combining the second weight / parameter matrix of the first adapter and the second weight / parameter matrix of the second adapter and generating an overall adapter weight / parameter matrix by combining the first combined weight / parameter matrix and the second combined weight / parameter matrix. The first / second weight matrix of the first adapter and the first / second weight matrix of the second adapter combination may be combined using a simple sum or a weighted sum. A weighted sum is a form of summing in which different weights are assigned to the attributes being summed. Combining the first combined weight / parameter matrix and the second combined weight / parameter matrix may be done by matrix multiplication. When using a weighted sum to generate the first and the second combined weight / parameter matrices, the training step may further comprise training the learnable weight, e.g. together with the learning of the calibration parameters.

[017] Adding the parameters may comprise sharing the parameters across different layers of the ML model and / or adapters. Adding the additional parameters may comprise adding the additional parameters to the overall adapter weight matrix, for example when using a simple sum to generate the first and the second combined weight matrices. Adding the additional parameters may comprise adding the additional parameters as a calibration matrix which may be factorised into a first weight matrix and a second weight matrix. Alternatively, adding the additional parameters may comprise adding the additional parameters as a column-wise set of parameters.

[018] In each of the variations above, the addition of the additional parameters (or calibration parameters - the terms may be used interchangeably) may be considered as a method to save storage for merged LoRAs. The method learns a small number of correction parameters for the existing LoRA adapters. This has the benefit of requiring storing only negligible amount of additional data on the disk, unlike an approach that would perform well but require storing a full additional LoRA.

[019] Training the added / calibration parameters may comprise obtaining a training dataset comprising input-output pairs of data items wherein each pair comprises an input and a desired output which is generated by performing the compositional task on the input. The set of calibration parameters which fine tune the calibrated ML model to generate the desired output for each input may be learnt during the training. Any suitable training technique may be used, e.g. using cross-entropy loss. When training the added parameters (and / or learnable weight), the parameters of the adapters and the ML model may be frozen.

[020] The training dataset may be generated as a benchmark, for example via the use of large LLMs, the training dataset may be generated for training additional parameters for compositional multi-tasking. In a second approach of the present techniques, there is provided a method of generating a training dataset for adapting the ML model as described above. The method may comprise selecting at least one dataset corresponding to the primary task and at least one dataset corresponding to the secondary task. The plurality of adapters may have been pre-trained using known techniques and may be pre-trained using the benchmark dataset. The benefits of our proposal include that by training on our benchmark, we obtain strong performance on compositional multi-tasking.

[021] Once the ML model is trained, the method may be used with only one inference for compositional multi-tasking. In a third approach of the present techniques, there is provided a method of using an ML model which has been adapted as described above. In other words, the method may comprise receiving an input; generating, using the adapted ML model, an output by applying a compositional task to the input; and outputting the output. The application of the compositional task is achieved by correcting the combination of adapters for each of the considered tasks via few additional parameters as described above and generating outputs for the compositional task.

[022] In more detail, we describe a method for using, on a user device, an adaptable ML model which comprises a base ML model, a plurality of adapters and at least one set of calibration parameters. The method comprises: receiving an input to be processed by the adaptable ML model; determining, in response to receiving the input, whether a compositional task comprising a first task and a second task is to be carried out on the input; when it is determined that a compositional task comprising a first task and a second task is to be carried out on the input: identifying a first adapter from the plurality of adapters, wherein the identified first adapter adapts the base ML model to perform the first task; identifying a second adapter from the plurality of adapters, wherein the identified second adapter adapts the base ML model to perform the second task; generating a calibrated adapter using the first adapter, the second adapter and a set of calibration parameters which is associated with the determined compositional task; combining the calibrated adapter with the base ML model to form an calibrated ML model; and generating, by inputting the input into the calibrated ML model, an output. In other words, the adaptable ML model is used to generate a calibrated ML model which comprises the base ML model, the necessary adapters and the set of calibration parameters which mean that the base ML model is adapted and fine tuned / calibrated to the specific compositional task.

[023] When the input is received, an indication of a compositional task to be performed on the input may also be received (simultaneously or sequentially). The received indication may be used simply to determine that a compositional task is to be carried out on the input. Alternatively, when the input is received, a selection of a first task to be performed on the input and a second task to be performed on the input may also be received. These selections may be received simultaneously with or sequentially with the input. The received selections are used to determine whether a compositional task is to be carried out on the input.

[024] Receiving an input may comprise receiving an input from a newly installed application and / or direct from a user, e.g. via a user interface.

[025] Receiving an input may comprise receiving a text input, an image input, a combination of text and image input. The output may be an image, a text output or a combination of text and image (e.g. a summary with an emoji). For example, the input may be an image with text and a desired output is a textual summary in another language or a reply to the given image and text in a specific language or tone. As another example, the compositional task may be to generate an image to support a text output, e.g. to add emojis or other images to the output text so that the added image / emoji may be considered to be providing tone. As another example, an image which is supported by text may be generated as the output, e.g. an image of a user’s pet with a caption title in a given tone. It will be appreciated that the output generated from the input will depend on the nature of the base ML model as well as the adapters which can be included in the base ML model. When the input comprises an image and the output comprises an image, the calibrated ML model (whether generated in training or at inference) may be considered to be generating a modified or altered image.

[026] The benefits of the proposal include using only one inference pass through the calibrated ML model, rather than two that would be needed by a simple two-step solution.

[027] The base ML model may be any suitable model, for example a base LLM model such as a Gauss LLM which has been compressed to fit the storage limitations on the device. LLMs can be used for lots of different applications and the multi-tasking of the present techniques can be applied to any combination. For example, the primary task may be summarization or reply, and the secondary task may be translation or tone adjustment. As a non-exhaustive list of tasks, the applications / uses of LLMs include Q&A, grammar correction, advanced tweet classifier, explain code, summarize in a simpler manner, natural language to OpenAI API, keywords, factual answering, text to command, English to other languages, Ad from product description, product name generator, natural language to Stripe API, SQL translate, TI;DR summarization, Python bug fixer, parse unstructured data, classification, spreadsheet creator, JavaScript helper chatbot, Python to natural language, Movie to Emoji, ML / AI language model tutor, Science fiction book list maker, calculate time complexity, translate programming language, tweet classifier, airport code extractor and so on. It will be appreciated that this is merely an indicative list, and other applications may be considered. This list is taken from https: / / abiaryan.com / posts / intro-llms. The base ML model, the set of adapters and the training dataset may be obtained from any suitable source, e.g. a database or library.

[028] The base model may be alternatively be any suitable generative ML model, e.g. an image generation model or image generative model, e.g. Stable Diffusion XL (SDXL), Flux, Koala. When generating an image, the first task may be generating an image containing an object and the second task may be generating an image in a specific style. Merely as examples, the object may be selected from a user’s dog, cat or another pet; a toy such as a robot, a teddy bear, a duck; apparel such as shoes, sunglasses, clothes; decoration such as vase or candle and so on. Similarly, merely as examples, the style may be selected from watercolour painting, oil painting, 3d rendering, flat cartoon, line drawing, sticker and so on.

[029] The base ML model may be a multi-modal ML model such as LLaVA, GPT-4o, Gemini 1.5 Pro, Llama 3.2 Vision. Tthe first task may be selected from summarization or reply (or any other language based task such as those listed above) and the second task may be generation of an image (e.g. an emoji).

[030] The features described above with respect to the first approach apply equally to the other approaches and therefore, for the sake of conciseness, are not repeated.

[031] The user device and / or electronic device comprise at least one processor and memory storing instructions that, when executed by the at least one processor individually or collectively, cause the user device and / or electronic device to carry out the methods described above.

[032] The user device may be a smart device. The user device may be a smartphone. A smartphone is an example of a smart device. The user device may be a smart appliance. A smart appliance is another example of a smart device. An example of a smart appliance is a smart television (TV), a smart fridge, a smart oven, a smart vacuum cleaner, a smart robotic device, a smart lawn mower, and so on. More generally, the user device may be a constrained-resource device, but which has the minimum hardware capabilities to personalise / adapt a trained ML model, and to use the adaptable ML model. The user device may be any one of: a smartphone, tablet, laptop, computer or computing device, virtual assistant device, a vehicle, an autonomous vehicle, a robot or robotic device, a robotic assistant, image capture system or device, an augmented reality system or device, a virtual reality system or device, a gaming system, an Internet of Things device, or a smart consumer device (such as a smart fridge, smart vacuum cleaner, smart lawn mower, smart oven, etc). It will be understood that this is a non-exhaustive and non-limiting list of example devices.

[033] The electronic device may be a server. The electronic device may be any suitable computer or computing device or combination of such devices.

[034] In a related approach of the present techniques, there is provided a computer-readable storage medium comprising instructions which, when executed by at least one processor, causes the processor to carry out any of the methods described herein.

[035] In the cases where the present techniques are implemented or executed on a device comprising multiple processors, the present techniques may be implemented by one or more of the multiple processors. That is, the present techniques may be implemented by or executed by the processors individually or collectively.

[036] As will be appreciated by one skilled in the art, the present techniques may be embodied as a system, method or computer program product. Accordingly, present techniques may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects.

[037] Furthermore, the present techniques may take the form of a computer program product embodied in a computer readable medium having computer readable program code embodied thereon. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. A computer readable medium may be, for example, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing.

[038] Computer program code for carrying out operations of the present techniques may be written in any combination of one or more programming languages, including object-oriented programming languages and conventional procedural programming languages. Code components may be embodied as procedures, methods or the like, and may comprise subcomponents which may take the form of instructions or sequences of instructions at any of the levels of abstraction, from the direct machine instructions of a native instruction set to high-level compiled or interpreted language constructs.

[039] Embodiments of the present techniques also provide a non-transitory data carrier carrying code which, when implemented on a processor, causes the processor to carry out any of the methods described herein.

[040] The techniques further provide processor control code to implement the abovedescribed methods, for example on a general-purpose computer system or on a digital signal processor (DSP). The techniques also provide a carrier carrying processor control code to, when running, implement any of the above methods, in particular on a non-transitory data carrier. The code may be provided on a carrier such as a disk, a microprocessor, CD- or DVD-ROM, programmed memory such as non-volatile memory (e.g. Flash) or read-only memory (firmware), or on a data carrier such as an optical or electrical signal carrier. Code (and / or data) to implement embodiments of the techniques described herein may comprise source, object or executable code in a conventional programming language (interpreted or compiled) such as Python, C, or assembly code, code for setting up or controlling an ASIC (Application Specific Integrated Circuit) or FPGA (Field Programmable Gate Array), or code for a hardware description language such as Verilog (RTM) or VHDL (Very high speed integrated circuit Hardware Description Language). As the skilled person will appreciate, such code and / or data may be distributed between a plurality of coupled components in communication with one another. The techniques may comprise a controller which includes a microprocessor, working memory and program memory coupled to one or more of the components of the system.

[041] It will also be clear to one of skill in the art that all or part of a logical method according to embodiments of the present techniques may suitably be embodied in a logic apparatus comprising logic elements to perform the steps of the above-described methods, and that such logic elements may comprise components such as logic gates in, for example a programmable logic array or application-specific integrated circuit. Such a logic arrangement may further be embodied in enabling elements for temporarily or permanently establishing logic structures in such an array or circuit using, for example, a virtual hardware descriptor language, which may be stored and transmitted using fixed or transmittable carrier media.

[042] In an embodiment, the present techniques may be realised in the form of a data carrier having functional data thereon, said functional data comprising functional computer data structures to, when loaded into a computer system or network and operated upon thereby, enable said computer system to perform all the steps of the above-described method.

[043] The method described above may be wholly or partly performed on an apparatus, i.e. an electronic device, using a machine learning or artificial intelligence model. The model may be processed by an artificial intelligence-dedicated processor designed in a hardware structure specified for artificial intelligence model processing. The artificial intelligence model may be obtained by training. Here, "obtained by training" means that a predefined operation rule or artificial intelligence model configured to perform a desired feature (or purpose) is obtained by training a basic artificial intelligence model with multiple pieces of training data by a training algorithm. The artificial intelligence model may include a plurality of neural network layers. Each of the plurality of neural network layers includes a plurality of weight values and performs neural network computation by computation between a result of computation by a previous layer and the plurality of weight values.

[044] As mentioned above, the present techniques may be implemented using an Al model. A function associated with Al may be performed through the non-volatile memory, the volatile memory, and the processor. The processor may include one or a plurality of processors. At this time, one or a plurality of processors may be a general-purpose processor, such as a central processing unit (CPU), an application processor (AP), or the like, a graphics-only processing unit such as a graphics processing unit (GPU), a visual processing unit (VPU), and / or an Al-dedicated processor such as a neural processing unit (NPU). The one or a plurality of processors control the processing of the input data in accordance with a predefined operating rule or artificial intelligence (Al) model stored in the non-volatile memory and the volatile memory. The predefined operating rule or artificial intelligence model is provided through training or learning. Here, being provided through learning means that, by applying a learning algorithm to a plurality of learning data, a predefined operating rule or Al model of a desired characteristic is made. The learning may be performed in a device itself in which Al according to an embodiment is performed, and / o may be implemented through a separate server / system.

[045] The Al model may consist of a plurality of neural network layers. Each layer has a plurality of weight values and performs a layer operation through calculation of a previous layer and an operation of a plurality of weights. Examples of neural networks include, but are not limited to, convolutional neural network (CNN), deep neural network (DNN), recurrent neural network (RNN), restricted Boltzmann Machine (RBM), deep belief network (DBN), bidirectional recurrent deep neural network (BRDNN), generative adversarial networks (GAN), and deep Q-networks.

[046] The learning algorithm is a method for training a predetermined target device (for example, a robot) using a plurality of learning data to cause, allow, or control the target device to make a determination or prediction. Examples of learning algorithms include, but are not limited to, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning. Brief description of the drawings

[047] Implementations of the present techniques will now be described, by way of example only, with reference to the accompanying drawings, in which:

[048] Figure 1 is a schematic representation of an enhanced / adaptable LLM of the present techniques that supports compositional multi-tasking;

[049] Figure 2a is an alternative simplified representation of the enhanced LLM of the present techniques;

[050] Figure 2b is a schematic illustration of an LLM model for multi-tasking according to the prior art for comparison;

[051] Figure 3a is a schematic illustration of a set of adapters for use with an LLM to achieve multi-tasking according to the prior art;

[052] Figure 3b is a schematic illustration of a set of adapters for use with an LLM to achieve multi-tasking according to the present techniques;

[053] Figure 4 is a flowchart of the steps carried out during server-side pre-training for the present techniques;

[054] Figures 5a to 5c illustrate three variations of the proposed approach which can be used in the method of Figure 4;

[055] Figure 6 is a schematic illustration comparing separate sets of calibration parameters and a shared set of calibration parameters;

[056] Figures 7 to 9 show three different variations of the steps which may be carried out by a user device at inference;

[057] Figure 10 is a schematic block diagram of a system for the proposed techniques.

[058] Figure 11 is a table showing the results of different techniques for performing four different compositional tasks;

[059] Figure 12 is a table showing that using the adapters already available helps improve the performance in compositional multi-tasking;

[060] Figure 13a plots count against update parameter value for the linear merge approach;

[061] Figure 13b plots count against update parameter value for the proposed approach using calibration parameters;

[062] Figure 13c plots the weight norm value against different components within the LoRA matrix for the linear merge approach;

[063] Figure 13d plots the weight norm value against different components for the proposed approach using calibration parameters;

[064] Figures 14a to 14d show the same results as Figures 13a to 13d when carrying out reply + tone adjustment;

[065] Figures 15a and 15b plot Rouge-L score and LLM judge score against model size for different approaches;

[066] Figures 16 and 17 provide an extended evaluation using all standard metrics, including ROUGE-1 (R1), ROUGE-2 (R2), ROUGE-3 (R3), and Weighted ROUGE (W-R)

[067] Figure 18a plots the loss against samples processed for the summarization main task;

[068] Figure 18b plots the loss against samples processed for the reply main task; and

[069] Figure 19 is a schematic representation of an architecture for an on-device use case for the proposed techniques. Detailed description of the drawings

[070] The present techniques generally relate to methods and systems in which machine learning (ML) models have been adapted for compositional tasks. In particular, the present techniques provide large language models (LLMs) or other generative models which have been adapted for diverse downstream tasks via parameter-efficient fine-tuning techniques such as Low-Rank Adapters (LoRA) or other similar adapter modules. This approach involves adding a learnable projection layer (may also be termed a learnable calibration adapter) on top of the combined adapters which are needed to handle the compositional task. The parameters for this learnable projection layer are adapted / learnt to improve the performance of the combination of the LLM and adapters when carrying out the compositional task. Advantageously, this approach enables effective integration while maintaining efficiency through reduced computational overhead compared to alternative strategies requiring extensive retraining or sequential processing.

[071] As explained in more detail below, given some task-specific adapter parameters, a compositional multi-tasking model can be built that is efficient and achieves comparable performance to inefficient but well-performing known approaches. Using the compositional multi-tasking, two or more tasks can be performed simultaneously, for example summarization combined with translation for LLMs or subject and style combined for image generation. For example, the text “a successful gold investor recently explained to me that stock prices lost steam for more than a decade before the Dow Jones index broke through the 1,000 mark in the early 1980s” can be summarized and translated from English to Spanish to give “Se tardo mas de una decada para que el indice Dow Jones cruzara la marca de los 1.000 a principios de los anos 1980”. The challenge is to perform both jointly for best efficiency, in contrast to doing them separately via separate inference passes. As described below, good performance is obtained with one inference pass through the final model only so that the end users do not need to wait an unnecessarily long time.

[072] Compositional multi-tasking has practically valuable use-cases yet has not been studied within the context of on-device large language models or natural language processing. In other words, compositional multi-tasking is a new problem setting that has not been exploited for on-device LLMs before and has large commercial value as the combined tasks have significant value for users. For example, the combination of summarization and translation could be valuable for people that move abroad and need to understand the overall message of long texts that are in the local language. Other examples of compositional multi-tasking include proposing suitable replies in cross-lingual scenarios, and combinations of summarization or reply with tone adjustment. The challenge is to perform both jointly for best efficiency, in contrast to doing them separately via two inference passes.

[073] Figure 1 is a schematic representation of the enhanced LLM 100 of the present techniques which may be used on a device for carrying out a compositional task. It will be appreciated that the LLM may be replaced by another generative model, e.g. for image processing. A generic task T maps the input text x to an output text y; i.e , T(-): x y. For example, x is either a long text or part of a conversation and then the goal is to provide a summary or suitable reply in a specific language or tone. Formally, we define a two stage compositional task as = x^yr^y2 where Tr is the primary task e.g. summarization and reply suggestion (often known as “smart reply”), T2 is the secondary task, e.g. translation or tone adjustment ( / .e , rewriting), x is the input, yr is an intermediate output from applying the primary task to the input and y2 is the final output generated by applying the secondary task to the intermediate output. More generally, N tasks can be used, for which the compositional task would be defined as where ^(■■■^(^iW)):^ ^Xi ^y2 ••• Input x is first processed by primary task Tr to produce yr, secondary task T2 subsequently transforms y± into y2, and so on. The number of tasks is typically N = 2, but results are also demonstrated for N = 3 and it is expected that more tasks could be combined.

[074] The base LLM 112 may be any suitable model, for example a Gauss LLM which has been compressed to fit the storage limitations on the device. Although Figure 1 shows a base LLM, the base model may also be an image generative model, e.g. Stable Diffusion XL (SDXL), Flux, Koala or a multimodal LLM such as LLaVA, GPT-4o, Gemini 1.5 Pro, Llama 3.2 Vision. As shown the base LLM 112 (or other base model) is enhanced by the insertion of one or more adapters into the model for example by inserting into the placeholder 110. Ina typical on-device scenario the LLM is assumed to be relatively small (e.g., of size between 1B and 3B parameters) and the adapters are stored on the device to support the individual tasks such as summarization, translation, etc. Each such adapter can be e.g., 20-100MB in size. To handle the compositional combinations of the tasks, a very small number of additional / calibration parameters 150 is also stored on the device. While prompting can be sufficient with large-scale models, adapters, specifically LoRAs, are commonly used in on-device settings to provide excellent performance in the desired tasks.

[075] A set 120 of adapters may be stored on the device and merely as examples, these adapters are low-rank adapters, for example as described in “Lora: Low-rank adaptation of large language models” by Hu et al. published on the arXiv in 2021 and in more detail below. There are four illustrative adapters: English summarization adapter 122, English reply adapter 124, English to Spanish translation adapter 126 and English to French translation adapter 128 but it will be appreciated that any number of adapters may be stored, subject to storage limitations on the device. The adapters in the set 120 of adapters may be selected to adapt the LLM to the most commonly used tasks on the device.

[076] LoRAs provide an efficient way to fine-tune LLMs by introducing learnable low-rank factorized matrices B e ]Rdxr,A e ]Rrxk for r « min (d,k) that are added to the weights Wo e IRdxk of the model. During training and inference, the weights Wo are frozen and the adjusted forward pass gives: h = Wox + AWx = Wox + BAx Typically, the adapters are stored on the device for both the primary task Ti (Bi, Ai) and the secondary task T2 (B2, A2). More generally, there are LoRAs {B^Ai}1^ for all N tasks {7)}" t available on-device.

[077] Figure 2a is another schematical illustration of the enhanced LLM 100 of the present techniques that supports compositional multi-tasking. In this case, the input to the enhanced LLM 100 is a “long English text” and the desired output is a “short Spanish text”. Thus, the compositional task comprises a first task (or main task) of summarisation and a second task (or auxiliary task) of translation. The main task determines the main functionality, while the auxiliary task modifies the output to meet additional requirements. This is achieved by combining the English summarization adapter and the English to French translation adapter to form a merged adapter 130 and learning only a small number of additional parameters (not shown) compared to a new adapter as described in more detail below. Thus, the output is generated from the input, using only a single inference pass.

[078] Merely as a comparison, Figure 2b illustrates how prior techniques could generate a similar output from the same input. In Figure 2b, there is a base LLM 200 which can also be enhanced using a LoRA adapter. The base LLM 200 may be same base LLM as in Figure 2a, e.g. a compressed Gauss LLM. However, in this example, there is no merging of LoRA adapters. Accordingly, the base LLM 200 must first be adapted to provide the English summarization and this is achieved in a first pass in which the base LLM 200 is enhanced by an English summarization adapter 222. Once the summarization output is produced, this can be passed back into the base LLM 200 which is now adapted to provide the English to French translation. This is achieved in a second pass in which the base LLM 200 is enhanced by an English to French translation adapter 228. Each of the adapters may be the same as the adapters shown in Figure 1.

[079] Figure 3a illustrates a known way of achieving a compositional task in a single pass using a “joint-expert” approach. For simplicity just the set of adapters 320 is shown but it will be appreciated that they will be incorporated into the base model as previously described. The set of adapters 320 may contain the same single task adapters as shown in Figure 1, e.g. English summarization adapter 222, English reply adapter 324, English to Spanish translation adapter 326 and English to French translation adapter 328. Additionally, the set of adapters 320 comprises combined or merged adapters which perform two or more tasks in a single inference pass. Each of these merged adapters has been separately fine-tuned. Merely as examples, the merged adapters may comprise a summarization and English to Spanish translation adapter 340 and a summarization and English to French translation adapter 342. Figure 3b schematically illustrates that in the present techniques, it is not necessary to store the merged adapters.

[080] The proposed LLM achieves higher accuracy than the single LoRA adapters shown in Figure 2a on compositional multi-tasking while being highly efficient. The enhanced LLM of Figure 1 is strong in multi-tasking, has the same inference time and similar storage as the comparison LLMs in Figures 2b and 3a, and has no need for specialized compositional multitasking LoRAs.

[081] Compositional multi-tasking is especially suitable for on device Al, where we can assume only restricted computational resources. Moreover, there is a desire to perform all processing directly on the device (e.g. a smartphone) without sending data to the cloud where it could be subject to privacy concerns. Deploying such ML systems on personal devices introduces the following desiderata’. 1) high accuracy: provide a well-performing solution; 2) low inference time: perform only one inference pass with the on-device LLM; 3) low storage: the solution should not require much additional storage and can instead benefit from reusing the existing task-specialized adapters. Therefore, training an additional specialized adapter as shown in Figure 3a to perform the compositional task is not a viable option, as each new adapter requires non-negligible storage. Nonetheless, the inefficient approaches that either (i) perform two inference passes using the different adapters (Figure 2b), or (ii) introduce an additional adapter to solve the compositional task (Figure 3a) are useful as reference points.

[082] As the problem setting of compositional multi-tasking is new, a new benchmark considering four compositional tasks is introduced: summarization and reply as primary tasks paired with translation and tone adjustment as secondary tasks is also described in more detail below. Primary tasks define evaluation of the performance and are performed first in a two-step solution, while secondary tasks modify the ground truth texts of the primary tasks. In the context of Samsung phones, the tasks may include summarization (Samsung Notes), translation (Samsung Translate), smart reply (Samsung Email / Messages / etc.) and tone adjustment (Samsung Chat Assist). With the new benchmark, existing solutions can also be evaluated and compared with the proposed solution. Our approach: Learnable Calibration

[083] The goal is to obtain strong performance on the compositional tasks while being efficient in terms of computations and storage. The proposed learnable approach utilizes data from compositional tasks during server-side pre-training of the additional parameters.

[084] Figure 4 is a flowchart of the steps carried out during server-side pre-training. In order to pre-train the additional / calibration parameters to enable solving compositional tasks, access to the following is required: a base LLM (0), at least primary and secondary task LoRAs Bi, Ai, B2, A2 or more generally task specific adapters and data D)c from the compositional task T^. In Figure 4, such data is shown as being obtained in separate steps S402, S404, S406 but it will be appreciated that the data can be obtained together. The required information is obtained from any suitable source. The training data is in the form of input-output pairs (x, y), where x is the input such as a long text, and y is the ground-truth output, such as the summary in another language. The pretraining of the additional parameters is done on the server, so that there is access to ample training data, i.e. data availability is not an issue in this training approach. Other training details may be obtained, e.g. in step S402. These include the number of training steps T and a learning rate. Thus, the first step of the method is to check whether the number of training steps has been carried and if not, proceed with the detailed training.

[085] The key idea of the proposed approach is relatively simple: use the combination of single task LoRAs (e.g. primary and secondary LoRAs) as the initial starting point that is then calibrated further via a relatively small number of additional parameters P specific to the given compositional task. In other words, as shown at step S412, each adapter which is required for the compositional task is merged with each other and the small number of additional parameters P. The number of additional parameters is negligible compared to using a new adapter specialized for the task, making it easy to store the additional parameters on-device for the different compositional tasks. In the initial step t=0, the additional parameters may be initialized to zero or another suitable initialization value as detailed below.

[086] At step S414, the calibrated adapter is combined with the base LLM to generate an enhanced LLM (which may also be termed a calibrated LLM and the terms may be used interchangeably). A sample from the training dataset D)c is obtained at step S416 and the predictions are then generated at step S418. When there are two tasks, the resulting forward pass can be expressed as: h = Wox + AWcx = Wox + f(P, B2, A2, Blt A^x where function f applies parameters P on top of the single task LoRAs. More generally, the resulting forward pass can be expressed as: h = Wox + AWcx = Wox + f(P, [Bit A^^x,

[087] At step S420, a loss, e.g. cross-entropy loss that is normally used fortraining / fine-tuning LLMs, may be calculated. The loss is the difference between the predicted output and the ground-truth output in the training dataset. The loss may be used at step S422 to update the calibration parameters, e.g. by selecting parameters which minimize the loss. The process then loops back to determining whether all the training steps have been carried out. If not, there is another iteration of the steps described above. Once training is complete, the trained / learnt calibration parameters are output at step S430. The complete process of Figure 4 is repeated for each pair or combination of adapters. The set of calibration parameters may be output from the server to a user device together with the base LLM and set of adapters. There are various options for how to utilize parameters P and we describe these next.

[088] Figures 5a to 5c illustrate three variations of the proposed Learnable Calibration approach, each of which allow control of the size-performance trade-off. In Figures 5a and 5b a calibration vector comprising a plurality of calibration vectors is used and in Figure 5c, a calibration matrix is used. In each of 5a to 5c, the adapters are linearly merged and calibrated via the use of parameters P. The parameters P are shared across all layers of the base LLM for parameter efficiency, but they may be specific to each component such as query or key projection, as these generally come in different sizes.

[089] In the variation shown in Figure 5a, the adapters are jointly merged with the same weighting. Thus, for the two adapter scenario, the merging may be expressed as: B' = 0BB1 + 0.5B2, A' = 0.52^ + 0.5A2 There is no need to learn any merging coefficients and thus the arrangement in Figure 5a uses the smallest number of Learnable Calibration parameters. The parameters / biases are in the form of a vector p that are applied to the LoRA update matrix AW' = B' A' as: AWC = p © B'A'= p © AW' = Xt=1 Pi&wi’ where © represents element-by-column addition and AWC is the calibrated adapter. The learnable biases p are initialized to 0. This variation is the most efficient and typically learns around 20k parameters (new LoRA would be e.g. 30M params.) The size on the disk would be around 0.05MB (compared to around 50MB needed by a new LoRA). In other words, there are large memory savings.

[090] The variation in Figure 5b is similar to the variation in Figure 5a in terms of efficiency as it learns a similar number of parameters overall. The key difference is that it also learns how to do a weighted average of the existing LoRAs. In other words, for the two-task scenario, the merging may be expressed as: B' = + 21'= £2^ + (1 - £)212 There is one weight p per each component and is otherwise shared across all layers of the base LLM. The weight is set to 0.5 at the start of the optimization process. The size on the disk would be around 0.05MB (compared to around 50MB needed by a new LoRA). In other words, there are large memory savings.

[091] The variation in Figure 5c introduces more parameters than Figure 5a or Figure 5a, but there are still significantly fewer parameters than what would be needed by a new LoRA. For example, it introduces 160k additional parameters instead of 30M. In this variation, a calibration matrix P is introduced. The calibration matrix is factorized into two low-rank matrices P2 e R dxs, Pi e IRsxk for s « min (d, k}. Merely as an example, the rank of the LoRA may be 32 and the rank of the calibration matrix may be 4. The calibration matrix can be understood as adding a new calibration LoRA on top of the combined LoRAs, resulting in the following overall update matrix: AWC = P2P1 + B'A' Compared to standard LoRAs it is shared across all layers but is initialized in the same way. For example, P± may be initialised as a random Gaussian matrix and P2 may be initialised with all values at zero.

[092] An experimental ablation study is described in more detail below showing that it is indeed beneficial to use the calibration parameters on top of the combination of adapters that were designed for single task use-cases - rather than simply omitting the existing adapters.

[093] In addition, a scenario where the parameters of Learnable Calibration across different compositional tasks is considered. This is schematically illustrated in Figure 6 which shows how the individual parameters Plt P2, P2 and P4 for four compositional tasks Tf, T2, T2 and Tf are shared as the same parameters P for the various considered combinations of tasks. In such a case, the dataset is modified to include examples from all four compositional tasks. By sharing additional parameters P across all compositional tasks, the overall number of additional parameters may be even smaller. There is a certain amount of performance loss from the parameter sharing though, which will be analysed in the results section.

[094] Figure 4 details the steps of the algorithm to train learnable calibration and this may be summarised as: Input: single-task LoRAs already stored on the device, LLM, dataset of examples from the compositional task (inputs x-e.g. long texts, outputs y- e.g. translated summaries) Initialize additional parameters PtoO (no impact) and insert them into the LLM While training: Sample new minibatch of examples from the compositional task Use the LLM with P on the given minibatch of examples and compute loss Update the additional parameters P via back-propagation Similarly, the algorithm used during deployment may be summarised as: Load relevant single-task LoRAs, average them, and transform via P to obtain the overall update weights.

[095] Figures 7 to 9 show in more detail the steps which may be carried out by the user device at inference. Typically, the user devices have restricted resources (e.g. restricted memory and / or computing power) and thus the proposed techniques will need to be efficient in terms of computation and storage. In general terms, the steps at the user device may be triggered when a new input (app installation, user input, etc.) is received. For example, as shown in Figure 7, inference may be triggered by receipt of a text input together with a primary task selection and a secondary task selection. In Figure 8, an example in which the user defines the compositional task rather than the individual tasks is the input, together with the text input. Figure 9 shows a variation of Figure 8 in which the text input is received from an app rather than a user and thus app installation may be the trigger. It will be appreciated that these are just examples. Once the input has been received, there is a step to determine the necessary tasks and then judge whether the tasks need to be combined to form a compositional task. When there is a compositional task, the merged and calibrated adapter may then be generated.

[096] As shown in Figure 7, the first step S700 is for the user device to receive (e.g. from a server) the base LLM, the set of adapters which can be added to the LLM to improve performance for certain, individual tasks and the set of calibration parameters to be used when combining the adapters to carry out compositional tasks. The base LLM may be any suitable on-device model such as LLaMA 3.2 1B, Qwen2.5 1.5B and StableLM2 1.6B models. The calibration parameters may have been generated on a server as described above. The received information may be stored on the user device for use at inference.

[097] Inference may be triggered by receipt of a text input from a user as shown at step S702. At the same time as the user sends the text input, the user may also send a primary task selection and a secondary task selection. In other words, the user device receives a primary task selection at step S704 and receives a secondary task selection at step S706. Thus, the compositional task to be carried out on the input is received from the user. These user inputs may be received in any standard way, e.g. via an appropriate user interface. For simplicity, a compositional task comprising only two sub-tasks is considered in the examples of Figures 7 to 9. It will also be appreciated that the user may define a compositional task which comprises more than two tasks and thus the user device may receive a primary task selection and a plurality of secondary task selections.

[098] The text generation system of the user device may, in response to receiving the primary and secondary task selections, obtain the corresponding primary and second task adapters as shown at steps S714 and S716. These adapters are typically downloaded from storage (e.g. a disk) on the user device because they were received in the initial step S700. Once the required adapters have been obtained, the system identifies, at step S718, the calibration parameters which are required to improve performance of the merged adapters. At step S720, the calibrated adapter is then generated and then combined with the base LLM to generate the enhanced LLM, for example as described in relation to Figure 4.

[099] The received test input is then input into the enhanced LLM and as shown at step S724 the output is generated. The response text may then be output at step S726 using any suitable user interface, e.g. oral, text or visual.

[100] Figure 8 shows a variation of Figure 7 in which the user defines the compositional task which is to be carried out on the text input rather than the individual tasks which are to be combined. The transfer of base LLM, adapters and calibration parameters at step S700 is assumed to have already taken place. Accordingly, the steps shown in Figure 8 are triggered when the user device receives a text input at step S802 and a task selection at step S804. At step S810, there is a determination as to whether the received task selection is a compositional task. If the task is compositional, the adapters which are required to carry out the compositional task are obtained at step S814. Thereafter, the method follows the same steps as Figure 7. In other words, at step S818, the calibration parameters are identified, at step S820, the calibrated adapter is generated and combined with the base LLM at step S822.

[101] When the received task is a single task, the system obtains the single required adapter at step S812. In this case, there is no need for merging and / or calibration parameters. Accordingly, as shown at step S821, the obtained adapter is simply combined with the base LLM. Regardless of which route is followed, there are the same final steps as shown in Figure 8. In other words, at step S824, the received text input is used to generate the output and the generated output is then output as a response at step S826.

[102] Figure 9 shows a variation of Figure 8 in which the text input is received from an app rather than a user. It will be appreciated that Figure 7 could be similarly adapted to be triggered by a text input from an app as shown at step S902. Merely as examples, the app could be a web browser which provides text in a specific language (e.g. English) or could be a chat assistant which provides a text message. The user defines, at step S904, the compositional task which is to be carried out on the text input from the app. The compositional task for the text input in a specific language could be to summarise the text in a different language (e.g. Spanish). The compositional task for the text message may be to generate a reply in a specific tone (e.g. a witty tone). Again for simplicity, only the compositional task scenario is shown but it will be appreciated that there could be a determination as to whether the user has defined a compositional task as in Figure 8. Similarly, the user could provide the relevant task adapters rather than a task definition as in Figure 7. Many other scenarios are possible.

[103] Once the system has received the compositional task selection from the user, at step S906, the system identifies the tasks which are included within the compositional task. At step S914, the adapter which corresponds to the primary task is obtained and similarly, at step S916, the adapter which corresponds to the secondary task is obtained. In this simple example, there are only two tasks to be combined but it will be appreciated that there may be more tasks. For the example compositional task of summarising the text in a different language, the first adapter may be a summarization adapter and the second adapter may be a translation adapter. For the compositional task of generate a reply in a specific tone, the first adapter may be a reply adapter and the second adapter may be a tone adj. adapter.

[104] Thereafter, the method follows the same steps as Figures 7 and 8. In other words, at step S918, the calibration parameters are identified, at step S920, the calibrated adapter is generated and combined with the base LLM at step S922. At step S924, the received text input is used to generate the output and the generated output is then output at step S926.

[105] Figure 10 is a block diagram of a system which may be used in the present techniques. The system comprises a user device 1050 which comprises: at least one processor 1052 coupled to a memory 1054. The at least one processor 1052 may comprise one or more of: a microprocessor, a microcontroller, and an integrated circuit. The memory 1054 may comprise volatile memory, such as random-access memory (RAM), for use as temporary memory, and / or non-volatile memory such as Flash, read only memory (ROM), or electrically erasable programmable ROM (EEPROM), for storing data, programs, or instructions, for example. The user device 1050 comprises a ML model 1060 for performing a composite task. The ML model 1060 comprises a base model 1060 (e.g. a base LLM, base generative model or base multi-modal ML model), a set of adapters 1064 and calibration parameters 1066 which may be received from a server 1000 via the input / output interface 1058 or another similar communication module. It will be appreciated that there may be other standard components of the user device which are omitted for simplicity. As explained above, the ML model 1060 may comprise a base model which is adaptable by using one or more of a plurality of adapters (e.g. LoRAs) which have been efficiently corrected for compositional multi-tasking. The correction may be performed on the user device and means that the calibrated / adapted LLM only needs one inference pass and also only has a small number of additional parameters. The corrected model is thus suitable to be deployed on the user device, even when the user device is resource constrained.

[106] The server 1000 comprises at least one processor 1002 coupled to a memory 1004. The at least one processor 1002 may comprise one or more of: a microprocessor, a microcontroller, and an integrated circuit. The memory 1004 may comprise volatile memory, such as random-access memory (RAM), for use as temporary memory, and / or non-volatile memory such as Flash, read only memory (ROM), or electrically erasable programmable ROM (EEPROM), for storing data, programs, or instructions, for example. The server 1000 may store the least one ML model 1010 before communicating to a user device 1050 via the input / output interface 1008 or another similar communication module. Each stored ML model 1010 may comprise a base model 1012 (e.g. a base LLM or other base ML model), a set of adapters 1014 and calibration parameters 1016. An optional dataset 1020 is also shown which can be used in the base training described above. It will be appreciated that there may be other standard components of the server which are omitted for simplicity.

[107] Experiments

[108] The experiments detailed below compare the proposed techniques to known techniques such as the two-step application of the adapters (shown in Figure 2b) or a new adapter trained specifically for the given compositional task (“joint-expert”) as shown in Figure 3a. One of the baselines that is considered is to simply use only one of the primary or secondary task LoRAs. As part of the evaluation, a number of merging strategies are also considered because these could potentially have some benefits also for compositional multi-tasking in addition to standard multi-tasking. In all cases, prompts specifying the task are included as part of the input. More specifically, we will consider the following merging strategies applied to the LoRA adapter parameters: 1) Linear, i.e., model soup (“Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time” by Wortsman et al. published in the International conference on machine learning 2022), 2) Concatenation (“State-of-the-art parameter- efficient fine-tuning methods” by Mangrulkar et al. published in 2022), 3) TIES (“Ties-merging: Resolving interference when merging models” by Yadav et al. published in Advances in Neural Information Processing Systems in 2024), 4) DARE (“Language models are super mario: Absorbing abilities from homologous models as a free lunch” by Yu et al. published by Forty-first International Conference on Machine Learning in 2024), 5) Slerp (“Sampling generative networks” by White published in arXiv in 2016), 6) LoraHub (“Lorahub: Efficient cross-task generalization via dynamic lora composition” by Huang et al., published in arXiv in 2023), 7) DAM (“Merging in a bottle: Differentiable adaptive merging (dam) and the path from averaging to automation” by Gauthier-Caron et al., published in arXiv in 2024) and 8) ZipLoRA (“Ziplora: Any subject in any style by effectively merging loras” by Shah et al., published in 2025) Each of these approaches is adapted to the use cases. DAM and ZipLoRA approaches learn column-wise scaling to merge multiple models. Note that these methods are typically used for merging whole models, rather than the LoRA adapters only. Implementation Details Models and LoRAs

[109] The models used are suitable for on-device settings, more specifically conversational versions of LLaMA 3.2 1B, Qwen2.5 1.5B and StableLM2 1.6B models. All of these models are multilingual and support the tested languages. The single-task LoRAs are trained as described earlier. The LoRAs are applied to both the attention components (query, key, value and output projections) and the MLP components (up, down, and gate projections), for example as described in “A note on lora” by Fomenko et al published in 2024. Each LoRA is trained for one epoch on the full training set, using Adam optimizer with learning rate 5*10 "5. A rank of 32 with a of 16 and dropout rate of 0.05 is used. Learnable Calibration Details

[110] The calibration parameters are learnt / trained using an Adam optimizer for one epoch on a randomly selected subset of 10,000 examples and with a learning rate of 5 x 10"4. A rank of 4 is used for the calibration matrix approach that includes factorization into two low-rank matrices. Benchmark

[111] The problem of on-device text-based compositional multi-tasking is a new practically valuable challenge that has not been studied before. Consequently, a benchmark to study this problem setting has also been developed.

[112] The benchmark features three translation settings (English to Spanish, French or German) and four tone adjustments (professional, casual, witty and paraphrase), resulting in 14 tasks in total (three languages for each of cross-lingual summarization and smart reply and four tones for each of cross-lingual summarization and smart reply). Existing summarization and dialogue datasets were re-purposed for compositional multi-tasking via the use of suitable specialized models. Details about the tasks and the number of examples are provided below in Table 1: Dataset Training Validation Test Summarization 12,460 500 1,500 Reply 225,061 1,000 1,000 Translation 196,026 4,231 5,571 Tone adjustment 2,245 321 642

[113] Table 1 shows the number of examples (input-output pairs) in the benchmark, across the different splits. Note that the compositional tasks have the same number of examples as the primary task, i.e. summarization + translation and summarization + tone adjustment have the same number of examples as the summarization itself and similarly for when reply is the primary task. The source datasets for the benchmark were selected based on their overall suitability for the proposed tasks as well as their licence types. The compositional tasks are listed below:

[114] Summarization + Translation. For this task, the aim is to provide summaries of dialogues. The DialogSum dataset is used as described in “Dialog- Sum: A real-life scenario dialogue summarization dataset” by Chen et al published in Findings of the Association for Computational Linguistics in 2021. The dataset is in English and to enable compositional multitasking, the ground-truth summaries are translated from English to the different languages via the popular Opus Machine Translation model that has around 70M parameters. The Opus model is decrisbed for example in “OPUS-MT - Building open translation services for the World” by Tiedemann et al published in Proceedings of the 22nd Annual Conference of the European Association for Machine Translation (EAMT) in 2020.

[115] Summarization + Tone Adjustment. The tone of the ground-truth summaries available in DialogSum is changed via the publicly available RedPajama-INCITE-Base 3B model described in “Redpajama: an open dataset for training large language models by Weber et al. published in 2024. This model has already been fine-tuned specifically for tone adjustment and paraphrasing.

[116] Reply + Translation. The Synthetic Persona Chat dataset described in “Faithful persona-based conversational dataset generation with large language models” by Jandaghi et al. published in arXiv in 2023 is used. This dataset contains a large number of dialogues. The dialogues are converted into pairs of consecutive sentences, where the first one acts as context and the second one acts as ground-truth reply. Similarly as before, the same machine translation models are used to translate the ground truth outputs into the various languages. As the number of validation and test examples is large and text generation takes time, only a subset of all the possible context-reply pairs is used in the benchmark.

[117] Reply + Tone Adjustment. For this combination we use the Synthetic Persona Chat dataset and the RedPajama-INCITE-Base 3B model specialized for tone adjustment and paraphrasing.

[118] In addition to the four compositional tasks, the single-task expert adapters also need to be trained on suitable datasets as described above. For summarization and reply tasks, the same datasets, i.e., DialogSum and Synthetic Persona Chat, are used respectively. For the translation task, the TED talks dataset described in “When and why are pre-trained word embeddings useful for neural machine translation?” by Qi et al. published in Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics is used for the English to Spanish, French and German language pairs. For the tone adjustment task, the Sound Natural content rephrasing dataset described in “Sound natural: Content rephrasing in dialog systems” by Einolghozati et al. published in 2020 is used. The rephrased content is taken as source and then transformed via the fine-tuned RedPajama-INCITEBase 3B model into the different tones or a paraphrase. Evaluation metrics

[119] The evaluation metrics are selected in line with the existing literature. The Rouge-1 / 2 / L metrics as described in “ROUGE: A package for automatic evaluation of summaries” by Lin et al as published in Text Summarization Branches Out in 2004 are used for the summarization task (%, f). This measures how much the corresponding n-grams overlap between the generated output and the ground truth summary. For reply compositional tasks, the Weighted Rouge score (%, $) which is computed as described below is used: r. , Rouge-1 Rouge-2 eishteu Rouge =---— -— " 6 3 Rouge-3 9 Weighed Rouge score resolves the limitations of Rouge score being sensitive to small changes in sequence lengths. Further, it has been shown to correlate well with the clickthrough rate, which reflects how often the users utilize the proposed reply.

[120] For summarization + translation, Rouge (1 / 2 / L) score (%, $) which measures how much the corresponding n-grams overlap between the generated output and the ground-truth summary can be used. For reply + translation, Weighted Rouge score (%, $) which correlates well with the click-through rate and which reflects how often the users utilize the proposed reply can be used.

[121] In addition, for summarization + translation, a Rouge L score of typically 20 or less is weaker and thus the result either does not do summary or translation. A stronger result is typically 25 and more. As examples, two predictions are considered: • Prediction 1: Hannah: I have Betty's number. I'll text him. Reference: Hannah necesita el numero de Betty pero Amanda no Io tiene. Score: 18.18 • Prediction 2: Rob y Eric estan en el estudio de Eric y Rob. Rob esta haciendo un show de humor. Reference: Eric y Rob van a ver un stand-up en YouTube. Score: 26.67

[122] The second prediction has the higher score. Similarly, considering summarization + tone, a Rouge L score of typically 15 or less is weaker and may mean that the prediction is continuing conversation instead of providing a summary. A stronger score is typically 20 and more. As examples, two predictions are considered: • Prediction 1: Eric: MACHINE! Reference: Eric and Rob will be viewing a live stand-up performance on the platform of their choosing, namely, YouTube. Score: 9.52 • Prediction: Rob and Eric are watching some Russian stand-up videos. Rob is impressed with the Russian humor. Reference: Eric and Rob will be viewing a live stand-up performance on the platform of their choosing, namely, YouTube. Score: 22.22

[123] The second prediction has the higher score. Similarly, for reply + translation, a weighted rouge score of close to 0, generally less than 1 is weaker and is often scored when replying in wrong language. A stronger result is 2 or more (and can be 100 for a perfect match). As examples, two predictions are considered: • Prediction 1: I will! Reference: Gracias por la recomendacion. Score: 0.00 • Prediction 2: ^Que haces para ganarte la vida? Reference: ^Que te gusta hacer por diversion? Score: 2.56

[124] The second prediction has the higher score. Similarly, for reply + tone adj, a weighted Rouge score of generally less than 5 is weaker and is often scored when replying in a way that does not fit the context. A stronger result is 10 or more (and can be 100 for a perfect match). As examples, two predictions are considered: • Prediction: You should! It's a beautiful place. Reference: I will certainly consider your recommendation. Thank you for the valuable input. Score: 1.75 • Prediction: I will definitely consider it. Reference: I will certainly consider your recommendation. Thank you for the valuable input. Score: 10.33 Again, the second prediction has the higher score. Our solution and the inefficient baselines typically reach the levels needed for stronger performance. LLM Judge Evaluation

[125] In addition, evaluations via LLM Judge (LLM-J) described in “Judging LLM as a judge with mt bench and chatbot arean” by Zheng et al published in NeurlPS 2023 are used. The LLM judge mimics human evaluation by assessing outputs using a pretrained LLM. The LLaMA 3.1 70B model with carefully designed prompts may be used and a binary output is assigned to each example. LLM judges may be relatively expensive to run locally, particularly when there are a large number of experiments and limited computational resources. Standard metrics may be used primarily, but the LLM Judge results provide an additional layer of interpretation, offering a practical perspective of the scores derived from the standard metrics.

[126] For all compositional tasks, a prompt explicitly specifying the desired combination of tasks may be used. For the example compositional tasks which have been discussed, the prompts may be summarized as: • Summarization + translation: Summarize the following text and translate it from English to {language} • Summarization + tone adjustment: Summarize the following text and change its tone to {tone} • Reply + translation: Suggest a reply for the following text and translate it from English to {language} • Reply + tone adjustment: Suggest a reply for the following text and change its tone to {tone}

[127] In more detail, a prompt which can be used by the LLM judge for cross-tone summarization may be: You are in a setting where you want to summarize a dialogue in a {tone} tone. Your task is to judge the quality of the summary, if it uses a {tone} tone. The summary should not take the form of a reply to the dialogue. I give you an example ground-truth summary that follows a {tone} tone and would be considered a good summary of the dialogue. Dialogue: {dialogue} Ground-truth summary: {ground-truth summary} Proposed summary: {proposed summary} Does the proposed summary summarize the dialogue and is it written in a {tone} tone in {language}? Please respond with 0 or 1. Only output 0 or 1 and nothing else. Main Results

[128] The results of the experiments are shown in Figure 11. The results are averaged across the three models as well as across the different languages or tones. From the results, there are the following observations. 1) Despite including prompts specifying the task in all approaches, the performance of zeroshot approach is generally poor (e.g. 0.44% LLM-J score on sum + translation). This suggests the small LLMs have limited ability to handle the tested compositional tasks. 2) Using the primary (main) or secondary (auxiliary) task LoRA generally improved performance compared to the zero-shot baseline (e.g. to 3.49% in LLM-J), but has limited usefulness. 3) Both simpler (e.g., TIES) and more advanced (e.g., LoraHub) merging strategies typically perform on par with using one LoRA at a time, suggesting they are not a suitable solution for the compositional multi-tasking. 4) Inefficient baselines such as the two-step LoRA application and joint-expert LoRA perform well, significantly better than all the other baseline approaches (e.g. 49.85% and 72.92% in LLM-J). This suggests that they are able to address compositional tasks, albeit inefficiently. 5) The proposed family of approaches achieved strong results (e.g. 65.15% in LLM-J), generally on par or even better than the inefficient baselines, while being efficient in terms of both additional storage required and inference passes. 6) Introducing learnable reweighting in addition to calibration via learnable biases improves performance, but overall the strongest results are obtained by the version of Learnable Calibration in which a matrix of parameters is used (Figure 5c). 7) The good performance of the proposed approaches generalizes across the different compositional tasks studied in the benchmark.

[129] In summary, the proposed techniques obtain strong performance across the four compositional tasks considered. The results are significantly stronger than the “efficient” baselines such as zero-shot or various merging strategies and have comparable or better performance than the “inefficient” baselines, while being much more efficient. Sharing the Learnable Calibration parameters generally works well but there is a performance loss. Sharing of Parameters Across Tasks

[130] In the main set of results, one set of additional parameters for each compositional task is learnt and to improve efficiency further, the sharing of these parameters as shown in Figure 6 was investigated. The results in the bottom block in Figure 11 suggest that there is a performance loss when sharing parameters compared to having separate sets of parameters. Nevertheless, the performance is comparable with the inefficient baselines that act as reference points for good performance. Note that the strategy that performs reweighting of existing adapters (Figure 5b) is not directly applicable to this scenario, and thus the results are shown for only the variations in Figures 5a and 5c (learnable calibration v learning calibration++, respectively). Efficiency Analysis

[131] The efficiency of the proposed solutions and the two well-performing but inefficient baselines is shown in the table below: Method Number of inferences Additional parameters Additional storage Two-step LoRA usage 2x 0 0.00MB Joint-expert LoRA 1x 30M 57.10MB Learning calibration (Figure 5a) 1x 23K 0.05MB Learning calibration (Figure 5b) 1x 23K 0.05MB Learning calibration (Figure 5c) 1x 166K 0.32MB

[132] . The results highlight that the proposed solutions require only a comparatively small number of additional parameters, between about 0.08-0.56% of the ones used by a jointexpert LoRA. The amount of additional storage on the disk is small, less than 0.5MB. In other words, the proposed Learnable Calibration solutions need only one inference pass and negligible amount of additional storage. Benefit of Using Existing Adapters

[133] To assess the impact of using existing adapters, performance is assessed when training only the Learnable Calibration parameters without leveraging pre-existing adapters. The results in the Table in Figure 12 confirm it is beneficial to use the existing adapters as a starting point and then calibrate them using the learnable parameters, either in the form of biases or low-rank factorized matrices. Without calibration parameters, the approach becomes linear merge that has weak performance, highlighting the critical role of calibration for achieving strong results. The best performance is obtained when the merged adapters are used as a starting point and subsequently refined with learnable calibration parameters. Interpretation of the Impact of Different Solutions

[134] Rouge score in general measures the overlap between the generated text and the reference ground-truth text. However, the results can be challenging to interpret and a qualitative analysis was also conducted.

[135] After inspecting the outputs of the model and the ground truth references, there are essentially two behavioural groups that can be observed. The first group typically does not perform one of the tasks, e.g., sometimes provides a summary but fails to translate it properly or provides a translated text but does not summarize. This group includes the approaches such as zero-shot, and existing merging strategies. The second group succeeds in performing both tasks. For instance, they correctly execute both translation and summarization. Representatives of this category are the two inefficient baselines as well as our Learnable Calibration solutions. The qualitative differences in the Rouge scores reflect these behaviours. Within the group itself, larger scores typically mean that the approach performs the compositional task in more of the examples, i.e. there is better execution quality and consistency across more examples. The frequency with which approaches successfully complete both tasks aligns with the LLM judge analysis, providing a quantitative measure of task success. Analysis of Changes in the Update Matrices

[136] To understand how the proposed methods address compositional tasks, the changes in the update matrices compared to simple merging have also been analysed. More specifically, the distribution of values of the parameters has been analysed via histograms and by evaluating the norms of the weights. The results for two different scenarios are compared, one for the combination of summarization and translation and another one for reply and tone adjustment, in Figures 13a to 14d.

[137] Figure 13a to 13d show results when carrying out summarization + translation using a Qwen2.5 1.5B model. Figure 13a plots count against update parameter value for the linear merge approach and for comparison Figure 13b plots count against update parameter value for the proposed approach using calibration parameters. Figure 13c plots the weight norm value against different components within the LoRA matrix for the linear merge approach and for comparison Figure 13d plots the weight norm value against different components for the proposed approach using calibration parameters. Figures 14a to 14d show the same results as Figures 13a to 13d when carrying out reply + tone adjustment using a Llama 3.2 1B model. As shown, the additional parameters make the overall LoRA update matrix significantly more diverse to take care of the additional secondary tasks. The weight norms of the update matrices also become significantly larger, reflecting the additional complexity required to address compositional tasks. These patterns are seen across both of the analysed scenarios, illustrating how learnable calibration adapts the adapters to accommodate multiple tasks effectively. Scalability to models of different sizes

[138] Figures 15a and 15b plot Rouge-L score and LLM judge score against model size for different approaches. For each plot, the compositional multi-task is the cross-lingual summarization task. The results show the scalability of the proposed techniques to Qwen2.5 model in the following sizes suitable for on-device deployment: 0.5B, 1.5B, 3B. The results confirm the proposed solutions consistently perform well with models of different sizes. Larger models typically obtain stronger performance. Extended evaluation with standard metrics

[139] In the examples above, the results have been evaluated using Rouge scores. The tables in Figures 16 and 17 provide an extended evaluation using all standard metrics, including ROUGE-1 (R1), ROUGE-2 (R2), ROUGE-3 (R3), and Weighted ROUGE (W-R). These results are consistent with those reported above and further validate the effectiveness and robustness of the proposed methods. Figure 16 evaluates compositional multi-tasking across different scenarios. The test results are reported as percentages (%, t) and averaged across different models and languages or tones. The proposed methods (learnable calibration and learnable calibration++) achieve comparable performance to inefficient baselines while being significantly more efficient in terms of inference and storage. Similarly fast baselines, such as various merging strategies generally fail in compositional multi-tasking. The bottom block includes the results for a version in which the additional calibration parameters are shared across all four tasks. Figure 17 evaluates ablation on usefulness of single-task LoRAs. In this evaluation, existing adapters are leveraged to enhance the performance of learnable calibration (LC) in compositional multi-tasking. As in Figure 16, the test results are reported as percentages (%, t) and averaged across different models and languages or tones. The top block of Figure 17 utilizes separate parameters for each compositional task, while the bottom block employs a single set of shared parameters. Out-of-domain generalization

[140] The generalization of LoRAs and additional parameters trained on the DialogSum dataset is evaluated by testing on a different dataset - SAMSum dataset as described in “Samsum corpus: A human-annotated dialogue dataset for abstractive summarization” by Gliwa et al published in ACL in 2019. The aim is to assess whether the proposed methods maintain strong performance and improvements when applied to data from a different domain. Specifically, DialogSum features spoken-language conversations from daily life scenarios, whereas Samsum includes written-style dialogues from online interactions. For instance, DialogSum contains sentences like “#Person_1#: Good morning. I wonder whether you have got an answer from your superior” whereas SamSum includes examples such as “Leo: BTW what are those pics?”. The results are shown in the table below. In-domain Out-of-domain R-L LLM-J R-L LLM-J Zero-shot 9.59 0.20 10.58 3.54 Main task LoRA 14.25 3.09 15.97 9.89 Auxiliary task LoRA 14.42 0.16 19.67 1.14 Linear merge 15.60 0.18 18.30 1.26 Multi-step LoRA usage 21.71 75.51 26.81 76.35 Joint-expert LoRA 23.47 52.51 25.38 56.33 Learnable Calibration 27.68 61.58 27.78 57.26 Learnable calibration++ 30.98 67.98 28.97 59.91

[141] The test results are reported as percentages (%, t) and averaged across models for the cross-lingual English-to-Spanish summarization task. The approaches work well even in the presence of mild domain shift. Compositional multi-tasking with three tasks

[142] In the benchmark above, the focus has been on combinations of two tasks. In this section, the joint tasks of summarization, professional tone adjustment and translation from English to Spanish, French or German are investigated. Data for this task was created by converting the ground-truth summaries into a professional tone and then translating them into multiple languages. The same models for tone adjustment and translation were applied in this set-up. Given the large number of possible combinations, the results focus on one tone and one main task for simplicity. Additionally, only a subset of the baseline approaches are evaluated.

[143] A prompt which can be used for the LLM judge for evaluating this scenario is: You are in a setting where you want to summarize a dialogue in a {tone} tone in another language. The dialogue is in English, but the summary should be in a {tone} tone in {language}. Your task is to judge the quality of the summary, if it uses a {tone} tone and if it is in {language}. The summary should not take the form of a reply to the dialogue. I given you an example ground-truth summary that follows a {tone} tone and is in {language}. The example would be considered a good summary of the dialogue. Dialogue: {dialogue} Ground-truth summary: {ground-truth summary} Proposed summary: {proposed summary} Does the proposed summary summarize the dialogue and is it written in a {tone} tone in {language}? Please respond with 0 or 1. Only output 0 or 1 and nothing else.

[144] The results are presented in the table below. R-L LLM-J Zero-shot 7.35 0.03 Summarization LoRA 9.41 0.00 Translation LoRA 10.95 0.00 Tone adj. LoRA 8.39 0.01 Linear merge 9.89 0.01 Multi-step LoRA usage 16.50 43.82 Joint-expert LoRA 14.35 12.75 Learnable Calibration 19.19 28.66 Learnable calibration++ 21.42 39.16

[145] The findings show that inefficient baselines outperform simple baselines in this more challenging scenario. Furthermore, the proposed learnable calibration approaches outperform the inefficient baselines both in terms of accuracy and footprints. The version using a calibration matrix performs better than its simple variant. While the overall performance is lower than that for the simpler combination of summarization and translation, these results confirm that the framework described above can successfully handle compositions involving more than two tasks, proving its flexibility. Learning curves analysis of the proposed approach

[146] The training process of learnable calibration is analysed to better understand its behaviour. For this analysis, a Qwen2.5 1.5B model with Spanish as the target language and a profession target tone is used. Figure 18a plots the loss against samples processed for the summarization main task and Figure 18b plots the loss against samples processed for the reply main task. The results in Figures 18a and 18b confirm that the loss consistently decreases as the training progresses and more samples are processed.

[147] Various use cases have been described above. The target use cases are combinations of the current target use cases for existing LLMs, for example the Gauss LLM on Samsung™ or other generative ML models, e.g. image generation, or multimodal LLMs. In each example, the proposed solution adds a few additional / calibration parameters to make the base Gauss LLM a more capable LLM model. One example use case has a base task which is a smart reply. Another example use case is one in which the base task is summarization / proofreading / rewriting (tone adjustment). Another example use case is one in which the base task is text translation. The compositional tasks may be Summarization &Translation, Summarization &Tone adjustment, Smart reply &Translation and / or Smart reply &Tone adjustment None of the known devices provides compositional multi-tasking, e.g., perform translation on top of summarization. For image generation, the base task may be generating an image including a specific object / subject (e.g. a dog) and the second task may be generating an image in a specific style (e.g. in a style of a specific artist, in the style of a cartoon and so on). For a multi-modal generative model, the primary / base task may be generate a summary / title and the second task may be to generate an image, e.g. an emoji. Existing solutions for multi-tasking can only perform multiple tasks separately, not if they are on top of each other.

[148] The solutions proposed above improve task performance for existing LLMs, particularly the Gauss LLM on Samsung ™. Using known techniques, multi-tasking using the Gauss LLM would either encompass one LoRA for each compositional task (extra storage needed) or apply LLM twice with different LoRAs (2 inference passes needed). As explained above, compositional multi-tasking is achieved using the present techniques by merging the different parameters and adding the Learnable Correction which has very few additional parameters. This leads to excellent performance and efficiency on compositional multi-tasking of Gauss LLM. Moreover, the solution proposed above is agnostic to the underlying LLM and framework. Thus, it can be employed to improve compositional multi-tasking of other LLMS, e.g. Gemini on Android Al Core and / or Apple Intelligence and so on.

[149] The table below conceptually compares the method described above with other known solutions: Method Performance Runtime (#inference) Additional storage Zero-shot or using one of the stored LoRAs paired with prompting Poor Excellent Excellent Existing merging strategies (e.g. DARE, Model Soup, TIES, ZipLoRA, LoraHub, ...) Poor Excellent Excellent Joint-expert LoRA Excellent Excellent Poor Two-step solution Excellent Poor Excellent Learnable calibration (as described above) Excellent Excellent Excellent

[150] More specifically the table below considers some of the advantages and disadvantages of the known methods for model merging techniques compared to the techniques described above. Method What it is Pros Cons Task arithmetic Directly adds taskspecific weight differences to a base model Simple &straightforward. Requires no additional parameter tuning Less nuances; may not fully capture complex model improvements. No redundancy handling TIES Focuses on significant weight changes by Reduces model complexity by focussing on impactful Potentially overlooks subtle but important adjustments. Requires magnitude, ignoring smaller ones changes. Addresses redundance to some extent choosing which changes to ignore. DARE Randomly drops weight changes to reduce redundancy Introduces randomness to prevent overfitting. Simplifies the model by reducing weight count Randomness can lead to loss of critical information. Less predictable outcomes DARE+TIES Combines the magnitude focus of TIES with the randomness of DARE Balances significant change focus with randomness to enhance model performance. Aims to capture the best of both methods Complexity increases with combining methods. Requires careful tuning of parameters.

[151] Figure 19 shows an example high-level architecture which could be implemented using the proposed techniques which is specifically tailored for compositional multi-tasking scenarios involving summarization and translation. A user interface either receives an input dialogue from a user or can load example dialogue from the dataset so that a user does not need to enter into long conversations. The user interface communicates with the LLM communication end-point which handles all communications between the front-end and backend of the application, e.g. it is connected to the dialogue emulator that loads example dialogues and realistically plays them. It is also connected to the inference API which is responsible for handling the LLM prompt, calculation of metrics, and most importantly doing the actual inference that outputs the translated summary of the dialogue. Finally, LLM setup and adapter handling tackles the task of loading the LLM model and adapters while allowing for the dynamic configuration of these components.

[152] As an example, the user interface may provide a configuration menu and may display the translated summary and metrics below the conversation. Key features include (i) method selection, (ii) task configuration, i.e. whether to run summarization only or with translation; (iii) optional random conversation generation; (iv) single or comparative method evaluation mode, i.e. whether to run one experiment with the selected method or with all methods; and (v) target language selection.

[153] In summary, an approach for solving the practically valuable problem of compositional multi-tasking for large language models is thus described above. In this problem setting, tasks are performed on top of each other, while being able to do so efficiently, for example when providing a summary of long text in another language. Compositional multi-tasking is especially interesting to study for on-device scenarios where resources are constrained and efficient solutions need to be developed. As the problem setting itself is new, a new benchmark has also been developed that enables the study of diverse approaches. The new benchmark shows that existing approaches are either inefficient or obtain low performance, necessitating the development of new methods. As a solution, a family of methods called Learnable Calibration is introduced for combining the adapters already available on the device and adapting them via a small number of additional parameters, resulting in excellent performance and efficiency.

[154] As described above, a compositional multi-tasking benchmark has been designed, which allows the evaluation of existing solutions on this highly practical use-case (i.e. compositional multi-tasking). A new strategy for model merging that is suitable to address compositional multi-tasking has been discussed. The proposed approach has been benchmarked on a wide range of setups considered, proving its superiority versus starting baseline models and state-of-the-art model merging approaches reused for our novel task.

[155] To summarize, the key contributions of the proposed techniques described above are as follows a. New practically useful problem setting of compositional multi-tasking for on-device LLM b. New benchmark to study the compositional multi-tasking in on-device LLMs and evaluate the performance of different approaches c. New approach, named Learnable Calibration, that successfully addresses the challenge by obtaining strong performance while being efficient in terms of both time and storage.

[156] In summary, unlike the known art, the present techniques tackle the problem of joint multi-tasking in LLMs, model merging via calibration parameters, model merging on on-device Al LLMs. Many of the known prior art documents which mention multi-tasking are for other applications (mainly computer vision based) and via other systems (like reinforcement learning, etc.). In such cases, the prior art focuses on different types of data and does not have task specialized LoRAs that should be merged for multi-tasking. As an example, such known solution may compute an expensive evolutionary optimization rather than a simpler calibration optimization of the merged models.

[157] References:

[158] Chen, Y., Liu, Y., Chen, L., and Zhang, Y. Dialog-Sum: A real-life scenario dialogue summarization dataset. In Findings of the Association for Computational Linguistics: ACL- IJCNLP 2021, pp. 5062-5074, Online, August 2021. Association for Computational Linguistics. doi: 10.18653 / v1 / 2021.findings-acl.449

[159] Ding, N., Qin, Y., Yang, G., Wei, F., Yang, Z., Su, Y., Hu, S., Chen, Y., Chan, C.-M., Chen, W., et al. Delta tuning: A comprehensive study of parameter efficient methods for pretrained language models. arXiv preprint arXiv:2203.06904, 2022.

[160] Einolghozati, A., Gupta, A., Diedrick, K., and Gupta, S. Sound natural: Content rephrasing in dialog systems. arXiv preprint arXiv:2011.01993, 2020.

[161] Fomenko, V., Yu, H., Lee, J., Hsieh, S., and Chen, W. A note on lora. arXiv preprint arXiv:2404.05086, 2024.

[162] Gauthier-Caron, T., Siriwardhana, S., Stein, E., Ehghaghi, M., Goddard, C., McQuade, M., Solawetz, J., and Labonne, M. Merging in a bottle: Differentiable adaptive merging (dam) and the path from averaging to automation. arXiv preprint arXiv:2410.08371, 2024.

[163] Han, Z., Gao, C., Liu, J., Zhang, J., and Zhang, S. Q. Parameter-efficient fine-tuning for large models: A comprehensive survey. arXiv preprint arXiv:2403.14608, 2024.

[164] Hu, E. J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., and Chen, W. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685, 2021.

[165] Huang, C., Liu, Q., Lin, B. Y., Pang, T., Du, C., and Lin, M. Lorahub: Efficient cross-task generalization via dynamic lora composition. arXiv preprint arXiv:2307.13269,2023.

[166] Jandaghi, P., Sheng, X., Bai, X., Pujara, J., and Sidahmed, H. Faithful persona-based conversational dataset generation with large language models. arXiv preprint arXiv:2312.10007, 2023.

[167] Lin, C.-Y. ROUGE: A package for automatic evaluation of summaries. In Text Summarization Branches Out, pp. 74-81, Barcelona, Spain, July 2004. Association for Computational Linguistics.

[168] Mangrulkar, S., Gugger, S., Debut, L., Belkada, Y., Paul, S., and Bossan, B. Peft: State-of-the-art parameter- efficient fine-tuning methods.

[169] Qi, Y., Sachan, D., Felix, M., Padmanabhan, S., and Neubig, G. When and why are pretrained word embeddings useful for neural machine translation? In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), pp. 529-535, New Orleans, Louisiana, June 2018. Association for Computational Linguistics, doi: 10.18653 / v1 / N 18-2084.

[170] Shah, V., Ruiz, N., Cole, F., Lu, E., Lazebnik, S., Li, Y., and Jampani, V. Ziplora: Any subject in any style by effectively merging loras. In European Conference on Computer Vision, pp. 422-438. Springer, 2025.

[171] Tiedemann, J. and Thottingal, S. OPUS-MT - Building open translation services for the World. In Proceedings of the 22nd Annual Conferenec of the European Association for Machine Translation (EAMT), Lisbon, Portugal, 2020.

[172] Weber, M., Fu, D., Anthony, Q., Oren, Y., Adams, S., Alexandrov, A., Lyu, X., Nguyen, H., Yao, X., Adams, V., etal. Redpajama: an open dataset for training large language models. arXiv preprint arXiv:2411.12372, 2024.

[173] Wolf, T. Huggingface's transformers: State-of-the- art natural language processing. arXiv preprint arXiv:1910.03771,2019.

[174] Wortsman, M., Ilharco, G., Gadre, S. Y., Roelofs, R., Gontijo-Lopes, R., Morcos, A. S., Namkoong, H., Farhadi, A., Carmon, Y., Kornblith, S., et al. Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time. In International conference on machine learning, pp. 23965-23998. PMLR, 2022.

[175] Yadav, P., Tam, D., Choshen, L., Raffel, C. A., and Bansal, M. Ties-merging: Resolving interference when merging models. Advances in Neural Information Processing Systems, 36, 2024.

[176] Yu, L., Yu, B., Yu, H., Huang, F., and Li, Y. Language models are super mario: Absorbing abilities from homologous models as a free lunch. In Forty-first International Conference on Machine Learning, 2024.

[177] Those skilled in the art will appreciate that while the foregoing has described what is considered to be the best mode and where appropriate other modes of performing present techniques, the present techniques should not be limited to the specific configurations and methods disclosed in this description of the preferred embodiment. Those skilled in the art will recognise that present techniques have a broad range of applications, and that the embodiments may take a wide range of modifications without departing from any inventive concept as defined in the appended claims.

Claims

1. A computer-implemented method for generating an adaptable machine learning, ML, model; the method comprising, on a server:obtaining a base ML model;obtaining a plurality of adapters each of which is combinable with the base ML model to adapt the base ML model to perform a specific task;defining multiple compositional tasks which are to be performed by the adaptable ML model on an input to generate an output, wherein each compositional task comprises a first task and a second task;for each of the defined compositional tasks:initializing a set of calibration parameters;identifying a first adapter from the plurality of adapters, wherein the identified first adapter adapts the base ML model to perform the first task;identifying a second adapter from the plurality of adapters, wherein the identified second adapter adapts the base ML model to perform the second task;generating a calibrated adapter by combining the first adapter, the second adapter and the initialized set of calibration parameters ;combining the calibrated adapter with the base ML model to form a calibrated ML model; andtraining the calibrated ML model by learning the set of calibration parameters which fine tunes the calibrated ML model to the defined compositional task; and outputting, to a user device, an adaptable ML model which comprises the base ML model, the plurality of adapters and the learnt calibration parameters whereby the adaptable ML model is usable by the user device to form multiple adapted ML models each of which is fine tuned to perform one of the multiple compositional tasks.

2. The method of claim 1, wherein defining multiple compositional task comprises defining at least one compositional task which further comprises a third task and for each defined compositional task comprising first, second and third tasks, the method further comprises:identifying a third adapter from the plurality of adapters, wherein the identified third adapter adapts the base ML model to perform the third task andwherein generating the calibrated adapter comprises combining the first adapter, the second adapter, the third adapter and the initialized set of calibration parameters.

3. The method of claim 1 or claim 2, wherein outputting the adaptable M L model comprising the learnt calibration parameters comprises outputting an adaptable ML model comprisingmultiple sets of calibration parameters with each set of calibration parameters associated with a specific compositional task from the multiple defined compositional tasks.

4. The method of claim 1 or claim 2, wherein prior to outputting the adaptable ML model comprising the learnt calibration parameters, the method further comprises:combining the learnt sets of calibration parameters to form a shared set of calibration parameters; andoutputting an adaptable ML model comprising the shared set of calibration parameters.

5. The method of any one of the preceding claims, wherein initializing a set of calibration parameters comprises initializing a set of calibration parameters in the form of a vector of calibration parameters andwherein generating the calibrated adapter comprises:merging the first adapter and the second adapter to forma merged adapter comprising a set of parameters arranged in a matrix; andcombining the merged adapter with the set of calibration parameters using matrix multiplication.

6. The method of any one of claims 1 to 4, wherein initializing a set of calibration parameters comprises initializing a set of calibration parameters in the form of a matrix of calibration parameters andwherein generating the calibrated adapter comprises:merging the first adapter and the second adapter to form a merged adapter comprising a set of parameters arranged in a matrix; andcombining the merged adapter with the set of calibration parameters using matrix multiplication.

7. The method according to claim 5 or claim 6, wherein the adapters are low-rank adaptersand merging the first and second adapters comprises:generating a first combined parameter matrix by summing a first parameter matrix of the first adapter and a first parameter matrix of the second adapter;generating a second combined parameter matrix by summing a second parameter matrix of the first adapter and a second parameter matrix of the second adapter;wherein the parameter matrix for the merged adapter comprises the first combined parameter matrix and the second combined parameter matrix.

8. The method according to claim 7, wherein summing the first parameter matrices and summing the second parameter matrices comprises summing using a weighted sum.

9. The method according to claim 8,wherein summing using a weighted sum comprises using a learnable weight in the weighted sum; andwherein training the adaptable ML model comprises learning the learnable weight.

10. The method according to any one of the preceding claims, wherein training the calibrated ML model comprises:obtaining a training dataset comprising a plurality of input-output pairs of data items wherein each pair comprises an input and a desired output which is generated by performing the compositional task on the input andlearning the set of calibration parameters which fine tune the calibrated ML model to generate the desired output for each input.

11. A computer-implemented method for using, on a user device, an adaptable ML model which comprises a base ML model, a plurality of adapters and at least one set of calibration parameters, the method comprising:receiving an input to be processed by the adaptable ML model;determining, in response to receiving the input, whether a compositional task comprising a first task and a second task is to be carried out on the input;when it is determined that a compositional task comprising a first task and a second task is to be carried out on the input:identifying a first adapter from the plurality of adapters, wherein the identified first adapter adapts the base ML model to perform the first task;identifying a second adapter from the plurality of adapters, wherein the identified second adapter adapts the base ML model to perform the second task;generating a calibrated adapter using the first adapter, the second adapter and a set of calibration parameters which is associated with the determined compositional task;combining the calibrated adapter with the base ML model to form a calibrated ML model; andgenerating, by inputting the input into the calibrated ML model, an output.

12. The method of claim 11, further comprising, when the input is received,receiving an indication of a compositional task to be performed on the input whereby the received indication is used to determine that a compositional task is to be carried out on the input.

13. The method of claim 12, further comprising, when the input is received, receiving a selection of a first task to be performed on the input; and receiving a selection of a second task to be performed on the input;whereby the received selections are used to determine whether a compositional task is to be carried out on the input.

14. The method of any one of claims 11 to 13, wherein receiving an input comprises receiving an input from a newly installed application.

15. The method according to any one of the preceding claims, wherein the base ML model is a large language model and the first task is selected from summarization or reply and the second task is selected from translation or tone adjustment.

16. The method according to any one of claims 1 to 14, wherein the base ML model is an image generation model and the first task is generating an image containing a specific object and the second task is generating an image in a specific style.

17. The method according to claim 16, wherein the input is an image or a text input and the output is an image.

18. The method according to any one of claims 1 to 14, wherein the base ML model is a multi-modal ML model and the first task is selected from summarization or reply and the second task is generation of an image.

19. The method according to claim 18, wherein the input comprises one or both of image and text and the output comprises an image.

20. A server comprising:at least one processor coupled to memory, wherein the at least one processor is for individually or collectively carrying out the method of any one of claims 1 to 10 and 15 to 19.

21. A user device comprising:at least one processor coupled to memory, wherein the at least one processor is for individually or collectively carrying out the method of any one of claims 11 to 19.

22. A computer-readable storage medium comprising instructions which, when executed by a processor, causes the processor to carry out the method of claims 1 to 10 or 11 to 19.

Images