Training-free machine learning model adapter transfer

Abstract

Certain aspects of the present disclosure provide techniques and apparatus for machine learning. In an example method, a first adapted machine learning model comprising a first base model and an adapter trained for the first base model is accessed. One or more adapter components are generated based on projecting the adapter to a range space and a null space of the first base model. A second base model is accessed, and a projected adapter is generated based on projecting the one or more adapter components to a range space and a null space of the second base model. A second adapted machine learning model comprising the second base model and the projected adapter is generated, and a machine learning model output is generated using the second adapted machine learning model.

Description

Client Ref. No.: 2405344B1WO TRAINING-FREE MACHINE LEARNING MODEL ADAPTER TRANSFER CROSS-REFERENCE TO RELATED APPLICATION(S)

[0001] This application claims priority to U.S. Patent Application No. 18 / 940,172, filed November 7, 2024, which is a continuation-in-part (CIP) of U.S. Patent Application No.18 / 882,595, filed September 11, 2024, which are both hereby expressly incorporated by reference herein in their entireties as if fully set forth below and for all applicable purposes. INTRODUCTION

[0002] Aspects of the present disclosure relate to machine learning.

[0003] A wide variety of machine learning model architectures have been trained to perform an assortment of diverse tasks, including computer vision tasks, language tasks, classification and regression tasks, and the like. Recently, research has yielded substantial success in using large language models (LLMs), large vison models (LVMs), and / or large multimodal models (LMMs) to process and generate output data. Often, machine learning models (especially LLMs, LVMs, and LMMs) have many parameters (e.g., millions or even billions), resulting in significant model size, as well as substantial computational expense in training the model. Further, once trained, such models are often difficult (or impossible) to fine-tune, as the vast number of parameters makes overfitting a major challenge (e.g., potentially relying on tremendous amounts of fine-tuning data to prevent overfitting).

[0004] One recent approach to enable fine-tuning or personalization of such generative models involves training relatively smaller model adapters for larger models. However, adapters trained for such models generally become intrinsically tied to the larger model and may not effectively be reused for other models (even highly similar models). That is, if a large model (e.g., an LLM) is modified even slightly, adapters trained for the original model are generally no longer useful and may not function properly with the modified model. BRIEF SUMMARY

[0005] Certain aspects of the present disclosure provide a processor-implemented method, comprising: accessing a first adapted machine learning model comprising a first base model and an adapter trained for the first base model; generating one or more adapter P+S Ref. No.: QUAL / 2405344PC02 1Client Ref. No.: 2405344B1WO components based on projecting the adapter to a range space and a null space of the first base model; accessing a second base model; generating a projected adapter based on projecting the one or more adapter components to a range space and a null space of the second base model; generating a second adapted machine learning model comprising the second base model and the projected adapter; and generating a machine learning model output using the second adapted machine learning model.

[0006] Other aspects provide processing systems configured to perform the aforementioned methods as well as those described herein; non-transitory, computer- readable media comprising instructions that, when executed by one or more processors of a processing system, cause the processing system to perform the aforementioned methods as well as those described herein; a computer program product embodied on a computer-readable storage medium comprising code for performing the aforementioned methods as well as those further described herein; and a processing system comprising means for performing the aforementioned methods as well as those further described herein.

[0007] The following description and the related drawings set forth in detail certain illustrative features of one or more aspects. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] The appended figures depict example features of certain aspects of the present disclosure and are therefore not to be considered limiting of the scope of this disclosure.

[0009] FIG. 1 depicts an example workflow for adapter reuse in machine learning models, according to some aspects of the present disclosure.

[0010] FIG.2 depicts an example architecture for effective adapter reuse in machine learning models, according to some aspects of the present disclosure.

[0011] FIG. 3 depicts an architecture for using a pre-trained adapter with a student machine learning model, according to some aspects of the present disclosure.

[0012] FIG. 4 is a flow diagram depicting an example method for reusing model adapters in machine learning models, according to some aspects of the present disclosure.

[0013] FIG.5 is a flow diagram depicting an example method for adapting machine learning models, according to some aspects of the present disclosure. P+S Ref. No.: QUAL / 2405344PC02 2Client Ref. No.: 2405344B1WO

[0014] FIG.6 depicts an example processing system configured to perform various aspects of the present disclosure.

[0015] FIG. 7 depicts an architecture for using a projected adapter with a student machine learning model, according to some aspects of the present disclosure.

[0016] FIG.8 is a flow diagram depicting an example method for projecting model adapters for reuse in machine learning models, according to some aspects of the present disclosure.

[0017] FIG.9 is a flow diagram depicting an example method for adapting machine learning models, according to some aspects of the present disclosure.

[0018] To facilitate understanding, identical reference numerals have been used, where possible, to designate identical elements that are common to the drawings. It is contemplated that elements and features of one aspect may be beneficially incorporated in other aspects without further recitation. DETAILED DESCRIPTION

[0019] Aspects of the present disclosure provide apparatuses, methods, processing systems, and non-transitory computer-readable mediums for providing improved machine learning. Specifically, in some aspects of the present disclosure, techniques for reusing model adapters in various machine learning models are provided.

[0020] Many model architectures, such as LLMs and LVMs have shown great promise in generating useful output data. In many cases, fine-tuning of such large models is difficult or impossible. Recently, low-rank adaptation (LoRA) adapters have been introduced to address many common challenges of fine-tuning such large models (where the larger model may be referred to as a “base model” that is adapted using an “adapter”). In some aspects, fine-tuning using adapters involves updating the parameters of the adapter(s) while retaining the parameters of the (larger) base model frozen. This can substantially reduce the memory and compute usages of the fine-tuning process. In some aspects, LoRA adapters can be applied to the cross-attention layers of the model, allowing the adapter to better learn to relate output representations (e.g., for images or text) with the prompts that describe the representations. For example, adapters can be trained to modify visual characteristics of the output of the base model, such as the color pallets used, the artistic style, and the like. Advantageously, training such LoRA adapters can P+S Ref. No.: QUAL / 2405344PC02 3Client Ref. No.: 2405344B1WO be performed substantially faster and with significantly reduced computation as compared to fine-tuning the base model itself.

[0021] A variety of base model architectures (e.g., LLMs, LVMs, and LMMs) have been trained for various tasks. For example, in some cases, a first base model may be modified somewhat to create a second base model (e.g., by modifying one or more hyperparameters or parameters). Similarly, a wide variety of adapters have been trained and made available for use for specific base models. However, an adapter trained for one base model is generally not useable with any other base models—even other base models that are highly similar to the base model for which the adapter was trained. For example, even if a first base model (referred to as a “teacher model”) is used to generate or train a smaller second base model (referred to as a “student model”), adapters trained for the teacher model cannot be readily used in conjunction with the student model.

[0022] Some conventional approaches have relied on training new adapters for the student model. However, this introduces inherent computational expense to attempt to recapture functionality that the teacher model (with an adapter) already had. Further, in many cases, the data used to train such adapters is kept private or is otherwise not available to train a new adapter. For example, suppose one entity grants access to a base model and an adapter, and a second entity adapts the base model (e.g., generating a student model). Without accessing the training data used by the first entity, the second entity may not successfully train a new adapter to perform similar functionality, and thus should not use the original adapter with the new base model. In some aspects of the present disclosure, techniques are provided to allow for distillation of knowledge from an adapted teacher model (e.g., a first base model with an adapter) to a student model (e.g., a (generally smaller) version of the first base model having a different architecture, a different number of sampling steps, and the like) without relying on access to training data (e.g., the data used to train the adapter). This allows for generation of an adapted student model that can re-use adapters previously trained for the teacher model without introducing the computational expense of further training.

[0023] In some aspects, the goal of this knowledge distillation is to cause intermediate outputs of the student model to be, in some way, similar to the intermediate outputs of the teacher model. For example, in some aspects, a projection (e.g., a linear projection) operation can be used to cause the student model’s outputs to more closely mirror the teacher model. This allows for adapters trained for the teacher model to be reused by the P+S Ref. No.: QUAL / 2405344PC02 4Client Ref. No.: 2405344B1WO student model, in some aspects. Advantageously, certain aspects of the present disclosure enable this reuse without relying on any further training or fine-tuning of the student or adapter. Instead, computationally inexpensive operations, such as linear algebra, can be used to enable re-use of the pretrained adapters, substantially increasing the flexibility of the student models. Example Workflow for Adapter Reuse in Machine Learning Models

[0024] FIG. 1 depicts an example workflow 100 for adapter reuse in machine learning models, according to some aspects of the present disclosure.

[0025] In the illustrated example, a first base model 105A is accessed by an adaptation system 115. As used herein, “accessing” data may generally include receiving, retrieving, requesting, obtaining, collecting, generating, training, or otherwise gaining access to the data. For example, the adaptation system 115 may itself train the base model 105A, or the adaptation system 115 may receive the base model 105A from another source (e.g., a dedicated training system). The base model 105A may generally be representative of any machine learning model architecture that can be adapted using adapter models (e.g., LoRA adapters). For example, as discussed above, the base model 105A may correspond to a large model such as an LLM, an LVM, and / or an LMM. As one example, the base model 105A may be an LVM trained to generate output images based on input textual prompts.

[0026] The adaptation system 115 is generally representative of any computing system capable of training model adapters for the base model 105A. Though depicted as a single discrete system for conceptual clarity, in some aspects, the operations of the adaptation system 115 may be combined or distributed across any number of systems and may be implemented using hardware, software, or a combination of hardware and software.

[0027] In the illustrated workflow 100, the adaptation system 115 also accesses a set of adaptation data 110 (also referred to in some aspects as an “adaptation dataset”). In some aspects, the adaptation data 110 can be used to train or refine an adapter (e.g., a LoRA adapter) for the base model 105A in order to refine or modify outputs (or intermediate tensors) of the base model. For example, as discussed above, the adaptation data 110 may be used to adjust the artistic style of the output images, the color pallet of the output images, the visuals that tend to be included in the images, and the like. In some P+S Ref. No.: QUAL / 2405344PC02 5Client Ref. No.: 2405344B1WO aspects, though the adaptation data 110 may include similar formatting and structure to the data used to train the base model 105A (e.g., the adaptation data 110 may include images having the desired features and text prompt(s) indicating the desired features), the adaptation data 110 may not have any overlap with the data used to train the base model 105A. That is, the adaptation system 115 may train an adapter without access to the original training data for the base model 105A.

[0028] As illustrated, the adaptation system 115 generates an adapted base model 120A. The adapted base model 120A generally includes the base model 105A and an adapter 125. As discussed above, in some aspects, the adaptation system 115 may freeze the parameters of the base model 105A and update one or more parameters of the adapter 125 using the adaptation data 110. This can cause the output of the adapted base model 120A to more accurately reflect the desired content indicated in the adaptation data 110 (e.g., the style).

[0029] In the depicted workflow 100, the adapted base model 120A is accessed by a distillation system 130. Although the illustrated example depicts the distillation system 130 accessing the adapted base model 120A directly, in some aspects, the distillation system 130 may access the base model 105A and the adapter 125 separately. For example, the distillation system 130 may access the base model 105A from the same source as the adaptation system 115 (e.g., a training system that trained the base model 105A), while accessing the adapter 125 from the adaptation system 115 itself. Though depicted as a single discrete system for conceptual clarity, in some aspects, the distillation system 130 (or the operations thereof) may be combined or distributed across any number of systems and may be implemented using hardware, software, or a combination of hardware and software.

[0030] In the illustrated example, the distillation system 130 includes a distillation component 135 and a projection component 140. Although depicted as discrete components for conceptual clarity, in some aspects, the distillation component 135 and the projection component 140 (or the operations thereof) may be combined or distributed across any number of systems and components.

[0031] In the illustrated example, the distillation component 135 may be used to modify the base model 105A to generate a second base model 105B (referred to in some aspects as a “student base model” and / or as “distilled base model”). That is, the student P+S Ref. No.: QUAL / 2405344PC02 6Client Ref. No.: 2405344B1WO base model 105B may be a modified version of the base model 105A. For example, the student base model 105B may have a different architecture, may use a different number of sampling or diffusion steps, and the like. For example, the distilled base model 105B may be generated by pruning or removing one or more operations, layers, parameters, attention mechanisms, and the like from the base model 105A to generate a somewhat smaller model that can be used with less computational expense. In many cases, despite the similarities between the student base model 105B and the original base model 105A, the adapter 125 (trained for the base model 105A) cannot be readily used with the student base model 105B.

[0032] In the illustrated example, the projection component 140 may be used to facilitate or enable this reuse of the adapter 125. Specifically, in some aspects, the projection component 140 may generate or determine linear projection operation(s) that cause tensors generated by the student base model 105B to better align with tensors generated by the base model 105A. For example, the intermediate tensors generated (in the latent space) by each iteration of the student base model 105B may be aligned using the projection(s). In some aspects, the projection component 140 may therefore project the parameters of the student base model 105B to generate a new (projected) base model 105C, as discussed in more detail below.

[0033] Although the illustrated example depicts the distillation system 130 as performing both model distillation (to generate the student base model 105B based on the base model 105A) as well as projection (to create the base model 105C based on the student base model 105B), in some aspects, the distillation and projection may be performed by different computing systems. For example, a first system (e.g., a distillation system) may generate a student base model 105B based on the base model 105A, and this student base model 105B may be accessed by a second system (e.g., a projection system) to generate the projected base model 105C.

[0034] In the illustrated example, the distillation system 130 generates an adapted base model 120B, which includes the (projected) base model 105C and the adapter 125. That is, the adapter 125, which was trained for the base model 105A and is generally incompatible with the distilled student base model 105B, may be combined with the projected version of the student base model (e.g., the base model 105C). This allows the adapter 125 to be reused without relying on any further training or refinement. That is, the distillation system 130 need not have access to (and does not use) the training data P+S Ref. No.: QUAL / 2405344PC02 7Client Ref. No.: 2405344B1WO used to train the base model 105A or the adaptation data 110 used to train the adapter 125. Instead, the distillation system 130 can use computationally inexpensive projection (e.g., linear projection) to enable the reuse. Example Architecture for Effective Adapter Reuse in Machine Learning Models

[0035] FIG. 2 depicts example architectures 200 for effective adapter reuse in machine learning models, according to some aspects of the present disclosure. Specifically, the illustrated example depicts an architecture 200A (which may correspond to all or a portion of a teacher model, such as the adapted base model 120A of FIG. 1) and an architecture 200B (which may correspond to all or a portion of a student base model, such as the student base model 105B discussed above with reference to FIG.1).

[0036] In the illustrated architecture 200A, a portion 210A of a teacher base machine learning model is depicted (designated as ^^^^^^^^in the illustrated example). That is, the portion 210A may correspond to the parameters of a portion of the teacher base model (e.g., the base model 105A of FIG.1), such as a single layer, an attention operation, and the like. In the illustrated example, the architecture 200A further includes an adapter 215 (designed as ^^^^^^^^^^^^in the illustrated example) that corresponds to the portion 210A of the base model. For example, the adapter 215 may correspond to the parameters of a model adapter (e.g., a LoRA adapter) such as the adapter 125 of FIG. 1. In some aspects, as discussed above, the parameters of the adapter 215 may be trained (e.g., modified, updated, or refined) while the parameters of the portion 210A of the teacher base model are frozen.

[0037] In the illustrated example, an input tensor 205A (designated as ^^^^^^^^in the illustrated example) for the portion 210A is also provided as input to the adapter 215. Based on the input tensor 205A, the portion 210A generates an output tensor 220A (designated as ^^^^^^^^in the illustrated example). As illustrated, the output tensor 220A from the portion 210A of the base model is aggregated with the output of the adapter 215 using an aggregation operation 225. The aggregation operation 225 may generally include a variety of operations, such as elementwise summation, to combine the tensors. In the illustrated example, aggregated tensor 230A (designated as ^^^^^^^^), generated by the aggregation operation 225, can then be used as the output of the architecture 200A (e.g., input to a subsequent component or layer, or output from the model). In some aspects, P+S Ref. No.: QUAL / 2405344PC02 8Client Ref. No.: 2405344B1WO the parameters of the architecture 200A (e.g., the portion 210A and the adapter 215) maybe defined

[0038] In the illustrated architecture 200B, a portion 210B of a student base machine learning model is depicted (designated as ^^^^^^^^in the illustrated example). That is, the portion 210B may correspond to the parameters of a portion of the distilled base model (e.g., the distilled base model 105B of FIG. 1), such as a single layer, an attention operation, and the like. In the illustrated example, the architecture 200B does not include or have a corresponding adapter. In some aspects, as discussed above, the parameters of the portion 210B may be generated based on distilling knowledge from the teacher base model. For example, in some aspects, the portion 210B of the student base model corresponds to the portion 210A of the teacher base model.

[0039] In the illustrated example, an input tensor 205B (designated as ^^^^^^^^in the illustrated example) for the portion 210B is provided as input to the portion 210B. Based on the input tensor 205B, the portion 210B generates an output tensor 220B (designated as ^^^^^^^^in the illustrated example). In some aspects, if no adapter is used, the output tensor 220B can then be used as the output of the architecture 200B (e.g., input to a subsequent component or layer, or output from the model).

[0040] As discussed above, in some aspects, a projection system (e.g., the projection component 140 of FIG.1) seeks to align the intermediate outputs of the student model (e.g., the output tensor 220B from the portion 210B) with the intermediate outputs of the corresponding portions of the teacher model (e.g., the output tensor 220A of the portion 210A). That is, the projection system may seek to make ^^^^^^^^andclose to each other byapplying linear projection ^^�^^ , allowing the adapter 215 from the teacher model to beadopted by the student model.

[0041] Specifically, in some aspects, the projection component may generate a projection ^^�^^ that causes the output of each portion of the student base model (e.g., the output tensor 220B) to align with or become more similar to the output of the teacher base model (e.g., the output tensor 220A). In some aspects, the projection component can useEquation 1 below to define the projection, where ^^�^^ is a linear projection, ^^^^^^^^ is theparameters (e.g., a set of weights) of the portion 210B of the student base model, ^^^^^^^^is the parameters (e.g., a set of weights) of the portion 210A of the teacher base model, and P+S Ref. No.: QUAL / 2405344PC02 9Client Ref. No.: 2405344B1WOthe superscript ^^^^ indicates transposition of the associated matrix (e.g., ^^^^^^^^^^^^ is thetransposed set of student weights). ^^�^^ = (^^^ ^^^^ −1 ^^^^^^^^ ^^^^^ ^^^^) ^^^^^^^^ ^^^^^^^^ (1)

[0042] That is, ^^�^^ is a linear projection from the parameters of the student base model (e.g., the portion 210B) to the parameters of the teacher base model (e.g., the portion 210A). Therefore, the projected version of the student base model may be defined as^^^^^^^^←^^^^ = ^^^^^^^^^^�^^. That is, the weights of a given portion of the student base model (e.g., theparameters ^^^^^^^^ of the portion 210B) may be multiplied by the projection ^^�^^ to yield a(portion of the) projected base model ^^^^^^^^←^^^^. Stated differently, one or more linear projections ^^�^^ may be applied to one or more portions of the student base model to yield a projected (student) base model.

[0043] Turning now to FIG. 3, an architecture 300 for using a pre-trained adapter with a student machine learning model, according to some aspects of the present disclosure, is depicted. In the illustrated architecture 300, the portion 210B of the student base model has been replaced with a portion 305 of the projected base model (e.g., a portion of the base model 105C of FIG. 1), designated as ^^^^^^^^←^^^^. As illustrated, this projection operation allows the adapter 215, which was trained for the portion 210A of the teacher base model, to be readily applied to the projected base model.

[0044] Specifically, as illustrated, an input tensor 205B (designated as ^^^^^^^^in the illustrated example) for the (projected) portion 305 is also provided as input to the adapter 215. Based on the input tensor 205B, the portion 305 generates an output tensor 220B (designated asin the illustrated example). As illustrated, the output tensor 220B from the portion 305 of the projected base model is aggregated with the output of the adapter 215 using an aggregation operation 225 (e.g., elementwise summation) to generate an aggregated tensor 230B (designated as ^^^^^^^^in the illustrated example). This aggregated tensor 230B can then be used as the output of the architecture 300 (e.g., input to a subsequent component or layer, or output from the model). In some aspects, theparameters of the architecture 300 may be defined as ^^^^∗^^^^ = ^^^^^^^^←^^^^ +That is, thearchitecture 300 may represent a portion of the adapted base model 120B of FIG.1, where ^^^^^^^^←^^^^is the parameters of at least a portion of the projected base model 105C and ^^^^^^^^^^^^is the parameters of at least a portion of the adapter 125. P+S Ref. No.: QUAL / 2405344PC02 10Client Ref. No.: 2405344B1WO

[0045] Advantageously, this projection can be implemented in a training-free manner using computationally inexpensive linear projections, allowing pre-trained adapters for a given base model to be reused by any number and variety of modified versions of the base model. Example Method for Reusing Model Adapters in Machine Learning Models

[0046] FIG.4 is a flow diagram depicting an example method 400 for reusing model adapters in machine learning models, according to some aspects of the present disclosure. In some aspects, the method 400 is performed by a computing system such as the distillation system 130 of FIG.1.

[0047] At block 405, the computing system accesses a first base model (e.g., the base model 105A of FIG. 1) and an adapter (e.g., adapter 125 of FIG. 1) for the first base model. In some aspects, as discussed above, the base model may generally correspond to a base model of a generative model such as an LLM, an LVM, an LMM, and the like. Further, as discussed above, the adapter may generally correspond to a relatively small set of parameters trained (based on a relatively small set of training data, as compared to the data used to train the base model) to modify the output of the base model (e.g., to modify the style of the outputs). In some aspects, the adapter corresponds to or comprises one or more LoRA adapters.

[0048] At block 410, the computing system generates a second base model (e.g., the student base model 105B of FIG.1) based on the first base model. For example, in some aspects, the second base model may correspond to a modified version of the first base model. The second base model may be generated by performing various actions, such as pruning one or more components of the first base model, reducing or modifying the number of sampling steps used to generate output, and the like. In some aspects, as discussed above, the second base model may be a distilled version of the first base model (e.g., intended to perform the same task or a similar task with reduced computational expense). Although the illustrated example depicts generating the second base model, in some aspects, the computing system may receive the second base model from another system, as discussed above.

[0049] At block 415, the computing system selects a layer (or other portion) of the second base model for which an adapter will be used. That is, the computing system may determine which layer(s) (or other portions) of the first base model have a corresponding P+S Ref. No.: QUAL / 2405344PC02 11Client Ref. No.: 2405344B1WO portion of the adapter (e.g., where the portion 210A of FIG. 2 has the corresponding adapter 215), and which layer(s) or portion(s) of the second base model correspond to these adapted portions of the first base model (e.g., where the portion 210B of FIG. 2 corresponds to the portion 210A because the portion 210B was generated based on distilling the portion 210A of the teacher model).

[0050] Generally, the computing system may use a variety of techniques to select the layer of the second base model at block 415, as the computing system will process each relevant (adapted) portion during the method 400.

[0051] At block 420, the computing system generates one or more linear projections for the selected layer (or other portion) of the second base model based on the corresponding layer (or other portion) of the first base model. For example, as discussed above, the computing system may generate the projection ^^�^^ using Equation 1 above. By multiplying this projection by the parameters of the selected portion of the second base model (e.g., ^^^^^^^^), the computing system can efficiently project the parameters of the second base model to align or be more similar to the parameters of the first base model (e.g., to generate ^^^^^^^^←^^^^). As discussed above, these projections can therefore be used to create a projected base model (e.g., by projecting the parameters of each layer based on the corresponding projection(s)).

[0052] At block 425, the computing system determines whether there is at least one additional adapted layer (or other portion) remaining in the second base model. If so, the method 400 returns to block 415. If not, the method 400 continues to block 430. Although the illustrated example depicts an iterative process (e.g., selecting and processing each layer of the second base model in sequence) for conceptual clarity, in some aspects, the computing system may process some or all of the layers of the second base model entirely or partially in parallel.

[0053] At block 430, the computing system deploys the projected second base model, along with the adapter of the first base model, as a projected and adapted base model (e.g., the adapted base model 120B of FIG.1). That is, the computing system may deploy the combination of the projected second base model and the adapter from the teacher. As used herein, “deploying” the model may generally include performing any operations used to prepare or provide the model for runtime use, including generating and / or P+S Ref. No.: QUAL / 2405344PC02 12Client Ref. No.: 2405344B1WO transmitting a model binary file, transferring the parameters to local memory for use, and the like. Example Method for Adapting Machine Learning Models

[0054] FIG.5 is a flow diagram depicting an example method for adapting machine learning models, according to some aspects of the present disclosure.

[0055] At block 505, a first adapted machine learning model (e.g., the adapted base model 120A of FIG.1) comprising a first base model (e.g., the base model 105A of FIG. 1) and an adapter trained for the first base model (e.g., the adapter 125 of FIG. 1) is accessed.

[0056] At block 510, a second base model (e.g., the base model 105B of FIG.1) is accessed.

[0057] At block 515, one or more linear projections for the second base model are generated based on the first base model, wherein the one or more linear projections align tensors generated by the second base model with tensors generated by the first base model.

[0058] At block 520, a projected base model (e.g., the base model 105C of FIG.1) is generated based on the second base model and the one or more linear projections.

[0059] At block 525, a second adapted machine learning model (e.g., the adapted base model 120B of FIG.1) comprising the projected base model and the adapter is generated.

[0060] In some aspects, the first base model comprises a generative model and wherein the adapter comprises a low-rank adaptation (LoRA) adapter.

[0061] In some aspects, generating the one or more linear projections comprises generating, for each respective layer of a plurality of layers in the second base model, a respective linear projection based on a respective corresponding layer in the first base model.

[0062] In some aspects, at least one of the one or more linear projections is definedas ^^�^^ = (^^^ ^^^^ −1 ^^^^^^^^ ^^^^^^^^) ^^^ ^^^^^^ ^^^^^^^^ , where: ^^�^^ is the at least one linear projection, ^^^^^^^^ is a set ofweights of the second base model, and^^^^^^^^is a set of weights of the first base model.

[0063] In some aspects, the projected base model is defined as ^^^^^^^^←^^^^ = ^^^^^^^^^^�^^, where^^^^^^^^←^^^^is the projected base model. P+S Ref. No.: QUAL / 2405344PC02 13Client Ref. No.: 2405344B1WO

[0064] In some aspects, the second adapted machine learning model is defined as^^^ ∗^^^^^where: ^^^ ∗^^^^^ is the second adapted machine learning model, and^^^^^^^^^^^^is the adapter.

[0065] In some aspects, the method 500 further includes deploying the second adapted machine learning model.

[0066] In some aspects, the method 500 further includes generating a model output based on processing a model input using the second adapted machine learning model.

[0067] In some aspects, generating the one or more linear projections, generating the projected base model, and generating the second adapted machine learning model are performed without processing data used to train the adapter (e.g., the adaptation data 110 of FIG.1).

[0068] In some aspects, the second base model corresponds to a modified version of the first base model. Example Processing System for Machine Learning

[0069] FIG. 6 depicts an example processing system 600 configured to perform various aspects of the present disclosure, including, for example, the techniques and methods described with respect to FIGS. 1-5 and / or FIGS. 7-9. In some aspects, the processing system 600 may correspond to a computing system. For example, the processing system 600 may correspond to the distillation system 130 of FIG.1 and / or the computing systems discussed above with reference to FIGS. 2-5 and / or below with reference to FIGS.7-9. Although depicted as a single system for conceptual clarity, in some aspects, as discussed above, the components described below with respect to the processing system 600 may be distributed across any number of devices or systems.

[0070] The processing system 600 includes a central processing unit (CPU) 602, which in some examples may be a multi-core CPU. Instructions executed at the CPU 602 may be loaded, for example, from a program memory associated with the CPU 602 or may be loaded from a memory partition (e.g., a partition of a memory 624).

[0071] The processing system 600 also includes additional processing components tailored to specific functions, such as a graphics processing unit (GPU) 604, a digital signal processor (DSP) 606, a neural processing unit (NPU) 608, a multimedia component 610 (e.g., a multimedia processing unit), and a wireless connectivity component 612. P+S Ref. No.: QUAL / 2405344PC02 14Client Ref. No.: 2405344B1WO

[0072] An NPU, such as the NPU 608, is generally a specialized circuit configured for implementing the control and arithmetic logic for executing machine learning algorithms, such as algorithms for processing artificial neural networks (ANNs), deep neural networks (DNNs), random forests (RFs), and the like. An NPU may sometimes alternatively be referred to as a neural signal processor (NSP), tensor processing unit (TPU), neural network processor (NNP), intelligence processing unit (IPU), vision processing unit (VPU), or graph processing unit.

[0073] NPUs, such as the NPU 608, are configured to accelerate the performance of common machine learning tasks, such as image classification, machine translation, object detection, and various other predictive models. In some examples, a plurality of NPUs may be instantiated on a single chip, such as a system on a chip (SoC), while in other examples the NPUs may be part of a dedicated neural-network accelerator.

[0074] NPUs may be optimized for training or inference, or in some cases configured to balance performance between both. For NPUs that are capable of performing both training and inference, the two tasks may still generally be performed independently.

[0075] NPUs designed to accelerate training are generally configured to accelerate the optimization of new models, which is a highly compute-intensive operation that involves inputting an existing dataset (often labeled or tagged), iterating over the dataset, and then adjusting model parameters, such as weights and biases, in order to improve model performance. Generally, optimizing based on a wrong prediction involves propagating back through the layers of the model and determining gradients to reduce the prediction error.

[0076] NPUs designed to accelerate inference are generally configured to operate on complete models. Such NPUs may thus be configured to input a new piece of data and rapidly process this piece of data through an already trained model to generate a model output (e.g., an inference). In some implementations, the NPU 608 is a part of one or more of the CPU 602, the GPU 604, and / or the DSP 606.

[0077] In some examples, the wireless connectivity component 612 may include subcomponents, for example, for third generation (3G) connectivity, fourth generation (4G) connectivity (e.g., Long-Term Evolution (LTE)), fifth generation (5G) connectivity (e.g., New Radio (NR)), Wi-Fi connectivity, Bluetooth connectivity, and other wireless P+S Ref. No.: QUAL / 2405344PC02 15Client Ref. No.: 2405344B1WO data transmission standards. The wireless connectivity component 612 is further coupled to one or more antennas 614.

[0078] The processing system 600 may also include one or more sensor processing units 616 associated with any manner of sensor, one or more image signal processors (ISPs) 618 associated with any manner of image sensor, and / or a navigation processor 620, which may include satellite-based positioning system components (e.g., GPS or GLONASS) as well as inertial positioning system components.

[0079] The processing system 600 may also include one or more input and / or output devices 622, such as screens, touch-sensitive surfaces (including touch-sensitive displays), physical buttons, speakers, microphones, and the like.

[0080] In some examples, one or more of the processors of the processing system 600 may be based on an ARM or RISC-V instruction set.

[0081] The processing system 600 also includes a memory 624, which is representative of one or more static and / or dynamic memories, such as a dynamic random access memory, a flash-based static memory, and the like. In this example, the memory 624 includes computer-executable components, which may be executed by one or more of the aforementioned processors of the processing system 600.

[0082] In particular, in this example, the memory 624 includes a distillation component 624A and a projection component 624B. Although not depicted in the illustrated example, the memory 624 may also include other components, such as a training component used to train or update machine learning model(s) or adapters, an inferencing component used to manage generation of model output during runtime, and the like. Though depicted as discrete components for conceptual clarity in FIG. 6, the illustrated components (and others not depicted) may be collectively or individually implemented in various aspects.

[0083] Further, in the illustrated example, the memory 624 also includes a set of model parameters 624C (e.g., parameters of one or more machine learning models, such as base models and / or adapters) and a set of projections 624D. In some aspects, the model parameters 624C may correspond to or include the parameters of one or more teacher and / or student base models (e.g., the base model 105A, 105B, and / or 105C, each of FIG. 1). In some aspects, the model parameters 624C may further include the parameters of one or more adapter models (e.g., the adapter 125 of FIG. 1 and / or the adapter 215 of P+S Ref. No.: QUAL / 2405344PC02 16Client Ref. No.: 2405344B1WO FIGS.2-3) and / or projected adapters (e.g., the adapter 705 of FIG.7). In some aspects, the projections 624D may indicate linear projections from the parameters of a distilled base model to the parameters of the original base model based on which the distilled model was generated (e.g., the projections ^^�^^). In some aspects, the projections 624D may include or indicate relevant components of the model parameters, such as left and / or right singular matrices of the teacher and / or student base model(s), decomposed matrices, and the like.

[0084] Although not depicted in the illustrated example, in some aspects, the memory 624 may include other data such as a training data for the machine learning model(s).

[0085] The processing system 600 further comprises a distillation circuit 626 and a projection circuit 627. The depicted circuits, and others not depicted (such as an inferencing circuit), may be configured to perform various aspects of the techniques described herein.

[0086] The distillation component 624A and / or the distillation circuit 626 (which may correspond to the distillation component 135 of FIG. 1) may be used to generate modified (e.g., distilled) versions of base machine learning models, as discussed above. For example, the distillation component 624A and / or the distillation circuit 626 may be used to generate modified architectures that use reduced sampling steps, reduced layers, and the like.

[0087] The projection component 624B and / or the projection circuit 627 (which may correspond to the projection component 140) may be used to generate and / or apply projections to distilled (e.g., student) base models, as discussed above, and / or to generate and / or apply projections to adapter models, as discussed in more detail below. For example, the projection component 624B and / or the projection circuit 627 may use Equation 1 above to generate the projection(s), and may then project the parameters of the distilled base model to allow adapters trained for the original teacher model to be re- used with the projected base model, as discussed above. As another example, the projection component 624B and / or the projection circuit 627 may use singular value decomposition and / or Equations 3, 4, 5, and / or 6 below to project the parameters of model adapters to different base models (allowing adapters trained for an original base model to be re-used with the various other base models), as discussed below in more detail. P+S Ref. No.: QUAL / 2405344PC02 17Client Ref. No.: 2405344B1WO

[0088] Though depicted as separate components and circuits for clarity in FIG.6, the distillation circuit 626 and the projection circuit 627 may collectively or individually be implemented in other processing devices of the processing system 600, such as within the CPU 602, the GPU 604, the DSP 606, the NPU 608, and the like.

[0089] Generally, the processing system 600 and / or components thereof may be configured to perform the methods described herein.

[0090] Notably, in other aspects, aspects of the processing system 600 may be omitted, such as where the processing system 600 is a server computer or the like. For example, the multimedia component 610, the wireless connectivity component 612, the sensor processing units 616, the ISPs 618, and / or the navigation processor 620 may be omitted in other aspects. Further, aspects of the processing system 600 may be distributed between multiple devices. Example Architecture for Using a Projected Adapter with a Student Machine Learning Model

[0091] FIG.7 depicts an architecture 700 for using a projected adapter with a student machine learning model, according to some aspects of the present disclosure. Specifically, the illustrated example depicts an architecture 700 (which may correspond to all or a portion of a student base model, such as the student base model 105B discussed above with reference to FIG.1).

[0092] In some aspects, rather than aligning the intermediate outputs of a student model (e.g., the output tensor 220B from the portion 210B of FIG. 2) with the intermediate outputs of the corresponding portions of the teacher model (e.g., the output tensor 220A of the portion 210A of FIG. 2), a projection system (e.g., the projection component 140 of FIG. 1) seeks to project an adapter (e.g., the adapter 215) from the space of the teacher model to the space of the student model. That is, rather than projecting the parameters of the student model, the projection system may project the parameters of the adapter, allowing the adapter from the teacher model to be adopted by the student model (or other base model).

[0093] Specifically, in some aspects, to transfer an adapter ^^^^^^^^^^^^(e.g., the adapter 215 of FIG.2) trained for a portion of a first base model (e.g., the portion 210A of FIG.2), the projection system can project the adapter to the range space (e.g., the column subspace P+S Ref. No.: QUAL / 2405344PC02 18Client Ref. No.: 2405344B1WO and / or the row subspace) and the null space of the base model (e.g., of ^^^^^^^^) using singular value decomposition (SVD).

[0094] Specifically, given an adapter ^^^^^^^^^^^^ ∈ ^^^^^^^^×^^^^, which is trained for a set of basemodel weights (e.g., for a layer of the base model) ^^^^ ∈ ^^^^^^^^×^^^^^^^^ of rank ^^^^ ≤ min (^^^^,^^^^),the projection component may apply SVD to the portion of the base model ^^^^^^^^to obtain^^^^ = ^^^^ ^^^^ ^^^^^^^^ , where ^^^^ ∈ ^^^^^^^^×^^^^ and ^^^^ ∈ ^^^^^^^^×^^^^ are left and right singular matrices,^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^respectively, and Σ ∈ ^^^^^^^^×^^^^t is a rectangular diagonal matrix of singular values.

[0095] In some aspects, the left singular matrix of the teacher model portion ^^^^^^^^canthen be decomposed as ^^^^ = [^^^^ , ^^^^ ], where ^^^^ ∈ ^^^^^^^^×^^^^^^^^ ^^^^,∥ ^^^^,⊥ ^^^^,∥ (which may be referredto as a “parallel matrix” in some aspects, and may correspond to the range space of the teacher base model) contains the orthonormal bases spanning the column subspace of ^^^^^^^^,and ^^^^^^^^,⊥ ∈ ^^^^^^^^×(^^^^−^^^^) (which may be referred to as a “normal matrix” in some aspects,and may correspond to the null space of the teacher base model) contains the orthonormal ^^^^bases spanning the null space of ^^^^ .^^^^

[0096] Similarly, in some aspects, the right singular matrix can be decomposed intothe range and null subspaces of the teacher base model as ^^^^ = [^^^^ , ^^^^ ] , where^^^^ ^^^^,∥ ^^^^,⊥^^^^×^^^^ ^^^^ ∈ ^^^^ contains the orthonormal bases spanning the row subspace of ^^^^ , and^^^^,∥ ^^^^^^^^ ×(^^^^−^^^^) ^^^^ ∈ ^^^^ contains the orthonormal bases spanning the null space of ^^^^ .^^^^,⊥ ^^^^

[0097] In some aspects, the projection system can then project the parameters of the adapter ^^^^^^^^ to the range space (e.g., the column and row subspaces) and the null space^^^^of ^^^^ by decomposing the adapter to one or more adapter components defined using^^^^Equations 2 and 3 below, where are first and secondcomponents (e.g., a parallel matrix and a normal matrix of the adapter), respectively.

[0098] That is, using Equation 2 above, the projection system may multiply the left and right parallel matrices ^^^^ and ^^^^ (and their transposes) of the first base model ^^^^^^^^,∥ ^^^^,∥ ^^^^with the adapter ^^^^^^^^ to project a first adapter component to the range subspacethe first base model. Similarly, using Equation 3 above, the projection system may multiply the left and right normal matrices ^^^^ and ^^^^ (and their transposes) of the first^^^^,⊥ ^^^^,⊥P+S Ref. No.: QUAL / 2405344PC02 19Client Ref. No.: 2405344B1WO base model ^^^^^^^^with the adapter ^^^^^^^^^^^^to project a second adapter componentto thenull subspace of the first base model. In some aspects, ^^^^^^^^^^^^ = ^^^^^^^^^^^^,‖ + ^^^^^^^^^^^^,⊥ (that is,the adapter components may be generated by projecting the adapter to the range and null space of the first base model).

[0099] In some aspects, the projection system can then transfer the projected adapter ^^^^^^^^^^^^to the target (e.g., student) base model. That is, the projection system may project the adapter components (e.g., ^^^^^^^^^^^^,‖and ^^^^^^^^^^^^,⊥) to the range and null space, respectively, of the second base model. In some aspects, the projection system seeks to transfer the adapter to the corresponding portion of the student model such that the transferred adapter ∈^^^^^^^^×^^^^ has the similar effect on the second model ^^^^^^^^ as the effect of the originaladapter Δ^^^^^^^^on the first base model ^^^^^^^^.

[0100] Specifically, the projection system may apply SVD to the correspondingportion of the second base model ^^^^ to obtain ^^^^ = ^^^^ ^^^^ ^^^^^^^^^^^^ ^^^^ ^^^^ ^^^^ ^^^^ , where ^^^^^^^^ and ^^^^^^^^ are left andright singular matrices, respectively, and Σsis a rectangular diagonal matrix of singular values.

[0101] In some aspects, as discussed above, the left singular matrix of the secondmodel portion ^^^^^^^^ can then be decomposed as ^^^^^^^^ = [^^^^^^^^,∥ , ^^^^^^^^,⊥] , where ^^^^^^^^,∥ may bereferred to as a “parallel matrix” in some aspects (and may correspond to the range space of the second base model), and contains the orthonormal bases spanning the column subspace of ^^^^^^^^. Similarly, ^^^^^^^^,⊥may be referred to as a “normal matrix” in some aspects (and may correspond to the null space of the second base model), and contains the orthonormal bases spanning the null space of ^^^^^^^^^^^^.

[0102] Similarly, in some aspects, the right singular matrix can be decomposed intothe range and null subspaces of the second base model as ^^^^^^^^ = [^^^^^^^^,∥, ^^^^^^^^,⊥], where ^^^^^^^^,∥contains the orthonormal bases spanning the row subspace of ^^^^^^^^, and ^^^^^^^^,⊥contains the orthonormal bases spanning the null space of ^^^^^^^^.

[0103] In some aspects, then, given the decomposed projected adapter ^^^^^^^^^^^^=^^^^^^^^ the projection system can use Equations 4 and 5 below to projectcomponents of the adapter to the range and null spaces of the second base model, where ^^^^^^^^^^^^←^^^^,‖and ^^^^^^^^^^^^←^^^^,⊥are first and second projected adapter components (e.g., a parallel matrix and a normal matrix of the projected adapter), respectively P+S Ref. No.: QUAL / 2405344PC02 20Client Ref. No.: 2405344B1WO (4)(5)

[0104] That is, using Equation 4 above, the projection system may multiply the left and right parallel matrices ^^^^^^^^,∥and ^^^^^^^^,∥(and their transposes) of the second base model ^^^^ with the first (parallel) adapter componentto project the first componentto the range subspace of the second base model. Similarly, Equation 5 above, the projection system may multiply the left and right normal matrices(and their transposes) of the second base model ^^^^^^^^with the second normal) adapter component ^^^^^^^^^^^^,⊥to project the second adapter component ^^^^^^^^^^^^,⊥to thenull subspace of the second base model. In some aspects,(that is, the projected adapter components may be aggregated to generate the projected adapter.

[0105] In this way, the projected adapter can be used alongside the unchanged student model in order to generate output data.

[0106] As noted, the architecture 700 for using a projected adapter with a pre-trained student machine learning model, according to some aspects of the present disclosure, is depicted. In the illustrated architecture 700, the portion 210B of the student base model (e.g., a portion of the base model 105C of FIG. 1) remains unchanged. As illustrated, this projection operation was used to project the adapter (e.g., the adapter 215) to the spaces of the student model, such that the adapter which was trained for the portion 210A of the teacher base model can be applied to the student base model. Specifically, as illustrated, the adapter for the portion 210B can be replaced with the projected adapter 705 (designated ^^^^^^^^^^^^←^^^^) in some aspects.

[0107] Specifically, as illustrated in the architecture 700, an input tensor 205B (designated as ^^^^^^^^in the illustrated example) for the portion 210B is also provided as input to the (projected) adapter 705. Based on the input tensor 205B, the portion 210B generates an output tensor 220B (designated as ^^^^^^^^in the illustrated example). As illustrated, the output tensor 220B from the portion 210B of the base model is aggregated with the output of the projected adapter 705 using an aggregation operation 225 (e.g., elementwise summation) to generate an aggregated tensor 230B (designated as ^^^^^^^^in the illustrated example). This aggregated tensor 230B can then be used as the output of the P+S Ref. No.: QUAL / 2405344PC02 21Client Ref. No.: 2405344B1WO architecture 700 (e.g., input to a subsequent component or layer, or output from the model). In some aspects, the parameters of the architecture 700 may be defined as ^^^^^^^^∗=^^^^^^^^ + ^^^^^^^^^^^^←^^^^. That is, the architecture 300 may represent a portion of the adapted basemodel 120B of FIG.1, where ^^^^^^^^is the parameters of at least a portion of the base model 105Cis the parameters of at least a portion of the adapter 125.

[0108] Advantageously, this projection can be implemented in a training-free manner using computationally inexpensive linear projections, allowing pre-trained adapters for a given base model to be projected and reused by any number and variety of base models (including modified versions of the original base model). Example Method for Projecting Model Adapters for Reuse in Machine Learning Models

[0109] FIG. 8 is a flow diagram depicting an example method 800 for projecting model adapters for reuse in machine learning models, according to some aspects of the present disclosure. In some aspects, the method 800 is performed by a computing system such as the distillation system 130 of FIG.1.

[0110] At block 805, the computing system accesses a first base model (e.g., the base model 105A of FIG. 1) and an adapter (e.g., adapter 125 of FIG. 1) for the first base model. In some aspects, as discussed above, the base model may generally correspond to a base model of a generative model, such as an LLM, an LVM, an LMM, and the like. Further, as discussed above, the adapter may generally correspond to a relatively small set of parameters trained (based on a relatively small set of training data, as compared to the data used to train the base model) to modify the output of the base model (e.g., to modify the style of the outputs). In some aspects, the adapter corresponds to or comprises one or more LoRA adapters.

[0111] At block 810, the computing system accesses a second base model (e.g., the student base model 105B of FIG. 1). For example, in some aspects, the second base model may correspond to a modified version of the first base model. The second base model may be generated by performing various actions, such as pruning one or more components of the first base model, reducing or modifying the number of sampling steps used to generate output, further training or refining the first base model, and the like. In some aspects, as discussed above, the second base model may be a distilled version of the first base model (e.g., intended to perform the same task or a similar task with reduced computational expense). P+S Ref. No.: QUAL / 2405344PC02 22Client Ref. No.: 2405344B1WO

[0112] At block 815, the computing system selects a layer (or other portion) of the first base model for which an adapter is used. That is, the computing system may determine which layer(s) (or other portions) of the first base model have a corresponding portion of the adapter (e.g., where the portion 210A of FIG. 2 has the corresponding adapter 215), and which layer(s) or portion(s) of the second base model correspond to these adapted portions of the first base model (e.g., where the portion 210B of FIG. 2 corresponds to the portion 210A because the portion 210B was generated based on distilling the portion 210A of the teacher model).

[0113] Generally, the computing system may use a variety of techniques to select the layer of the first base model at block 815, as the computing system will process each relevant (adapted) portion during the method 800.

[0114] At block 820, the computing system applies SVD to the parameter tensor of the selected layer (e.g., to ^^^^^^^^) to generate a first left singular matrix (e.g., ^^^^^^^^) and a first right singular matrix (e.g., ^^^^^^^^), as discussed above.

[0115] At block 825, the computing system decomposes the first left singular matrix to generate a first parallel matrix (e.g., ^^^^^^^^,‖) and a first normal matrix (e.g., ^^^^^^^^,⊥), and further decomposes the first right singular matrix to generate a second parallel matrix (e.g., and a second normal matrix (e.g., ^^^^^^^^,⊥), as discussed above.

[0116] At block 830, the computing system projects the adapter to the range subspaces (e.g., the column and row subspaces) and the null subspace of the selected layer of the teacher base model (e.g., using Equations 2 and 3 above) to generate a pair of adapter components, as discussed above.

[0117] At block 835, the computing system can apply SVD to the parameter tensor of the corresponding layer in the student base model (e.g., to ^^^^^^^^) to generate a second left singular matrix (e.g., ^^^^^^^^) and a second right singular matrix (e.g., ^^^^^^^^), as discussed above.

[0118] At block 840, the computing system decomposes the second left singular matrix to generate a third parallel matrix (e.g., ^^^^^^^^,‖) and a third normal matrix (e.g., ^^^^^^^^,⊥), and further decomposes the second right singular matrix to generate a fourth parallel matrixand a fourth normal matrix (e.g., ^^^^^^^^,⊥), as discussed above.

[0119] At block 845, the computing system projects the adapter (e.g., the adapter components) to the range subspaces (e.g., the column and row subspaces) and the null P+S Ref. No.: QUAL / 2405344PC02 23Client Ref. No.: 2405344B1WO subspace of the corresponding layer in the second (e.g., student) base model (e.g., using Equations 4 and 5 above).

[0120] At block 850, the computing system determines whether there is at least one additional adapted layer (or other portion) remaining in the first (teacher) base model. If so, the method 800 returns to block 815. If not, the method 800 continues to block 830. Although the illustrated example depicts an iterative process (e.g., selecting and processing each layer of the first base model in sequence) for conceptual clarity, in some aspects, the computing system may process some or all of the layers of the base model entirely or partially in parallel.

[0121] At block 855, the computing system deploys the second base model, along with the projected adapter from the first base model, as a projected and adapted base model (e.g., the adapted base model 120B of FIG.1). That is, the computing system may deploy the combination of the second base model and the projected adapter from the teacher. As used herein, “deploying” the model may generally include performing any operations used to prepare or provide the model for runtime use, including generating and / or transmitting a model binary file, transferring the parameters to local memory for use, and the like. Example Method for Adapting Machine Learning Models

[0122] FIG. 9 is a flow diagram depicting an example method 900 for adapting machine learning models, according to some aspects of the present disclosure. In some aspects, the method 900 is performed by a computing system such as the distillation system 130 of FIG.1.

[0123] At block 905, a first adapted machine learning model (e.g., the adapted base model 120A of FIG.1) comprising a first base model (e.g., the base model 105A of FIG. 1) and an adapter (e.g., the adapter 125 of FIG. 1) trained for the first base model is accessed.

[0124] At block 910, one or more adapter components are generated based on projecting the adapter to a range space and a null space of the first base model (e.g., using Equations 2 and 3 above).

[0125] At block 915, a second base model (e.g., the base model 105B of FIG.1) is accessed. P+S Ref. No.: QUAL / 2405344PC02 24Client Ref. No.: 2405344B1WO

[0126] At block 920, a projected adapter (e.g., corresponding to the adapter 705 of FIG.7) is generated based on projecting the one or more adapter components to a range space and a null space of the second base model (e.g., using Equations 4 and 5 above).

[0127] At block 925, a second adapted machine learning model (e.g., the adapted base model 120B of FIG.1) comprising the second base model and the projected adapter is generated.

[0128] At block 930, a machine learning model output is generated using the second adapted machine learning model.

[0129] In some aspects, generating the one or more adapter components comprises applying singular value decomposition (SVD) to the first base model to generate a left singular matrix and a right singular matrix for the first base model.

[0130] In some aspects, generating the one or more adapter components further comprises decomposing the left singular matrix to generate a first parallel matrix corresponding to the range space of the first base model and a first normal matrix corresponding to the null space of the first base model, as well as decomposing the right singular matrix to generate a second parallel matrix corresponding to the range space of the first base model and a second normal matrix corresponding to the null space of the first base model.

[0131] In some aspects, a first adapter component of the one or more adapter ^^^^ ^^^^ components is generated according tois thefirst adapter component, ^^^^^^^^^^^^is the adapter, ^^^^^^^^,‖is the first parallel matrix,is a transpose of the first parallel matrix, ^^^^^^^^,‖is the second parallel matrix, andis a transpose of the second parallel matrix.

[0132] In some aspects, a second adapter component of the one or more adaptercomponents is generated according to ^^^^^^^^ = ^^^^ ^^^^^^^^ ^^^^^^^^ ^^^^^^^^^^^^,⊥ ^^^^,⊥ ^^^^,⊥ ^^^^ ^^^^,⊥ ^^^^^^^^,⊥, where ^^^^^^^^^^^^,⊥ is thesecond adapter component, ^^^^^^^^,⊥is the first normal matrix,is a transpose of the first normal matrix, ^^^^^^^^,⊥is the second normal matrix, and ^^^^^^^^^^,^⊥^is a transpose of the second normal matrix.

[0133] In some aspects, generating the projected adapter comprises applying singular value decomposition (SVD) to the second base model to generate a left singular matrix and a right singular matrix for the second base model. P+S Ref. No.: QUAL / 2405344PC02 25Client Ref. No.: 2405344B1WO

[0134] In some aspects, generating the projected adapter further comprises decomposing the left singular matrix to generate a first parallel matrix corresponding to the range space of the second base model and a first normal matrix corresponding to the null space of the second base model, as well as decomposing the right singular matrix to generate a second parallel matrix corresponding to the range space of the second base model and a second normal matrix corresponding to the null space of the second base model.

[0135] In some aspects, the projected adapter is generated according to^^^^ ^^^^+ ^^^^^^^^,⊥^^^^^^^^,⊥ ^^^^^^^^^^^^,⊥^^^^^^^^,⊥ ^^^^^^^^,⊥ , where ^^^^^^^^^^^^←^^^^ is the projected adapter,^^^^ is the second parallel matrix,is a transpose of the second parallel matrix, is a parallel matrix of the adapter projected to the range space of the first base model,is the second parallel matrix,is a transpose of the second parallel matrix, ^^^^ is second normal matrix, ^^^^^^^^^^^,^⊥ is a transpose of the second normal matrix, ^^^^^^^^^^^^,⊥is a normal matrix of the adapter projected to the null space of the first base model, ^^^^^^^^,⊥is the second normal matrix, andis a transpose of the second normal Example Clauses

[0136] Implementation examples are described in the following numbered clauses:

[0137] Clause 1: A method, comprising: accessing a first adapted machine learning model comprising a first base model and an adapter trained for the first base model; accessing a second base model; generating one or more linear projections for the second base model based on the first base model, wherein the one or more linear projections align tensors generated by the second base model with tensors generated by the first base model; generating a projected base model based on the second base model and the one or more linear projections; and generating a second adapted machine learning model comprising the projected base model and the adapter.

[0138] Clause 2: A method according to Clause 1, wherein the first base model comprises a generative model and wherein the adapter comprises a low-rank adaptation (LoRA) adapter.

[0139] Clause 3: A method according to any of Clauses 1-2, wherein generating the one or more linear projections comprises generating, for each respective layer of a P+S Ref. No.: QUAL / 2405344PC02 26Client Ref. No.: 2405344B1WO plurality of layers in the second base model, a respective linear projection based on a respective corresponding layer in the first base model.

[0140] Clause 4: A method according to any of Clauses 1-3, wherein at least one ofthe one or more linear projections is defined as ^^�^^where: ^^�^^ is the atleast one linear projection, ^^^^^^^^is a set of weights of the second base model, and ^^^^^^^^is a set of weights of the first base model.

[0141] Clause 5: A method according to Clause 4, wherein the projected base modelis defined as ^^^^^^^^←^^^^ = ^^^^^^^^^^�^^, where ^^^^^^^^←^^^^ is the projected base model.

[0142] Clause 6: A method according to Clause 5, wherein the second adaptedmachine learning model is defined as ^^^ ∗^^^^^ = ^^^^ ∗^^^^←^^^^ + ^^^^^^^^^^^^ , where: ^^^^^^^^ is the secondadapted machine learning model, and ^^^^^^^^^^^^is the adapter.

[0143] Clause 7: A method according to any of Clauses 1-6, further comprising deploying the second adapted machine learning model.

[0144] Clause 8: A method according to any of Clauses 1-7, further comprising generating a model output based on processing a model input using the second adapted machine learning model.

[0145] Clause 9: A method according to any of Clauses 1-8, wherein generating the one or more linear projections, generating the projected base model, and generating the second adapted machine learning model are performed without processing data used to train the adapter.

[0146] Clause 10: A method according to any of Clauses 1-9, wherein the second base model corresponds to a modified version of the first base model.

[0147] Clause 11: A method, comprising: accessing a first adapted machine learning model comprising a first base model and an adapter trained for the first base model; generating one or more adapter components based on projecting the adapter to a range space and a null space of the first base model; accessing a second base model; generating a projected adapter based on projecting the one or more adapter components to a range space and a null space of the second base model; generating a second adapted machine learning model comprising the second base model and the projected adapter; and generating a machine learning model output using the second adapted machine learning model. P+S Ref. No.: QUAL / 2405344PC02 27Client Ref. No.: 2405344B1WO

[0148] Clause 12: A method according to Clause 11, wherein generating the one or more adapter components comprises applying singular value decomposition (SVD) to the first base model to generate a left singular matrix and a right singular matrix for the first base model.

[0149] Clause 13: A method according to Clause 12, wherein generating the one or more adapter components further comprises: decomposing the left singular matrix to generate a first parallel matrix corresponding to the range space of the first base model and a first normal matrix corresponding to the null space of the first base model; and decomposing the right singular matrix to generate a second parallel matrix corresponding to the range space of the first base model and a second normal matrix corresponding to the null space of the first base model.

[0150] Clause 14: A method according to Clause 13, wherein a first adapter component of the one or more adapter components are generated according to= ^^^^^^^^,‖^^^^^^^^^^^,^‖ ^^^^^^^^^^^^^^^^^^^^^^,^‖^^^^^^^^^,‖, wherein: ^^^^^^^^^^^^,‖is the first adapter component, ^^^^^^^^^^^^is the adapter, ^^^^^^^^,‖is the first parallel matrix,is a transpose of the first parallel matrix,is the second parallel matrix, and ^^^^^^^^^^,^‖^is a transpose of the second parallel matrix.

[0151] Clause 15: A method according to Clause 13, wherein a second adapter component of the one or more adapter components are generated according to ^^^^^^^^^^^^,⊥= ^^^^^^^^,⊥^^^^^^^^^^^,^⊥ ^^^^^^^^^^^^^^^^^^^^^^,^⊥^^^^^^^^^,⊥, wherein: ^^^^^^^^^^^^,⊥is the second adapter component, ^^^^^^^^,⊥is the first normal matrix,is a transpose of the first normal matrix,is the second normal matrix, andis a transpose of the second normal matrix.

[0152] Clause 16: A method according to any of Clauses 11-15, wherein generating the projected adapter comprises applying singular value decomposition (SVD) to the second base model to generate a left singular matrix and a right singular matrix for the second base model.

[0153] Clause 17: A method according to Clause 16, wherein generating the projected adapter further comprises: decomposing the left singular matrix to generate a first parallel matrix corresponding to the range space of the second base model and a first normal matrix corresponding to the null space of the second base model; and decomposing the right singular matrix to generate a second parallel matrix corresponding to the range space P+S Ref. No.: QUAL / 2405344PC02 28Client Ref. No.: 2405344B1WO of the second base model and a second normal matrix corresponding to the null space of the second base model.

[0154] Clause 18: A method according to Clause 17, wherein the projected adapter isgenerated accordingwherein:^^^^^^^^ is the projected adapter, ^^^^^^^^,‖is the second parallel matrix,is a transpose the second parallel matrix, ^^^^^^^^^^^^,‖is a parallel matrix of the adapter projected to the range space of the first base model, ^^^^ is the second parallel matrix,is a transpose of second parallel matrix, ^^^^ is the second normal matrix,is a transpose of the normal matrix, ^^^^^^^^^^^^,⊥is a normal matrix of the adapter projected to the null space of the first base model, ^^^^^^^^,⊥is the second normal matrix, and ^^^^^^^^^^,^⊥^is a transpose of the second normal matrix.

[0155] Clause 19: A processing system comprising: a memory comprising processor- executable instructions; and one or more processors coupled to the one or more memories and configured to execute the processor-executable instructions and cause the processing system to perform a method in accordance with any of Clauses 1-18.

[0156] Clause 20: A processing system comprising means for performing a method in accordance with any of Clauses 1-18.

[0157] Clause 21: A non-transitory computer-readable medium comprising computer-executable instructions that, when executed by one or more processors of a processing system, cause the processing system to perform a method in accordance with any of Clauses 1-18.

[0158] Clause 22: A computer program product embodied on a computer-readable storage medium comprising code for performing a method in accordance with any of Clauses 1-18. Additional Considerations

[0159] The preceding description is provided to enable any person skilled in the art to practice the various aspects described herein. The examples discussed herein are not limiting of the scope, applicability, or aspects set forth in the claims. Various modifications to these aspects will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other aspects. For example, changes may be made in the function and arrangement of elements discussed without departing P+S Ref. No.: QUAL / 2405344PC02 29Client Ref. No.: 2405344B1WO from the scope of the disclosure. Various examples may omit, substitute, or add various procedures or components as appropriate. For instance, the methods described may be performed in an order different from that described, and various steps may be added, omitted, or combined. Also, features described with respect to some examples may be combined in some other examples. For example, an apparatus may be implemented or a method may be practiced using any number of the aspects set forth herein. In addition, the scope of the disclosure is intended to cover such an apparatus or method that is practiced using other structure, functionality, or structure and functionality in addition to, or other than, the various aspects of the disclosure set forth herein. It should be understood that any aspect of the disclosure disclosed herein may be embodied by one or more elements of a claim.

[0160] As used herein, the word “exemplary” means “serving as an example, instance, or illustration.” Any aspect described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other aspects.

[0161] As used herein, a phrase referring to “at least one of” a list of items refers to any combination of those items, including single members. As an example, “at least one of: a, b, or c” is intended to cover a, b, c, a-b, a-c, b-c, and a-b-c, as well as any combination with multiples of the same element (e.g., a-a, a-a-a, a-a-b, a-a-c, a-b-b, a-c- c, b-b, b-b-b, b-b-c, c-c, and c-c-c or any other ordering of a, b, and c).

[0162] As used herein, the term “determining” encompasses a wide variety of actions. For example, “determining” may include calculating, computing, processing, deriving, investigating, looking up (e.g., looking up in a table, a database or another data structure), ascertaining, and the like. Also, “determining” may include receiving (e.g., receiving information), accessing (e.g., accessing data in a memory), and the like. Also, “determining” may include resolving, selecting, choosing, establishing, and the like.

[0163] The methods disclosed herein comprise one or more steps or actions for achieving the methods. The method steps and / or actions may be interchanged with one another without departing from the scope of the claims. In other words, unless a specific order of steps or actions is specified, the order and / or use of specific steps and / or actions may be modified without departing from the scope of the claims. Further, the various operations of methods described above may be performed by any suitable means capable of performing the corresponding functions. The means may include various hardware P+S Ref. No.: QUAL / 2405344PC02 30

Claims

1. Client Ref. No.: 2405344B1WO and / or software component(s) and / or module(s), including, but not limited to a circuit, an application specific integrated circuit (ASIC), or processor. Generally, where there are operations illustrated in figures, those operations may have corresponding counterpart means-plus-function components with similar numbering. [0164] The following claims are not intended to be limited to the aspects shown herein, but are to be accorded the full scope consistent with the language of the claims. Within a claim, reference to an element in the singular is not intended to mean “one and only one” unless specifically so stated, but rather “one or more.” Unless specifically stated otherwise, the term “some” refers to one or more. No claim element is to be construed under the provisions of 35 U.S.C. § 112(f) unless the element is expressly recited using the phrase “means for” or, in the case of a method claim, the element is recited using the phrase “step for.” All structural and functional equivalents to the elements of the various aspects described throughout this disclosure that are known or later come to be known to those of ordinary skill in the art are expressly incorporated herein by reference and are intended to be encompassed by the claims. Moreover, nothing disclosed herein is intended to be dedicated to the public regardless of whether such disclosure is explicitly recited in the claims. P+S Ref. No.: QUAL / 2405344PC02 31 Client Ref. No.: 2405344B1WO WHAT IS CLAIMED IS:

1. A processing system for machine learning comprising: one or more memories comprising processor-executable instructions; and one or more processors coupled to the one or more memories and configured to execute the processor-executable instructions and cause the processing system to: access a first adapted machine learning model comprising a first base model and an adapter trained for the first base model; generate one or more adapter components based on projecting the adapter to a range space and a null space of the first base model; access a second base model; generate a projected adapter based on projecting the one or more adapter components to a range space and a null space of the second base model; generate a second adapted machine learning model comprising the second base model and the projected adapter; and generate a machine learning model output using the second adapted machine learning model.

2. The processing system of claim 1, wherein, to generate the one or more adapter components, the one or more processors are configured to execute the processor- executable instructions and cause the processing system to apply singular value decomposition (SVD) to the first base model to generate a left singular matrix and a right singular matrix for the first base model.

3. The processing system of claim 2, wherein, to generate the one or more adapter components, the one or more processors are configured to execute the processor- executable instructions and further cause the processing system to: decompose the left singular matrix to generate a first parallel matrix corresponding to the range space of the first base model and a first normal matrix corresponding to the null space of the first base model; and decompose the right singular matrix to generate a second parallel matrix corresponding to the range space of the first base model and a second normal matrix corresponding to the null space of the first base model. P+S Ref. No.: QUAL / 2405344PC02 32 Client Ref. No.: 2405344B1WO 4. The processing system of claim 3, wherein a first adapter component of the oneor more adapter components is generated according to wherein: is the first adapter ^^^^^^^^^^^^is the adapter, ^^^^^^^^,‖is the first parallel matrix, ^^^^^^^^^^^,^‖ is a transpose of the first parallel matrix, ^^^^^^^^,‖is the second parallel matrix, and ^^^^^^^^^^,^‖^is a transpose of the second parallel matrix.

5. The processing system of claim 4, wherein a second adapter component of the one or more adapter components is generated according to ^^^^^^^^^^^^,⊥= ^^^^^^^^,⊥^^^^^^^^^^^,^⊥ ^^^^^^^^^^^^^^^^^^^^^^,^⊥^^^^^^^^^,⊥, wherein: is the second adapter ^^^^^^^^,⊥is the first normal matrix, ^^^^^^^^^^^,^⊥ is a transpose of the first normal matrix, ^^^^^^^^,⊥is the second normal matrix, and ^^^^^^^^^^,^⊥^is a transpose of the second normal matrix.

6. The processing system of claim 1, wherein, to generate the projected adapter, the one or more processors are configured to execute the processor-executable instructions and cause the processing system to apply singular value decomposition (SVD) to the second base model to generate a left singular matrix and a right singular matrix for the second base model.

7. The processing system of claim 6, wherein, to generate the projected adapter, the one or more processors are configured to execute the processor-executable instructions and further cause the processing system to: decompose the left singular matrix to generate a first parallel matrix corresponding to the range space of the second base model and a first normal matrix corresponding to the null space of the second base model; and P+S Ref. No.: QUAL / 2405344PC02 33 Client Ref. No.: 2405344B1WO decompose the right singular matrix to generate a second parallel matrix corresponding to the range space of the second base model and a second normal matrix corresponding to the null space of the second base model.

8. The processing system of claim 7, wherein the projected adapter is generatedaccording wherein:^^^^^^^^^^^^←^^^^is the projected adapter, ^^^^^^^^,‖is the second parallel matrix, ^^^^^^^^^^^,^‖ is a transpose of the second parallel matrix, ^^^^^^^^^^^^,‖is a parallel matrix of the adapter projected to the range space of the first base model, ^^^^^^^^,‖is the second parallel matrix, ^^^^^^^^^^,^‖^is a transpose of the second parallel matrix, ^^^^^^^^,⊥is the second normal matrix, is a transpose of the second normal matrix, ^^^^^^^^^^^^,⊥is a normal matrix of the adapter projected to the null space of the first base model, ^^^^^^^^,⊥is the second normal matrix, and is a transpose of the second normal matrix.

9. A processor-implemented method of machine learning, comprising: accessing a first adapted machine learning model comprising a first base model and an adapter trained for the first base model; generating one or more adapter components based on projecting the adapter to a range space and a null space of the first base model; accessing a second base model; generating a projected adapter based on projecting the one or more adapter components to a range space and a null space of the second base model; generating a second adapted machine learning model comprising the second base model and the projected adapter; and generating a machine learning model output using the second adapted machine learning model. P+S Ref. No.: QUAL / 2405344PC02 34 Client Ref. No.: 2405344B1WO 10. The processor-implemented method of claim 9, wherein generating the one or more adapter components comprises applying singular value decomposition (SVD) to the first base model to generate a left singular matrix and a right singular matrix for the first base model.

11. The processor-implemented method of claim 10, wherein generating the one or more adapter components further comprises: decomposing the left singular matrix to generate a first parallel matrix corresponding to the range space of the first base model and a first normal matrix corresponding to the null space of the first base model; and decomposing the right singular matrix to generate a second parallel matrix corresponding to the range space of the first base model and a second normal matrix corresponding to the null space of the first base model.

12. The processor-implemented method of claim 11, wherein a first adapter component of the one or more adapter components is generated according to = ^^^^^^^^,‖^^^^^^^^^^^,^‖ ^^^^^^^^^^^^^^^^^^^^^^,^‖^^^^^^^^^,‖, wherein: ^^^^^^^^^^^^,‖is the first adapter component, ^^^^^^^^^^^^is the adapter, ^^^^^^^^,‖is the first parallel matrix, ^^^^^^^^^^^,^‖ is a transpose of the first parallel matrix, ^^^^^^^^,‖is the second parallel matrix, and is a transpose of the second parallel matrix.

13. The processor-implemented method of claim 12, wherein a second adapter component of the one or more adapter components is generated according to ^^^^^^^^^^^^,⊥= wherein: adapter component, ^^^^^^^^,⊥is the first normal matrix, ^^^^^^^^^^^,^⊥ is a transpose of the first normal matrix, ^^^^^^^^,⊥is the second normal matrix, and is a transpose of the second normal matrix. P+S Ref. No.: QUAL / 2405344PC02 35 Client Ref. No.: 2405344B1WO 14. The processor-implemented method of claim 9, wherein generating the projected adapter comprises applying singular value decomposition (SVD) to the second base model to generate a left singular matrix and a right singular matrix for the second base model.

15. The processor-implemented method of claim 14, wherein generating the projected adapter further comprises: decomposing the left singular matrix to generate a first parallel matrix corresponding to the range space of the second base model and a first normal matrix corresponding to the null space of the second base model; and decomposing the right singular matrix to generate a second parallel matrix corresponding to the range space of the second base model and a second normal matrix corresponding to the null space of the second base model.

16. The processor-implemented method of claim 15, wherein the projected adapteris generated according wherein: ^^^^^^^^^^^^←^^^^is the projected adapter, ^^^^^^^^,‖is the second parallel matrix, ^^^^^^^^^^^,^‖ is a transpose of the second parallel matrix, is a parallel matrix of the adapter projected to the range space of the first base model, ^^^^^^^^,‖is the second parallel matrix, ^^^^^^^^^^,^‖^is a transpose of the second parallel matrix, ^^^^^^^^,⊥is the second normal matrix, is a transpose of the second normal matrix, is a normal matrix of the adapter projected to the null space of the first base model, ^^^^^^^^,⊥is the second normal matrix, and ^^^^^^^^^^,^⊥^is a transpose of the second normal matrix. P+S Ref. No.: QUAL / 2405344PC02 36 Client Ref. No.: 2405344B1WO 17. A processing system comprising: means for accessing a first adapted machine learning model comprising a first base model and an adapter trained for the first base model; means for generating one or more adapter components based on projecting the adapter to a range space and a null space of the first base model; means for accessing a second base model; means for generating a projected adapter based on projecting the one or more adapter components to a range space and a null space of the second base model; means for generating a second adapted machine learning model comprising the second base model and the projected adapter; and means for generating a machine learning model output using the second adapted machine learning model.

18. The processing system of claim 17, wherein the means for generating the one or more adapter components comprise: means for applying singular value decomposition (SVD) to the first base model to generate a left singular matrix and a right singular matrix for the first base model; means for decomposing the left singular matrix to generate a first parallel matrix corresponding to the range space of the first base model and a first normal matrix corresponding to the null space of the first base model; and means for decomposing the right singular matrix to generate a second parallel matrix corresponding to the range space of the first base model and a second normal matrix corresponding to the null space of the first base model.

19. The processing system of claim 17, wherein the means for generating the projected adapter comprises: means for applying singular value decomposition (SVD) to the second base model to generate a left singular matrix and a right singular matrix for the second base model; means for decomposing the left singular matrix to generate a first parallel matrix corresponding to the range space of the second base model and a first normal matrix corresponding to the null space of the second base model; and P+S Ref. No.: QUAL / 2405344PC02 37 Client Ref. No.: 2405344B1WO means for decomposing the right singular matrix to generate a second parallel matrix corresponding to the range space of the second base model and a second normal matrix corresponding to the null space of the second base model.

20. The processing system of claim 19, wherein the projected adapter is generatedaccording wherein:^^^^^^^^^^^^←^^^^is the projected adapter, ^^^^^^^^,‖is the second parallel matrix, ^^^^^^^^^^^,^‖ is a transpose of the second parallel matrix, ^^^^^^^^^^^^,‖is a parallel matrix of the adapter projected to the range space of the first base model, ^^^^^^^^,‖is the second parallel matrix, ^^^^^^^^^^,^‖^is a transpose of the second parallel matrix, ^^^^^^^^,⊥is the second normal matrix, is a transpose of the second normal matrix, ^^^^^^^^^^^^,⊥is a normal matrix of the adapter projected to the null space of the first base model, ^^^^^^^^,⊥is the second normal matrix, and ^^^^^^^^^^,^⊥^is a transpose of the second normal matrix. P+S Ref. No.: QUAL / 2405344PC02 38

Images