Method and apparatus for enabling efficient fine tuning of unstructured sparse low-precision large pre-trained base model

Abstract

An example apparatus includes an interface circuit, machine readable instructions, and at least one processor circuit, the at least one processor circuit is programmed by the machine readable instructions to sparse a base model of the base model to generate a sparse base model, apply a neural low rank adapter search to the sparse base model, and output a trimmed base model based on application of the neural low rank adapter search to the sparse base model.

Description

Background Technology

[0001] Base models (e.g., pre-trained large language models (LLMs)) are neural networks that perform AI-based tasks. These models utilize millions or billions of parameters, which may need to be fine-tuned for new datasets or downstream tasks, such as mathematical reasoning. LLMs include encoder-only models for classification tasks, decoder-only models for content generation tasks, and encoder-decoder models for content evaluation and content generation tasks such as translation and summarization. Attached Figure Description

[0002] Figure 1 illustrates the existing limitations of known methods for fine-tuning sparse quantization models and merging low-rank (LoRA) adapters.

[0003] Figure 2 This is a block diagram of an exemplary implementation of a model tuner circuit for fine-tuning a pre-trained LLM on a downstream task, constructed in accordance with the teachings of this disclosure.

[0004] Figure 3 This indicates that it can be implemented, instantiated, and / or accomplished by an exemplary programmable circuit. Figure 2 A flowchart of exemplary machine-readable instructions and / or exemplary operations for the exemplary model tuner circuit shown.

[0005] Figure 4 This indicates that it can be implemented, instantiated, and / or accomplished by an exemplary programmable circuit. Figure 2 The flowchart illustrates exemplary machine-readable instructions and / or operations of an exemplary model tuner circuit to perform sparsification and quantization of a basic model.

[0006] Figure 5 This indicates that it can be implemented, instantiated, and / or accomplished by an exemplary programmable circuit. Figure 2 The flowchart illustrates exemplary machine-readable instructions and / or exemplary operations of an exemplary model tuner circuit to recover model accuracy using Neural Low-Rank Adapter Search (NLS).

[0007] Figure 6 An exemplary pipeline configuration that can be instantiated to efficiently fine-tune large models is shown, comprising (1) a first pipeline for parametrically efficient fine-tuning (PEFT) of a sparse quantized model using a flexible adapter to obtain an unmerged model and adapter, (2) a second pipeline for parametrically efficient fine-tuning of a sparse model using sparse-aware (sparse PEFT) to allow subsequent merging of the model and adapter, and (3) a third pipeline for parametrically efficient fine-tuning of a sparse quantized model by merging quantization and sparse-aware adapters.

[0008] Figure 7 An example of a known low-rank (LoRA) adapter is shown in contrast to an elastic adapter associated with the neural low-rank search (NLS) disclosed herein.

[0009] Figure 8 This demonstrates a model-based fine-tuning of pre-trained LLM using sparsity optimization. Figure 7 The NLS adapter shown provides an exemplary overview of restoring the accuracy of the basic model and identifying the sparsed fine-tuned architecture based on sub-adaptor search.

[0010] Figure 9 Efficient fine-tuning of sparse parameters using a binary mask obtained from sparsified weights (Sparse PEFT) is demonstrated.

[0011] Figure 10 This paper demonstrates an exemplary reduction in the parameters required to fine-tune an LLM while achieving higher accuracy using the methods disclosed herein, compared to known sparse fine-tuning methods.

[0012] Figure 11A The results of fine-tuning of the exemplary first model are shown for comparison with known fine-tuning methods disclosed herein, such as fine-tuning of a sparse quantization model (SQFT), SQFT combined with sparse PEFT, and SQFT combined with sparse PEFT including quantization and sparse perception adapter merging (QA-sparse PEFT).

[0013] Figure 11B This demonstrates the use of the same method when evaluating fine-tuning of the SQFT disclosed in this paper, including SQFT with sparse PEFT and SQFT with quantization-aware sparse PEFT. Figure 7 The ablation experiment results show the results of ablation experiments where the elastic adapter associated with neural low-rank search (NLS) was fine-tuned compared to a known low-rank (LoRA) adapter.

[0014] Figure 12 Exemplary cost analyses of different pipelines associated with model fine-tuning are shown, including assessments of model storage, fine-tuning time, and accuracy.

[0015] Figure 13 This is a block diagram of an exemplary processing platform including programmable circuitry configured to execute, instantiate, and / or perform... Figures 3-5 The exemplary machine-readable instructions shown and / or execution Figures 3-5 The exemplary operation shown is used to implement Figure 2 The model tuner circuit shown.

[0016] Figure 14 yes Figure 13 A block diagram illustrating an exemplary implementation of the processor circuit.

[0017] Figure 15 yes Figure 13 A block diagram of another exemplary embodiment of the programmable circuit shown.

[0018] Figure 16 It is used to store software, instructions, and / or firmware (e.g., corresponding to...). Figures 3-5 The diagram illustrates an exemplary software / firmware / instruction distribution platform (e.g., one or more servers) that distributes exemplary machine-readable instructions to client devices associated with end users and / or consumers (e.g., for licensing, selling, and / or using), retailers (e.g., for selling, reselling, licensing, and / or sublicensing), and / or original equipment manufacturers (OEMs) (e.g., for inclusion in products to be distributed to other end users such as retailers and / or direct-purchase customers).

[0019] Generally, the same reference numerals are used throughout the accompanying drawings and written descriptions to refer to the same or similar parts. Drawings are not necessarily to scale. Detailed Implementation

[0020] Large Language Models (LLMs) comprise large transformer-based models that demonstrate capabilities across a variety of tasks, from language understanding to text generation. State-of-the-art LLMs often involve billions of parameters for understanding natural language. For example, LLMs are pre-trained with billions or trillions of tokens to improve their robustness in zero-shot evaluations used to measure performance. However, LLMs need to be adaptable to new tasks and / or datasets to improve performance. Fine-tuning is required to increase accuracy, but this effort demands resource-intensive methods. Known methods for inducing sparsity and fine-tuning in large base models include post-training pruning methods (e.g., Sparse GPT) for compressing LLMs and fine-tuning sparse models. These known methods require considerable computational resources and runtime that can be utilized with the resulting sparse weight matrices.

[0021] Furthermore, Parametric Efficient Fine-Tuning (PEFT) techniques have been developed to reduce the number of parameters used to fine-tune a model. For example, Low-Rank (LoRA) adapters are a popular PEFT-based technique for adjusting the selection of linear layers given a pre-trained model. LoRA techniques adjust the selection of linear layers, each with a weight matrix. Additionally, LoRA uses a low-rank adapter to expand the linear projection of the layers, allowing the original model weights to remain frozen while only the additional adapters are fine-tuned. While LoRA-based techniques have proven effective in model adaptation, combining LoRA with model compression techniques (e.g., sparsity, quantization, etc.) introduces challenges associated with incorporating adapters into a single compressed fine-tuned model. For example, merging dense adapters with sparse model weights results in a loss of sparsity in the updated base model. Similarly, adapter merging is discouraged when model weights are used with a different numerical precision compared to the adapter's numerical precision.

[0022] The methods and apparatus disclosed herein achieve efficient fine-tuning of large pre-trained base models with sparse quantization. In the examples disclosed herein, fine-tuning of LLMs is improved while maintaining a low count of trainable parameters and resource requirements and mitigating memory bandwidth challenges. In the examples disclosed herein, the resulting fine-tuned model accelerates inference by leveraging existing runtimes that support unstructured sparsity. Thus, the methods and apparatus disclosed herein introduce efficient fine-tuning superior to conventional PEFT methods. In the examples disclosed herein, LLM-based fine-tuning is performed using (1) model sparsification and / or quantization, (2) accuracy recovery using Neural Low-Rank Adapter Search (NLS), and (3) identification of high-performance sparse fine-tuning architectures. For example, NLS is performed on a resilient adapter attached to the base LLM. In the examples disclosed herein, performance is improved while using parameters that are orders of magnitude smaller than known sparse fine-tuning methods. The methods and apparatus disclosed herein can be further extended to other converter-based architectures (e.g., not limited to LLM) that use parameterized linear transformations and / or other architectures that are not converter-based (e.g., selective state-space models (SSM)).

[0023] Furthermore, the methods and apparatus disclosed herein merge sparse weights and dense adapters while avoiding the loss of sparsity when merging sparse and / or dense matrices. Thus, machine learning-based inference can be accelerated by using fewer parameters than other known commercial methods for sparse fine-tuning. For example, the methods and apparatus disclosed herein outperform low-rank (LoRA) adapters in both dense and sparse configurations. In the examples disclosed herein, computational challenges associated with memory bandwidth are addressed by applying weight compression for sparse fine-tuning. The sparsity obtained in LLMs can be used to improve execution time. For example, significant speedups can be achieved by employing model sparsity with almost no (e.g., minimal) compression loss, demonstrating that the resulting sparse models can alleviate existing memory bandwidth challenges. Compared to known state-of-the-art techniques for LLM fine-tuning, the methods and apparatus disclosed herein use fewer parameters to fine-tune pre-trained LLMs on downstream tasks, require less time to sparsify a given LLM, require less memory during fine-tuning, and can be applied to a wider range of hardware.

[0024] Figure 1 illustrates the existing limitations of 100 known methods for fine-tuning sparsely quantized models and incorporating low-rank (LoRA) adapters. Large pre-trained base models or cutting-edge models are essential components of artificial intelligence applications (e.g., language understanding, code generation, etc.). These models are trained using thousands of hardware accelerators (e.g., graphics processing units (GPUs)) to achieve excellent zero-shot performance (e.g., performance associated with task completion without receiving any training instances). However, such models often require further adaptation to improve their performance on new tasks and / or new data. While low-rank (LoRA) adapters can be used for model adaptation, combining such adapters with model compression techniques (e.g., sparsification, quantization, etc.) prevents these adapters from being incorporated into a single compressed fine-tuned model. For example, incorporating dense adapters results in a loss of sparsity in the base model and / or adapter merging is not possible due to differing numerical precision. As illustrated in the example of Figure 1, full parameter fine-tuning is computationally expensive, and LoRA, used for parametrically efficient fine-tuning (PEFT) of sparse or quantized models, cannot be easily incorporated with compressed weights due to the loss from previously induced sparsity and / or varying numerical accuracy. For example, Figure 1 includes a base model 102, a sparse base model 104, a quantized base model 106, and a LoRA-based adapter 108. A sparse base model 104 can be obtained from the initial base model 102 using sparsification 110. While this version of the exemplary sparse full parameter fine-tuning 112 allows for the generation of trainable sparse models, such models are computationally expensive.

[0025] Sparsity encompasses a range of techniques for compressing and optimizing neural networks. By removing and / or reducing the importance of less critical connections and information within a given model, sparsity typically results in a decrease in accuracy while producing smaller model size, faster inference, and / or reduced energy consumption. For example, compared to sparse full-parameter fine-tuning 112, using LoRA for PEFT on a sparse model 114 results in a lack of merging between the sparse base model 104 and the LoRA-based adapter 108 due to the loss of sparsity, while using LoRA for PEFT on a quantized model 120 results in a lack of merging between the quantized base model 106 and the adapter 108 due to the different numerical precision. In the examples disclosed herein, quantization reduces the precision of weights and activations in the neural network (e.g., from 32-bit floating-point numbers to 8-bit integers). For example, quantization reduces model size and / or memory usage, resulting in faster inference.

[0026] In the examples disclosed herein, adapter 108 is used to reduce the number of parameters in the fine-tuning model. For example, as in combination with Figure 7 In more detail, for a given pre-trained model, adapter 108 can be used to adjust the selection of linear layers, each with a weight matrix. As shown in the example in Figure 1, combining LoRA for PEFT with sparse model 114 and LoRA for PEFT with quantized model 120, low-rank adapters can be used to keep the weights of the original model frozen, while only the inserted adapters are fine-tuned.

[0027] Figure 2 This is a block diagram 200 illustrating an exemplary implementation of a model tuner circuit 205 constructed according to the teachings of this disclosure to fine-tune a pre-trained LLM on a downstream task. Figure 2 The model tuner circuit 205 shown can be instantiated by a programmable circuit, such as a central processing unit (CPU) that executes the first instruction (e.g., creating an instance of, generating, materializing, implementing, etc., over any time period). Additionally or alternatively, Figure 2 The model tuner circuit 205 shown can be instantiated (e.g., created, generated, materialized, implemented, etc.) by (i) an application-specific integrated circuit (ASIC) and / or (ii) a field-programmable gate array (FPGA), which is constructed and / or configured to perform the operation corresponding to the first instruction in response to the execution of the second instruction. It should be understood that Figure 2 Some or all of the circuits shown can therefore be instantiated simultaneously or at different times. For example, Figure 2Some or all of the circuits shown can be instantiated in one or more threads that execute in parallel and / or serially on hardware. Furthermore, in some instances, Figure 2 Some or all of the circuits shown can be implemented by microprocessor circuits that execute instructions and / or FPGA circuits that perform operations to implement one or more virtual machines and / or containers.

[0028] exist Figure 2 In the example shown, the model tuner circuit 205 includes an exemplary sparsity trigger circuit 210, an exemplary quantization applyer circuit 215, an exemplary neural low-rank search (NLS) trainer circuit 220, an exemplary weight combiner circuit 225, an exemplary model generator circuit 230, and an exemplary data storage 235. Figure 2 In the example shown, the sparsity trigger circuit 210, the quantization applicator circuit 215, the NLS trainer circuit 220, the weight merger circuit 225, the model generator circuit 230, and the data memory 235 communicate with the exemplary bus 240.

[0029] The sparsification trigger circuit 210 performs sparsification on the base model (e.g., base model 102 shown in Figure 1) to obtain a sparse model. For example, the resulting sparse model can serve as the base model for applying Neural Low-Rank Search (NLS) using the NLS trainer circuit 220. In the example disclosed herein, the sparsification trigger circuit 210 assigns an arbitrary scoring function (Ψ) to the solution proposed for a given matrix (W), where And W = (w i，j For entry w i，j , 1≤i≤m, 1≤j≤n. This function obtains w. i，j The importance of the scoring function (Ψ) relative to other weights in W. In the examples disclosed herein, the scoring function (Ψ) can be expressed in any given number of ways. In some instances, Ψ(W) = |W|·||X||2, where X represents the sampled feature input activation values. However, the end-to-end model fine-tuning solution disclosed herein can use any other scoring function. For example, the sparsity trigger circuit 210 uses the score from the scoring function Ψ and the desired sparsity level(s) to obtain a given matrix (W) with a sparsity pattern. p This sparsity pattern corresponds to Must satisfy |S{W p The condition}|≤|S{W}|. For example, combining... Figure 6As described, the sparsification trigger circuit 210 performs sparsification of the base model 102, independent of whether the sparse base model is subsequently used in a first pipeline for fine-tuning the sparse quantization model, a second pipeline for efficient parameter fine-tuning of the sparse model, and / or a third pipeline for fine-tuning the sparse quantization model using quantization and sparsity-aware adapter merging. As described in conjunction with the examples disclosed herein, large pre-trained models (LPMs) can tolerate higher levels of sparsity compared to smaller, converter-based models of previous generations.

[0030] The quantization applicator circuit 215 performs the quantization of the sparsity weights. For example, the quantization applicator circuit 215 quantizes the sparsity weights (W... p Quantization is reduced to a lower precision to alleviate memory bandwidth challenges. Thus, once the sparsity trigger circuit 210 has induced sparsity in the pre-trained weights, the numerical precision of these weights can be optionally reduced using the quantization applyer circuit 215. Figure 6 In the example shown, the quantization applicator circuit 215 performs quantization of the sparse basic model in a first pipeline for fine-tuning the sparse quantization model and in a third pipeline for fine-tuning the sparse quantization model using a merging of quantization and sparsity-aware adapters, but not in a second pipeline for efficient parameter fine-tuning of the sparse model. In some instances, the quantization applicator circuit 215 applies layer-by-layer one-time quantization. For example, the quantization applicator circuit 215 uses quantization from... Induced sparsity often reduces the accuracy of the model, thus requiring fine-tuning to improve performance (e.g., using NLS trainer circuit 220).

[0031] In some instances, the quantization applicator circuit 215 performs quantization in conjunction with a quantization and sparsity-aware parameter efficient fine-tuning (QA-Sparse PEFT) pipeline, such as when combined with Figure 6 As shown, this QA-Sparse PEFT represents an extension of Sparse PEFT (Sparse Parameter Efficient Fine-Tuning) for sparse quantization models. In the examples disclosed herein, QA-Sparse PEFT integrates quantization awareness into Sparse PEFT. In most common quantization schemes, the zeros and scaling factors of the target quantization tensor can be determined during the quantization process. Within the framework of QA-Sparse PEFT, with an adapter (e.g., Figure 6 The adapter 606 shown shares sparse quantization weights. The zeros and scaling factors. For example, the quantization applyer circuit 215 uses shared fixed zeros and scaling factors to quantize the adapter, thereby enabling quantization-aware fine-tuning. For example, given sparsed pre-trained weights (W... p ), sparsed adapter weights (L) p (e.g., obtained from sparse PEFT) and from (Wp If the quantization zero z and scaling factor s are obtained, then the quantization applicator circuit 215 performs quantization processing according to Equation 1.

[0032]

[0033] In the example of Equation 1, Corresponding to sparse quantization (e.g., merged) weights, while Q p =2 n-1 -1, where n represents the bit width of the quantization value. Conversely, the quantization applicator circuit 215 can perform dequantization according to Equation 2.

[0034]

[0035] In the example of Equation 2, the zero point z and the scaling factor s are applied to approximate... In some instances, the NLS trainer circuit 220 can perform NLS training simultaneously, thus allowing the merging of adapters once the optimal configuration is identified.

[0036] NLS trainer circuit 220 trains a set of adapters with variable configurations. For example, given sparse quantization weights... The NLS trainer circuit 220 then recovers any accuracy degradation caused by compression and fine-tunes the weights for a specific downstream task. Unlike NLS, traditional LoRA adapters require assigning values ​​to multiple hyperparameters, including the associated rank(r) and / or subsets of modules fitted into these adapters. In the example disclosed herein, the NLS trainer circuit 220 extends the weight-sharing technique to facilitate the discovery of optimal adapter configurations from the space of flexible adapter configurations. For example, instead of having a fixed value for the rank(r), the NLS trainer circuit 220 applies weights that satisfy r←c based on the activation of the corresponding sub-adaptors. i Flexible configuration of conditions C = [c1,...,c n In some instances, the NLS trainer circuit 220 can also instantiate arbitrary configurations from a feasible configuration design space. For example, techniques from weight-sharing supernetworks can be used to fine-tune the adapter so that W p ←Sparseness and quantization (W), where and Thus, L δ1 and L δ2 This refers to a resilient adapter that can be configured differently without requiring more storage than its static counterparts (e.g., L1 and L2).

[0037] The weighted combiner circuit 225 identifies a binary mask from the sparse weights and applies this binary mask to the adapter, thereby allowing subsequent combining of the adapter without sparsity loss. (As in combination) Figure 9In more detail, the sparse PEFT applies a binary mask (M) derived from the initial sparsification of matrix W. For example, the weight combiner circuit 225 applies the binary mask to sparsify the adapter-based matrix (e.g., denoted as BA) into adapter weights (L). p ), where L is used p =(BA)⊙M determines the adapter weights and activates them during the fine-tuning process for sparsity awareness. In the example disclosed herein, the weight combiner circuit 225 combines the sparsified weights (W) without sacrificing the sparsity induced earlier in the compressed pipeline. p ) and adapter weights (L p (For example, W) p ←W p +L p ). (e.g., combined) Figure 9 The sparse PEFT described shows improved accuracy compared to fine-tuning using a dense adapter.

[0038] Model generator circuit 230 generates the final model obtained by merging adapters. While sparsity and quantization techniques demonstrate significant effectiveness in accelerating model serviceability and inference across various hardware platforms and kernels, adding adapter models introduces computational overhead during inference due to the non-mergeability of adapter models, which is crucial for efficient parameter fine-tuning using sparsed or quantized models. As described earlier in conjunction with weight merger circuit 225, adapters can be merged into sparse and / or quantized models, reducing adapter redundancy and computational overhead, thus achieving a simpler and more efficient inference process. Figure 6 In more detail, the model generator circuit 230 outputs a final merged model obtained by (1) a sparse basic model and an adapter or by (2) merging a basic model and an adapter after sparsification and quantization.

[0039] Data memory 235 can be used to store any information associated with sparsity trigger circuit 210, quantization applicator circuit 215, NLS trainer circuit 220, weight combiner circuit 225 and / or model generator circuit 230. Figure 2 The data storage device 235 shown in the example can be implemented by any memory, storage device, and / or disk used for storing data, such as flash memory, magnetic media, optical media, etc. Furthermore, the data stored in the data storage device 235 can be in any data format, such as binary data, comma-separated data, tab-separated data, Structured Query Language (SQL) structures, image data, etc.

[0040] In some instances, the device includes means for sparsifying the basic model. For example, the means for sparsifying the basic model can be implemented by a sparsification trigger circuit 210. In some instances, the sparsification trigger circuit 210 can be implemented by, for example... Figure 13 The exemplary programmable circuit 1312 shown is an example of a programmable circuit instantiation. For example, the sparse trigger circuit 210 can be instantiated by... Figure 14 The exemplary microprocessor 1400 shown is instantiated, and the microprocessor 1400 performs, for example, actions by... Figure 4 Those machine-executable instructions implemented in at least blocks 405, 410, and / or 415. In some instances, the sparse flip-flop circuit 210 can be instantiated by hardware logic circuitry, which can be... Figure 15 The sparse trigger circuit 210 is implemented by an ASIC, XPU, or FPGA circuit 1500 configured to perform operations corresponding to machine-readable instructions. Additionally or alternatively, the sparse trigger circuit 210 can be instantiated by any other combination of hardware, software, and / or firmware. For example, the sparse trigger circuit 210 can be implemented by at least one or more hardware circuits (e.g., processor circuitry, discrete circuitry and / or integrated circuits, analog circuitry and / or digital circuitry, FPGA, ASIC, XPU, comparator, operational amplifier, logic circuitry, etc.) configured to perform some or all of the operations corresponding to machine-readable instructions without executing software or firmware, but other configurations are equally suitable.

[0041] In some instances, the device includes means for performing quantization. For example, the means for performing quantization may be implemented by quantization applicator circuitry 215. In some instances, quantization applicator circuitry 215 may be, for example... Figure 13 The exemplary programmable circuit 1312 shown is an example of a programmable circuit instantiated from such a circuit. For example, the quantization applicator circuit 215 can be derived from... Figure 14 The exemplary microprocessor 1400 shown is instantiated, and the microprocessor 1400 performs, for example, actions by... Figure 4 Those machine-executable instructions implemented in at least blocks 420, 425, and / or 430. In some instances, the quantization applicator circuit 215 can be instantiated by hardware logic circuitry, which can be... Figure 15The quantization applicator circuit 215 is implemented by an ASIC, XPU, or FPGA circuit 1500 configured to perform operations corresponding to machine-readable instructions. Additionally or alternatively, the quantization applicator circuit 215 may be instantiated by any other combination of hardware, software, and / or firmware. For example, the quantization applicator circuit 215 may be implemented by at least one or more hardware circuits (e.g., processor circuitry, discrete circuitry and / or integrated circuits, analog circuitry and / or digital circuitry, FPGA, ASIC, XPU, comparator, operational amplifier, logic circuitry, etc.) configured to perform some or all of the operations corresponding to machine-readable instructions without executing software or firmware, but other configurations are equally suitable.

[0042] In some instances, the device includes means for applying neural low-rank adapter search. For example, the means for applying neural low-rank adapter search can be implemented by neural low-rank search (NLS) trainer circuitry 220. In some instances, NLS trainer circuitry 220 can be implemented by, for example... Figure 13 The exemplary programmable circuit 1312 shown is an example of a programmable circuit instantiation. For example, the NLS trainer circuit 220 can be instantiated by... Figure 14 The exemplary microprocessor 1400 shown is instantiated, and the microprocessor 1400 performs, for example, actions by... Figure 3 At least those machine-executable instructions implemented in block 315. In some instances, the NLS trainer circuit 220 can be instantiated by hardware logic circuitry, which can be... Figure 15 The NLS trainer circuit 220 is implemented by an ASIC, XPU, or FPGA circuit 1500 configured to perform operations corresponding to machine-readable instructions. Additionally or alternatively, the NLS trainer circuit 220 can be instantiated by any other combination of hardware, software, and / or firmware. For example, the NLS trainer circuit 220 can be implemented by at least one or more hardware circuits (e.g., processor circuitry, discrete circuitry and / or integrated circuits, analog circuitry and / or digital circuitry, FPGA, ASIC, XPU, comparator, operational amplifier, logic circuitry, etc.) configured to perform some or all of the operations corresponding to machine-readable instructions without executing software or firmware, but other configurations are equally suitable.

[0043] In some instances, the device includes means for merging weights. For example, the means for merging weights can be implemented by a weight merger circuit 225. In some instances, the weight merger circuit 225 can be, for example... Figure 13 The exemplary programmable circuit 1312 shown is an example of a programmable circuit instantiated from such a circuit. For example, the weighted combiner circuit 225 can be derived from... Figure 14 The exemplary microprocessor 1400 shown is instantiated, and the microprocessor 1400 performs, for example, actions by... Figure 5 At least those machine-executable instructions implemented in block 520. In some instances, the weighted combiner circuit 225 can be instantiated by hardware logic circuitry, which can be... Figure 15 The weight combiner circuit 225 is implemented by an ASIC, XPU, or FPGA circuit 1500 configured to perform operations corresponding to machine-readable instructions. Additionally or alternatively, the weight combiner circuit 225 can be instantiated by any other combination of hardware, software, and / or firmware. For example, the weight combiner circuit 225 can be implemented by at least one or more hardware circuits (e.g., processor circuitry, discrete and / or integrated analog and / or digital circuitry, FPGA, ASIC, XPU, comparator, operational amplifier, logic circuitry, etc.) configured to perform some or all of the operations corresponding to machine-readable instructions without executing software or firmware, but other configurations are equally suitable.

[0044] In some instances, the device includes means for outputting a fine-tuned base model. For example, the means for outputting the fine-tuned base model can be implemented by model generator circuitry 230. In some instances, model generator circuitry 230 can be implemented by, for example... Figure 13 The exemplary programmable circuit 1312 shown is an example of a programmable circuit instantiation. For example, the model generator circuit 230 can be instantiated by... Figure 14 The exemplary microprocessor 1400 shown is instantiated, and the microprocessor 1400 performs, for example, actions by... Figure 3 At least those machine-executable instructions implemented in block 320. In some instances, the model generator circuit 230 can be instantiated by hardware logic circuitry, which can be... Figure 15 The model generator circuit 230 is implemented by an ASIC, XPU, or FPGA circuit 1500 configured to perform operations corresponding to machine-readable instructions. Additionally or alternatively, the model generator circuit 230 may be instantiated by any other combination of hardware, software, and / or firmware. For example, the model generator circuit 230 may be implemented by at least one or more hardware circuits (e.g., processor circuitry, discrete circuitry and / or integrated circuits, analog circuitry and / or digital circuitry, FPGA, ASIC, XPU, comparator, operational amplifier, logic circuitry, etc.) configured to perform some or all of the operations corresponding to machine-readable instructions without executing software or firmware, but other configurations are equally suitable.

[0045] Although Figure 2 The diagram illustrates an exemplary manner of implementing the model tuner circuit 205, but it can be combined, divided, rearranged, omitted, eliminated, and / or implemented in any other manner. Figure 2 One or more of the elements, processes, and / or devices shown. Furthermore, Figure 2 The exemplary sparsity trigger circuit 210, exemplary quantization applyer circuit 215, exemplary NLS trainer circuit 220, exemplary weight combiner circuit 225, exemplary model generator circuit 230, and / or more generally, model tuner circuit 205 shown can be implemented by hardware, software, firmware, and / or any combination of hardware, software, and / or firmware. Therefore, for example, Figure 2 Any of the exemplary sparsity trigger circuit 210, exemplary quantization applyer circuit 215, exemplary NLS trainer circuit 220, exemplary weight combiner circuit 225, exemplary model generator circuit 230, and / or more generally model tuner circuit 205 shown can be implemented by programmable circuitry, processor circuitry, analog circuitry, digital circuitry, logic circuitry, programmable processors, programmable microcontrollers, graphics processing units (GPUs), digital signal processors (DSPs, ASICs), programmable logic devices (PLDs), and / or field-programmable logic devices (FPLDs) such as FPGAs, combined with machine-readable instructions (e.g., firmware or software). Furthermore, Figure 2 The tuner circuit 205 shown may include additions or replacements. Figure 2 One or more of the elements, processes and / or devices shown, and / or may include any part or all of the elements, processes and devices shown.

[0046] exist Figures 3-5 The diagram illustrates how this can be implemented and / or instantiated by programmable circuitry. Figure 2 The exemplary machine-readable instructions of the tuner circuit 205 shown herein and / or represent instructions that can be executed by programmable circuitry to implement and / or instantiate it. Figure 2 The flowchart illustrates an exemplary operation of the tuner circuit 205 shown. Machine-readable instructions may be used, for example, in conjunction with the following... Figure 13 The programmable circuitry, such as programmable circuitry 1312 shown in the exemplary processor platform 1300 discussed herein, executes one or more executable programs or a portion thereof, and / or may be to be combined with the following Figure 14 and / or Figure 15 The exemplary programmable circuits discussed (e.g., FPGAs) perform one or more functions or a portion of functions. In some instances, machine-readable instructions enable the implementation and / or execution of operations, tasks, etc., in a real-world manner. As used herein, “automation” means without human intervention.

[0047] The program can be implemented in instructions (e.g., software and / or firmware) stored on one or more non-transitory computer-readable storage media and / or non-transitory machine-readable storage media, such as cache memory, magnetic storage devices or disks (e.g., floppy disks, hard disk drives (HDDs) etc.), optical storage devices or optical discs (e.g., Blu-ray discs, CDs, DVDs etc.), redundant arrays of independent disks (RAID), registers, ROM, solid-state drives (SSDs), SSD memory, non-volatile memory (e.g., electrically erasable programmable read-only memory (EEPROM), flash memory, etc.), volatile memory (e.g., random access memory (RAM) of any type, etc.) and / or any other storage device or disk. Instructions on non-transitory computer-readable media and / or non-transitory machine-readable media can be programmed to and / or executed by programmable circuitry in one or more hardware devices, but the entire program and / or a portion thereof can alternatively be executed and / or instantiated and / or implemented in dedicated hardware by one or more hardware devices other than programmable circuitry. Machine-readable instructions can be distributed across multiple hardware devices and / or executed by two or more hardware devices (e.g., server and client hardware devices). For example, client hardware devices can be implemented by endpoint client hardware devices (e.g., hardware devices associated with human users and / or machine users) or intermediate client hardware device gateways (e.g., radio access networks (RAN)) that facilitate communication between the server and endpoint client hardware devices. Similarly, non-transitory computer-readable storage media can include one or more media. Furthermore, although references... Figures 3-5 The flowchart shown illustrates an exemplary procedure; however, an alternative implementation can be used. Figure 2Many other methods are possible with the exemplary model tuner circuit 205 shown. For example, the execution order of the blocks in the flowchart can be changed, and / or a portion of the described blocks can be altered, eliminated, or combined. Additionally or alternatively, any part or all of the blocks in the flowchart can be implemented by one or more hardware circuits (e.g., processor circuitry, discrete circuitry and / or integrated circuits, analog circuitry and / or digital circuitry, FPGA, ASIC, comparator, operational amplifier, logic circuitry, etc.) configured to perform the corresponding operations without executing software or firmware. Programmable circuitry can be distributed across different network locations and / or local to one or more hardware devices (e.g., a single-core processor (e.g., a single-core CPU), a multi-core processor (e.g., a multi-core CPU, XPU, etc.)). For example, programmable circuitry can be a CPU and / or FPGA in the same package (e.g., the same integrated circuit (IC) package or two or more separate housings), one or more processors in a single machine, multiple processors distributed across multiple servers spanning a server rack, multiple processors distributed across one or more server racks, and / or any combination thereof.

[0048] The machine-readable instructions described herein can be stored in one or more of the following formats: compressed format, encrypted format, segmented format, compiled format, executable format, packaged format, etc. Machine-readable instructions as described herein can be stored as data (e.g., computer-readable data, machine-readable data, one or more bits (e.g., one or more computer-readable bits, one or more machine-readable bits, etc.), bit streams (e.g., computer-readable bit streams, machine-readable bit streams, etc.)) or data structures (e.g., as part of instructions, code, code representations, etc.), which can be used to create, manufacture, and / or produce machine-executable instructions. For example, machine-readable instructions can be segmented and stored on one or more storage devices, disks, and / or computing devices (e.g., servers) located at the same or different locations (e.g., in the cloud, at an edge device, etc.) within a network or set of networks. Machine-readable instructions may require one or more of the following operations: installation, adjustment, adaptation, updating, combination, supplementation, configuration, decryption, decompression, unpacking, distribution, reallocation, compilation, etc., so that the machine-readable instructions can be directly read, interpreted, and / or executed by computing devices and / or other machines. For example, machine-readable instructions may be stored in multiple parts that are individually compressed, encrypted, and / or stored on separate computing devices, wherein these parts, when decrypted, decompressed, and / or combined, form a set of computer-executable instructions and / or machine-executable instructions that together can form one or more functions and / or operations of a program, such as those described herein.

[0049] In another instance, machine-readable instructions can be stored in a state where they are readable by programmable circuitry, but libraries (e.g., dynamic link libraries (DLLs)), software development kits (SDKs), application programming interfaces (APIs), etc., need to be added to execute the machine-readable instructions on a specific computing device or other device. In yet another instance, machine-readable instructions (e.g., stored settings, data inputs, recorded network addresses, etc.) may need to be configured before they can be executed in whole or in part. Therefore, as used herein, machine-readable, computer-readable, and / or machine-readable media can include instructions and / or programs, regardless of their specific format or state.

[0050] The machine-readable instructions described in this article can be represented by any past, present, or future instruction language, scripting language, programming language, etc. For example, machine-readable instructions can be represented using any of the following languages: C, C++, Java, C#, Perl, Python, JavaScript, Hypertext Markup Language (HTML), Structured Query Language (SQL), Swift, etc.

[0051] As mentioned above, Figures 3-5The exemplary operations shown can be implemented using executable instructions (e.g., computer-readable instructions and / or machine-readable instructions) stored on one or more non-transitory computer-readable media and / or non-transitory machine-readable media. As used herein, the terms non-transitory computer-readable media, non-transitory computer-readable storage media, non-transitory machine-readable media and / or non-transitory machine-readable storage media are explicitly defined as including any type of computer-readable storage device and / or storage disk, and excluding propagation signals and transmission media. Examples of such non-transitory computer-readable media, non-transitory computer-readable storage media, non-transitory machine-readable media and / or non-transitory machine-readable storage media include optical storage devices, magnetic storage devices, HDDs, flash memory, read-only memory (ROM), CDs, DVDs, caches, any type of RAM, registers, and / or any other storage device or storage disk where information is stored for any duration (e.g., for extended periods, permanently, for transient situations, for temporary buffering, and / or for cached information). As used herein, the terms "non-transitory computer-readable storage device" and "non-transitory machine-readable storage device" are defined as any physical (mechanical, magnetic, and / or electrical) hardware that stores information for a period of time, but excludes propagating signals and transmission media. Examples of non-transitory computer-readable storage devices and / or non-transitory machine-readable storage devices include any type of random access memory, any type of read-only memory, solid-state memory, flash memory, optical discs, magnetic disks, disk drives, and / or redundant arrays of independent disks (RAID) systems. As used herein, the term "device" means a physical structure, such as mechanical and / or electrical equipment, hardware, and / or circuitry, that can be configured or not configured by computer-readable instructions, machine-readable instructions, etc., and / or manufactured to execute computer-readable instructions, machine-readable instructions, etc.

[0052] "Comprising" and "including" (and all forms and tenses thereof) are used herein as open-ended terms. Therefore, whenever a claim uses any form of "comprising" or "including" (e.g., including, having, etc.) as a preamble or within the scope of any type of claim, it should be understood that other elements, terms, etc., may exist that do not fall outside the scope of the corresponding claim or the scope of the statement. As used herein, the phrase "at least" is also open-ended when used, for example, as a transitional term in the preamble of a claim, in the same way that the terms "comprising" and "including" are open-ended. The term "and / or," when used in the form of, for example, A, B, and / or C, refers to any combination or subset of A, B, and C, such as (1) A alone, (2) B alone, (3) C alone, (4) A and B, (5) A and C, (6) B and C, or (7) A and B and C. As used herein in the context of describing structures, components, articles, objects, and / or things, the phrase “at least one of A and B” is intended to refer to an implementation that includes (1) at least one A, (2) at least one B, or (3) at least one A and at least one B. Similarly, as used herein in the context of describing structures, components, articles, objects, and / or things, the phrase “at least one of A or B” is intended to refer to an implementation that includes (1) at least one A, (2) at least one B, or (3) at least one A and at least one B. As used herein in the context of describing the completion or execution of processes, instructions, actions, activities, and / or steps, the phrase “at least one of A and B” is intended to refer to an implementation that includes (1) at least one A, (2) at least one B, or (3) at least one A and at least one B. Similarly, as used herein in the context of describing the completion or execution of processes, instructions, actions, activities and / or steps, the phrase “at least one of A or B” is intended to refer to an implementation including (1) at least one A, (2) at least one B, or (3) at least one A and at least one B.

[0053] As used herein, singular references (e.g., "a," "an," "first," "second," etc.) do not exclude plurals. As used herein, the term "a" or "an" refers to one or more of that object. The terms "a" (or "an"), "one or more," and "at least one" are used interchangeably herein. Furthermore, although listed separately, multiple means, elements, or actions may be implemented by, for example, the same entity or object. Additionally, although individual features may be included in different embodiments or claims, these features may be combined, and inclusion in different embodiments or claims does not imply that the combination of features is impractical and / or advantageous.

[0054] Figure 3This means that it can be executed, instantiated, and / or completed by a programmable circuit. Figure 2 A flowchart of exemplary machine-readable instructions and / or exemplary operations 300 for the exemplary model tuner circuit shown. Figure 3 The machine-readable instructions and / or operations 300 shown begin at block 305. The sparsity trigger circuit 210 identifies the basic model at block 305 and continues to perform basic model sparsification at block 310, as combined with... Figure 4 More detailed description. Once model sparsification and / or model quantization are complete, the NLS trainer circuit 220 uses a neural low-rank adapter search at box 315 to recover model accuracy, as combined with Figure 5 More detailed description. After model accuracy is restored, the model generator circuit 230 outputs a sparsed and / or sparsed and quantized fine-tuned architecture of the initial input base model at block 320. For example, as combined with Figure 6 In more detail, the model generator circuit 230 outputs a final merged model obtained by (1) a sparse basic model and a corresponding adapter or by (2) merging a basic model that has been sparsified and quantized and a corresponding adapter.

[0055] Figure 4 This is a flowchart illustrating exemplary machine-readable instructions and / or exemplary operations 310, which can be executed, instantiated, and / or performed by programmable circuitry. Figure 2 The exemplary model tuner circuit 205 shown performs basic model sparsification and quantization. Figure 3 The machine-readable instructions and / or operations 310 shown begin at block 405. The sparsity trigger circuit 210 receives pre-trained weights at block 405 and continues at block 410 to assign an arbitrary scoring function (Ψ) based on, for example, sampled feature input activation values. At block 415, the sparsity trigger circuit 210 determines the sparsity weights (W) based on the arbitrary scoring function and the desired sparsity level. p Subsequently, the quantization applicator circuit 215 determines at block 420 whether to perform quantization. For example, if the quantization applicator circuit 215 determines that quantization is not required (e.g., when performing efficient fine-tuning of the parameters of the sparse model), the sparsity trigger circuit 210 outputs the sparse base model at block 422. In some instances, triggered quantization is part of fine-tuning the sparse quantization model, or as part of fine-tuning the sparse quantization model with quantization and a sparse-aware adapter combined. In such instances, the quantization applicator circuit 215 applies layer-by-layer one-time quantization at block 425. The quantization applicator circuit 215 then quantizes the sparsity weights to a lower precision at block 430 to alleviate memory bandwidth challenges, resulting in the sparsed and quantized base model at block 435.

[0056] Figure 5 This is a flowchart illustrating exemplary machine-readable instructions and / or exemplary operations 315, which can be executed, instantiated, and / or performed by programmable circuitry. Figure 2 The exemplary model tuner circuit 205 shown is used to restore model accuracy using Neural Low-Rank Adapter Search (NLS). Figure 3 The machine-readable instructions and / or operations 315 shown begin at block 505, the NLS trainer circuit 220 retrieves the sparsified and / or sparsified and quantized base model at block 505 and retrieves the search space configuration of the elastic adapter at block 510. In some instances, such as combining Figure 2 As described, the NLS trainer circuit 220 uses techniques from weight-sharing supernetworks to fine-tune the adapter. After NLS training, if the weight merger circuit 225 determines that sparse weights and adapter weights need to be merged, the weight merger circuit 225 generates a binary mask derived from the initial sparsification of the weight matrix (W). For example, as described above... Figure 2 In more detail, the weighted combiner circuit 225 applies a binary mask to the adapter, thereby allowing subsequent combining of the adapter without sparsity loss. Figure 5 In the example shown, the NLS trainer circuit 220 retrieves a binary mask generated from the sparse weights in the base model at box 515. Figure 5 In the example shown, quantization applyer circuit 215 retrieves the zeros and scaling factors associated with the quantized base model at box 520. In some instances, model generator circuit 230 triggers fine-tuning of the model at box 525. For example, as part of the fine-tuning, weight combiner circuit 225 determines at box 530 whether to trigger the merging of base model and adapter weights to avoid sparsity loss after fine-tuning. If weight combiner circuit 225 triggers the merging of base model and adapter weights, control proceeds to box 535, where weight combiner circuit 225 aligns the sparse model weights with the sparse adapter weights as part of parameter-efficient fine-tuning (Sparse PEFT) of the sparse model. In some instances, sparsification trigger circuit 210 sparsifies the adapter matrix during fine-tuning at box 540 to achieve sparsity awareness.

[0057] Once the base model and adapter weights are merged and / or if the weight merger circuit 225 determines that merging of the base model and adapter weights is not triggered, control proceeds to block 545. For example, at block 545, the weight merger circuit 225 determines whether merging of the base model and adapter weights with different numerical precisions should be triggered. If the weight merger circuit 225 determines to merge the base model and adapter weights with different numerical precisions, the quantization applicator circuit 215 performs quantization-aware sparse PEFT (QA-Sparse PEFT) at block 550 to align the low-precision model weights with the floating-point precision adapter weights. For example, the quantization applicator circuit 215 performs quantization using sparsed pre-trained weights and sparsed adapter weights as part of fine-tuning the sparsely quantized model using quantization and sparse-aware adapter merging. Figure 5 In the example shown, model generator circuit 230 determines at block 555 whether to perform fine-tuning using frozen model weights and elastic adapters. If model generator circuit 230 determines at block 560 that model fine-tuning is complete, it outputs the fine-tuned model at block 565. If model generator circuit 230 determines based on the generated output that additional model fine-tuning is needed, control returns to block 530 until the model fine-tuning process is complete.

[0058] Figure 6 An exemplary pipeline configuration 600, which can be instantiated for efficient fine-tuning of large models, is shown. Pipeline configuration 600 includes (1) a first pipeline 610 for parametrically efficient fine-tuning of a sparse quantized model using a resilient adapter to obtain an unmerged model and adapter; (2) a second pipeline 620 for parametrically efficient fine-tuning of a sparse model using sparsity-aware (Sparse PEFT) to allow subsequent merging of the model and adapter; and (3) a third pipeline 630 for parametrically efficient fine-tuning of the sparse quantized model using quantization and sparsity-aware adapter merging. While LLM fine-tuning is widely used to align base and / or frontier models to specific tasks or datasets, fine-tuning these large models remains a resource-intensive task. Figure 6 In the example shown, fine-tuning of the sparse quantized model (SQFT) results in a fine-tuned large pre-trained model (LPM) obtained in an efficient multi-stage method comprising (1) sparsification with optional reduction of numerical precision (e.g., quantization), (2) neural low-rank adapter search (NLS), and (3) efficient fine-tuning of sparse parameters with optional quantization awareness (QA-Sparse PEFT) (Sparse PEFT). Figure 6In the examples shown, alternatives to LPM compression and model adaptation pipelines 610, 620, and 630 can be selected based on whether quantization and / or sparse-aware adapter merging are applied. For example, the choice between sparsity level and whether quantization is applied can depend on the specific deployment scenario (e.g., task requirements, resource constraints, etc.) which includes the expected trade-offs between model performance, inference speed, and / or memory efficiency.

[0059] exist Figure 6 In the example shown, the first pipeline 610 for fine-tuning the sparse quantization model (SQFT) involves... Figure 2 The sparsity trigger circuit 210, which performs sparsification of the model, identifies the base model 102. The quantization applyer circuit 215 further quantizes the sparse model, resulting in a sparsified and quantized base model 604. The base model 604 is then processed using the NLS trainer circuit 220 to obtain an unmerged adapter. However, the sparsified and quantized base model 604 and adapter 606 remain unmerged in the first pipeline 610, while the combination of the second pipeline 620 and the third pipeline 630 shows that the adapter 606 is efficiently merged into the sparse and / or quantized model without sparsity loss. For example, the second pipeline 620 (Sparse PEFT), which includes components from the first pipeline 610 (SQFT), addresses several limitations in existing efficient parameter fine-tuning methods for sparsely quantized models, including reduced fine-tuning costs, efficient merging of the adapter into the sparse model without sparsity loss, and efficient merging of components operating with different numerical precisions. In the second pipeline 620 instance, after the sparse base model 602 is generated and the NLS trainer circuit 220 performs training to obtain an unmerged adapter, the weight merger circuit 225 identifies a binary mask from the sparse weights and applies the binary mask to the adapter, thereby allowing subsequent merging of the adapter 606 without sparsity loss, resulting in the output of a sparsely fine-tuned architecture 608 of the original base model.

[0060] While sparse PEFT effectively preserves the sparsity of the model, this pipeline presents additional challenges when merging with quantized models, primarily due to the need for the adapter and pre-trained weights to have the same numerical accuracy. Figure 6 In the example shown, the third pipeline 630 represents an extension of the sparse PEFT used for the sparse quantization model. For example, the quantization applicator circuit 215 identifies the sparsification weights 622 and the corresponding zeros and scaling factors 624 of the sparse quantization weights shared with the adapter 606. Thus, the quantization applicator circuit 215 quantizes the adapter 606 using the shared fixed zeros and scaling factors, thereby achieving quantization-aware fine-tuning of the sparsely and quantized fine-tuned architecture 628 that produces the original basic model.

[0061] Figure 7 An exemplary comparison 700 of a known low-rank (LoRA) adapter and a resilient adapter associated with the neural low-rank search (NLS) disclosed herein is shown. For example, the LoRA adapter is associated with an efficient fine-tuning technique for adjusting the selection of linear layers, each with a weight matrix (e.g., ...). In some instances, LoRA adapters are used (e.g., first adapter A and second adapter B, where...). and The linear projection of the extended layer (e.g., extending the projection from H = W0X to H = W0X + BAX) allows the weights (W0) of the original model to remain frozen, with only the inserted adapters being fine-tuned. Although Figure 7 The original LoRA adapter shown includes the use of a first rank 705, where the rank is maintained at the same level (e.g., r = 32). However, the NLS-based LoRA adapter is a flexible adapter with variable ranks, allowing the rank to be adjusted as needed (e.g., a second rank 710, a third rank 715). For example, as combined with... Figure 2 In more detail, the NLS trainer circuit 220 does not have a fixed value for rank (r), but instead applies a flexible configuration based on the activation of the corresponding sub-adapters. The flexible LoRA adapter using variable rank evaluation described herein outperforms conventional LoRA adapters.

[0062] Figure 8 This demonstrates a model-based fine-tuning of pre-trained LLM using sparsity optimization. Figure 7 The NLS adapter shown provides an exemplary overview 800 of the accuracy of restoring the basic model and identifying the sparsed fine-tuned architecture based on sub-adaptor search. Figure 8 In the example shown, for a given matrix (W) 805, Figure 2 The sparsity trigger circuit 210 shown identifies the sparsity weights (W). p 810. The sparsity trigger circuit 210 receives the original weight matrix 815 of the dense model weights and applies an arbitrary scoring function (Ψ) 820 to select the most critical weights, thereby obtaining a sparse output matrix 825. The quantization applyer circuit 215 performs the quantization of the sparse weights, while the NLS trainer circuit 220 combines the pre-trained weights 830 of the sparse model with a set of elastic adapters 835 trained with variable configurations. Figure 7 As described, the NLS trainer circuit 220 applies a flexible configuration of the adapters determined based on the activation of the corresponding sub-adapters 840, such that the rank can be changed to facilitate the identification of the optimal adapter configuration from the space of flexible adapter configurations.

[0063] Figure 9This illustrates efficient fine-tuning of sparse parameters (Sparse PEFT) using a binary mask obtained from sparsified weights. For example, as combined with... Figure 2 As described, the weighted combiner circuit 225 identifies a binary mask from the sparse weights and applies that binary mask to the adapter, thereby allowing subsequent combining of the adapter without loss of sparsity. Figure 9 In the example shown, the sparsity trigger circuit 210 receives the matrix (W) 905 and identifies the sparsity weights (W). p )910. The weight combiner circuit 225 performs adapter weight identification 920 by applying a binary mask (M) 925 to sparsify the adapter matrix (e.g., denoted as BA 930) into adapter weights (L p )915, enabling the use of L p =(BA)⊙M Identify adapter weights 915. Weight merger circuit 225 merges sparse weights (W) without sacrificing the sparsity induced earlier in the compressed pipeline. p )910 and adapter weights (L p )915.

[0064] Figure 10 An exemplary reduction of 1000 parameters required for fine-tuning an LLM is shown while achieving higher accuracy using the method disclosed herein, compared to known sparse fine-tuning methods. Figure 10 In the examples shown, the fine-tuning methods and devices disclosed herein (e.g., the proposed solution 1002) are compared with known methods for fine-tuning models (e.g., sparse fine-tuning 1005 from NeuralMagic). For example, as in Figure 10 The examples shown validate the SQFT pipeline (e.g., in three experimental settings including elementary math 8K (GSM8K)). Figure 6 The first pipeline 610 in the model is used. The proposed solution 1002 is compared with sparse fine-tuning 1005 using measurements of GSM8K accuracy (%) 1010, trainable parameter ratio 1015, and sparsity (%) 1020. Figure 10 In the example shown, compared to sparse fine-tuning 1005, the proposed solution 1002 achieves higher accuracy while requiring approximately 83.1 times fewer parameters to perform fine-tuning.

[0065] Figure 11AExemplary results 1100 are shown for evaluating fine-tuning of an exemplary first model 1102 (e.g., LLAMA-3-8B) using known fine-tuning methods compared to those disclosed herein (e.g., fine-tuning of a sparse quantization model (SQFT), SQFT combined with sparse PEFT, and SQFT combined with sparse PEFT including quantization and sparse perceptual adapter merging (QA-sparse PEFT)). In the examples disclosed herein, SQFT is evaluated on multiple large pre-trained models and datasets that reach state-of-the-art levels. For example, SQFT is evaluated using the Llama-3-8B model and the Phi-3-Mini-4K-Instruct model. In the examples disclosed herein, SQFT is validated on three experimental settings: 1) Elementary Math 8K (GSM8K), 2) Math reasoning with instruction tuning, including three math reasoning datasets (e.g., GSM8K, Math Word Problems, and Simple Variations of Arithmetic Math Word Problems (SVAMP)), and 3) Commonsense Reasoning datasets (e.g., Boolean Problems (BoolQ), Physical Interaction: Question Answering (PIQA), Large-Scale Winograd Pattern Challenge (WinoGrande), AI2 Reasoning Challenge (Arc-e, Arc-c), and Open Book Question Answering (OBQA)). In the examples disclosed herein, the use of the disclosed pipeline (e.g., Figure 6 The analysis compares the results obtained by using the first pipeline 610, the second pipeline 620, and / or the third pipeline 630 in this paper, and comparing this pipeline with the results obtained by comparing it with a common low-rank (LoRA) adapter, the Shears method (e.g., a parameter-efficient fine-tuning method for sparse models), and GPTQ+LoRA, where all methods are run in the same environment with the same configuration. In the examples disclosed herein, SQFT also uses an implementation of the Wanda algorithm as the default method for sparsity and uses GPTQ from Huggingface 3 for quantization of the LPM and adapter.

[0066] exist Figure 11A In the example shown, the Llama-3-8B model is used to evaluate pipelines (e.g., SQFT and SQFT+QA-SparsePEFT), assessing accuracy after inducing 50% sparsity in dense mode without fine-tuning the GSM8K dataset. Figure 11AIn the examples, the results include induced sparsity level 1104, pipeline method selection 1106, mergeability results 1108, final accuracy results 1110, and GSM8K test accuracy percentage results 1112. For example, SQFT recovers the model's accuracy from 12.5% ​​to 52.5% without quantization, while allowing adapters to be merged (using sparse PEFT) and further incorporating quantization into the pipeline without sacrificing sparsity, resulting in a slight decrease in accuracy to 50.2% when tuning is enabled to merge adapters (using QA-sparse PEFT). Furthermore, SQFT with sparse PEFT and QA-sparse PEFT exhibits performance comparable to its corresponding non-mergeable methods. These results demonstrate that SQFT with sparse PEFT (e.g., QA-sparse PEFT) effectively addresses the limitations of merging problems encountered when fine-tuning adapters into sparse models (or sparse quantized models) without a reduction in accuracy. Furthermore, comparisons between LoRA and SQFT with sparse PEFT (or Shears method) without adapter merging, and between GPTQ+LoRA and SQFT with QA-sparse PEFT, highlight the superior performance of NLS (e.g., performed using elastic rank evaluation) compared to LoRA (e.g., performed using fixed rank).

[0067] Figure 11B This demonstrates the use of the same method when evaluating fine-tuning of the SQFT disclosed in this paper, including SQFT with sparse PEFT and SQFT with quantization-aware sparse PEFT. Figure 7 The exemplary ablation study results shown are 1160, demonstrating the results of a fine-tuning of a known low-rank adapter (LoRA) compared to an elastic adapter associated with neural low-rank search (NLS). Figure 11B In the example shown, the results for model 1102 include induced sparsity level 1104 for a specific dataset, pipeline method selection 1106, mergeability results 1108, final accuracy results 1110, fine-tuning method selection 1165, and test accuracy percentage results 1112. Figure 11B In the examples shown, NLS demonstrates significantly better accuracy performance across all pipelines and different sparsity levels of SQFT compared to LoRA. For example, ablation studies using 30%, 50%, and 70% sparsity highlight the benefits of a resilient adapter (e.g., applied using NLS) that enhances the performance of the SQFT pipeline disclosed herein.

[0068] Figure 12 An exemplary cost analysis 1200 for different pipelines associated with model fine-tuning is shown, which includes assessments of model storage, fine-tuning time, and accuracy. Figure 12In the example shown, identifiers 1205 represent different pipelines, where ID=1 corresponds to LoRA / Shears, ID=2 to SQFT, ID=3 to SQFT+Sparse PEFT, and ID=4 to SQFT+QA-Sparse PEFT. Different versions of the SQFT pipeline incur different costs, allowing users to choose based on their fine-tuning budget. Based on whether the adapter can be merged (e.g., mergeable indicator 1210), the final accuracy of the base model and adapter (e.g., final accuracy 1215), and the cost of each configuration, in Figure 12 The examples shown detail the characteristics of each pipeline configuration. Figure 12 In the example shown, model storage 1220, fine-tuning time 1225, fine-tuning memory 1230, inference acceleration 1235, inference memory 1240, and accuracy 1245 are compared for all pipelines. Regarding model storage, inference acceleration, and memory, it can be assumed that merging is better than not merging due to the overhead from the unmerged adapter, and quantization mode is better than full-precision mode. Regarding fine-tuning time, the merging method is slightly slower than the non-merging method due to additional mask and adapter computations. Based on the results, SQFT with sparse PEFT is the optimal choice for full-precision mode without sacrificing accuracy. If memory usage during fine-tuning is the priority of quantization mode, then plain SQFT is the optimal choice because the use of quantized models makes the overhead associated with different precision adapters minimal. Otherwise, SQFT with QA-sparse PEFT produces a more efficient model compared to other fine-tuning pipelines.

[0069] Figure 13 It is constructed to execute and / or instantiate Figures 3-5 The exemplary machine-readable instructions and / or exemplary operations shown are for implementation Figure 2 The block diagram shown is of an exemplary programmable circuit platform 1300 for an exemplary model tuner circuit 205. The programmable circuit platform 1300 can be, for example, a server, personal computer, workstation, self-learning machine (e.g., neural network), or mobile device (e.g., cellular phone, smartphone, such as iPad). TM Tablet computers, personal digital assistants (PDAs), internet devices, DVD players, CD players, digital video recorders, Blu-ray players, game consoles, personal video recorders, set-top boxes, headphones (e.g., augmented reality (AR) headphones, virtual reality (VR) headphones, etc.) or other wearable devices, or any other type of computing device and / or electronic device.

[0070] The programmable circuit platform 1300 in the illustrated example includes programmable circuitry 1312. Programmable circuitry 1312 in the illustrated example is hardware. For example, programmable circuitry 1312 can be implemented by one or more integrated circuits, logic circuits, FPGA microprocessors, CPUs, GPUs, DSPs, and / or microcontrollers from any desired family or manufacturer. Programmable circuitry 1312 can be implemented by one or more semiconductor-based (e.g., silicon-based) devices. In this example, processor circuitry 1312 implements sparsity trigger circuitry 210, quantization applyer circuitry 215, NLS trainer circuitry 220, weight combiner circuitry 225, and / or model generator circuitry 230.

[0071] The programmable circuit 1312 in the illustrated example includes local memory 1313 (e.g., cache, registers, etc.). The programmable circuit 1312 in the illustrated example communicates via bus 1318 with main memory, which includes volatile memory 1314 and non-volatile memory 1316. The volatile memory 1314 may be synchronous dynamic random access memory (SDRAM), dynamic random access memory (DRAM), etc. Dynamic Random Access Memory And / or any other type of RAM device. The non-volatile memory 1316 can be implemented by flash memory and / or any other desired type of memory device. Access to the main memories 1314, 1316 in the illustrated example is controlled by a memory controller 1317. In some instances, the memory controller 1317 can be implemented by one or more integrated circuits, logic circuits, microcontrollers, or any other type of circuit from any desired family or manufacturer to manage the flow of data into and out of the main memories 1314, 1316.

[0072] The programmable circuit platform 1300 in the illustrated example also includes interface circuitry 1320. Interface circuitry 1320 can be implemented by hardware conforming to any type of interface standard, such as an Ethernet interface, a Universal Serial Bus (USB) interface, etc. Interfaces include Near Field Communication (NFC) interfaces, Peripheral Component Interconnect (PCI) interfaces, and / or Peripheral Component Interconnect Fast (PCIe) interfaces.

[0073] In the illustrated example, one or more input devices 1322 are connected to interface circuitry 1320. Input devices 1322 allow users (e.g., human users, machine users, etc.) to input data and / or commands into programmable circuitry 1312. Input devices 1322 can be implemented using, for example, audio sensors, microphones, cameras (still or video), keyboards, buttons, mice, touchscreens, trackpads, trackballs, etc. (center) point devices and / or voice recognition systems.

[0074] One or more output devices 1324 are also connected to the interface circuitry 1320 in the illustrated example. The output devices 1324 may be implemented as, for example, display devices (e.g., light-emitting diode (LED), organic light-emitting diode (OLED), liquid crystal display (LCD), cathode ray tube (CRT) display, in-plane switching (IPS) display, touchscreen, etc.), haptic output devices, printers, and / or speakers. Therefore, the interface circuitry 1320 in the illustrated example typically includes a graphics driver card, a graphics driver chip, and / or graphics processor circuitry such as a GPU.

[0075] The interface circuit 1320 in the illustrated example also includes communication devices such as transmitters, receivers, transceivers, modems, residential gateways, wireless access points, and / or network interfaces to facilitate data exchange with external machines (e.g., any type of computing device) via network 1326. Communication can be achieved via, for example, Ethernet connections, digital subscriber line (DSL) connections, telephone line connections, coaxial cable systems, satellite systems, line-of-sight wireless systems, cellular telephone systems, optical connections, etc.

[0076] The programmable circuit platform 1300 in the illustrated example also includes one or more mass storage devices 1328 for storing software and / or data. Examples of such mass storage devices 1328 include magnetic storage devices (e.g., floppy disks, drives, HDDs, etc.), optical storage devices (e.g., Blu-ray discs, CDs, DVDs, etc.), RAID systems, and / or solid-state storage disks or storage devices such as flash memory devices and / or SSDs.

[0077] It can be by Figures 3-5 The machine-executable instructions 1332 implemented by the machine-readable instructions shown may be stored in a mass storage device 1328, a volatile memory 1314, a non-volatile memory 1316, and / or stored on at least one non-transitory computer-readable storage medium, such as a removable CD or DVD.

[0078] Figure 14 yes Figure 13 A block diagram illustrating an exemplary implementation of the programmable circuit 1312. In this example, Figure 13 The programmable circuit 1312 shown is implemented by a microprocessor 1400. For example, the microprocessor 1400 may be a general-purpose microprocessor (e.g., a general-purpose microprocessor circuit). The microprocessor 1400 executes... Figures 3-5 The flowchart shown contains some or all of the machine-readable instructions to effectively instantiate... Figure 2 The circuitry shown is a logic circuit designed to perform operations corresponding to machine-readable instructions. In some such instances, Figure 2The circuitry shown is instantiated from the instruction-coupled hardware circuitry of microprocessor 1400. For example, microprocessor 1400 can implement multi-core hardware circuitry such as a CPU, DSP, GPU, XPU, etc. Although microprocessor 1400 may include any number of exemplary cores 1402 (e.g., one core), the microprocessor 1400 in this example is a multi-core semiconductor device comprising N cores. The cores 1402 of microprocessor 1400 can operate independently or collaboratively to execute machine-readable instructions. For example, machine code corresponding to firmware, embedded software programs, or software programs can be executed by one of the cores 1402, or can be executed simultaneously or at different times by multiple cores 1402. In some instances, the machine code corresponding to firmware, embedded software programs, or software programs is split into threads and executed in parallel by two or more cores 1402. The software program can be coupled with... Figures 3-5 The flowchart shown corresponds to some or all of the machine-readable instructions and / or operations.

[0079] Core 1402 can communicate via an exemplary first bus 1404. In some instances, the first bus 1404 can implement a communication bus to enable communication associated with one of the cores 1402. For example, the first bus 1404 can implement at least one of an Inter-Integrated Circuit (I2C) bus, a Serial Peripheral Interface (SPI) bus, a PCI bus, or a PCIe bus. Additionally or alternatively, the first bus 1404 can implement any other type of computing bus or electrical bus. Core 1402 can obtain data, instructions, and / or signals from one or more external devices via exemplary interface circuitry 1406. Core 1402 can output data, instructions, and / or signals to one or more external devices via interface circuitry 1406. Although the core 1402 in this example includes an exemplary local memory 1420 (e.g., a Level 1 (L1) cache that can be split into an L1 data cache and an L1 instruction cache), the microprocessor 1400 also includes an exemplary shared memory 1410 (e.g., a Level 2 (L2) cache) that can be shared by the cores for high-speed access to data and / or instructions. Data and / or instructions can be transferred (e.g., shared) by writing to and / or reading from the shared memory 1410. The local memory 1420 and shared memory 1410 in each core 1402 can be multi-level cache memory and main memory (e.g., Figure 13 This is part of the storage device hierarchy of the main memories (1314, 1316). Typically, higher-level memories in the hierarchy exhibit shorter access times and have smaller storage capacities compared to lower-level memories. Variations within the cache hierarchy are managed by cache coherence policies (e.g., reconciliation).

[0080] Each core 1402 may be referred to as a CPU, DSP, GPU, etc., or may be any other type of hardware circuitry. Each core 1402 includes control unit circuitry 1414, arithmetic logic (AL) circuitry (sometimes referred to as ALU) 1416, multiple registers 1418, an L1 cache 1420, and an exemplary second bus 1422. Other structures may also be present. For example, each core 1402 may include vector unit circuitry, single instruction multiple data (SIMD) unit circuitry, load / store unit (LSU) circuitry, branch / jump unit circuitry, floating-point unit (FPU) circuitry, etc. Control unit circuitry 1414 includes semiconductor-based circuitry configured to control (e.g., coordinate) the movement of data within the corresponding core 1402. AL circuitry 1416 includes semiconductor-based circuitry configured to perform one or more mathematical and / or logical operations on the data within the corresponding core 1402. In some instances, AL circuitry 1416 performs integer-based operations. In other instances, AL circuitry 1416 also performs floating-point operations. In other instances, AL circuit 1416 may include a first AL circuit that performs integer-based operations and a second AL circuit that performs floating-point operations. In some instances, AL circuit 1416 may be referred to as an arithmetic logic unit (ALU).

[0081] Register 1418 is a semiconductor-based structure used to store data and / or instructions that are the result of one or more operations performed, for example, by the AL circuit 1416 of the corresponding core 1402. For example, register 1418 may include vector registers, SIMD registers, general-purpose registers, flag registers, segment registers, machine-specific registers, instruction pointer registers, control registers, debug registers, memory management registers, machine check registers, etc. Register 1418 may be arranged in a manner such as... Figure 14 The storage bank shown. Alternatively, register 1418 can be organized in any other arrangement, format, or structure, including being distributed throughout the core 1402 to reduce access time. The second bus 1422 can be implemented by at least one of an I2C bus, an SPI bus, a PCI bus, or a PCIe bus.

[0082] Each core 1402 and / or more generally, the microprocessor 1400 may include additional and / or alternative structures to the structures shown and described above. For example, there may be one or more clock circuits, one or more power supplies, one or more power gating mechanisms, one or more cache home agents (CHAs), one or more convergence / common grid termination structures (CMS), one or more shifters (e.g., barrel shifters), and / or other circuitry. The microprocessor 1400 is a semiconductor device fabricated to include a plurality of transistors interconnected in one or more integrated circuits (ICs) housed in one or more packages to implement the structures described above.

[0083] Microprocessor 1400 may include one or more accelerators (e.g., acceleration circuitry, hardware accelerators, etc.) and / or cooperate with one or more accelerators. In some instances, accelerators are implemented by logic circuitry to perform certain tasks faster and / or more efficiently than a general-purpose processor can. Examples of accelerators include, for example, ASICs and FPGAs discussed herein. GPUs, DSPs, and / or other programmable devices may also be accelerators. Accelerators may be onboard with microprocessor 1400, in the same chip package as microprocessor 1400, and / or in one or more packages separate from microprocessor 1400.

[0084] Figure 15 yes Figure 13 A block diagram of another exemplary embodiment of the programmable circuit shown. In this example, the programmable circuit 1312 is implemented by an FPGA circuit 1500. For example, the FPGA circuit 1500 may be implemented by an FPGA. For example, the FPGA circuit 1500 may be used to perform operations that can be otherwise controlled by... Figure 14 The exemplary microprocessor 1400 shown executes the corresponding machine-readable instructions. However, once configured, the FPGA circuitry 1500 instantiates the operations and / or functions corresponding to the machine-readable instructions in hardware, and thus can generally execute the operations / functions faster than a general-purpose microprocessor executing the corresponding software.

[0085] More specifically, as described above Figure 14 The microprocessor 1400 shown is a programmable microprocessor that can execute commands generated by... Figures 3-5 The flowchart shown represents some or all of the general-purpose devices in machine-readable instructions (but the interconnections and logic circuitry of these general-purpose devices are fixed once manufactured). Figure 15 The FPGA circuit 1500 in the example shown includes interconnect and logic circuitry that can be configured, structured, programmed, and / or interconnected in different ways after manufacturing to, for example, connect with... Figures 3-5The flowchart shown represents the instantiation of some or all of the machine-readable instructions corresponding to the operations / functions. Specifically, the FPGA 1500 can be considered as an array of logic gates, interconnects, and switches. Switches can be programmed to change the way logic gates are interconnected, effectively forming one or more dedicated logic circuits (unless or until the FPGA circuit 1500 is reprogrammed). The configured logic circuits enable logic gates to cooperate in different ways to perform different operations on data received from the input circuits. Those operations can be combined with... Figures 3-5 The flowchart shown corresponds to some or all of the instructions (e.g., software and / or firmware). Thus, the FPGA circuit 1500 can be configured and / or constructed to effectively connect with... Figures 3-5 In the flowchart shown, some or all of the machine-readable instructions corresponding to the operations / functions are instantiated as dedicated logic circuits. These dedicated logic circuits execute the operations / functions corresponding to those software instructions in a dedicated manner similar to that of an ASIC. Therefore, the FPGA circuit 1500 can execute operations / functions faster than a general-purpose microprocessor. Figures 3-5 Some or all of the machine-readable instructions in the code correspond to the operations / functions.

[0086] exist Figure 15 In the example shown, the FPGA circuit 1500 is configured and / or constructed to be programmed (and / or reprogrammed once or multiple times) in response to a binary file. In some instances, the binary file can be compiled and / or generated based on instructions in a hardware description language (HDL) such as Lucid, VHSIC (Very High Speed ​​Integrated Circuit) Hardware Description Language (VHDL), or Verilog. For example, a user (e.g., a human user, a machine user, etc.) can write code or programs corresponding to one or more operations / functions in the HDL; the code / program can be translated into a low-level language as needed; and the code / program (e.g., code / program in a low-level language) can be converted (e.g., by a compiler, software application, etc.) into a binary file. In some instances, Figure 15 The FPGA circuit 1500 shown can access and / or load binary files to enable... Figure 15 The FPGA circuit 1500 shown is configured and / or constructed to perform one or more operations / functions. For example, a binary file may consist of a bit stream (e.g., one or more computer-readable bits, one or more machine-readable bits, etc.), data (e.g., computer-readable data, machine-readable data, etc.), and / or may be generated by... Figure 15 The illustrated FPGA circuit 1500 accesses machine-readable instructions to enable... Figure 15 The FPGA circuit 1500 or a portion thereof shown is configured and / or structured.

[0087] In some instances, binary files are compiled, generated, translated, and / or otherwise output from a unified software platform used for programming FPGAs. For example, the unified software platform can translate first instructions (e.g., code or program) corresponding to one or more operations / functions in a high-level language (e.g., C, C++, Python, etc.) into second instructions corresponding to one or more operations / functions in an HDL. In some such instances, binary files are compiled, generated, and / or otherwise output from the unified software platform based on the second instructions. In some instances, Figure 15 The FPGA circuit 1500 shown can access and / or load binary files to enable... Figure 15 The FPGA circuit 1500 shown is configured and / or constructed to perform one or more operations / functions. For example, a binary file may consist of a bit stream (e.g., one or more computer-readable bits, one or more machine-readable bits, etc.), data (e.g., computer-readable data, machine-readable data, etc.), and / or may be generated by... Figure 15 The illustrated FPGA circuit 1500 accesses machine-readable instructions to enable... Figure 15 The FPGA circuit 1500 or a portion thereof shown is configured and / or structured.

[0088] Figure 15 The illustrated FPGA circuit 1500 includes exemplary input / output (I / O) circuitry 1502, which is used to obtain data from and / or output data to exemplary configuration circuitry 1504 and / or external hardware 1506. For example, configuration circuitry 1504 may be implemented by interface circuitry that can obtain a binary file to configure FPGA circuitry 1500 or a portion thereof, which may be implemented as a bitstream, data, and / or machine-readable instructions. In some such instances, configuration circuitry 1504 may obtain the binary file from a user, a machine (e.g., hardware circuitry that can implement an artificial intelligence (AI) model / machine learning (ML) model to generate a binary file (e.g., programmable circuitry or dedicated circuitry)), and / or any combination thereof. In some instances, external hardware 1506 may be implemented by external hardware circuitry. For example, external hardware 1506 may be implemented by… Figure 14 The microprocessor 1400 shown is implemented.

[0089] The FPGA circuit 1500 also includes an array of exemplary logic gates 1508, a plurality of exemplary configurable interconnects 1510, and exemplary memory circuitry 1512. The logic gates 1508 and the configurable interconnects 1510 can be configured to interact with... Figures 3-5Instantiate at least some of the machine-readable instructions in the instruction set corresponding to one or more operations / functions and / or other expected operations. Figure 15 The logic gate circuits 1508 shown are fabricated as blocks or groups. Each block includes a semiconductor-based electrical structure that can be configured into a logic circuit. In some instances, the electrical structure includes logic gates (e.g., AND gates, OR gates, NOR gates, etc.) that provide basic building blocks for the logic circuit. Electrically controllable switches (e.g., transistors) are present within each logic gate circuit 1508 to enable the configuration of the electrical structure and / or logic gates to form a circuit for performing a desired operation / function. The logic gate circuits 1508 may include other electrical structures such as lookup tables, registers (e.g., flip-flops or latches), multiplexers, etc.

[0090] The configurable interconnect 1510 in the illustrated example may include conductive paths, traces, vias, etc. of electrically controllable switches (e.g., transistors), and the state of the electrically controllable switches can be changed by programming (e.g., using an HDL instruction language) to activate or deactivate one or more connections between one or more logic gates 1508 to program the desired logic circuit.

[0091] The storage circuit 1512 in the illustrated example is configured to store the results of one or more operations performed by the corresponding logic gates. The storage circuit 1512 can be implemented using registers, etc. In the illustrated example, the storage circuit 1512 is distributed among the logic gates 1508 for easy access and to improve execution speed.

[0092] Figure 15 The exemplary FPGA circuit 1500 shown also includes exemplary dedicated operating circuitry 1514. In this example, dedicated operating circuitry 1514 includes special-purpose circuitry 1516, which can be invoked to perform common functions to avoid the need for field programming of those functions. Examples of such special-purpose circuitry 1516 include memory (e.g., DRAM) controller circuitry, PCIe controller circuitry, clock circuitry, transceiver circuitry, memory, and multiplier-accumulator circuitry. Other types of special-purpose circuitry may also be present. In some instances, FPGA circuitry 1500 may also include exemplary general-purpose programmable circuitry 1518, such as exemplary CPU 1520 and / or exemplary DSP 1522. Additionally or alternatively, other general-purpose programmable circuitry 1518, such as GPU, XPU, etc., that can be programmed to perform other operations may be present.

[0093] although Figure 14 and Figure 15 It shows Figure 13The two exemplary implementations of the programmable circuit 1312 shown are examples; however, many other methods are also conceivable. For instance, the FPGA circuit may include, for example... Figure 15 One or more exemplary CPUs, such as CPU1520, are shown as onboard CPUs. Therefore, Figure 13 The programmable circuit 1312 shown can also be used by at least... Figure 14 The exemplary microprocessor 1400 shown is Figure 15 The exemplary FPGA circuit 1500 shown is combined to implement this. In some such hybrid instances, Figure 15 One or more cores 1502 shown can perform the functions of Figures 3-5 The flowchart shown represents the first part of a machine-readable instruction to perform a first operation / function. Figure 15 The FPGA circuit 1500 shown can be configured and / or constructed to perform operations related to... Figures 3-5 The flowchart shown represents the second operation / function corresponding to the second part of the machine-readable instruction, and / or the ASIC can be configured and / or constructed to perform the operation / function described by the instruction. Figures 3-5 The flowchart shown represents the third operation / function corresponding to the third part of the machine-readable instruction.

[0094] It should be understood that Figure 2 Some or all of the circuits shown can therefore be instantiated simultaneously or at different times. For example, Figure 14 The same and / or different parts of the microprocessor 1400 shown can be programmed to execute parts of machine-readable instructions at the same and / or different times. In some instances, Figure 15 The same and / or different parts of the FPGA circuit 1500 shown can be configured and / or constructed to perform operations / functions corresponding to the parts of the machine-readable instructions at the same and / or different times.

[0095] In some instances, Figure 2 Some or all of the circuits shown can be instantiated in one or more threads, for example, executing in parallel and / or serially. For example, Figure 14 The microprocessor 1400 shown can execute machine-readable instructions in one or more threads that execute in parallel and / or serially. In some instances, Figure 15 The FPGA circuit 1500 shown can be configured and / or constructed to perform operations / functions in parallel and / or serially. Furthermore, in some instances, Figure 2 Some or all of the circuits shown can be implemented within one or more virtual machines and / or containers, which in Figure 14 It is executed on the microprocessor 1400 shown.

[0096] In some instances, Figure 13The programmable circuit 1312 shown can be housed in one or more packages. For example, Figure 14 The microprocessor 1400 and / or shown Figure 15 The FPGA circuit 1500 shown can be housed in one or more packages. In some instances, the XPU can be... Figure 13 The programmable circuit 1312 shown can be implemented in one or more packages. For example, the XPU may include a CPU in one package (e.g., Figure 14 The microprocessor 1400 shown Figure 15 The CPU1520 shown), and the DSP in another package (e.g., Figure 15 The DSP 1522 shown is in another package, the GPU is in yet another package, and the FPGA is in yet another package (e.g., Figure 15 The FPGA circuit shown is 1500.

[0097] exist Figure 16 A block diagram illustrating an exemplary software distribution platform 1605 is provided, which is used to distribute, for example... Figure 13 The illustrated machine-readable instructions 1332 and other software are distributed to other hardware devices (e.g., hardware devices owned and / or operated by a third party from the owner and / or operator of the software distribution platform). The exemplary software distribution platform 1605 can be implemented by any computer server, data facility, cloud service, etc., capable of storing and transferring software to other computing devices. The third party can be a customer of the entity that owns and / or operates the software distribution platform 1605. For example, the entity that owns and / or operates the software distribution platform 1605 can be, for example... Figure 13 The illustrated machine-readable instructions 1332 and other software developers, sellers, and / or licensors are shown. Third parties can be consumers, users, retailers, OEMs, etc., who purchase and / or license the software for use and / or resell and / or sublicense. In the illustrated example, the software distribution platform 1305 includes one or more servers and one or more storage devices. As described above, the storage devices can store data that can be stored in conjunction with… Figures 3-5The exemplary machine-readable instruction shown corresponds to machine-readable instruction 1332. One or more servers of the exemplary software distribution platform 1305 communicate with an exemplary network 1310, which may correspond to any one or more networks on the Internet and / or any of the exemplary networks described above. In some instances, as part of a business transaction, one or more servers respond to a request and transmit software to a requesting party. Payment for the delivery, sale, and / or licensing of the software may be processed by one or more servers of the software distribution platform and / or through a third-party payment entity. The server enables purchasers and / or licensees to download machine-readable instruction 1332 from the software distribution platform 1605. For example, it may be compatible with... Figures 3-5 The software corresponding to the exemplary machine-readable instructions in the example can be downloaded to the exemplary programmable circuit platform 1300, which executes the machine-readable instructions 1332 to implement... Figure 2 The model tuner circuit 205 is shown. In some instances, one or more servers of the software distribution platform 1605 periodically provide, transmit, and / or force software (e.g., Figure 13 The exemplary machine-readable instructions (1332) shown are updated to ensure that improvements, patches, updates, etc., are distributed and applied to the software at the end-user device. Although referred to as software above, the distributed “software” may alternatively be firmware.

[0098] "Comprising" and "including" (and all forms and tenses thereof) are used herein as open-ended terms. Therefore, whenever a claim uses any form of "comprising" or "including" (e.g., including, having, etc.) as a preamble or within the scope of any type of claim, it should be understood that other elements, terms, etc., may exist that do not fall outside the scope of the corresponding claim or the scope of the statement. As used herein, the phrase "at least" is also open-ended when used, for example, as a transitional term in the preamble of a claim, in the same way that the terms "comprising" and "including" are open-ended. The term "and / or," when used in the form of, for example, A, B, and / or C, refers to any combination or subset of A, B, and C, such as (1) A alone, (2) B alone, (3) C alone, (4) A and B, (5) A and C, (6) B and C, or (7) A and B and C. As used herein in the context of describing structures, components, articles, objects, and / or things, the phrase “at least one of A and B” is intended to refer to an implementation that includes (1) at least one A, (2) at least one B, or (3) at least one A and at least one B. Similarly, as used herein in the context of describing structures, components, articles, objects, and / or things, the phrase “at least one of A or B” is intended to refer to an implementation that includes (1) at least one A, (2) at least one B, or (3) at least one A and at least one B. As used herein in the context of describing the completion or execution of processes, instructions, actions, activities, etc., the phrase “at least one of A and B” is intended to refer to an implementation that includes (1) at least one A, (2) at least one B, or (3) at least one A and at least one B. Similarly, as used herein in the context of describing the completion or execution of processes, instructions, actions, activities, etc., the phrase “at least one of A or B” is intended to refer to an implementation including (1) at least one A, (2) at least one B, or (3) at least one A and at least one B.

[0099] As used herein, singular references (e.g., "a," "an," "first," "second," etc.) do not exclude plurals. As used herein, the term "a" or "an" refers to one or more of that object. The terms "a" (or "an"), "one or more," and "at least one" are used interchangeably herein. Furthermore, although listed separately, multiple means, elements, or actions may be implemented by, for example, the same entity or object. Additionally, although individual features may be included in different embodiments or claims, these features may be combined, and inclusion in different embodiments or claims does not imply that the combination of features is impractical and / or advantageous.

[0100] As used herein, the phrase “communication” includes variations thereof covering direct communication and / or indirect communication via one or more intermediate components, and does not require direct physical (e.g., wired) communication and / or continuous communication, but additionally includes selective communication with periodic intervals, scheduling intervals, non-periodic intervals and / or manifested as one-off events.

[0101] As used herein, “programmable circuit” is defined as including (i) one or more special-purpose circuits (e.g., application-specific integrated circuits (ASICs)) configured to perform specific operations and including one or more semiconductor-based logic devices (e.g., electrical hardware implemented by one or more transistors), and / or (ii) one or more semiconductor-based general-purpose circuits that can be programmed with instructions to perform specific functions and / or operations and include one or more semiconductor-based logic devices (e.g., electrical hardware implemented by one or more transistors). Examples of programmable circuits include programmable microprocessors, such as central processing units (CPUs) capable of executing first instructions to perform one or more operations and / or functions; field-programmable gate arrays (FPGAs) programmable with second instructions to be configured and / or constructed to instantiate one or more operations and / or functions corresponding to the first instructions; graphics processing units (GPUs) capable of executing first instructions to perform one or more operations and / or functions; digital signal processors (DSPs) capable of executing first instructions to perform one or more operations and / or functions; XPUs; network processing units (NPUs); one or more microcontrollers capable of executing first instructions to perform one or more operations and / or functions; and / or integrated circuits such as application-specific integrated circuits (ASICs). For example, an XPU can be implemented by a heterogeneous computing system comprising various types of programmable circuits (e.g., one or more FPGAs, one or more CPUs, one or more GPUs, one or more NPUs, one or more DSPs, etc., and / or any combination thereof) and orchestration techniques (e.g., application programming interfaces (APIs)) that can assign computational tasks to any of the various types of programmable circuits suitable for and available to perform the computational task.

[0102] As used herein, an integrated circuit / integrated circuit is defined as one or more semiconductor packages containing one or more circuit elements such as transistors, capacitors, inductors, resistors, current paths, diodes, etc. For example, an integrated circuit can be implemented as one or more of ASICs, FPGAs, chips, microchips, programmable circuits, semiconductor substrates coupling multiple circuit elements, systems-on-chips (SoCs), etc.

[0103] As described above, it can be understood that the exemplary systems, methods, apparatuses, and artifacts disclosed herein allow for unstructured sparsity in large, low-precision pre-trained base models. Fine-tuning of LLMs is improved while maintaining a low count of trainable parameters and resource requirements. Thus, machine learning-based inference can be accelerated by using fewer parameters compared to known sparse fine-tuning. For example, significant speedups can be achieved by leveraging model sparsity with almost no (e.g., minimal) compression loss, demonstrating that the resulting sparse models can alleviate existing memory bandwidth challenges. Therefore, the instances disclosed herein result in improvements to the operation of machines.

[0104] This document discloses exemplary methods, apparatuses, systems, and artifacts for achieving unstructured sparsity in large pre-trained base models with low precision. Further embodiments and combinations thereof include the following:

[0105] Example 1 includes an apparatus comprising interface circuitry, machine-readable instructions, and at least one processor circuitry, the at least one processor circuitry being programmed by the machine-readable instructions to sparsify a basic model of a base model to generate a sparse basic model, apply a neural low-rank adapter search to the sparse basic model, and output a fine-tuned basic model based on the application of the neural low-rank adapter search to the sparse basic model.

[0106] Example 2 includes the device of Example 1, wherein one or more of the at least one processor circuitry are used to sparsify the base model by identifying sparsity patterns associated with the sparsification weights of the base model.

[0107] Example 3 includes a device comprising one or more of Examples 1-2, wherein one or more of the at least one processor circuitry are used to identify the sparse weights based on a scoring function applied to the pre-trained weights of the base model.

[0108] Example 4 includes a device comprising one or more of Examples 1-3, wherein when the fine-tuned basic model is a sparsified and quantized basic model, one or more of the at least one processor circuitry are configured to identify the sparsified and quantized basic model by quantizing the sparsification weights to a lower precision.

[0109] Example 5 includes the device of Example 1, wherein one or more of the at least one processor circuitry are configured to generate a binary mask based on the sparse basic model, the binary mask being derived from the initial sparsification of the weight matrix of the basic model.

[0110] Example 6 includes the device of Example 1, wherein one or more of the at least one processor circuitry are used to apply the neural low-rank adapter search to training a resilient adapter with variable configuration to improve the accuracy of the fine-tuned base model.

[0111] Example 7 includes a device from one or more of Examples 1-6, wherein the variable configuration represents a variable rank rating compared to a fixed rank rating.

[0112] Example 8 includes a device comprising one or more of Examples 1-7, wherein the neural low-rank adapter search is used to apply the variable rank rating to the resilient adapter to identify a single resilient adapter configuration from the space of resilient adapter configurations.

[0113] Example 9 includes a device comprising one or more of Examples 1-6, wherein one or more of the at least one processor circuitry are configured to merge the model weights of the elastic adapter and the base model after fine-tuning, while maintaining the sparsity of the model weights.

[0114] Example 10 includes at least one non-transient machine-readable medium comprising machine-readable instructions for causing at least one processor circuit to at least sparsify a basic model of a base model to generate a sparse basic model, apply a neural low-rank adapter search to the sparse basic model, and output a fine-tuned basic model based on the application of the neural low-rank adapter search to the sparse basic model.

[0115] Example 11 includes at least one non-transitory machine-readable medium of Example 10, wherein the machine-readable instructions are used to cause one or more of the at least one processor circuitry to sparsify the base model by identifying sparsity patterns associated with sparsification weights of the base model.

[0116] Example 12 includes at least one non-transitory machine-readable medium comprising one or more of Examples 10-11, wherein the machine-readable instructions are used to enable one or more of the at least one processor circuitry to identify the sparse weights based on a scoring function applied to the pre-trained weights of the base model.

[0117] Example 13 includes at least one non-transitory machine-readable medium comprising one or more of Examples 10-12, wherein the fine-tuned basic model is a sparsified and quantized basic model, and the machine-readable instructions are used to enable one or more of the at least one processor circuitry to identify the sparsified and quantized basic model by quantizing the sparsification weights to a lower precision.

[0118] Example 14 includes at least one non-transitory machine-readable medium of Example 10, wherein the machine-readable instructions are used to cause one or more of the at least one processor circuitry to generate a binary mask based on the sparse basic model, the binary mask being derived from the initial sparsification of the weight matrix of the basic model.

[0119] Example 15 includes at least one non-transitory machine-readable medium of Example 10, wherein the machine-readable instructions are used to cause one or more of the at least one processor circuitry to apply the neural low-rank adapter search to training a resilient adapter with variable configuration to improve the accuracy of the fine-tuned base model.

[0120] Example 16 includes at least one non-transitory machine-readable medium of one or more of Examples 10-15, wherein the variable configuration represents a variable rank rating compared to a fixed rank rating.

[0121] Example 17 includes at least one non-transitory machine-readable medium of one or more of Examples 10-16, wherein the neural low-rank adapter search is used to apply the variable rank rating to the resilient adapter to identify a single resilient adapter configuration from the space of resilient adapter configurations.

[0122] Example 18 includes at least one non-transitory machine-readable medium of one or more of Examples 10-16, wherein the machine-readable instructions are used to cause one or more of the at least one processor circuitry to merge the model weights of the resilient adapter and the base model after fine-tuning, while maintaining the sparsity of the model weights.

[0123] Example 19 includes an apparatus comprising means for sparsifying a base model of a base model to generate a sparse base model, means for applying a neural low-rank adapter search to the sparse base model, and means for outputting a fine-tuned base model based on the application of the neural low-rank adapter search to the sparse base model.

[0124] Example 20 includes the device of Example 19, wherein the means for sparsification includes identifying sparsity patterns associated with sparsification weights of the base model.

[0125] Example 21 includes a device from one or more of Examples 19-20, wherein the means for sparsification includes identifying the sparsification weights based on a scoring function applied to the pre-trained weights of the base model.

[0126] Example 22 includes a device from one or more of Examples 19-21, wherein the fine-tuned basic model is a sparsified and quantized basic model, and the device includes means for performing quantization to identify the sparsified and quantized basic model by quantizing the sparsification weights to a lower precision.

[0127] Example 23 includes the apparatus of Example 19, the apparatus including means for merging weights to generate a binary mask based on the sparse basic model, the binary mask being derived from an initial sparsification of the weight matrix of the basic model.

[0128] Example 24 includes the device of Example 19, wherein the means for applying neural low-rank adapter search includes training a flexible adapter with variable configuration to improve the accuracy of the fine-tuned base model.

[0129] Example 25 includes a device from one or more of Examples 19-24, wherein the variable configuration represents a variable rank rating compared to a fixed rank rating.

[0130] Example 26 includes a device from one or more of Examples 19-25, wherein the means for applying the neural low-rank adapter search includes applying the variable rank rating to the resilient adapter to identify a single resilient adapter configuration from the space of resilient adapter configurations.

[0131] Example 27 includes a device from one or more of Examples 19-24, wherein the means for applying the neural low-rank adapter search includes merging the model weights of the elastic adapter and the base model after fine-tuning, while maintaining the sparsity of the model weights.

[0132] Example 28 includes a method comprising: sparsifying a base model of a base model to generate a sparse base model; applying a neural low-rank adapter search to the sparse base model; and outputting a fine-tuned base model based on the application of the neural low-rank adapter search to the sparse base model.

[0133] Example 29 includes the method of Example 28, and further includes sparsifying the base model by identifying sparsity patterns associated with the sparsification weights of the base model.

[0134] Example 30 includes one or more of the methods in Examples 28-29, and further includes identifying the sparse weights based on a scoring function applied to the pre-trained weights of the base model.

[0135] Example 31 includes one or more of the methods in Examples 28-30, wherein when the fine-tuned basic model is a sparsified and quantized basic model, the method further includes identifying the sparsified and quantized basic model by quantizing the sparsification weights to a lower precision.

[0136] Example 32 includes the method of Example 28, and further includes generating a binary mask based on the sparse basic model, the binary mask being derived from the initial sparsification of the weight matrix of the basic model.

[0137] Example 33 includes the method of Example 28, and further includes applying the neural low-rank adapter search to train a resilient adapter with variable configuration to improve the accuracy of the fine-tuned base model.

[0138] Example 34 includes one or more of the methods in Examples 28-33, wherein the variable configuration represents a variable rank rating compared to a fixed rank rating.

[0139] Example 35 includes one or more of the methods in Examples 28-34, wherein the neural low-rank adapter search is used to apply the variable rank rating to the resilient adapter to identify individual resilient adapter configurations from the space of resilient adapter configurations.

[0140] Example 36 includes one or more of the methods in Examples 28-33, and further includes merging the model weights of the elastic adapter and the base model after fine-tuning, while maintaining the sparsity of the model weights.

[0141] The following claims are incorporated herein by reference. Although some exemplary systems, methods, apparatuses, and articles of art have been disclosed herein, the scope of this patent is not limited thereto. Rather, this patent covers all systems, methods, apparatuses, and articles of art that reasonably fall within the scope of the claims of this patent.

Claims

1. An apparatus comprising: Interface circuit; Machine-readable instructions; as well as At least one processor circuit, which is programmed by the machine-readable instructions to: Sparsify the basic model of the base model to generate a sparse basic model; The neural low-rank adapter search is applied to the sparse basic model; as well as The application of the neural low-rank adapter search to the sparse basic model outputs a finely tuned basic model.

2. The apparatus of claim 1, wherein one or more of the at least one processor circuitry are configured to sparsify the base model by identifying sparsity patterns associated with the sparsification weights of the base model.

3. The device of claim 2, wherein one or more of the at least one processor circuitry are used to identify the sparse weights based on a scoring function applied to the pre-trained weights of the base model.

4. The device of claim 3, wherein when the fine-tuned basic model is a sparsified and quantized basic model, one or more of the at least one processor circuitry are configured to identify the sparsified and quantized basic model by quantizing the sparsification weights to a lower precision.

5. The device according to any one of claims 1, 2, 3 or 4, wherein one or more of the at least one processor circuitry are configured to generate a binary mask based on the sparse basic model, the binary mask being derived from the initial sparsification of the weight matrix of the basic model.

6. The device according to any one of claims 1, 2, 3, 4 or 5, wherein one or more of the at least one processor circuitry are configured to apply the neural low-rank adapter search to training a resilient adapter with variable configuration to improve the accuracy of the fine-tuned base model.

7. The device of claim 6, wherein the variable configuration represents a variable rank rating compared with a fixed rank rating.

8. The device of claim 7, wherein the neural low-rank adapter search is used to apply the variable rank rating to the resilient adapter to identify a single resilient adapter configuration from the space of resilient adapter configurations.

9. The device of claim 6, wherein one or more of the at least one processor circuitry are configured to merge the model weights of the elastic adapter and the base model after fine-tuning, while maintaining the sparsity of the model weights.

10. At least one machine-readable medium comprising machine-readable instructions, said machine-readable instructions being configured to cause at least one processor circuitry to at least: Sparsify the basic model of the base model to generate a sparse basic model; The neural low-rank adapter search is applied to the sparse basic model; and The application of the neural low-rank adapter search to the sparse basic model outputs a finely tuned basic model.

11. The at least one machine-readable medium of claim 10, wherein the machine-readable instructions are configured to cause one or more of the at least one processor circuitry to sparsify the basic model by identifying sparsity patterns associated with the sparsification weights of the basic model.

12. The at least one machine-readable medium of claim 11, wherein the machine-readable instructions are configured to enable one or more of the at least one processor circuitry to identify the sparse weights based on a scoring function applied to the pre-trained weights of the base model.

13. The at least one machine-readable medium of claim 12, wherein the fine-tuned basic model is a sparsified and quantized basic model, and the machine-readable instructions are used to cause one or more of the at least one processor circuitry to identify the sparsified and quantized basic model by quantizing the sparsification weights to a lower precision.

14. At least one machine-readable medium according to any one of claims 10, 11, 12 or 13, wherein the machine-readable instructions are used to cause one or more of the at least one processor circuitry to generate a binary mask based on the sparse basic model, the binary mask being derived from the initial sparsification of the weight matrix of the basic model.

15. At least one machine-readable medium according to any one of claims 10, 11, 12, 13 or 14, wherein the machine-readable instructions are configured to cause one or more of the at least one processor circuitry to apply the neural low-rank adapter search to training a resilient adapter with variable configuration to improve the accuracy of the fine-tuned base model.

16. The at least one machine-readable medium of claim 15, wherein the variable configuration represents a variable rank rating compared to a fixed rank rating.

17. The at least one machine-readable medium of claim 16, wherein the neural low-rank adapter search is used to apply the variable rank rating to the resilient adapter to identify a single resilient adapter configuration from the space of the resilient adapter configurations.

18. The at least one machine-readable medium of claim 16, wherein the machine-readable instructions are configured to cause one or more of the at least one processor circuitry to merge the model weights of the resilient adapter and the base model after fine-tuning, while maintaining the sparsity of the model weights.

19. An apparatus comprising: A device for sparsifying the basic model of a base model to generate a sparse basic model; A means for applying neural low-rank adapter search to the sparse basic model; as well as A device for applying the sparse basic model to the search based on the neural low-rank adapter to output a finely tuned basic model.

20. The apparatus of claim 19, wherein the means for sparsification includes identifying sparsity patterns associated with sparsification weights of the base model.

21. A method, the method comprising: Sparsify the basic model of the base model to generate a sparse basic model; The neural low-rank adapter search is applied to the sparse basic model; as well as The application of the neural low-rank adapter search to the sparse basic model outputs a finely tuned basic model.

22. The method of claim 21, further comprising sparsifying the base model by identifying sparsity patterns associated with the sparsification weights of the base model.

23. The method of claim 22, further comprising identifying the sparsified weights based on a scoring function applied to the pre-trained weights of the base model.

24. The method of claim 23, wherein when the fine-tuned basic model is a sparsified and quantized basic model, the method further comprises identifying the sparsified and quantized basic model by quantizing the sparsification weights to a lower precision.

25. The method according to any one of claims 21, 22, 23 or 24, further comprising generating a binary mask based on the sparse basic model, the binary mask being derived from the initial sparsification of the weight matrix of the basic model.

Images