METHOD AND DEVICES FOR EFFICIENT FINE-TUNING OF UNSTRUCTURED, SPARY AND INACCURATE LARGE PRE-TRAINED FOUNDATION MODELS

Abstract

An exemplary device includes an interface circuit, machine-readable instructions, and at least one processor circuit programmed by the machine-readable instructions to sparsify a basic model of a foundation model, to generate a sparse basic model, to apply a neural low-rank adapter search to the sparse basic model, and to output a fine-tuned basic model based on the application of the neural low-rank adapter search to the sparse basic model.

Description

BACKGROUND

[0001] Foundational models (e.g., pre-trained Large Language Models (LLMs)) are neural networks that perform artificial intelligence (AI)-based tasks. These models use millions or billions of parameters that may require fine-tuning for a new dataset or a downstream task, such as mathematical reasoning. LLMs include encoder-only models for classification tasks, decoder-only models for content generation tasks, and encoder-decoder models for content evaluation and generation tasks, such as translation and summarization. BRIEF DESCRIPTION OF THE DRAWINGS Fig. Figure 1 illustrates the existing limitations of known approaches for fine-tuning sparse and quantized models and for fusion of low-rank adapters (LoRA). Fig. Figure 2 is a block diagram of an exemplary implementation of a model tuner circuit constructed in accordance with the teachings of this revelation to fine-tune pre-trained LLMs for downstream tasks. Fig. Figure 3 is a flowchart that represents exemplary machine-readable instructions and / or exemplary operations that can be executed, instantiated, and / or performed by exemplary programmable circuits to implement the exemplary model tuner circuit of Fig. 2 to implement. Fig. Figure 4 is a flowchart that represents exemplary machine-readable instructions and / or exemplary operations that can be executed, instantiated, and / or performed by exemplary programmable circuits to implement the exemplary model tuner circuit of Fig. 2, to perform sparsification and quantization of the base model. Fig. Figure 5 is a flowchart that represents exemplary machine-readable instructions and / or exemplary operations that can be executed, instantiated, and / or performed by exemplary programmable circuits to implement the exemplary model tuner circuit of Fig. 2. To implement the restoration of model accuracy using neural low-rank adapter search (NLS). Fig. Figure 6 illustrates exemplary pipeline configurations that can be used for efficient fine-tuning of large models, including (1) a first pipeline for parameter-efficient fine-tuning of sparse and quantized models using elastic adapters, resulting in a non-fused model and adapters, (2) a second pipeline for parameter-efficient fine-tuning of sparse models using Sparse Awareness (SparsePEFT), which enables subsequent fusion of model and adapters, and (3) a third pipeline for parameter-efficient fine-tuning of sparse and quantized models using quantization and Sparse-Aware Adapter Merging. Fig. Figure 7 shows an example of known low-rank adapters (LoRA) compared to elastic adapters connected to a neural low-rank search (NLS) disclosed herein. Fig. Figure 8 shows an exemplary overview of the fine-tuning of pre-trained LLMs based on model sparsification, restoring the accuracy of the base model using the NLS adapters of Fig. 7, and identification of a sparsified, finely tuned architecture based on a subadapter search. Fig. Figure 9 illustrates sparsified parameter-efficient fine-tuning (SparsePEFT) using a binary mask derived from sparsified weights. Fig. Figure 10 illustrates an exemplary reduction of the parameters required for the fine-tuning of an LLM while simultaneously achieving higher accuracy using the methods disclosed herein compared to known sparse fine-tuning methods. Fig. Figure 11A illustrates the results for the evaluation of the fine-tuning of a first exemplary model using known fine-tuning compared to the approaches disclosed herein (e.g., fine-tuning of sparse and quantized models (SQFT), SQFT in combination with SparsePEFT, and SQFT in combination with SparsePEFT including quantization and Sparse-Aware Adapter Merging (QA-SparsePEFT)). Fig. Figure 11B shows the results of an ablation fine-tuning study using known low-rank adapters (LoRA) compared to elastic adapters using neural low-rank search (NLS) by Fig. 7 in the evaluation of the fine-tuning disclosed herein, including SQFT associated with SparsePEFT and SQFT associated with quantizable SparsePEFT. Fig. Figure 12 illustrates an exemplary cost analysis for different pipelines related to model fine-tuning, including an evaluation of model storage, fine-tuning time, and accuracy. Fig. Figure 13 is a block diagram of an exemplary processing platform with a programmable circuit structured to execute, instantiate, and / or perform the exemplary machine-readable instructions and / or operations of the Fig. 3-5 for the implementation of the model tuner circuit of Fig. 2. Perform. Fig. 14 is a block diagram of an exemplary implementation of the processor circuit of Fig. 13. Fig. Figure 15 is a block diagram of another exemplary implementation of the programmable circuit of Fig. 13. Fig. Figure 16 is a block diagram of an exemplary software / firmware / instruction distribution platform (e.g., one or more servers) for distributing software, instructions, and / or firmware (e.g., according to the exemplary machine-readable instructions of Fig. 3-5) to client devices associated with end users and / or consumers (e.g., for licensing, sale, and / or use), retailers (e.g., for sale, resale, licensing, and / or sublicensing), and / or original equipment manufacturers (OEMs) (e.g., for inclusion in products intended for distribution to retailers and / or other end users such as direct customers).

[0002] Generally, the same reference symbols are used throughout the drawing(s) and accompanying written description to refer to the same or similar parts. The figures are not necessarily to scale. DETAILED DESCRIPTION

[0003] Large Language Models (LLMs) comprise large transformer-based models that exhibit capabilities in a wide variety of tasks, from language comprehension to text generation. State-of-the-art LLMs contain billions of parameters for natural language understanding. LLMs are trained, for example, on billions or trillions of tokens to improve the model's robustness in zero-shot evaluations, which are used to measure model performance. However, LLMs need to be adapted to improve model performance through new tasks and / or datasets. Fine-tuning the models is required to increase accuracy, but such measures are resource-intensive. Known approaches for generating sparsity and fine-tuning in large foundational models include, for example, a post-training clipping method to compress LLMs (e.g.,SparseGPT) and the fine-tuning of sparse models. These well-known methods require significant computational resources and a runtime that can take advantage of the resulting sparse weighting matrices.

[0004] Furthermore, parameter-efficient fine-tuning techniques (PEFT) have been developed to reduce the number of parameters used for fine-tuning a model. For example, low-rank adapters (LoRA) are a popular PEFT-based technique for modifying a selection of linear layers from a pre-trained model. The LoRA technique modifies a selection of linear layers, each with a weight matrix. Additionally, the LoRA technique uses low-rank adapters to extend the linear projection of a layer, so the original model weights remain frozen, and only the attached adapters are fine-tuned. While the LoRA-based technique has proven effective for model fitting, combining LoRA with model compression techniques (e.g., sparsity, quantization, etc.) has proven even more effective.This presents challenges associated with fusing adapters into a single, compressed, and finely tuned model. For example, fusing dense adapters with sparse model weights results in a loss of sparsity in the updated base model. Similarly, adapter fusing is prevented if model weights with a different numerical accuracy than the adapters are used.

[0005] The methods and devices disclosed herein enable the efficient fine-tuning of sparse and quantized large, pre-trained foundational models. In the examples disclosed here, the fine-tuning of LLMs is improved while maintaining a small number of trainable parameters and resource requirements, and mitigating the challenge of memory bandwidth. In these examples, the resulting fine-tuned models accelerate inference by leveraging existing runtimes that support unstructured sparsity. The methods and devices presented here enable efficient fine-tuning that surpasses traditional PEFT approaches.In the examples disclosed here, LLM-based fine-tuning is performed using (1) model sparsification and / or quantization, (2) accuracy restoration using neural low-rank adapter search (NLS), and (3) identification of a high-performance sparsified fine-tuning architecture. NLS is performed, for example, on elastic adapters connected to basic LLMs. In the examples disclosed here, performance is improved while using an order of magnitude fewer parameters compared to known sparse fine-tuning methods. The methods and devices disclosed here can also be generalized to other transformer-based architectures (e.g., not limited to LLMs) and / or other non-transformer-based architectures (e.g., selective state-space models (SSMs)) that use parameterized linear transformations.

[0006] Furthermore, the methods and devices described herein fuse sparse weights and dense adapters while avoiding the loss of sparse matrices when fusing sparse and / or dense matrices. This allows machine learning-based inferences to be accelerated by using fewer parameters than other known commercial sparse fine-tuning methods. For example, the methods and devices disclosed herein outperform low-rank (LoRA) adapters in both dense and sparse configurations. In the examples disclosed herein, computational challenges associated with memory bandwidth are addressed by applying weight compression for sparse fine-tuning. The resulting sparsity in LLMs can be used to improve execution times. For example, the sparsity of the model can be reduced with low (e.g.,The minimal compression loss is exploited, leading to a significant speedup and demonstrating that the resulting sparse model can alleviate existing memory bandwidth issues. Compared to known state-of-the-art LLM fine-tuning techniques, the methods and devices disclosed here use fewer parameters for fine-tuning pre-trained LLMs for downstream tasks, require less time for sparsifying a given LLM, consume less memory during fine-tuning, and can be deployed in a wider range of hardware.

[0007] Fig. Figure 1 illustrates the existing limitations of known approaches 100 for fine-tuning sparse and quantized models and for fusion of low-rank adapters (LoRA). Large, pre-trained foundational or boundary models are an essential component of artificial intelligence applications (e.g., natural language understanding, code generation, etc.). These models are trained using thousands of hardware accelerators (e.g., graphics processing units (GPUs)), resulting in outstanding zero-shot performance across a variety of tasks and datasets (e.g., performance associated with completing a task without having been given training examples). However, such models often require further adjustment to improve their performance on new tasks and / or data. While low-rank adapters (LoRA) can be used for model adjustment, combining such adapters with model compression techniques (e.g., sparsity, quantization, etc.) prevents) the fusion of these adapters into a single compressed, finely tuned model. For example, the fusion of dense adapters leads to a loss of sparsity in the base model and / or the fusion of adapters may not be achievable due to differing numerical accuracy. As in the example of . Fig. As shown in Figure 1, full fine-tuning is computationally intensive, while LoRA for parameter-efficient fine-tuning (PEFT) with sparse or quantized models cannot be readily fused with the compressed weights due to the loss of previously induced sparsity and / or differing numerical precision. For example, Fig. 1 a base model 102, a sparse base model 104, and a quantized base model 106, and a LoRA-based adapter 108. The sparse base model 104 can be obtained from the original base model 102 by sparsification 110. While this version of an example of a sparse complete vote 112 allows the creation of a trainable sparse model, such a model is computationally expensive.

[0008] Sparsification encompasses a range of techniques for compressing and optimizing neural networks. By removing and / or reducing the significance of less important connections and information within a given model, sparsification often leads to a decrease in accuracy while simultaneously achieving a smaller model size, faster inferences, and / or lower energy consumption. For example, the use of LoRA for PEFT on sparse models 114, compared to sparse full tuning 112, results in a lack of fusion between the sparse base model 104 and the LoRA-based adapter(s) 108 due to the loss of sparsity, whereas the use of LoRA for PEFT on quantized models 120 results in a lack of fusion between the quantized base model 106 and the adapter(s) 108 due to differing numerical accuracy.In the examples revealed here, quantization reduces the precision of the weights and activations in a neural network (e.g., from 32-bit floating-point numbers to 8-bit integers). Thus, quantization reduces, for example, the model size and / or memory requirements, leading to faster inference.

[0009] In the examples presented here, the Adapter 108 is used to reduce the number of parameters for fine-tuning a model. For example, given a pre-trained model, the Adapter 108 can be used to modify a selection of linear layers, each with a weighting matrix, as in conjunction with Fig. 7 described in more detail. Low-rank adapters can be used to leave the weighting of the original model unchanged, while only the inserted adapters are fine-tuned, as in connection with the use of LoRA for PEFT with sparse models 114 and the use of LoRA for PEFT with quantized models 120 in the example of Fig. 1 shown.

[0010] Fig. Figure 2 is a block diagram 200 showing an exemplary implementation of a model tuner circuit 205, constructed in accordance with the teachings of this revelation to fine-tune pre-trained LLMs for downstream tasks. The model tuner circuit 205 of Fig. 2 can be instantiated by programmable circuits such as a central processing unit (CPU) that executes initial instructions (e.g., create an instance, launch it for any desired duration, materialize it, implement it, etc.). Additionally or alternatively, the model tuner circuit 205 of Fig. 2. The second instruction is instantiated (e.g., created, launched for any given time period, materialized, implemented, etc.) by (i) an application-specific integrated circuit (ASIC) and / or (ii) a field-programmable gate array (FPGA) that is structured and / or trained in response to the execution of the second instruction to perform operations corresponding to the first instruction. It is understood that some or all of the circuits of Fig. 2 can therefore be instantiated at the same time or at different times. Part or all of the circuit of Fig. 2 can, for example, be instantiated in one or more threads that run concurrently on hardware and / or sequentially on hardware. Furthermore, in some examples, some or all of the circuits of Fig. 2. can be implemented by a microprocessor circuit that executes instructions and / or an FPGA circuit that performs operations to implement one or more virtual machines and / or one or more virtual containers.

[0011] In the example of Fig. 2, the model tuner circuit 205 includes an exemplary sparsification initiator circuit 210, an exemplary quantization application circuit 215, an exemplary neural low-rank search (NLS) trainer circuit 220, an exemplary weight fusion circuit 225, an exemplary model generator circuit 230, and an exemplary data store 235. In the example of Fig. 2 The sparsification initiator circuit 210, the quantization application circuit 215, the NLS trainer circuit 220, the weight fusion circuit 225, the model generator circuit 230 and the data storage 235 are connected to an exemplary bus 240.

[0012] The sparsification initiator circuit 210 performs sparsification on a base model (e.g., base model 102 of Fig. 1) to obtain a sparse model. The resulting sparse model can, for example, serve as a basis model for the application of neural low-rank search (NLS) using the NLS trainer circuit 220. In the examples disclosed herein, the sparsification initiator circuit 210 assigns an arbitrary evaluation function (Ψ) to a proposed solution for a given matrix (W), where W∈ℝm×n and W=(wi,j),1≤i≤m,1≤j≤n, for entries w i,j This function determines the importance of wi,j in relation to the other weights in W. In the examples disclosed here, the weighting function Ψ can be formulated in any number of ways. In some examples, Ψ(W) = |W| · ||X||², where X represents sampled feature input activations. However, the end-to-end fine-tuning solution described here can use any other weighting function. For example, using the results of the weighting function Ψ and a desired sparsity level (s), the sparsification initiator circuit 210 obtains a given matrix (W p ) with a sparsity pattern that S{Wp}={(i,j)|Wi,jp≠0,1≤i≤m,1≤j≤n}, depending on |S{W p}| ≤ |S{W}| corresponds. As in the context of Fig. As described in Figure 6, the sparsification initiator circuit 210 performs sparsification of the base model 102, regardless of whether the sparse base model is subsequently used in a first pipeline for fine-tuning sparse and quantized models, a second pipeline for parameter-efficient fine-tuning of sparse models, and / or a third pipeline for fine-tuning sparse and quantized models with quantization and sparse-aware adapter merging. As described in connection with the examples disclosed here, large pretrained models (LPMs) can tolerate a higher degree of sparsity compared to previous generations of smaller transformer-based models.

[0013] The quantization application circuit 215 performs the quantization of the sparsified weights. For example, the quantization application circuit 215 quantizes the sparsified weights (W p) with lower accuracy to reduce memory bandwidth issues. Once the sparsification initiator circuit 210 has induced sparsity in the pretrained weights, an optional reduction in the numerical precision of these weights can be performed using the quantization application circuit 215. In the example of Fig. 6. The quantization application circuit 215 performs the quantization of the sparse base model(s) in the first pipeline for fine-tuning sparse and quantized models and in the third pipeline for fine-tuning sparse and quantized models with quantization and domain-specific adapter fusion, but not in the second pipeline for parameter-efficient fine-tuning of sparse models. In some examples, the quantization application circuit 215 applies layer-wise one-shot quantization. For example, using a selection from the most advanced quantization approaches, the quantization application circuit 215 identifies the low-accuracy sparsified weights (Ŵ) after training. p ), which, given an input X, returns the values ​​of argminW^p‖WpX−W^pX‖22 minimize. However, reducing the numerical accuracy and the variance of the weights often decreases the accuracy of the model, so fine-tuning is required to improve performance (e.g., using the NLS trainer circuit 220).

[0014] In some examples, the quantization application circuit 215 performs quantization in conjunction with the quantization- and sparsity-aware parameter-efficient fine-tuning (QA-SparsePEFT) pipeline, as in conjunction with Fig. Figure 6 shows an extension of sparse parameter-efficient fine-tuning (SparsePEFT) for sparsely quantized models. In the examples disclosed here, QA-SparsePEFT integrates quantization awareness into SparsePEFT. In most common quantization procedures, zeros and scales for the quantized target tensor can be determined during the quantization process. In QA-SparsePEFT, the zeros and scales of the sparsely quantized weights (W) are determined p ) together with the adapter (e.g. adapter 606 from Fig. 6) used. For example, the quantization application circuit 215 quantizes the adapters with the common fixed zeros and scales, resulting in quantization-aware fine-tuning. For example, given the sparsified pre-trained weight (W P ), the reduced adapter weight (L p) (e.g. obtained from SparsePEFT) and the zeros z and scale zeros s from the quantization of W p , the processing circuit 215 performs the quantization process in accordance with equation 1: W^mp=clamp(round((Wp+Lp)s)+z,0,Qp)

[0015] In the example, equation 1 corresponds W^mp the sparsely quantized (e.g., fused) weight, while Qp=2n−1−1, where n represents the bit width of the quantized values. Conversely, the quantization application circuit 215 can perform dequantization according to equation 2: W˜mp=s(W^mp−z)

[0016] In the example of equation 2, the zeros z and the scales s are used to refer to Wmp to approximate. In some examples, the NLS trainer circuit 220 can continue with NLS training simultaneously, which allows the fusion of adapters once an optimal configuration has been identified.

[0017] The NLS trainer circuit 220 trains a collection of adapters with variable configurations. With the sparsely quantized weights (Ŵ pFor example, the NLS Trainer Circuit 220 can compensate for accuracy losses caused by compression and fine-tune the weights for a specific downstream task. Unlike NLS, conventional LoRA adapters require values ​​to be assigned to several hyperparameters, including an associated rank (r) and / or the subset of modules in which these adapters are to be placed. In the examples disclosed here, the NLS Trainer Circuit 220 extends the weight distribution techniques to facilitate the discovery of optimal adapter configurations from a space of elastic adapter configurations. For example, instead of a fixed value for the rank (r), the NLS Trainer Circuit 220 applies elastic configurations C = [c1, ... , c n ], depending on r ← c i, depending on the activation of the corresponding subadapter. In some examples, the NLS trainer circuit 220 can also instantiate any configuration from a design space of possible configurations. For example, adapters can be fine-tuned using techniques in supernetworks with weight distribution, so that W p ← SparsityAndQuantization(W), where L δ1 ∈ ℝ h × {s0,...,sn} ← L1 ∈ ℝ h × r and L δ2 ∈ ℝ h × {p0,...,pm} ← L2 ∈ ℝ r × o ...As such, L δ1 and L δ2 These are elastic adapters that can be configured differently without requiring more memory than their static counterparts (e.g., L1 and L2).

[0018] The weight fusion circuit 225 identifies a binary mask from the sparsified weights and applies the binary mask to the adapters, enabling subsequent fusion of the adapters without loss of sparsity. As in Fig. As described in more detail in section 9, SparsePEFT applies a binary mask (M) derived from the initial sparsification of the matrix W. For example, the weight fusion circuit 225 applies the binary mask to convert the adapter-based matrix (e.g., designated BA) into an adapter weight (L). p ) to save, reducing the adapter weight using L p = (BA) ⊙ M is determined and activated during the fine-tuning process for sparsity awareness. In the examples revealed here, the weight fusion circuit 225 fuses the sparsified weights (W p ) and the adapter weight (L p ), without sacrificing the sparsity induced early in the compression pipeline (e.g. W p ← W p + L P ). As in connection with Fig. As described in section 9, SparsePEFT exhibits improved accuracy compared to fine-tuning with dense adapters.

[0019] The model generator circuit 230 produces the final model resulting from the fusion of the adapters. While sparsification and quantization techniques significantly accelerate model provisioning and inference across various hardware platforms and kernels, the addition of adapter models during inference leads to additional computational overhead for parameter-efficient fine-tuning with sparsified or quantized models due to the non-fusionability of adapter models. As previously described in connection with the weight fusion circuit 225, adapters can be integrated into the sparse and / or quantized model, thereby reducing adapter redundancy and computational overhead, resulting in leaner inference processes.The model generator circuit 230 outputs the final fused model resulting from the fusion of (1) a sparse base model and one or more adapters or (2) a sparse and quantized base model and the adapter(s), as described in conjunction with . Fig. 6 described in more detail.

[0020] The data storage device 235 can be used to store all information connected to the sparsification initiator circuit 210, the quantization application circuit 215, the NLS trainer circuit 220, the weight fusion circuit 225, and / or the model generator circuit 230. The data storage device 235 of the illustrated example of Fig. 2 can be implemented by any storage device, storage medium and / or storage disk for storing data, such as flash memory, magnetic media, optical media, etc. Furthermore, the data stored in data storage 235 can be in any data format, e.g., binary data, comma-separated data, tab-separated data, structured query language (SQL) structures, image data, etc.

[0021] In some examples, the device includes means for sparsifying a basic model. For example, the means for sparsifying a basic model can be implemented by the sparsification initiator circuit 210. In some examples, the sparsification initiator circuit 210 can be instantiated by a programmable circuit, such as the programmable circuit 1312 of Fig. 13. For example, the sparsification initiator circuit 210 can be replaced by the exemplary microprocessor 1400 from Fig. 14, which execute machine-executable instructions as defined at least in blocks 405, 410 and / or 415 of Fig. 4 are implemented. In some examples, the sparsification initiator circuit 210 can be instantiated by a hardware logic circuit, which is implemented by an ASIC, an XPU, or the FPGA circuit 1500. Fig. 15 can be implemented, structured to perform operations corresponding to the machine-readable instructions. Additionally or alternatively, the sparsification initiator circuit 210 can be instantiated by any other combination of hardware, software, and / or firmware. For example, the sparsification initiator circuit 210 can be implemented by at least one or more hardware circuits (e.g., processor circuits, discrete and / or integrated analog and / or digital circuits, an FPGA, an ASIC, an XPU, a comparator, an operational amplifier, a logic circuit, etc.) structured to execute some or all of the machine-readable instructions and / or to perform some or all of the operations corresponding to the machine-readable instructions without executing any software or firmware; however, other structures are also suitable.

[0022] In some examples, the device includes means for performing quantization. These means for performing quantization can be implemented, for example, by the quantization application circuit 215. In some examples, the quantization application circuit 215 can be instantiated by programmable circuits, such as the programmable circuit 1312 in Fig. 13. For example, the quantization application circuit 215 can be implemented by the one described in Fig. The exemplary microprocessor 1400 shown in Figure 14 is instantiated and executes machine-executable instructions, such as those provided by at least blocks 420, 425 and / or 430 in Fig. 4 are implemented. In some examples, the quantization application circuit 215 can be instantiated by a hardware logic circuit implemented by an ASIC, an XPU, or the FPGA circuit 1500 from Fig. 15 can be implemented, structured to perform operations corresponding to the machine-readable instructions. Additionally or alternatively, the quantization application circuit 215 can be instantiated by any other combination of hardware, software, and / or firmware. For example, the quantization application circuit 215 can be implemented by one or more hardware circuits (e.g., processor circuits, discrete and / or integrated analog and / or digital circuits, an FPGA, an ASIC, an XPU, a comparator, an operational amplifier, a logic circuit, etc.) structured to execute some or all of the machine-readable instructions and / or to perform some or all of the operations corresponding to the machine-readable instructions without executing any software or firmware; however, other structures are also suitable.

[0023] In some examples, the apparatus includes means for applying a neural low-rank adapter search. The means for applying a neural low-rank adapter search can be implemented, for example, by the neural low-rank search (NLS) trainer circuit 220. In some examples, the NLS trainer circuit 220 can be implemented by a programmable circuit such as the exemplary programmable circuit 1312 from Fig. 13 can be instantiated. For example, the NLS trainer circuit 220 can be instantiated by the one in Fig. The example microprocessor 1400 shown in Figure 14 is instantiated, which executes machine-executable instructions, such as those provided by at least block 315 in Figure 14. Fig. 3 are implemented. In some examples, the NLS trainer circuit 220 can be instantiated by a hardware logic circuit, which is implemented by an ASIC, an XPU, or the FPGA circuit 1500. Fig. 15 can be implemented, structured to perform operations corresponding to the machine-readable instructions. Additionally or alternatively, the NLS trainer circuit 220 can be instantiated by any other combination of hardware, software, and / or firmware. For example, the NLS trainer circuit 220 can be implemented by one or more hardware circuits (e.g., processor circuits, discrete and / or integrated analog and / or digital circuits, an FPGA, an ASIC, an XPU, a comparator, an operational amplifier, a logic circuit, etc.) structured to execute some or all of the machine-readable instructions and / or to perform some or all of the operations corresponding to the machine-readable instructions without executing any software or firmware; however, other structures are also suitable.

[0024] In some examples, the device includes means for weight fusion. The means for fusion of the weights can be implemented, for example, by the weight fusion circuit 225. In some examples, the weight fusion circuit 225 can be implemented by a programmable circuit, such as the programmable circuit 1312 from Fig. 13 can be instantiated. For example, the weight fusion circuit 225 can be instantiated by the exemplary microprocessor 1400. Fig. 14 will be instantiated, which executes machine-executable instructions, such as those provided by at least block 520 from Fig. 5 are implemented. In some examples, the weight fusion circuit 225 can be instantiated by a hardware logic circuit implemented by an ASIC, an XPU, or the FPGA circuit 1500 from Fig. 15 can be implemented, structured to perform operations corresponding to the machine-readable instructions. Additionally or alternatively, the weight fusion circuit 225 can be instantiated by any other combination of hardware, software, and / or firmware. For example, the weight fusion circuit 225 can be implemented by at least one or more hardware circuits (e.g., processor circuits, discrete and / or integrated analog and / or digital circuits, an FPGA, an ASIC, an XPU, a comparator, an operational amplifier, a logic circuit, etc.) structured to execute some or all of the machine-readable instructions and / or to perform some or all of the operations corresponding to the machine-readable instructions without executing any software or firmware; however, other structures are also suitable.

[0025] In some examples, the apparatus includes means for outputting a finely tuned basic model. The means for outputting a finely tuned basic model can be implemented, for example, by the model generator circuit 230. In some examples, the model generator circuit 230 can be replaced by a programmable circuit, such as the programmable circuit 1312 from Fig. 13 can be instantiated. For example, the model generator circuit 230 can be instantiated by the one in Fig. The exemplary microprocessor 1400 shown in Figure 14 is instantiated and executes machine-executable instructions, such as those provided by at least block 320 in Fig. 3 are implemented. In some examples, the model generator circuit 230 can be instantiated by a hardware logic circuit, which is implemented by an ASIC, an XPU, or the FPGA circuit 1500 from Fig. 15 can be implemented in a way that is structured to perform operations corresponding to the machine-readable instructions. Additionally or alternatively, the model generator circuit 230 can be instantiated by any other combination of hardware, software, and / or firmware. For example, the model generator circuit 230 can be implemented by at least one or more hardware circuits (e.g., processor circuits, discrete and / or integrated analog and / or digital circuits, an FPGA, an ASIC, an XPU, a comparator, an operational amplifier, a logic circuit, etc.) that are structured to execute some or all of the machine-readable instructions and / or to perform some or all of the operations corresponding to the machine-readable instructions without executing any software or firmware. However, other structures are also suitable.

[0026] While in Fig. 2 An example of the implementation of the model tuner circuit 205 is illustrated; one or more of the components shown in Fig. The elements, processes, and / or devices shown in Figure 2 can be combined, divided, rearranged, omitted, eliminated, and / or implemented in any other way. Furthermore, the exemplary sparsification initiator circuit 210, the exemplary quantization application circuit 215, the exemplary NLS trainer circuit 220, the exemplary weight fusion circuit 225, the exemplary model generator circuit 230, and / or more generally, the model tuner circuit 205, can be combined, divided, rearranged, omitted, eliminated, and / or implemented in any other way. Fig. 2. can be implemented by hardware, software, firmware, and / or any combination of hardware, software, and / or firmware. For example, any of the exemplary sparsification initiator circuit 210, the exemplary quantization application circuit 215, the exemplary NLS trainer circuit 220, the exemplary weight fusion circuit 225, the exemplary model generator circuit 230, and / or, more generally, the model tuner circuit 205 could be implemented. Fig. 2. can be implemented by a programmable circuit in combination with machine-readable instructions (e.g., firmware or software), a processing circuit, one or more analog circuits, one or more digital circuits, one or more logic circuits, one or more programmable processors, one or more programmable microcontrollers, one or more graphics processing units (GPUs), digital signal processors (DSPs, ASICs), programmable logic devices (PLDs), and / or field-programmable logic devices (FPLDs) such as FPGAs. Furthermore, the model tuner circuit 205 can be implemented from Fig. 2 one or more elements, methods and / or devices in addition to or instead of those in Fig. 2. The product includes, and / or may include more than one or all of the elements, methods and devices shown.

[0027] Flowcharts representing exemplary machine-readable instructions that can be executed by a programmable circuit to design the model tuner circuit 205 by Fig. 2 to implement and / or instantiate, and / or which represent exemplary operations that can be performed by a programmable circuit to implement the model tuner circuit 205 of Fig. 2. To implement and / or instantiate are in Fig. Figures 3-5 illustrate the machine-readable instructions. The machine-readable instructions can be one or more executable programs or one or more parts of one or more executable programs for execution by a programmable circuit, such as the programmable circuit 1312 shown below in conjunction with Fig. 13 discussed exemplary processor platform 1300 is shown, and / or may be one or more functions or one or more parts of functions that are shown by the below in conjunction with the Fig. The examples discussed in sections 14 and / or 15, which are programmable circuits (e.g., an FPGA), are to be implemented. In some examples, the machine-readable instructions cause an operation, task, etc., to be performed automatically in the real world. As used here, "automated" means without human intervention.

[0028] The program can be embodied in instructions (for example, software and / or firmware) stored on one or more non-volatile computer-readable and / or machine-readable storage media, such as a buffer, a magnetic storage device or magnetic storage disk (for example, a floppy disk, a hard disk drive (HDD), etc.), an optical storage device or optical storage disk (for example, a Blu-ray disc, a compact disc (CD), a digital versatile disc (DVD), etc.), a redundant array of independent disks (RAID), a register, a ROM, a solid-state drive (SSD), an SSD memory, non-volatile memory (for example, electrically erasable programmable read-only memory (EEPROM), flash memory, etc.), volatile memory (for example, random access memory (RAM) of any type, etc.).) and / or any other storage device or storage disk. The instructions of the non-volatile computer-readable and / or machine-readable medium can be programmed and / or executed by a programmable circuit located in one or more hardware devices; however, the entire program and / or parts thereof could alternatively be executed and / or instantiated by one or more hardware devices that are not part of the programmable circuit and / or are embodied in permanently assigned hardware. The machine-readable instructions can be distributed across multiple hardware devices and / or executed by two or more hardware devices (e.g., a server and a client hardware device). For example, the client hardware device can be executed by an endpoint client hardware device (e.g.,a hardware device associated with a human and / or machine user) or an inter-client hardware device gateway (e.g., a wireless access network (RAN)) that can facilitate communication between a server and an endpoint client hardware device. Similarly, the non-volatile, machine-readable storage medium can include one or more media. Furthermore, although the example program refers to the one in . Fig. The flowcharts illustrated in Figures 3-5 describe, or alternatively, a variety of other methods for implementing the exemplary model tuner circuit 205 are also described. Fig. 2. For example, the execution order of the flowchart blocks can be changed, and / or some of the described blocks can be modified, removed, or combined. Additionally or alternatively, some or all of the flowchart blocks can be implemented by one or more hardware circuits (e.g., processor circuits, discrete and / or integrated analog and / or digital circuits, an FPGA, an ASIC, a comparator, an operational amplifier, a logic circuit, etc.) structured to perform the corresponding operation without executing any software or firmware. The programmable circuits can be distributed across different network locations and / or locally distributed on one or more hardware devices (e.g., a single-core processor (e.g., a single-core CPU), a multi-core processor (e.g., a multi-core CPU, an XPU, etc.)).The programmable circuits can be, for example, a CPU and / or an FPGA located in the same package (e.g., in the same IC package or in two or more separate packages), one or more processors in a single machine, or multiple processors distributed across multiple servers in a server rack, multiple processors distributed across one or more server racks, etc., and / or any combination thereof.

[0029] The machine-readable instructions described here can be stored in one or more of the following formats: compressed format, encrypted format, fragmented format, compiled format, executable format, packaged format, etc. Machine-readable instructions, as described here, can be stored as data (e.g., computer-readable data, machine-readable data, one or more bits (e.g., one or more computer-readable bits, one or more machine-readable bits, etc.), a bitstream (e.g., a computer-readable bitstream, a machine-readable bitstream, etc.), or a data structure (e.g., as part(s) of instructions, code, representations of code, etc.) that can be used to generate, manufacture, and / or produce machine-executable instructions. For example, the machine-readable instructions can be fragmented and stored on one or more storage devices, disks, and / or computing devices (e.g.,Machine-readable instructions are stored on servers located at the same or different locations within a network or collection of networks (e.g., in the cloud, on edge devices, etc.). These instructions may require one or more of the following operations: installation, modification, adaptation, updating, combination, augmentation, configuration, decryption, decompression, unpacking, distribution, reassignment, compilation, etc., to make them directly readable, interpretable, and / or executable by a computing device and / or another machine.For example, the machine-readable instructions can be stored in multiple parts, which are individually compressed, encrypted and / or stored on separate computers, the parts, when decrypted, decompressed and / or combined, forming a set of computer-executable and / or machine-executable instructions that implement one or more functions and / or operations which together can form a program such as the one described here.

[0030] In another example, the machine-readable instructions might be stored in a state where they can be read by a programmable circuit, but require adding them to a library (e.g., a Dynamic Link Library (DLL)), a Software Development Kit (SDK), an Application Programming Interface (API), etc., to execute the machine-readable instructions on a dedicated computing device or other device. In yet another example, the machine-readable instructions might need to be configured (e.g., settings saved, data entered, network addresses recorded, etc.) before the machine-readable instructions and / or the corresponding program(s) can be executed, either wholly or partially.Thus, machine-readable, computer-readable and / or machine-readable media, as used here, can include instructions and / or one or more programs, regardless of the specific format or state of the machine-readable instructions and / or the one or more programs.

[0031] The machine-readable instructions described herein can be represented by any past, present, or future instruction language, scripting language, programming language, etc. For example, the machine-readable instructions can be represented using any of the following languages: C, C++, Java, C#, Perl, Python, JavaScript, HyperText Markup Language (HTML), Structured Query Language (SQL), Swift, etc.

[0032] As mentioned above, the exemplary operations of Fig. 3-5 are implemented using executable instructions (e.g., computer-readable and / or machine-readable instructions) stored on one or more non-volatile computer-readable and / or machine-readable media. As used herein, the terms non-volatile computer-readable medium, non-volatile computer-readable storage medium, non-volatile machine-readable medium, and / or non-volatile machine-readable storage medium are expressly defined to include any type of computer-readable storage device and / or storage disk, excluding signal propagation and transmission media.Examples of such non-volatile computer-readable media, non-volatile computer-readable storage media, non-volatile machine-readable media and / or non-volatile machine-readable storage media include optical storage devices, magnetic storage devices, a hard disk drive (HDD), flash memory, read-only memory (ROM), a CD, a DVD, a cache, RAM of any type, a register and / or any other storage device or storage disk in which information is stored for any duration (e.g., for extended periods, permanently, for short periods, for temporary buffering and / or intermediate storage of information).As used herein, the terms "non-volatile computer-readable storage medium" and "non-volatile machine-readable storage medium" are defined to include any physical (mechanical, magnetic, and / or electrical) hardware for storing information for a specified period of time, but excluding signal propagation and transmission media. Examples of non-volatile computer-readable storage devices and / or non-volatile machine-readable storage devices include random-access memory of any type, read-only memory of any type, solid-state memory, flash memory, optical disks, magnetic disks, floppy disk drives, and / or RAID (Redundant Array of Independent Disks) systems. As used herein, the term "device" refers to a physical structure such as mechanical and / or electrical equipment, hardware, and / or circuitry that is controlled by computer-readable instructions, machine-readable instructions, etc.can be configured and / or manufactured to execute computer-readable instructions, machine-readable instructions, etc., or not.

[0033] "Including" and "comprising" (and all forms and tenses thereof) are used here as open terms. It is understood that whenever a claim uses any form of "including" or "comprising" (e.g., encompasses, exhibits, encompassing, including, exhibiting, etc.) as a preamble or within any kind of claim statement, additional elements, terms, etc., may be present without falling outside the scope of protection of the claim or statement in question. When used in this document, the expression "at least" is used as the transitional term in, for example, a preamble of a claim, it is open in the same way as the terms "comprising" and "including" are open.The term "and / or," when used in a form such as A, B and / or C, refers to any combination or subset of A, B, C, such as (1) A alone, (2) B alone, (3) C alone, (4) A with B, (5) A with C, (6) B with C, or (7) A with B and with C. As used here in the context of describing structures, components, items, objects, and / or things, the phrase "at least one of A and B" refers to implementations that contain (1) at least one A, (2) at least one B, or (3) at least one A and at least one B. Similarly, as used here in the context of describing structures, components, items, objects, and / or things, the phrase "at least one of A or B" refers to implementations that contain (1) at least one A, (2) at least one B, or (3) at least one A and at least one B.As used here in the context of describing the execution or implementation of processes, instructions, actions, activities, and / or steps, the phrase "at least one of A and B" refers to implementations that include (1) at least one A, (2) at least one B, or (3) at least one A and at least one B. Similarly, the phrase "at least one of A or B," as used here in the context of describing the execution or implementation of processes, instructions, actions, activities, and / or steps, refers to implementations that include (1) at least one A, (2) at least one B, or (3) at least one A and at least one B.

[0034] As used herein, singular designations (e.g., "one," "a," "first," "second," etc.) do not preclude a plurality. The term "one" object, as used in this document, refers to one or more of this object. The terms "one" (or "one / s"), "one or more," and "at least one" are used interchangeably here. Furthermore, a plurality of means, elements, or actions, even if listed individually, may be performed, for example, by the same location or object. Additionally, although individual features may be included in different examples or claims, they may potentially be combined, and inclusion in different examples or claims does not imply that a combination of features is not feasible and / or advantageous.

[0035] Fig. Figure 3 is a flowchart that represents exemplary machine-readable instructions and / or exemplary operations 300 that can be executed, instantiated, and / or performed by a programmable circuit to implement the exemplary model tuner circuit of Fig. 2 to implement. The machine-readable instructions and / or operations 300 of Fig. 3 begin at block 305, where the sparsification initiator circuit 210 identifies the base model and proceeds with the base model sparsification at block 310, as in conjunction with Fig. 4 described in more detail. Once the model sparsification and / or model quantization is complete, the NLS trainer circuit 220 restores the model accuracy using a neural low-rank adapter search in block 315, as described in conjunction with Fig. 5 described in more detail. Following the restoration of model accuracy, the model generator circuit 230 in block 320 outputs the sparsified and / or sparsified and quantized fine-tuned architecture of the initial input base model. For example, the model generator circuit 230 outputs the final fused model resulting from the fusion of (1) a sparse base model and the appropriate adapter(s) or (2) a sparsified and quantized base model and the appropriate adapter(s), as described in conjunction with Fig. 6 described in more detail.

[0036] Fig. Figure 4 is a flowchart that represents exemplary machine-readable instructions and / or exemplary operations 310 that can be executed, instantiated and / or performed by a programmable circuit to implement the exemplary model tuner circuit 205 of Fig. 2, to perform sparsification and quantization of the base model. The machine-readable instructions and / or operations 310 of Fig. The 3 process begins in block 405, where the sparsification initiator circuit 210 receives pre-trained weights and assigns an arbitrary evaluation function (Ψ) in block 410, which is based, for example, on sampled feature input activations. The sparsification initiator circuit 210 determines the sparsified weight(s) (W) in block 415. p) based on the arbitrary evaluation function and a desired sparsity level. Subsequently, the quantization application circuit 215 in block 420 determines whether quantization should be performed. For example, if the quantization application circuit 215 determines that quantization is not required (e.g., when performing parameter-efficient fine-tuning of sparse models), the sparsification initiator circuit 210 outputs the sparse base model in block 422. In some examples, quantization is introduced as part of fine-tuning sparse and quantized models, or as part of fine-tuning sparse and quantized models using quantization and sparse-aware adapter merging. In such examples, the quantization application circuit 215 in block 425 applies layer-wise one-shot quantization.The quantization application circuit 215 then quantizes the sparsified weights with lower accuracy to reduce memory bandwidth problems in block 430, resulting in a sparsified and quantized base model in block 435.

[0037] Fig. Figure 5 is a flowchart that represents exemplary machine-readable instructions and / or exemplary operations 315 that can be executed, instantiated and / or performed by programmable circuits to implement the exemplary model tuner circuit 205 of Fig. 2 to implement the restoration of model accuracy using neural low-rank adapter search (NLS). The machine-readable instructions and / or operations 315 of Fig. The 3 examples begin in block 505, where the NLS trainer circuit 220 retrieves the sparsified and / or sparsified and quantized base model and, in block 510, trains search space configurations for elastic adapters. In some examples, the NLS trainer circuit 220 fine-tunes adapters by using supernetwork weight distribution techniques, as described in conjunction with Fig. 2 described. If, after NLS training, the weight fusion circuit 225 determines that sparsified weights and adapter weights are to be fused, the weight fusion circuit 225 generates a binary mask derived from an initial sparsification of the weight matrix (W). For example, as previously described in conjunction with Fig. 2, the weight fusion circuit 225 applies the binary mask to the adapters, which allows subsequent fusion of the adapters without loss of sparsity. In the example of Fig. In block 515, the NLS trainer circuit 220 retrieves the binary mask(s) generated from sparse weights in a base model. In the example of Fig. Block 520 retrieves the quantization application circuit 215, which retrieves zeros and scales associated with a quantized base model. In some examples, the model generator circuit 230 in Block 525 initiates the fine-tuning of the model(s). As part of the fine-tuning, for example, the weight fusion circuit 225 in Block 530 determines whether to initiate the fusion of the base model and adapter weights to avoid loss of dispersion after fine-tuning. If the weight fusion circuit 225 initiates the fusion of the base model and adapter weights, the controller proceeds to Block 535, where the weight fusion circuit 225 adjusts the sparse model weights and the sparse adapter weights as part of the parameter-efficient fine-tuning of sparse models (SparsePEFT).In some examples, the sparsification initiator circuit 210 sparsifies the adapter matrix during fine-tuning to achieve sparsity awareness in block 540.

[0038] Once the base model and adapter weights have been fused, and / or if the weight fusion circuit 225 determines that the fusion of the base model and adapter weights should not be initiated, the controller proceeds to block 545. For example, in block 545, the weight fusion circuit 225 determines whether the fusion of the base model and adapter weights with different numerical accuracies should be initiated. If the weight fusion circuit 225 determines to proceed with the fusion of the base model and adapter weights with different numerical accuracies, the quantization application circuit 215 performs quantization-sensitive sparsePEFT (QA-sparsePEFT) in block 550 to align low-precision model weights with float-precision adapter weights.For example, the quantization application circuit 215 performs quantization by using the sparse pre-trained weight(s) and the sparse adapter weight(s) as part of fine-tuning sparse and quantized models with quantization and sparse-conscious adapter fusion. In the example of . Fig. In block 555, model generator circuit 230 determines whether fine-tuning with frozen model weights and elastic adapters should be performed. If model generator circuit 230 in block 560 determines that fine-tuning of the model is complete, it outputs the fine-tuned model in block 565. If model generator 230 determines that further fine-tuning of the model is required based on the generated output, the control returns to block 530 until the fine-tuning process is complete.

[0039] Fig. Figure 6 illustrates exemplary pipeline configurations 600 that can be instantiated for efficient fine-tuning of large models, including (1) a first pipeline 610 for parameter-efficient fine-tuning of sparse and quantized models using elastic adapters, resulting in an unfused model and adapters, (2) a second pipeline 620 for parameter-efficient fine-tuning of sparse models using Sparse Awareness (SparsePEFT), which allows subsequent fusion of the model and adapters, and (3) a third pipeline 630 for parameter-efficient fine-tuning of sparse and quantized models using quantization and sparse-aware adapter merging. While LLM fine-tuning is frequently used to fit foundational and / or boundary models to a specific task or dataset, fine-tuning these large models is a resource-intensive undertaking.In the example of . Fig. 6. The fine-tuning of sparse and quantized models (SQFT) leads to fine-tuned large pretrained models (LPMs) obtained in an efficient multi-step approach that includes (1) sparsification with an optional reduction in numerical precision (e.g., quantization), (2) neural low-rank adapter search (NLS), and (3) sparse parameter-efficient fine-tuning (SparsePEFT) with optional quantization awareness (QA-SparsePEFT). In the example of Fig. 6. Alternative LPM compression and model fitting pipelines 610, 620, 630 can be selected depending on whether quantization and / or adapter fusion with sparse awareness is to be applied. For example, the choice between the degree of sparsity and the application of quantization may depend on the specific use case (e.g., task requirements, resource constraints, etc.), including the expected trade-offs between model performance, inference speed, and / or storage efficiency.

[0040] In the example of Fig. 6 includes the first pipeline 610 for the fine-tuning of sparse and quantized models (SQFT) and the identification of a base model 102 by the sparsification initiator circuit 210 of Fig. 2, which performs model sparsification. The quantization application circuit 215 quantizes the sparse model, resulting in a sparse and quantized base model 604, which is processed by the NLS trainer circuit 220 to obtain unfused adapters. However, the sparsified and quantized base model 604 and the adapter(s) 606 remain unfused in the first pipeline 610, while the effective fusion of the adapter(s) 606 into the sparse and / or quantized model(s) without loss of sparsity is shown in conjunction with the second and third pipeline(s) 620, 630.The second pipeline 620 (SparsePEFT), which contains components of the first pipeline 610 (SQFT), addresses, for example, several limitations in existing parameter-efficient fine-tuning approaches for sparse and quantized models, including reducing the cost of fine-tuning, effectively fusing adapters into the sparse model without loss of sparsity, and effectively fusing components that operate at different numerical accuracies.In the example of the second pipeline 620, after the sparse base model 602 has been generated and the NLS trainer circuit 220 performs training to obtain non-fused adapters, the weight fusion circuit 225 identifies a binary mask from the sparse weights and applies the binary mask to the adapters, enabling a subsequent fusion of the adapters 606 without loss of sparsity, resulting in the output of a sparsified, fine-tuned architecture 608 of the original base model.

[0041] Although SparsePEFT can effectively preserve the sparsity of the model, this pipeline presents an additional challenge when fusion with quantized models, primarily because the adapter and the pre-trained weights must have identical numerical precision. In the example of Fig. Section 6, the third pipeline 630, represents an extension of SparsePEFT for sparsely quantized models. For example, the quantization application circuit 215 identifies the sparsified weights 622 and the corresponding zeros and scales 624 of the sparsified quantized weights that are shared with the adapter(s) 606. Thus, the quantization application circuit 215 quantizes the adapters 606 with the shared fixed zeros and scales, resulting in quantization-aware fine-tuning that yields a sparsified and quantized fine-tuned architecture 628 of the original base model.

[0042] Fig. Figure 7 illustrates an exemplary comparison of 700 known low-rank adapters (LoRA) with elastic adapters connected to a neural low-rank search (NLS) disclosed herein. For example, LoRA adapters are connected to parameter-efficient fine-tuning techniques used to modify a selection of linear layers, each containing a weighting matrix (e.g., W0 ∈ ℝ). d×k exhibit )). In some examples, LoRA adapters (e.g., a first adapter A, where A ℝ) are used. r × k and a second adapter B, where B ∈ ℝ d × r ) is used to extend the linear projection of the layer (e.g., extending the projection from H = W0X to H = W0X + BAX), so that the weights (W0) of the original model can remain frozen and only the inserted adapters are fine-tuned. While the original LoRA adapters from Fig. While 7 involves the use of a first rank of 705, where the rank is kept at the same level (e.g., r = 32), NLS-based LoRA adapters are elastic adapters with variable ranks, so the ranks (e.g., second rank 710, third rank 715) can be changed as needed. Instead of a fixed value for the rank (r), the NLS trainer circuit 220, for example, applies elastic configurations depending on the activation of the corresponding sub-adapter, as in the context of Fig. 2 described in more detail. The elastic LoRA adapters that use the variable ranking described here outperform the traditional LoRA adapters.

[0043] Fig. Figure 8 shows an exemplary overview of 800 of the fine-tuning of pre-trained LLMs based on model sparsification, the restoration of base model accuracy using the NLS adapters of Fig. 7, and identification of a sparsified, finely tuned architecture based on a subadapter search. In the example of Fig. 8 identifies the energy saving initiator circuit 210 from Fig. 2 for a given matrix (W) 805 the sparsified weights (W p ) 810. The sparsification initiator circuit 210 receives the original weighting matrix 815 of the dense model weights and applies an arbitrary evaluation function (Ψ) 820 to select the most critical weights, resulting in a sparse output matrix 825. The quantization application circuit 215 performs the quantization of the sparsified weights, while the NLS trainer circuit 220 combines the pre-trained weights 830 of the sparse model with a collection of elastic adapters 835 trained with variable configurations. As in the context of Fig. As described in Figure 7, the NLS trainer circuit applies 220 elastic configurations of adapters determined on the basis of the activation of a corresponding sub-adapter 840, such that the rank is varied to facilitate the identification of optimal adapter configurations from a space of elastic adapter configurations.

[0044] Fig. Figure 9 illustrates the sparsified parameter-efficient fine-tuning (SparsePEFT) 900 using a binary mask derived from sparsified weights. For example, as in the context of Fig. As described in section 2, the weight fusion circuit 225 identifies a binary mask from the sparsified weights and applies the binary mask to the adapters, enabling subsequent fusion of the adapters without loss of sparsity. In the example of Fig. 9 The sparsification initiator circuit 210 receives a matrix (W) 905 and identifies sparsified weights (W p ) 910. The weight fusion circuit 225 performs an adapter weight identification 920 by applying a binary mask (M) 925 to convert the adapter matrix (e.g., designated BA 930) into an adapter weight (L p ) 915 to save, so that the adapter weight 915 using L p = (BA) ⊙ M is identified. The weight fusion circuit 225 fuses the sparsified weights (W). p ) 910 and the adapter weight (L p ) 915, without sacrificing the sparsity induced early in the compression pipeline.

[0045] Fig. Figure 10 illustrates an example of reducing the number of parameters 1000 required for fine-tuning an LLM, achieving higher accuracy with the methods disclosed here than with known sparse fine-tuning methods. In the example of Fig. 10. The methods and devices disclosed herein for fine-tuning (e.g., the proposed solution 1002) are compared with a known method for fine-tuning models (e.g., the sparse fine-tuning 1005 of Neural Magic). The SQFT pipeline (e.g., the first pipeline 610 in Fig. 6) is validated, for example, on three experimental settings containing Grade School Math 8K (GSM8K), as in the example of Fig. 10 shown. The proposed solution 1002 is compared with the sparse fine-tuning 1005 based on measurements of the GSM8K accuracy (%) 1010, the ratio of trainable parameters 1015, and the sparsity (%) 1020. In the example of Fig. 10. The proposed solution 1002 requires approximately 83.1 times fewer parameters for fine-tuning, while achieving higher accuracy compared to the sparse fine-tuning 1005.

[0046] Fig. Figure 11A illustrates exemplary results 1100 for the evaluation of the fine-tuning of a first exemplary model 1102 (e.g., LLAMA-3-8B) using known fine-tuning compared to approaches disclosed herein (e.g., fine-tuning of sparsified and quantized models (SQFT), SQFT in combination with SparsePEFT, and SQFT in combination with SparsePEFT including quantization and sparse-aware adapter merging (QA-SparsePEFT)). In the examples presented here, SQFT is evaluated on several large, pre-trained models and datasets that are state of the art. For example, SQFT is evaluated using Llama-3-8B and Phi-3-Mini-4K-Instruct. In the examples presented here, SQFT is validated in three experimental setups: 1) Grade School Math 8K (GSM8K), 2) Mathematical reasoning with instruction matching, including three datasets of mathematical reasoning (e.g.GSM8K, Mathematical Word Problems, Simple Variations of Arithmetic Word Problems (SVAMP)), and 3) General Reasoning datasets (e.g., Boolean Questions (BoolQ), Physical Interaction: Question Answering (PIQA), Largescale Winograd Schema Challenge (WinoGrande), AI2 Reasoning Challenges (Arc-e, Arc-c), and Open Book Question Answering (OBQA)). In the examples disclosed here, a comparative analysis of the results obtained using the disclosed pipelines (e.g., first pipeline 610, second pipeline 620, and / or third pipeline 630 of ) can be performed. Fig. 6) is presented and compared with vanilla low-rank adapters (LoRA), Shears (e.g., a parameter-efficient fine-tuning method for sparse models), and GPTQ + LoRA, with all methods running in the same environment and with the same configuration. In the examples disclosed here, SQFT also uses the Wanda implementation as the standard method for sparsification and GPTQ in Huggingface 3 for quantizing the LPMs and adapters.

[0047] In the example of Fig. 11A The evaluation of the pipelines (e.g., SQFT and SQFT + QA-SparsePEFT) is performed with Llama-3-8B, assessing accuracy in a dense mode and after induction of 50% sparsity without fine-tuning on the GSM8K dataset. In the example of Fig. Section 11A includes the results on the degree of induced sparsity (1104), the pipeline method selection (1106), the fusionability results (1108), the final precision results (1110), and the percentage results of the GSM8K test accuracy (1112). Thus, SQFT increases the model accuracy from 12.5% ​​to 52.5% without quantization, while adapter fusion is possible without loss of sparsity (using SparsePEFT). Further integration of quantization into the pipeline results in a slight decrease in accuracy to 50.2% when adapter fusion adaptation is enabled (using QA-SparsePEFT). Furthermore, SQFT with SparsePEFT and QA-SparsePEFT demonstrate comparable performance to the corresponding non-fusionable approaches. These results suggest that SQFT with SparsePEFT (e.g.,QA-SparsePEFT effectively addresses the limitation of the merging problem that arises when fine-tuning adapters in sparse models (or sparse and quantized models) without compromising accuracy. Furthermore, comparisons between LoRA and SQFT using SparsePEFT (or Shears), as well as between GPTQ + LoRA and SQFT using QA-SparsePEFT without adapter merging, demonstrate the superior performance of NLS (e.g., using elastic ranking) compared to LoRA (e.g., using a fixed rank).

[0048] Fig. Figure 11B shows exemplary ablation study results 1160 for fine-tuning using known low-rank adapters (LoRA) compared to elastic adapters using neural low-rank search (NLS) by Fig. 7 in the evaluation of the fine-tuning disclosed herein, including SQFT with SparsePEFT and SQFT with quantizable SparsePEFT. In the example of Fig. 11B includes the results for model 1102, a level of induced sparsity 1104, a selection of the pipeline method 1106, the results of the fusionability 1108, the final precision results 1110, the selection of the fine-tuning approach 1165, and the results of the test accuracy in percent 1112 for specific datasets. In the example of Fig. Figure 11B demonstrates significantly improved accuracy performance with NLS compared to LoRA across all SQFT pipelines and various sparsity levels. Ablation studies with 30%, 50%, and 70% sparsity, for example, highlight the advantages of elastic adapters (e.g., using NLS) that enhance the performance of the SQFT pipeline(s) disclosed here.

[0049] Fig. Figure 12 shows an example of the cost analysis 1200 for various pipelines related to model fine-tuning, including an evaluation of model storage, fine-tuning time, and accuracy. In the example of Fig. The identifier 1205 represents different pipelines, where ID = 1 stands for LoRA / Shears, ID = 2 for SQFT, ID = 3 for SQFT + SparsePEFT, and ID = 4 for SQFT + QA-SparsePEFT. The different versions of the SQFT pipelines have varying costs, allowing users to fine-tune their configurations according to their budget. The properties of each pipeline configuration are shown in the example of... Fig. 12 described in detail, based on whether adapters can be merged (e.g., fusionability indicator 1210), as well as on the final accuracy of the base model and the adapters (e.g., final accuracy 1215) and the cost of each configuration. In the example of Fig. In section 12, model memory (1220), fine-tuning time (1225), fine-tuning memory (1230), inference acceleration (1235), inference memory (1240), and accuracy (1245) are compared for all pipelines. Regarding model memory, inference acceleration, and memory, fusion is expected to perform better than non-fusion due to the overhead of the non-fused adapters, and the quantization mode is expected to perform better than the full-precision mode. As for fine-tuning time, the fusion method is slightly slower than the non-fusion method due to the additional mask and adapter calculations. Based on these results, SQFT with SparsePEFT is an optimal choice for full-precision mode without compromising accuracy.If memory usage during fine-tuning is a priority for the quantization mode, Vanilla SQFT is the optimal choice due to its use of the quantized model and the low overhead associated with various precision adapters. Otherwise, SQFT with QA-SparsePEFT produces a more efficient model than the other fine-tuning pipelines.

[0050] Fig. Figure 13 is a block diagram of an exemplary programmable circuit platform 1300, structured to represent the exemplary machine-readable instructions and / or the exemplary operations of the Fig. 3-5 executes and / or instantiates to run the exemplary model tuner circuit 205 from Fig. 2 to implement. For example, the Processor Platform 1300 could be a server, a personal computer, a workstation, a self-learning machine (e.g., a neural network), a mobile device (e.g., a mobile phone, a smartphone, a tablet such as an iPad™), a personal digital assistant (PDA), an internet device, a DVD player, a CD player, a digital video recorder, a Blu-ray player, a game console, a personal video recorder, a set-top box, a headset (e.g., an augmented reality (AR) headset, a virtual reality (VR) headset, etc.), or any other portable device or any other type of computing device.

[0051] The programmable circuit platform 1300 of the illustrated example contains a programmable circuit 1312. The programmable circuit 1312 of the illustrated example is hardware. For example, the programmable circuit 1312 can be implemented by one or more integrated circuits, logic circuits, FPGAs, microprocessors, CPUs, GPUs, DSPs, and / or microcontrollers of any family or manufacturer. The programmable circuit 1312 can be implemented by one or more semiconductor devices (e.g., silicon-based). In this example, the processor circuit 1312 implements the sparsification initiator circuit 210, the quantization application circuit 215, the NLS trainer circuit 220, the weight fusion circuit 225, and / or the model generator circuit 230.

[0052] The programmable circuit 1312 of the illustrated example includes a local memory 1313 (e.g., a cache, registers, etc.). The programmable circuit 1312 of the illustrated example communicates via a bus 1318 with a main memory, which comprises a volatile memory 1314 and a non-volatile memory 1316. The volatile memory 1314 can be accessed by Synchronous Dynamic Random Access Memory (SDRAM), Dynamic Random Access Memory (DRAM), or RAMBUS. ® Dynamic Random Access Memory (RDRAM ®The non-volatile memory 1316 can be implemented as flash memory and / or any other desired type of storage device. Access to the main memory 1314, 1316 of the illustrated example is controlled by a memory controller 1317. In some examples, the memory controller 1317 can be implemented by one or more integrated circuits, logic circuits, microcontrollers of any desired family or manufacturer, or any other type of circuit to manage the data flow to and from the main memory 1314, 1316.

[0053] The programmable circuit platform 1300 of the illustrated example also includes the interface circuit 1320. The interface circuit 1320 can be implemented in hardware in accordance with any interface standard, such as an Ethernet interface, a USB (Universal Serial Bus) interface, or Bluetooth. ® -interface, an NFC interface (Near Field Communication), a PCI interface (Peripheral Component Interconnect) and / or a PCIe interface (Peripheral Component Interconnect Express).

[0054] In the example shown, one or more input devices 1322 are connected to the interface circuit 1320. The input device(s) 1322 enable(s) a user (e.g., a human user, a machine user, etc.) to input data and / or instructions into the programmable circuit 1312. The input device(s) 1322 can be implemented, for example, by an audio sensor, a microphone, a camera (still image or video), a keyboard, a button, a mouse, a touchscreen, a trackpad, a trackball, an isopoint, and / or a speech recognition system.

[0055] One or more output devices 1324 are also connected to the interface circuit 1320 of the illustrated example. The output devices 1324 can be implemented, for example, by display devices (e.g., a light-emitting diode (LED), an organic light-emitting diode (OLED), a liquid crystal display (LCD), a cathode ray tube (CRT) display, an in-place switching (IPS) display, a touchscreen, etc.), a tactile output device, a printer, and / or a loudspeaker. The interface circuit 1320 of the illustrated example thus typically includes a graphics driver card, a graphics driver chip, and / or a graphics processing unit (GPU) circuit.

[0056] The interface circuit 1320 of the illustrated example also includes a communication device, such as a transmitter, a receiver, a transceiver, a modem, a residential gateway, a wireless access point, and / or a network interface, to facilitate the exchange of data with external machines (e.g., computing devices of any type) via a network 1326. Communication can be effected, for example, through an Ethernet connection, a digital subscriber line (DSL) connection, a telephone line connection, a coaxial cable system, a satellite system, a wireless line-of-sight system, a mobile phone system, an optical connection, etc.

[0057] The programmable circuit platform 1300 of the illustrated example also includes one or more mass storage devices 1328 for storing software and / or data. Examples of such mass storage devices 1328 include magnetic storage devices (e.g., floppy disks, drives, HDDs, etc.), optical storage devices (e.g., Blu-ray discs, CDs, DVDs, etc.), RAID systems, and / or solid-state storage disks or devices such as flash memory devices and / or SSDs.

[0058] The machine-executable instructions 1332, which are replaced by the machine-readable instructions of Fig. 3-5 can be implemented, can be stored in the mass storage device 1328, in volatile memory 1314, in non-volatile memory 1316 and / or on at least one non-volatile computer-readable storage medium such as a CD or DVD.

[0059] Fig. Figure 14 is a block diagram of an exemplary implementation of the programmable circuit 1312 from Fig. 13. In this example, the programmable circuit 1312 is used. Fig. 13 is implemented by a microprocessor 1400. The microprocessor 1400 can, for example, be a general-purpose microprocessor (e.g., a general-purpose microprocessor circuit). The microprocessor 1400 executes some or all of the machine-readable instructions of the flowcharts of Fig. 3-5 out to complete the circuit of Fig. 2. to effectively instantiate logic circuits in order to perform the operations corresponding to these machine-readable instructions. In some of these examples, the circuit is made up of Fig. 2. Instantiated by the hardware circuitry of the microprocessor 1400 in combination with the instructions. For example, the microprocessor 1400 can implement a multi-core hardware circuit, such as a CPU, DSP, GPU, XPU, etc. Although it can contain any number of exemplary cores 1402 (e.g., 1 core), the microprocessor 1400 in this example is a multi-core semiconductor device containing N cores. The cores 1402 of the microprocessor 1400 can operate independently or can work together to execute machine-readable instructions. For example, machine code corresponding to a firmware program, an embedded software program, or a software program can be executed by one of the cores 1402 or can be executed by several of the cores 1402 at the same or different times.In some examples, the machine code corresponding to the firmware program, the embedded software program, or the software program is divided into threads and executed in parallel by two or more of the 1402 cores. The software program can correspond to a section or all of the machine-readable instructions and / or operations defined by the flowcharts of [missing information]. Fig. 3-5 are represented.

[0060] The 1402 cores can communicate through a first example bus 1404. In some examples, the first bus 1404 can implement a communication bus to enable communication between one or more of the 1402 cores. For example, the first bus 1404 can implement an I2C (Inter-Integrated Circuit) bus and / or an SPI (Serial Peripheral Interface) bus and / or a PCI bus and / or a PCIe bus. Additionally or alternatively, the first bus 1404 can implement any other type of computational or electrical bus. The 1402 cores can receive data, instructions, and / or signals from one or more external devices through an example interface circuit 1406. The 1402 cores can output data, instructions, and / or signals through the interface circuit 1406 to the one or more external devices.Although the cores 1402 of this example include exemplary local memory 1420 (for example, Level 1 (L1) cache, which may be divided into an L1 data cache and an L1 instruction cache), the microprocessor 1400 also includes exemplary shared memory 1410 (for example, Level 2 (L2) cache) that can be shared by the cores for high-speed access to data and / or instructions. Data and / or instructions can be transferred (e.g., shared) by writing to and / or reading from the shared memory 1410. The local memory 1420 of each of the cores 1402 and the shared memory 1410 can be part of a hierarchy of memory devices, including multiple levels of cache memory and main memory (e.g., main memory 1314, 1316 of ). Fig. 13) Typically, higher storage levels in the hierarchy have lower access times and smaller storage capacities than lower storage levels. Changes at the various levels of the cache hierarchy are managed (e.g., coordinated) by a cache coherence policy.

[0061] Each 1402 core can be referred to as a CPU, DSP, GPU, or any other type of hardware circuit. Each 1402 core comprises a control unit circuit 1414, an arithmetic and logic (AL) circuit (sometimes called an ALU) 1416, several registers 1418, the L1 cache 1420, and a second example bus 1422. Other structures may be present. For example, each 1402 core may include a vector unit circuit, a SIMD (Single Instruction Multiple Data) unit circuit, an LSU (Load / Store Unit) circuit, a branch / jump unit circuit, a floating-point unit (FPU), and so on. The control unit circuit 1414 comprises semiconductor-based circuits designed to control (e.g., coordinate) the movement of data within the corresponding 1402 core.The AL circuit 1416 comprises semiconductor-based circuits structured to perform one or more mathematical and / or logical operations on the data within the corresponding core 1402. In some examples, the AL circuit 1416 performs integer operations. In other examples, the AL circuit 1416 also performs floating-point operations. In still other examples, the AL circuit 1416 may include a first AL circuit that performs integer-based operations and a second AL circuit that performs floating-point operations. In some examples, the AL circuit 1416 may be referred to as an arithmetic logic unit (ALU).

[0062] The 1418 registers are semiconductor-based structures for storing data and / or instructions, such as the results of one or more of the operations performed by the AL circuit 1416 of the corresponding 1402 core. The 1418 registers can include, for example, vector registers, SIMD registers, general-purpose registers, flag registers, segment registers, machine-specific registers, instruction pointer registers, control registers, debug registers, memory management registers, machine check registers, etc. The 1418 registers can be arranged in a bank, as shown in Fig. Figure 14 shows the following. Alternatively, the registers 1418 can be organized in any other arrangement, format, or structure, including distribution within the core 1402 to reduce access time. The second bus 1422 can be implemented by at least one I2C bus, one SPI bus, one PCI bus, or one PCIe bus.

[0063] Each Core 1402 and / or, more generally, the Microprocessor 1400 may include additional and / or alternative structures to those shown and described above. For example, it may include one or more clock circuits, one or more power supplies, one or more power gates, one or more cache home agents (CHAs), one or more converged / common mesh stops (CMSs), one or more sliders (e.g., barrel sliders), and / or other circuitry. The Microprocessor 1400 is a semiconductor device manufactured to include many transistors interconnected to implement the structures described above in one or more integrated circuits (ICs) contained within one or more packages.

[0064] The Microprocessor 1400 can contain and / or work in conjunction with one or more accelerators (e.g., accelerator circuits, hardware accelerators, etc.). In some examples, accelerators are implemented by logic circuitry to perform certain tasks faster and / or more efficiently than is possible with a general-purpose processor. Examples of accelerators include ASICs and FPGAs, such as those discussed herein. A GPU, DSP, and / or other programmable device can also be an accelerator. Accelerators can be integrated into the Microprocessor 1400, in the same chip package as the Microprocessor 1400, and / or in one or more packages separate from the Microprocessor 1400.

[0065] Fig. Figure 15 is a block diagram of another exemplary implementation of the programmable circuit of Fig. 13. In this example, the programmable circuit 1312 is implemented by the FPGA circuit 1500. For example, the FPGA circuit 1500 can be implemented by an FPGA. The FPGA circuit 1500 can be used, for example, to perform operations that would otherwise be performed by the exemplary microprocessor 1400. Fig. 14 operations could be performed, which executes the corresponding machine-readable instructions. However, after configuration, the FPGA 1500 circuit instantiates the operations and / or functions corresponding to the machine-readable instructions in the hardware and can therefore often execute the operations / functions faster than a general-purpose microprocessor executing the corresponding software.

[0066] More precisely, in contrast to the microprocessor 1400 described above from Fig. 14 (which is a universal device that can be programmed to execute some or all of the machine-readable instructions provided by the flowcharts of Fig. 3-5 are represented, but their intermediate connections and logic circuits are fixed after manufacturing) the FPGA circuit 1500 of the example of Fig. 15 intermediate connections and logic circuits that can be configured, structured, programmed and / or interconnected in various ways after manufacturing, for example to implement some or all of the operations / functions that correspond to the machine-readable instructions provided by the flowchart of Fig. 3-5 are represented, to instantiate. In particular, the FPGA 1500 can be viewed as an array of logic gates, interconnects, and switches. The switches can be programmed to change how the logic gates are interconnected by the interconnects, effectively forming one or more dedicated logic circuits (unless and until the FPGA 1500 circuit is reprogrammed). The configured logic circuits allow the logic gates to interact in different ways to perform different operations on data received by the input circuit. These operations may correspond to some or all of the instructions (e.g., software and / or firmware) shown in the flowcharts of Fig. 3-5 are shown. Therefore, the FPGA circuits 1500 can be designed and / or structured to perform some or all of the operations / functions that correspond to the machine-readable instructions of the flowcharts of Fig. 3-5 correspond to effectively instantiating dedicated logic circuits to perform the operations / functions corresponding to these software instructions in a dedicated manner analogous to an ASIC. Therefore, the FPGA 1500 circuit can perform the operations corresponding to some or all of the machine-readable instructions of Fig. 3-5, perform faster than the universal microprocessor can execute them.

[0067] In the example of Fig. 15. The FPGA 1500 circuit is configured and / or structured in response to being programmed (and / or reprogrammed one or more times) based on a binary file. In some examples, the binary file may be compiled and / or generated based on instructions in a hardware description language (HDL) such as Lucid, Very High Speed ​​Integrated Circuits (VHSIC) Hardware Description Language (VHDL), or Verilog. For example, a user (e.g., a human user, a machine user, etc.) may write code or a program corresponding to one or more operations / functions in an HDL; the code / program may be translated into a low-level language as needed; and the code / program (e.g., the code / program in the low-level language) may be converted into the binary file (e.g., by a compiler, a software application, etc.). In some examples, the FPGA 1500 circuit may be configured and / or structured based on instructions in a hardware description language (HDL) such as Lucid, Very High Speed ​​Integrated Circuits (VHSIC), or Verilog. Fig. 15 access and / or load the binary file to control the FPGA circuit 1500 from Fig. 15 to cause, configure, and / or structure the data to perform one or more operations / functions. For example, the binary file can be implemented by a bitstream (e.g., one or more computer-readable bits, one or more machine-readable bits, etc.), data (e.g., computer-readable data, machine-readable data, etc.), and / or machine-readable instructions to which the FPGA 1500 circuit can respond. Fig. 15 can access the configuration and / or structuring of the FPGA circuit 1500 of Fig. 15 or of a part or several parts thereof.

[0068] In some examples, the binary file is compiled, generated, transformed, and / or otherwise output by a unified software platform used for programming FPGAs. The unified software platform can, for example, translate first instructions (e.g., code or a program) corresponding to one or more operations / functions in a high-level language (e.g., C, C++, Python, etc.) into second instructions corresponding to the one or more operations / functions in an HDL. In some of these examples, the binary file is compiled, generated, and / or otherwise output by the unified software platform based on the second instructions. In some examples, the FPGA circuit 1500 of the Fig. 15 access and / or load the binary file to control the FPGA circuit 1500 from Fig. 15 to cause, configure, and / or structure the data to perform one or more operations / functions. For example, the binary file can be implemented by a bitstream (e.g., one or more computer-readable bits, one or more machine-readable bits, etc.), data (e.g., computer-readable data, machine-readable data, etc.), and / or machine-readable instructions to which the FPGA 1500 circuit can respond. Fig. 15 can access the configuration and / or structuring of the FPGA circuit 1500 of Fig. 15 or of a part or several parts thereof.

[0069] The FPGA circuit 1500 from Fig. Reference 15 includes an exemplary input / output (I / O) circuit 1502 for receiving and / or outputting data to / from an exemplary configuration circuit 1504 and / or external hardware 1506. The configuration circuit 1504 can, for example, be implemented by an interface circuit capable of receiving a binary file, which can be implemented by a bitstream, data, and / or machine-readable instructions to configure the FPGA circuit 1500 or parts thereof. In some examples, the configuration circuit 1504 can receive the binary file from a user, a machine (e.g., a hardware circuit (such as a programmable or dedicated circuit) capable of implementing an artificial intelligence / machine learning (AI / ML) model to generate the binary file), etc., and / or any combination thereof.In some examples, the external hardware 1506 can be implemented by an external hardware circuit. For example, the external hardware 1506 can be implemented by the microprocessor 1400. Fig. 14 will be implemented.

[0070] The FPGA circuit 1500 also includes an array consisting of an example logic gate circuit 1508, several example configurable interconnects 1510, and an example memory circuit 1512. The logic gate circuit 1508 and the configurable interconnects 1510 are configurable to instantiate one or more operations / functions that perform at least some of the machine-readable instructions of Fig. 3-5 and / or desired operations correspond to the following. Fig. The 1508 logic gate circuit shown is manufactured in blocks or groups. Each block contains semiconductor-based electrical structures that can be configured into logic circuits. In some examples, the electrical structures include logic gates (e.g., AND gates, OR gates, NOR gates, etc.) that provide basic building blocks for logic circuits. Electrically controllable switches (e.g., transistors) are present in each of the 1508 logic gate circuits to allow the configuration of the electrical structures and / or logic gates to form circuits for performing desired operations / functions. The 1508 logic gate circuit may include other electrical structures, such as lookup tables (LUTs), registers (e.g., flip-flops or latches), multiplexers, etc.

[0071] The configurable intermediate connections 1510 of the illustrated example are conductive paths, tracks, vias or the like, which may have electrically controllable switches (e.g. transistors) whose state can be changed by programming (e.g. using an HDL instruction language) to enable or disable one or more connections between one or more of the logic gate circuits 1508 in order to program desired logic circuits.

[0072] The memory circuit 1512 of the illustrated example is structured to store one or more results of the one or more operations performed by corresponding logic gates. The memory circuit 1512 can be implemented using registers or similar mechanisms. In the illustrated example, the memory circuit 1512 is distributed among the logic gate circuit 1508 to facilitate access and increase execution speed.

[0073] The exemplary FPGA circuit 1500 from Fig. Figure 15 also includes an exemplary dedicated operating circuit 1514. In this example, the dedicated operating circuit 1514 includes a special circuit 1516 that can be called upon to implement frequently used functions, thus avoiding the need to program these functions locally. Examples of such a special circuit 1516 include a memory (e.g., DRAM) control circuit, a PCIe control circuit, a clock circuit, a transceiver circuit, a memory, and a multiplier-accumulator circuit. Other types of special circuits may be present. In some examples, the FPGA circuit 1500 may also include an exemplary programmable general-purpose circuit 1518, such as an exemplary CPU 1520 and / or an exemplary DSP 1522. Another programmable general-purpose circuit 1518 may be present additionally or alternatively, such as a GPU, an XPU, etc., which can be programmed to perform other operations.

[0074] Although Fig. 14 and Fig. 15 Two exemplary implementations of the programmable circuit 1312 from Fig. As illustrated in Figure 13, many other approaches are considered. For example, an FPGA circuit can contain an integrated CPU, such as one or more of the example CPUs 1520 from [reference missing]. Fig. 15. Therefore, the programmable circuit 1312 can be made from Fig. 13 additionally by combining at least the exemplary microprocessor 1400 from Fig. 14 and the exemplary FPGA circuit 1500 from Fig. 15 be implemented. In some such hybrid examples, one or more cores may be 1502 of Fig. 15. Execute a first section of the machine-readable instructions, which are defined by one or more flowcharts of the Fig. 3-5 are shown to perform one or more initial operations / functions, with the FPGA circuit 1500 of Fig. 15 may be designed and / or structured to perform one or more second operations / functions corresponding to a second section of the machine-readable instructions provided by the flowcharts of the Fig. 3-5 are represented, and / or an ASIC can be configured and / or structured to perform one or more third operations / functions corresponding to a third section of the machine-readable instructions represented by the flowcharts of the Fig. 3-5 are shown.

[0075] It goes without saying that some or all of the circuits of Fig. 2. Therefore, they can be instantiated at the same time or at different times. For example, identical and / or different parts of the 1400 microprocessor can be instantiated. Fig. 14 can be programmed to execute parts of the machine-readable instructions simultaneously and / or at different times. In some examples, one or more identical and / or different parts of the FPGA circuit 1500 can be used. Fig. 15. be designed and / or structured to perform operations / functions according to one or more parts of machine-readable instructions at the same and / or different times.

[0076] In some examples, some or all of the circuits of Fig. 2. For example, they can be instantiated in one or more threads that are executed simultaneously and / or sequentially. For example, the 1400 microprocessor from Fig. Execute 14 machine-readable instructions in one or more threads, which are executed concurrently and / or sequentially. In some examples, the FPGA circuit can 1500 of Fig. 15. be configured and / or structured to execute operations / functions simultaneously and / or sequentially. Furthermore, in some examples, some or all of the circuits of Fig. 2 be implemented in one or more virtual machines and / or containers running on the 1400 microprocessor Fig. 14 will be executed.

[0077] In some examples, the programmable circuit 1312 can be used by Fig. 13 are located in one or more enclosures. For example, the 1400 microprocessor from Fig. 14 and / or the FPGA circuit 1500 of Fig. 15 in one or more enclosures. In some examples, an XPU can be provided by the programmable circuit 1312 from Fig. 13 can be implemented, which can be housed in one or more packages. The XPU can, for example, be a CPU (e.g., the 1400 microprocessor from Fig. 14, the CPU 1520 from Fig. 15 etc.) in a housing, a DSP (e.g. the DSP 1522 from Fig. 15) in another package, a GPU in another package and an FPGA (e.g. the FPGA 1500 circuit from Fig. 15) contained in yet another housing.

[0078] A block diagram illustrating an exemplary software distribution platform 1605 to distribute software such as the exemplary machine-readable instructions 1332 from Fig. 13. Distributing to other hardware devices (e.g., hardware devices owned by third parties and / or operated by the owner and / or operator of the software distribution platform) is in Fig. Figure 16 illustrates the exemplary software distribution platform 1605, which can be implemented by any computer server, data facility, cloud service, etc., capable of storing software and transferring it to other computing devices. These third parties can be customers of the entity that owns and / or operates the software distribution platform 1605. The entity that owns and / or operates the software distribution platform 1605 can be, for example, a developer, vendor, and / or licensor of software such as the exemplary machine-readable instructions 1332. Fig. 13. Third parties may be consumers, users, retailers, OEMs, etc., who acquire and / or license and / or resell and / or sublicense the software for use. In the illustrated example, the software distribution platform 1305 includes one or more servers and one or more storage devices. The storage devices store the machine-readable instructions 1332, which correspond to the exemplary machine-readable instructions of Fig. 3-5 can correspond to, as described above. The one or more servers of the exemplary software distribution platform 1305 communicate with an exemplary network 1310, which can correspond to any one or more of the Internet and / or any of the exemplary networks described above. In some examples, the one or more servers respond to requests to send the software to a requesting party as part of a commercial transaction. Payment for the delivery, sale, and / or licensing of the software can be processed through the one or more servers of the software distribution platform and / or through a third-party payment agent. The servers enable purchasers and / or licensors to download the machine-readable instructions 1332 from the software distribution platform 1605. For example, the software, which corresponds to the exemplary machine-readable instructions of the Fig. 3-5 can be downloaded to the exemplary programmable circuit platform 1300, which is intended to execute the machine-readable instructions 1332 to control the model tuner circuit 205 of Fig. 2 to implement. In some examples, one or more servers of the Software Distribution Platform 1605 periodically provide updates for the software (e.g., the exemplary machine-readable instructions 1332 of Fig. 13) to, transmit and / or enforce such to ensure that improvements, patches, updates, etc. are distributed and applied to the software on end-user devices. Although referred to above as software, the distributed “software” could alternatively be firmware.

[0079] "Including" and "comprising" (and all forms and tenses thereof) are used here as open terms. It is understood that whenever a claim uses any form of "including" or "comprising" (e.g., encompasses, exhibits, encompassing, including, exhibiting, etc.) as a preamble or within any kind of claim statement, additional elements, terms, etc., may be present without falling outside the scope of protection of the claim or statement in question. When used in this document, the expression "at least" is used as the transitional term in, for example, a preamble of a claim, it is open in the same way as the terms "comprising" and "including" are open.The term "and / or," when used in a form such as A, B and / or C, refers to any combination or subset of A, B, C, such as (1) A alone, (2) B alone, (3) C alone, (4) A with B, (5) A with C, (6) B with C, or (7) A with B and with C. As used here in the context of describing structures, components, items, objects, and / or things, the phrase "at least one of A and B" refers to implementations that contain (1) at least one A, (2) at least one B, or (3) at least one A and at least one B. Similarly, as used here in the context of describing structures, components, items, objects, and / or things, the phrase "at least one of A or B" refers to implementations that contain (1) at least one A, (2) at least one B, or (3) at least one A and at least one B.As used here in the context of describing the execution or performance of processes, instructions, actions, activities, etc., the phrase "at least one of A and B" refers to implementations that include any one of (1) at least one A, (2) at least one B, and (3) at least one A and at least one B. Similarly, the phrase "at least one of A or B," as used here in the context of describing the execution or performance of processes, instructions, actions, activities, and / or steps, refers to implementations that include (1) at least one A, (2) at least one B, or (3) at least one A and at least one B.

[0080] As used herein, singular designations (e.g., "one," "a," "first," "second," etc.) do not preclude a plurality. The term "one" object, as used in this document, refers to one or more of this object. The terms "one" (or "one / s"), "one or more," and "at least one" are used interchangeably here. Furthermore, a plurality of means, elements, or actions, even if listed individually, may be performed, for example, by the same location or object. Additionally, although individual features may be included in different examples or claims, they may potentially be combined, and inclusion in different examples or claims does not imply that a combination of features is not feasible and / or advantageous.

[0081] As used herein, the term “in communication”, including variations thereof, includes direct communication and / or indirect communication through one or more intermediaries, and does not require direct physical (e.g., wired) communication and / or continuous communication, but additionally includes selective communication at periodic, planned, aperiodic intervals and / or one-off events.

[0082] The term "programmable circuit" as used here includes (i) one or more special-purpose electrical circuits (e.g., an application-specific integrated circuit (ASIC)) structured to perform specific operations and containing one or more semiconductor-based logic devices (e.g., electrical hardware implemented by one or more transistors), and / or (ii) one or more general-purpose semiconductor-based electrical circuits that can be programmed with instructions to perform specific functions and / or operations and containing one or more semiconductor-based logic devices (e.g., electrical hardware implemented by one or more transistors). Examples of programmable circuits include programmable microprocessors, such as central processing units (CPUs).that can execute first instructions to perform one or more operations and / or functions, field-programmable gate arrays (FPGAs) that can be programmed with second instructions to configure and / or structure the FPGAs in order to instantiate one or more operations and / or functions according to the first instructions, graphics processing units (GPUs) that can execute first instructions to perform one or more operations and / or functions, digital signal processors (DSPs) that can execute first instructions to perform one or more operations and / or functions, XPUs, network processing units (NPUs), one or more microcontrollers that can execute first instructions to perform one or more operations and / or functions,and / or integrated circuits such as application-specific integrated circuits (ASICs). For example, an XPU can be implemented by a heterogeneous computing system comprising several types of programmable circuitry (e.g., one or more FPGAs, one or more CPUs, one or more GPUs, one or more NPUs, one or more DSPs, etc., and / or any combination thereof) and an orchestration technology (e.g., an application programming interface (API) that can assign a computational task to any of the several types of programmable circuitry that is suitable and available to perform the computational task).

[0083] Integrated circuits / circuit arrangements are defined here as one or more semiconductor packages containing one or more circuit elements such as transistors, capacitors, inductors, resistors, current paths, diodes, etc. An integrated circuit can be implemented, for example, as an ASIC, FPGA, chip, microchip, programmable circuit, semiconductor substrate coupling multiple circuit elements, system-on-chip (SoC), etc.

[0084] It is evident from the foregoing that the systems, methods, devices, and manufactured items disclosed here enable unstructured sparsity in large, pre-trained, low-accuracy foundational models. The fine-tuning of the LLMs is improved while keeping the number of trainable parameters and resource requirements low. Thus, machine learning-based inference can be accelerated by using fewer parameters compared to the known sparse fine-tuning. For example, the sparsity of the model can be exploited with low (e.g., minimal) compression loss, leading to a significant speedup and demonstrating that the resulting sparse model can alleviate existing memory bandwidth problems. In this way, the examples disclosed here lead to improvements in machine operation.

[0085] Exemplary methods, devices, systems, and fabrication articles for enabling unstructured sparsity in large, pre-trained foundation models with low accuracy are disclosed here. Further examples and combinations thereof include the following:

[0086] Example 1 comprises a device comprising an interface circuit, machine-readable instructions, and at least one processor circuit to be programmed by the machine-readable instructions to sparsify a basic model of a foundation model, to generate a sparse basic model, to apply a neural low-rank adapter search to the sparse basic model, and to output a fine-tuned basic model based on the application of the neural low-rank adapter search to the sparse basic model.

[0087] Example 2 comprises the device from Example 1, wherein one or more of the at least one processor circuit(s) sparsify the base model by identifying a sparsity pattern associated with sparsified weights of the base model.

[0088] Example 3 comprises the apparatus of one or more of Examples 1 to 2, wherein one or more of the at least one processor circuit(s) identify the sparsified weights on the basis of an evaluation function applied to pretrained weights of the base model.

[0089] Example 4 comprises the apparatus of one or more of Examples 1 to 3, wherein, if the fine-tuned base model is a sparsified and quantized base model, one or more of the at least one processor circuit(s) shall identify the sparsified and quantized base model by quantizing the sparsified weights to a lower accuracy.

[0090] Example 5 comprises the device from Example 1, wherein one or more of the at least one processor circuit(s) generate a binary mask based on the sparse basic model, the binary mask being derived from an initial sparsification of a weighting matrix of the basic model.

[0091] Example 6 comprises the device from Example 1, wherein one or more of the at least one processor circuit(s) apply neural low-rank adapter search to train elastic adapters with variable configurations to improve the accuracy of the fine-tuned base model.

[0092] Example 7 comprises the apparatus of one or more of Examples 1-6, wherein the variable configurations represent variable rank values ​​as opposed to fixed rank values.

[0093] Example 8 comprises the apparatus of one or more of Examples 1-7, wherein the neural low-rank adapter search is used to apply the variable rank values ​​to the elastic adapters in order to identify a single elastic adapter configuration from a space of elastic adapter configurations.

[0094] Example 9 comprises the apparatus of one or more of Examples 1 to 6, wherein one or more of the at least one processor circuit(s) fuse the elastic adapters and model weights of the base model after fine-tuning while maintaining the sparsity of the model weights.

[0095] Example 10 contains at least one non-volatile machine-readable medium containing machine-readable instructions to cause at least one processor circuit to sparsify at least one base model of a foundation model to generate a sparse base model, to apply a neural low-rank adapter search to the sparse base model, and to output a fine-tuned base model based on the application of the neural low-rank adapter search to the sparse base model.

[0096] Example 11 comprises the at least one non-volatile machine-readable medium of Example 10, wherein the machine-readable instructions are intended to cause one or more of the at least one processor circuit to sparsify the base model by identifying a sparsity pattern associated with sparsified weights of the base model.

[0097] Example 12 comprises the at least one non-volatile machine-readable medium of one or more of Examples 10-11, wherein the machine-readable instructions are to cause one or more of the at least one processor circuit to identify the sparsified weights on the basis of an evaluation function applied to pre-trained weights of the base model.

[0098] Example 13 comprises the at least one non-volatile machine-readable medium of one or more of Examples 10-12, wherein the fine-tuned base model is a sparsified and quantized base model and the machine-readable instructions are to cause one or more of the at least one processor circuit to identify the sparsified and quantized base model by quantizing the sparsified weights to a lower precision.

[0099] Example 14 comprises the at least one non-volatile machine-readable medium of Example 10, wherein the machine-readable instructions are to cause one or more of the at least one processor circuit to generate a binary mask based on the sparse base model, wherein the binary mask is derived from an initial sparsification of a weighting matrix of the base model.

[0100] Example 15 comprises the at least one non-volatile machine-readable medium of Example 10, wherein the machine-readable instructions are intended to cause one or more of the at least one processor circuit to apply neural low-rank adapter search to train elastic adapters with variable configurations to improve the accuracy of the fine-tuned base model.

[0101] Example 16 comprises the at least one non-volatile machine-readable medium of one or more of Examples 10-15, wherein the variable configurations represent variable rank values ​​as opposed to fixed rank values.

[0102] Example 17 comprises the at least one non-volatile machine-readable medium of one or more of Examples 10-16, wherein neural low-rank adapter search is used to apply the variable rank values ​​to the elastic adapters in order to identify a single elastic adapter configuration from a space of elastic adapter configurations.

[0103] Example 18 comprises the at least one non-volatile machine-readable medium of one or more of Examples 10-16, wherein the machine-readable instructions are to cause one or more of the at least one processor circuit to fuse the elastic adapters and model weights of the base model after fine-tuning, while maintaining the sparsity of the model weights.

[0104] Example 19 comprises a device comprising means for sparsifying a basic model of a foundation model to generate a sparse basic model, means for applying a neural low-rank adapter search to the sparse basic model, and means for outputting a fine-tuned basic model based on the application of the neural low-rank adapter search to the sparse basic model.

[0105] Example 20 comprises the apparatus of Example 19, wherein the means for sparsifying comprise the identification of a sparsity pattern associated with sparsified weights of the base model.

[0106] Example 21 comprises the apparatus of one or more of Examples 19-20, wherein the means for sparsifying includes identifying the sparsified weights based on an evaluation function applied to pre-trained weights of the base model.

[0107] Example 22 comprises the apparatus of one or more of Examples 19-21, wherein the fine-tuned base model is a sparsified and quantized base model and comprises means for performing quantization to identify the sparsified and quantized base model by quantizing the sparsified weights to a lower accuracy.

[0108] Example 23 comprises the apparatus from Example 19, including means for weight fusion to generate a binary mask based on the sparse base model, wherein the binary mask is derived from an initial sparsification of a weighting matrix of the base model.

[0109] Example 24 comprises the apparatus from Example 19, wherein the means for applying a neural low-rank adapter search includes training elastic adapters with variable configurations to improve the accuracy of the finely tuned base model.

[0110] Example 25 comprises the apparatus of one or more of Examples 19-24, wherein the variable configurations represent variable rank values ​​as opposed to fixed rank values.

[0111] Example 26 comprises the apparatus of one or more of Examples 19-25, wherein the means for applying neural low-rank adapter search includes applying the variable rank values ​​to the elastic adapters to identify a single elastic adapter configuration from a space of elastic adapter configurations.

[0112] Example 27 comprises the apparatus of one or more of Examples 19-24, wherein the means for applying neural low-rank adapter search includes fusing the elastic adapters and model weights of the base model after fine-tuning, while maintaining the sparsity of the model weights.

[0113] Example 28 includes a procedure that involves sparsifying a basic model of a foundation model to generate a sparse basic model, applying a neural low-rank adapter search to the sparse basic model, and outputting a fine-tuned basic model based on the application of the neural low-rank adapter search to the sparse basic model.

[0114] Example 29 comprises the procedure of Example 28, wherein the base model is further simplified by identifying a sparsity pattern associated with sparsified weights of the base model.

[0115] Example 30 includes the procedure of one or more of Examples 28-29, wherein the sparsified weights are identified on the basis of a rating function applied to pre-trained weights of the base model.

[0116] Example 31 comprises the procedure of one or more of Examples 28-30, wherein, if the fine-tuned base model is a sparsified and quantized base model, identifying the sparsified and quantized base model by quantizing the sparsified weights to a lower level of accuracy.

[0117] Example 32 includes the procedure of Example 28, which further includes the generation of a binary mask based on the sparse basis model, wherein the binary mask is derived from an initial sparsification of a weighting matrix of the basis model.

[0118] Example 33 incorporates the procedure from Example 28, applying neural low-rank adapter search to train elastic adapters with variable configurations to improve the accuracy of the finely tuned base model.

[0119] Example 34 comprises the procedure of one or more of Examples 28-33, where the variable configurations represent variable rank values ​​as opposed to fixed rank values.

[0120] Example 35 comprises the procedure of one or more of Examples 28-34, wherein the neural low-rank adapter search is used to apply the variable rank values ​​to the elastic adapters in order to identify a single elastic adapter configuration from a space of elastic adapter configurations.

[0121] Example 36 includes the procedure of one or more of Examples 28-33, wherein the elastic adapters and model weights of the base model are fused after fine-tuning, while preserving the sparsity of the model weights.

[0122] The following claims are hereby incorporated into this detailed description by reference. Although certain exemplary systems, processes, devices, and articles are disclosed herein, the scope of protection of this patent is not limited to them. Rather, this patent covers all systems, processes, devices, and articles that reasonably fall within the scope of protection of the claims of this patent.

Claims

[1] Device comprising the following: an interface circuit; machine-readable instructions; and at least one processor circuit that is programmed by machine-readable instructions to perform the following: Sparsifying a base model of a foundation model to generate a sparse base model; Applying a neural low-rank adapter search to the sparse base model; and Outputting a finely tuned base model based on the application of neural low-rank adapter search to the sparse base model. [2] Device according to claim 1, wherein one or more of the at least one processor circuit(s) sparsify the base model by identifying a sparsity pattern associated with sparsified weights of the base model. [3] Device according to claim 2, wherein one or more of the at least one processor circuit(s) identify the sparsified weights on the basis of an evaluation function applied to previously trained weights of the base model. [4] Device according to claim 3, wherein, if the finely tuned base model is a sparsified and quantized base model, one or more of the at least one processor circuit(s) shall identify the sparsified and quantized base model by quantizing the sparsified weights to a lower accuracy. [5] Device according to one of claims 1, 2, 3 or 4, wherein one or more of the at least one processor circuit(s) generate a binary mask based on the sparse base model, wherein the binary mask is derived from an initial sparsification of a weighting matrix of the base model. [6] Device according to one of claims 1, 2, 3, 4 or 5, wherein one or more of the at least one processor circuit(s) apply neural low-rank adapter search to train elastic adapters with variable configurations to improve the accuracy of the fine-tuned base model. [7] Device according to claim 6, wherein the variable configurations represent variable rank values ​​compared to fixed rank values. [8] Device according to claim 7, wherein the neural low-rank adapter search applies the variable rank values ​​to the elastic adapters to identify a single elastic adapter configuration from a space of elastic adapter configurations. [9] Device according to claim 6, wherein one or more of the at least one processor circuit(s) serve to fuse the elastic adapters and model weights of the base model after fine-tuning while maintaining the sparsity of the model weights. [10] At least one machine-readable medium containing machine-readable instructions to to cause at least one processor circuit to do at least the following: Sparsifying a base model of a foundation model to generate a sparse base model; Applying a neural low-rank adapter search to the sparse base model; and Outputting a finely tuned base model based on the application of neural low-rank adapter search to the sparse base model. [11] At least one machine-readable medium according to claim 10, wherein the machine-readable instructions are to cause one or more of the at least one processor circuit to sparsify the base model by identifying a sparsity pattern associated with sparsified weights of the base model. [12] At least one machine-readable medium according to claim 11, wherein the machine-readable instructions are to cause one or more of the at least one processor circuit to identify the sparsified weights on the basis of an evaluation function applied to pre-trained weights of the base model. [13] At least one machine-readable medium according to claim 12, wherein the fine-tuned base model is a sparsified and quantized base model and the machine-readable instructions are intended to cause one or more of the at least one processor circuit to identify the sparsified and quantized base model by quantizing the sparsified weights to a lower accuracy. [14] At least one machine-readable medium according to one of claims 10, 11, 12 or 13, wherein the machine-readable instructions are to cause one or more of the at least one processor circuit to generate a binary mask based on the sparse base model, wherein the binary mask is derived from an initial sparsification of a weighting matrix of the base model. [15] At least one machine-readable medium according to one of claims 10, 11, 12, 13 or 14, wherein the machine-readable instructions are to cause one or more of the at least one processor circuit to apply neural low-rank adapter search to train elastic adapters with variable formations to improve the accuracy of the fine-tuned base model. [16] At least one machine-readable medium according to claim 15, wherein the variable configurations represent variable rank values ​​compared to fixed rank values. [17] At least one machine-readable medium according to claim 16, wherein the neural low-rank adapter search applies the variable rank values ​​to the elastic adapters to identify a single elastic adapter configuration from a space of elastic adapter configurations. [18] At least one machine-readable medium according to claim 16, wherein the machine-readable instructions are to cause one or more of the at least one processor circuit to fuse the elastic adapters and model weights of the base model after fine-tuning, while maintaining the sparsity of the model weights. [19] Device comprising the following: Means for sparsifying a basic model of a foundational model to generate a sparse basic model; Means of applying a neural low-rank adapter search to the sparse base model; and Means of outputting a finely tuned base model based on the application of neural low-rank adapter search to the sparse base model. [20] Device according to claim 19, wherein the means for sparsifying comprise identifying a sparsity pattern associated with sparsified weights of the base model.

Images