Machine learning model parameter compression

Abstract

Certain aspects of the present disclosure provide techniques and apparatus for improved machine learning model compression. A set of parameters for the machine learning model is accessed, wherein the set of parameters is formatted according to the first coding. A converted set of parameters is generated based on applying a conversion operation to format the set of parameters according to a second encoding. A set of bit planes is generated based on applying a bit plane transform to the converted set of parameters, and a compressed set of parameters for the machine learning model is generated based on applying a bit mask operation to one or more of the set of bit planes.

Description

[0001] Cross-references to related applications

[0002] This application claims priority to U.S. Patent Application No. 18 / 458,786, filed August 30, 2023, which has been assigned to the assignee of this application and is expressly incorporated herein by reference in its entirety, as fully set forth below and for all applicable purposes.

[0003] introduction

[0004] This disclosure relates to various aspects of machine learning.

[0005] A wide variety of machine learning model architectures have proliferated and have been used to provide solutions to a wide range of prediction problems. While specific architectures may vary, machine learning models generally rely on a set of model parameters with values ​​learned or trained based on training data, which can include labeled and / or unlabeled data. In many architectures (e.g., deep learning models), a large number of such parameters (in some cases, well over billions) are used to provide better utility. Additionally, in many cases, larger models (e.g., models with more parameters) tend to perform better (e.g., have higher prediction accuracy) and / or tend to be better suited to more complex prediction tasks. However, even relatively small models typically have a relatively large number of parameters and a considerable memory footprint.

[0006] Such a large number of parameters inherently leads to significant memory and / or storage device footprint, as well as similarly large usage of other computing resources. Model size becomes particularly problematic in resource-constrained scenarios where it is desirable to deploy trained models on devices with relatively limited resources (e.g., mobile devices, embedded devices, intelligent transportation vehicles, etc.). Some conventional approaches to improve this problem involve parameter compression. However, such compression-based solutions often rely on lossy compression (involving approximation of the original parameters), which results in reduced model accuracy and reliability. Summary of the Invention

[0007] Some aspects provide a method comprising: accessing a set of parameters for a machine learning model, wherein the set of parameters is formatted according to a first encoding; generating a transformed set of parameters based on applying a transformation operation to format the set of parameters according to a second encoding; generating a set of bit planes based on applying a bit plane transformation to the transformed set of parameters; and generating a compressed set of parameters for the machine learning model based on applying a bit mask operation to one or more bit planes in the set of bit planes.

[0008] Other aspects include: processing systems configured to perform the foregoing methods and those methods described herein; a non-transitory computer-readable medium including instructions that, when executed by one or more processors of the processing system, cause the processing system to perform the foregoing methods and those methods described herein; a computer program product embodied on a computer-readable storage medium including code for performing the foregoing methods and those methods further described herein; and a processing system including components for performing the foregoing methods and those methods further described herein.

[0009] The following description and related figures illustrate certain exemplary features of one or more aspects. Attached Figure Description

[0010] The accompanying drawings depict certain features of one or more aspects of this disclosure and are therefore not intended to limit the scope of this disclosure.

[0011] Figure 1 This example demonstrates a workflow for compressing parameters in machine learning models.

[0012] Figure 2 Example operations of bit-plane transformations used to provide parameter compression for machine learning models are shown.

[0013] Figure 3 This is a flowchart depicting an example method for compressing parameters in machine learning models.

[0014] Figure 4 This is a flowchart depicting an example method for bit-plane transformation operations used to provide parameter compression for machine learning models.

[0015] Figure 5 This is a flowchart depicting an example method for bitmasking operations used to provide compression of machine learning model parameters.

[0016] Figure 6 This is a flowchart depicting an example method for data compression.

[0017] Figure 7 Example processing systems configured to perform various aspects of this disclosure are described.

[0018] For ease of understanding, the same reference numerals are used where possible to designate common elements shared by all figures. Elements and features conceived in one aspect can be beneficially incorporated into other aspects without further elaboration. Detailed Implementation

[0019] This disclosure provides apparatus, methods, processing systems, and computer-readable media for compressing parameters of machine learning models.

[0020] In some respects, lossless parameter compression techniques are provided to substantially reduce the model's data size (e.g., memory footprint) without affecting model accuracy or performance. In some respects, parameters can be compressed before runtime (e.g., by a compiler) and decompressed at runtime (e.g., using hardware). In some respects, by reducing model size, memory usage is reduced, and the effective capacity of the processing hardware that uses the model to process the data is increased. This enables the efficient use of machine learning models on resource-constrained devices and / or at reduced computational cost.

[0021] In some respects, the lossless compression techniques disclosed herein can be combined with lossy compression methods to further reduce model size, which has a reduced impact on model performance (e.g., compared to using lossy techniques alone, as used in some conventional methods).

[0022] In some respects, model compression is achieved by transforming or reshaping model data using a series of operations to achieve efficient, lossless compression by leveraging the inherent distributions present in the data. In other respects, model parameters are transformed from an initial format (e.g., two's complement encoding) to a target format (e.g., signed value encoding). This transformation allows the system to more easily utilize the naturally occurring sparsity in the data. That is, parameters using two's complement encoding typically exhibit a uniform value distribution across bit positions, while the same data using signed value encoding often exhibits a significantly different value distribution, as discussed in more detail below.

[0023] In some respects, one or more bit-plane transformations can then be used to process such transformed parameters (encoded in a symbolic value format) to produce a set of bit planes. Because symbolic value encoding reveals the underlying data distribution (e.g., where a large amount of model sparsity exists in one or more specific locations across the parameters), such bit-plane transformations enable targeted extraction of these sparse regions, as discussed in more detail below.

[0024] In some aspects, bitmasking operations can then be used to compress the bit plane, where portions of data (e.g., words) with values ​​of zero (or some other value) can be removed, and a single mask bit can be used to indicate that the word has been omitted. In some aspects, words with non-zero values ​​(or values ​​that do not match the masked value) can have their mask bits set to defined values ​​that indicate that the word is non-zero. By applying this masking operation to the bit plane (which itself is based on reformatted data that utilizes inherent sparsity), the techniques described herein can significantly reduce the size of the data (e.g., the number of bits used to encode model parameters).

[0025] Example workflow for machine learning model parameter compression

[0026] Figure 1 Example workflow 100 for compressing parameters of machine learning models.

[0027] In the illustrated example, compression system 110 accesses a set of input parameters 105 to produce a set of compressed parameters 140. In some aspects, compression system 110 is a computational system used to compress machine learning models. In some aspects, compression system 110 corresponds to... Figure 7 The processing system 700. As used herein, “accessing” data can generally include receiving, requesting, capturing, obtaining, collecting, generating, or otherwise acquiring access to data. For example, compression system 110 may receive input parameters 105 from another system or component (e.g., a training system), or compression system 110 may generate input parameters 105 itself (e.g., compression system 110 may train a model).

[0028] Input parameter 105 typically corresponds to parameters or variables of one or more machine learning models. For example, parameters may be weights and / or hyperparameters of a deep neural network. In some aspects, input parameter 105 has learned values. For example, compression system 110 may access input parameter 105 after training is complete and before deploying the model for runtime inference. While some aspects of this disclosure describe machine learning model parameters as an example set of data that can be compressed, aspects of this disclosure can be readily applied to provide efficient compression of a wide variety of data.

[0029] In some respects, compression system 110 is used to compress input parameters 105 and prepare a model for deployment during or as part of compiling a trained model. Although depicted as a discrete system for clarity of concept, in some respects, the operation of compression system 110 can be combined or distributed across any number and variety of components and systems. The operation of compression system 110 can typically be implemented using hardware, software, or a combination of hardware and software.

[0030] In the illustrated example, compression system 110 includes a variety of components, including (but not limited to) a transformation component 115, a bit plane component 125, and a mask component 135. Although depicted as discrete components for clarity of concept, the operations performed by the depicted components can similarly be combined or distributed across any number of components.

[0031] As illustrated, conversion component 115 evaluates input parameter 105 (encoded or formatted according to a first format) to produce a set of converted parameters 120 (encoded or formatted according to a second format). For example, in some aspects, input parameter 105 may be encoded using a two's complement format, a common system used to encode or represent parameters of machine learning models. While the two's complement format is used as an example format that can be used to encode input parameter 105, input parameter 105 may be encoded using any other alternative format, such as one's complement. In some aspects, converted parameters 120 may use a different format, such as a signed value encoding format (or any other format different from the format used to encode input parameter 105).

[0032] Both two's complement and signed value formats can be used to encode positive and negative values ​​for input parameter 105. For example, in either encoding format, each input parameter in input parameter 105 typically includes a sign bit and a set of bits used to encode the value. In the signed value format, the sign bit indicates the sign of the parameter (e.g., a value of 1 indicates a negative value, and a value of 0 indicates a positive value). The value of the parameter is encoded using the remaining bits in binary form (e.g., for an 8-bit value, the first bit can be the sign bit, and the following seven bits can encode the value).

[0033] In two's complement format, positive numbers can be encoded in the same way as in signed value format. For negative numbers, the value bits can store the two's complement of the positive version of the number (with the sign bit indicating the transformation). In some respects, two's complement formatting is frequently used in some conventional systems because such formatting significantly simplifies addition operations (eliminating the evaluation of the sign bit used to handle negative numbers during addition and subtraction). Furthermore, such formatting has only a single representation of the value 0 (while signed value format has two: positive zero and negative zero).

[0034] However, as discussed above, data encoded using two's complement formatting typically has a relatively uniform value distribution across bit positions. That is, suppose input parameter 105 comprises a set of 8-bit values. While input parameter 105 itself may have a Gaussian value distribution, the average binary value across each bit position is often uniform. For example, the first, second, third, and so on positions typically each have an average value of approximately 0.5 (across input parameter 105) (e.g., for each bit position, approximately half of input parameter 105 has a value of 0 and approximately half has a value of 1). In contrast, parameters encoded with sign magnitudes can reveal a fundamental bit distribution. For example, while the sign bit (the first or most significant bit) may have an average value of 0.5, the preceding bits of the magnitude portion (e.g., the second, third, and fourth positions in a word, where the first position is the sign bit) may tend to be substantially skewed toward an average value of 0 (e.g., because model parameters often have relatively small magnitudes).

[0035] As illustrated, the bit plane assembly 125 evaluates the converted parameters 120 to generate a set of bit planes 130. Each bit plane in the bit plane 130 contains a binary value from the corresponding bit position in the converted parameters 120. For example, if the converted parameters 120 are encoded in an 8-bit signed value format, the bit plane assembly 125 may generate eight bit planes 130: one bit plane including the most significant bit (the sign bit across all parameters), one bit plane including the least significant bit across all parameters, and six bit planes corresponding to the remaining six bit positions.

[0036] As discussed in more detail below, in some respects, bit-plane component 125 can use different word lengths to encode each bit-plane in bit-plane 130, regardless of the word length used to encode input parameter 105 and / or transformed parameter 120. For example, transformed parameter 120 can each be encoded using eight bits, and bit-plane component 125 can use different word lengths for each bit-plane in bit-plane 130 (e.g., using a four-bit word length to encode the first bit-plane, using a sixteen-bit word length to encode the second bit-plane, and so on). By dynamically selecting or determining the word length used to encode each bit-plane in bit-plane 130, compression system 110 can achieve enhanced compression, as discussed in more detail below.

[0037] In the illustrated example, bit plane 130 is then evaluated by masking component 135 to produce the set of compressed parameters 140. In some aspects, masking component 135 may typically append or prepend mask bits to each word used to encode bit plane 130, setting the value of the mask bits to indicate whether the accompanying word has a value of 0 (or some other defined value, discussed in more detail below). If so (e.g., if the mask bit is set to 1), the word can be omitted. For example, an 8-bit word with a value of 0 (e.g., "00000000") can be replaced with a single bit with a value of 1 (e.g., "1"), while an 8-bit word with a non-zero value (e.g., "10010100") can be replaced with a 9-bit word corresponding to the original eight bits (e.g., "010010100") plus a mask bit with a value of 0. See below for further details. Figure 5 To further explain this aspect.

[0038] Typically, the average number of bits per word used to encode compression parameters 140 depends primarily on the sparsity of bit plane 130. Higher sparsity (e.g., more words with values ​​of 0) generally results in increased compression (a reduction in the average number of bits per parameter), while lower sparsity can actually result in an increased model size (e.g., a higher average number of bits per parameter than the average number of bits per parameter used to encode the original input parameters 105). However, as discussed in more detail above and below, transformation component 115 and bit plane component 125 can typically represent input parameters 105 such that sparsity is arranged using masking operations, resulting in a substantially reduced memory footprint.

[0039] For example, in some aspects, the number of bits used to encode the compressed parameter 140 may be approximately 40%, approximately 50%, or approximately 60% of the number of bits used to encode the input parameter 105, depending on the specific architecture and implementation.

[0040] The compressed parameters 140 can then be used to deploy a machine learning model (locally or on another system). For example, at runtime, an inference system (e.g., a computing system that uses the machine learning model to perform runtime inference, which may be the same system as compression system 110 and / or a different computing system) can capture portions of the compressed parameters 140 because such portions are used to process data (e.g., extracting parameters of each layer of a deep neural network from memory to the processing chip when data reaches a given layer during forward pass through the network). The inference system can decode the compressed parameters 140 to recover the original input parameters 105 (e.g., on-chip) because such original parameters are used to process the input data. This substantially reduces the memory footprint used to store the model and reduces the bandwidth used on the links between memory and on-chip components used to process the data.

[0041] For example, the inference system may apply the inverse of a bitmasking operation (e.g., removing mask bits and inserting a word with a value of 0, where the mask bits indicate that the original word was removed), the inverse of a bit-plane transformation (e.g., rearranging bits so that each word includes bits from a given model parameter, rather than bits from multiple parameters), and / or the inverse of a transformation transformation (e.g., if two's complement encoding is preferred or used during inference).

[0042] Additionally, since each of the transformations described can typically be implemented efficiently (e.g., using a relatively simple hardware decoder that does not consume substantial computational resources or time), the decoding process can be easily completed in real time during inference (using hardware, software, or a combination of hardware and software) without burdening the inference system.

[0043] Example operations for bit-plane transformations used to provide parameter compression for machine learning models

[0044] Figure 2 Example operation 200 of bit-plane transformation is illustrated for providing parameter compression of machine learning models. In some aspects, the illustrated example depicts a bit-plane transformation that can be generated by bit-plane components (such as...) Figure 1 The conceptual operation performed by the bit plane component 125.

[0045] In the illustrated example, input words 205A to 205H (for example, used to encode) Figure 1 The words (transformed by parameter 120) are arranged horizontally (e.g., input words 205A are stacked on top of input words 205B, etc.) to form a data matrix. In the illustrated example, each of input words 205A through 205H (collectively referred to as input words 205) has a word length of eight (e.g., each is encoded using eight bits), and each bit is depicted as a square with rounded corners in the illustrated example. Each input word 205 typically corresponds to a corresponding parameter in a machine learning model. That is, each input word 205 includes a set of eight bits encoding the value of the corresponding machine learning model parameter. Although the illustrated example depicts an eight-bit word length for input words 205, in all respects, input words of any length can be used. Similarly, although the illustrated example depicts eight input words 205 for conceptual clarity, any number of input words 205 can be used.

[0046] As illustrated, input word 205 is used to generate a set of bit planes 210A to 210H (e.g., Figure 1 Bit plane 130). Specifically, each bit plane 210A to 210H (collectively referred to as bit plane 210) includes a single bit from each of one or more input words 205 at the corresponding bit position (e.g., where bit plane 210A includes the most significant bit of each input word and bit plane 210H includes the least significant bit of each input word 205).

[0047] In the illustrated example, different word lengths can be used to encode each bit plane 210 by selectively choosing how many input words 205 to use to generate bit plane words. In the illustrated example, such word lengths are depicted using shaded boxes. For example, as illustrated, a word length of eight is used to encode bit plane 210A (e.g., using the most significant bit of one most significant bit from each of the eight input words 205A-205H to form a single word encoding bit plane 210A). That is, when generating bit plane 210A, the bit plane component can extract one bit from each of the eight input words 205 to generate each word encoding bit plane 210A. Similarly, in the illustrated example, a word length of eight is used to encode bit plane 210E.

[0048] Additionally, in the illustrated example, bit planes 210B, 210G, and 210H are encoded with a word length of four. That is, each word in each of bit planes 210B, 210G, and 210H is generated based on a set of four input words 205. Specifically, the first word of bit plane 210B is generated based on input words 205A, 205B, 205C, and 205D, and the second word of bit plane 210B is generated based on input words 205E, 205F, 205G, and 205H.

[0049] As another example, a word length of two is used to encode bit plane 210F, and a word length of six is ​​used to encode bit plane 210C (in the illustrated example, the bits in the third position of input words 205G and 205H are still included in bit plane 210C, but are included in a separate word (not depicted) that is different from the words used to cover input words 205A, 205B, 205C, 205D, 205E and 205F).

[0050] In this way, by selecting the number of input words 205 to be evaluated when generating words for each bit plane 210, the bit plane component can dynamically determine the word size for each bit plane in a way that enhances the compressibility of the bit plane 210. For example, a larger word length allows the system to better amortize the cost of adding mask bits, resulting in a greater reduction in word size when replacing (longer) words with a single mask bit. However, longer word lengths also tend to result in reduced average word sparsity (e.g., the probability of a given word having a reduction of value 0), which adds complexity and reduces (or eliminates) the benefits of bitmasking. As discussed in more detail below, in some aspects, the bit plane component can adaptively select the optimal word size for each bit plane 210 based on bit plane sparsity to maximize (or at least increase) the compression achievable using bitmasking operations.

[0051] Although the illustrated example depicts an input word 205 with a word length of eight, aspects of this disclosure can be readily applied to input words 205 of any length. Furthermore, while the illustrated example depicts eight input words 205, any number of input words 205 can exist, and the word length used for each bit plane 210 can be less than, the same as, or greater than the word length of the input words 205. For example, even using an input word 205 with a word length of eight, the words given to the bit plane 210 can typically be of any length (e.g., one bit plane 210 can use 256-bit words to encode data, while another plane can use 2-bit words to encode data).

[0052] Example methods for compressing parameters in machine learning models

[0053] Figure 3This is a flowchart depicting an example method 300 for parameter compression in machine learning models. In some respects, method 300 targets... Figure 1 Workflow 100 provides additional details. In some respects, method 300 is composed of a compression system (such as...) Figure 1 The compression system 110) is executed.

[0054] In box 305, the compression system accesses a set of (trained) model parameters (e.g., Figure 1 The input parameters are 105. As discussed above, model parameters typically include the parameters (or hyperparameters) of a machine learning model, each with a value learned during the training phase based on the training data. For example, for a neural network, model parameters may include a set of weights for each layer in the model. In some respects, these class of model parameters are encoded according to a first format (e.g., two's complement format).

[0055] At box 310, the compression system converts the received parameters to a second format (e.g., symbolic value encoding) to establish the converted parameters (e.g., Figure 1 The converted parameters (120). As discussed above, a second format (also referred to as the target format) can be selected to enhance the compressibility of the model, such as by arranging sparsity (e.g., bits with values ​​of 0) in one or more specific locations within each parameter. In some aspects, the compression system may select and evaluate multiple encoding formats. For example, the compression system may use any suitable technique (including randomly or pseudo-randomly) to select an alternative target encoding format, encode the parameters using the selected format, and compress the converted data to evaluate the compressibility of the selected format. The compression system may evaluate each alternative (sequentially or in parallel) to determine which target format should be used for the specific set of model parameters accessed at box 305. In some aspects, the compression system may similarly select the target format based at least in part on the architecture of the model (e.g., where each type of machine learning architecture may have different encoding formats that are most suitable, or at least more suitable, for compression). For example, the compression system may refer to a defined mapping (which indicates the appropriate target encoding for each type of model architecture).

[0056] In some respects, when converting to a symbolic value format, the compression system may selectively compress the sign bit where possible. For example, symbolic value encoding has representations of both positive and negative zero. Therefore, in some respects, the compression system may omit or remove the sign bit if the magnitude of the parameter is zero. For example, the compression system may produce a converted parameter where the magnitude portion precedes the sign bit (e.g., the sign bit is the least significant bit, not the most significant bit). At runtime, the inference system may first decode the magnitude (e.g., the first seven bits). If the value is 0, the inference system can understand that there is no sign bit (e.g., the next word or parameter begins immediately after the magnitude portion of the current word). If the value is non-zero, the compression system can determine that the next bit is the sign bit of the current word / parameter.

[0057] At box 315, the compression system generates a set of bit planes based on the transformed parameters (e.g., Figure 1 (Bit plane 130). For example, as discussed above, each bit plane may correspond to or include bits in a given location within a word used to encode the transformed parameters. In some aspects, as discussed above, the compression system may determine which word length should be used to encode the bit plane for each bit plane (e.g., each bit location). For example, the compression system may use heuristic or rule-based methods (e.g., where data scientists pre-specify the desired word length for each bit plane based on evaluating other models), or the compression system itself may dynamically evaluate the sparsity of each bit plane to select the word length for each bit plane. See below for reference. Figure 4 Let's discuss an example method for generating a bit plane in more detail.

[0058] At box 320, the compression system applies one or more bit masks to a bit plane (e.g., to the words used to encode the bit plane). For example, as discussed above, the compression system may prepend or append mask bits to each bit plane word, and the compression system may set the value of the mask bit to indicate whether the corresponding word has a defined value (e.g., value 0). If so, the word itself may be removed, and only the mask bit may be retained and stored. In some aspects, value 0 is used as a defined value (e.g., where all words with value 0 are replaced with the corresponding sign bit). In some aspects, the compression system may choose another value to mask. For example, the compression system may determine a statistical pattern of words (e.g., the most common value given to bit plane words in the localization plane), and the compression system may use this pattern as the masked value for the bit plane (e.g., where words equal to or matching the pattern value are masked). This maximizes (or at least increases) the compression achieved by using masking operations.

[0059] Additionally, in some aspects, the compression system can selectively determine whether to apply a bitmask to each bit plane (e.g., to words in each bit plane) based on the sparsity of the bit planes, as discussed in more detail below. See below for reference. Figure 5Let's discuss an example method for mask bit plane words in more detail.

[0060] At box 325, the compression system optionally performs bit-plane trimming on the masked bit planes. While trimming is typically a lossy form of compression, accuracy loss can be reduced, as compared to some conventional methods, by trimming bit-plane words (each corresponding to bits from multiple parameters) rather than directly trimming the parameters themselves. In some aspects, to trim bit planes, the compression system may set or determine a threshold (e.g., defined as a hyperparameter), and the compression system may trim (e.g., set to zero) all bit-plane words with a magnitude (e.g., absolute value) less than the threshold. Such trimmed bit-plane words may then be masked (e.g., replaced with a single mask bit during subsequent masking operations) or removed.

[0061] In some respects, after pruning, the compression system can perform fine-tuning on the model to recover any accuracy loss. That is, the compression system can update one or more of the pruned parameters (e.g., using a relatively small set of training data). For example, in some respects, the compression system can decode the pruned bit plane (e.g., invert the encoding operations used to compress the data to produce the bit plane) to produce the pruned parameters; update such parameters using a set of training data (e.g., using training or model refinement operations); and then re-encode such updated pruned parameters (e.g., by applying transformation operations, bit plane transformations, and bitmasking operations again) to produce the updated pruned bit plane.

[0062] In the illustrated example, at box 330, the compression system provides compressed model parameters (e.g., masked bit-plane words generated at box 320 and / or trimmed masked bit-plane words generated at box 325). As discussed above, providing compressed parameters can typically include compilation parameters or otherwise provide access to parameters for inference. For example, the compression system could send compressed parameters to one or more dedicated inference systems, store such parameters locally for local inference, etc.

[0063] Example methods for providing bit-plane transformation operations to compress parameters for machine learning models.

[0064] Figure 4 This is a flowchart depicting an example method 400 for providing bit-plane transformation operations to compress parameters of a machine learning model. In some aspects, method 400 provides... Figure 3 Additional details for box 315. In some respects, method 400 is composed of a compression system (such as...) Figure 1 The compression system 110) is executed.

[0065] At box 405, the compression system selects the bit positions to encode the words with transformed parameters. As discussed above, selecting the bit positions can also be referred to as selecting the bit plane. Typically, the compression system can use any suitable technique to select the bit positions (including randomly or pseudo-randomly), since each bit position will be similarly processed during method 400. Although depicted as an iterative process (where each bit position is selected and evaluated sequentially) for conceptual clarity, in some respects, the compression system can process some or all bit positions in parallel.

[0066] At box 410, the compression system selects the word length for the selected bit position. That is, the compression system selects the word length used to encode the bit plane corresponding to the selected bit position. Typically, any suitable technique can be used to select the word length. In some aspects, the compression system may refer to a defined set or list of alternative word lengths (e.g., defined as hyperparameters) and select a word length (at box 410) from that list (e.g., randomly or pseudo-randomly). For example, in some aspects, the set of alternative word lengths includes word lengths ranging from a minimum length (e.g., a word length of 2 or other values) to a maximum length (e.g., a word length of 256 or other values).

[0067] Although described as an iterative process for clarity (where each alternative word length is selected and evaluated sequentially), in some respects, the compression system can process some or all word lengths in parallel. In some aspects, the compression system can refer to a defined mapping to identify which word length should be used at the selected bit position (without actively evaluating alternatives).

[0068] At box 415, the compression system generates a bit plane for selected bit positions based on the selected word length. That is, the compression system uses words with selected word lengths to encode the selected bit plane.

[0069] At box 420, the compression system determines the sparsity of the selected bit plane encoded at the selected word length. In some respects, this is referred to as bit plane sparsity or bit plane sparsity value. In some respects, bit plane sparsity indicates the percentage of words (in the selected bit plane encoded at the selected word length) that have a value that matches a defined value (such as 0, or a value equal to the pattern value of a word in the selected bit plane).

[0070] At box 425, the compression system determines whether there are any additional word lengths that have not yet been used to evaluate the current position. If yes, method 400 returns to box 410. If no, method 400 continues to box 430.

[0071] At box 430, the compression system determines the word length to be used for the selected bit position (e.g., once the compression system has evaluated all alternative word lengths for the current bit position). In some aspects, the compression system selects the word length that results in the highest bit-plane sparsity. In some aspects, the compression system selects the maximum word length that results in bit-plane sparsity above a defined threshold. In some aspects, the compression system selects the word length that results in maximum compression when a bitmask is applied. For example, the compression system may select a word length that ensures the total number of bits used to encode the bit plane (after adding mask bits and removing words with defined values ​​such as 0) is minimized, or at least close to the minimum. In some aspects, if the minimum bit-plane sparsity is above a threshold (e.g., no word length results in sparsity above the threshold), the compression system may determine that a bitmasking operation will not be used for that bit plane (discussed in more detail below), and therefore the compression system may use one or more other criteria to determine the word length (e.g., based on hardware configuration or other preferences).

[0072] At box 435, the compression system determines whether there is at least one additional bit position that has not yet been evaluated. If so, method 400 returns to box 405. If not, method 400 terminates at box 440. In this way, the compression system can adaptively select or determine the word length for each bit position (e.g., each bit plane) that maximizes (or at least increases) the compressibility of the model.

[0073] Example methods for bitmasking operations to provide compression of machine learning model parameters

[0074] Figure 5 This is a flowchart depicting an example method 500 for bitmasking operations used to provide compression of machine learning model parameters. In some aspects, method 500 provides... Figure 3 Additional details for box 320. In some respects, method 500 is composed of a compression system (such as...) Figure 1 The compression system 110) is executed.

[0075] At box 505, the compression system selects a bit plane. As discussed above, selecting a bit plane can also be referred to as selecting a bit location. Typically, the compression system can use any suitable technique to select the bit plane (e.g., randomly or pseudo-randomly), since each bit plane will be similarly processed during method 500. Although depicted as an iterative process (where each bit plane is selected and evaluated sequentially) for clarity of concept, in some respects, the compression system can process some or all bit planes in parallel.

[0076] At box 510, the compression system determines the sparsity of the selected bit plane. For example, as discussed above, the compression system may determine the percentage of words (used to encode the selected bit plane) that match or are equal to a defined mask value (e.g., a value of 0, or a value equal to the statistical pattern of the words used to encode the bit plane).

[0077] At box 515, the compression system determines whether one or more sparsity criteria are met. Sparsity criteria are typically used to determine whether a masking operation should be performed on a bit plane. In some aspects, sparsity criteria include a threshold amount of sparsity. For example, in some aspects, if the bit plane sparsity is below a threshold, adding mask bits can effectively increase the average size of each word in the bit plane, and the compression system can therefore determine that the sparsity criteria are not met. As an example, if the bit plane word is eight bits long, the sparsity value should be at least 12.5%. If the sparsity is less than this value, the bit mask will increase the size of the bit plane (e.g., increase the total number of bits used to encode the bit plane), and may not meet the sparsity criteria. If the sparsity is greater than this value, the masking operation will reduce the size of the bit plane, and the sparsity criteria will be met. Generally, higher sparsity is associated with higher compressibility (e.g., 62.5% sparsity for an 8-bit word can result in a compression factor of two, where the size of the bit plane is halved). Specific sparsity criteria can be varied depending on the specific implementation of the word size and / or mask used to encode the bit plane.

[0078] If at box 515 the compression system determines that the criteria are not met (e.g., the bit plane is not sparse enough to guarantee the bitmask), then method 500 continues to box 530. If the compression system determines that the criteria are met, then method 500 continues to box 520.

[0079] At box 520, the compression system determines a pattern value (e.g., the most common value) for the words used to encode the bit plane. At box 525, the compression system then masks the bit plane based on the determined pattern. For example, as discussed above, the compression system can replace all words having that pattern value with a single mask bit having a defined value (e.g., value 1), and the compression system can append or prepend all other words with a single mask bit having another defined value (e.g., value 0). For example, suppose the pattern value for the bit plane is “01010101”. In some aspects, the compression system can replace any bit plane word with the pattern value “01010101” with a single mask bit having a value of “1”. Similarly, the compression system can append a mask bit having a value of zero to any bit plane word having a value different from the pattern value (e.g., replacing the bit plane word “10101010” with “010101010”).

[0080] At box 530, the compression system determines whether there is at least one additional bit plane that has not yet been evaluated. If yes, method 500 returns to box 505. If no, method 500 terminates at box 535. In this way, the compression system can adaptively determine whether to use a bitmask for each bit plane (e.g., for each bit position) to maximize (or at least increase) model compression.

[0081] Example methods for data compression

[0082] Figure 6 This is a flowchart depicting an example method 600 for data compression. In some aspects, method 600 is composed of a compression system, such as... Figure 1 Compression system 110, execution Figure 2 Operating the 200 system, executing Figure 3 Method 300 system, execution Figure 4 Method 400 system and / or execution Figure 5 The method is executed by the 500 system.

[0083] At box 605, access a set of parameters used for the machine learning model, wherein the set of parameters is formatted according to a first encoding format.

[0084] At box 610, a converted set of parameters is generated based on applying a conversion operation to format the set of parameters according to a second encoding format.

[0085] In some respects, applying the conversion operation includes identifying one or more parameters in the set of parameters that have a value of zero; and encoding each of the one or more parameters without including a corresponding sign bit, wherein one or more other parameters that have a non-zero value are encoded with a sign bit.

[0086] At box 615, a set of bit planes is generated by applying a bit plane transformation based on the transformed set of parameters.

[0087] In some aspects, applying the bit-plane transformation (at box 615) includes determining, for each corresponding bit-plane in the set of bit-planes, the corresponding word length for encoding that corresponding bit-plane.

[0088] In some aspects, determining the corresponding word length for encoding each corresponding bit plane includes adaptively selecting a first word length for the first bit plane of the set of bit planes. This adaptive selection may include: encoding the first bit plane based on the first word length; determining a first sparsity value for the first bit plane encoded based on the first word length; encoding the first bit plane based on a second word length; determining a second sparsity value for the first bit plane encoded based on the second word length; and selecting the first word length for the first bit plane based on the first sparsity value and the second sparsity value.

[0089] At box 620, a compressed set of parameters for the machine learning model is generated by applying a bitmasking operation to one or more of the bit planes in the set of bit planes.

[0090] In some respects, generating the compressed set of parameters (at box 620) includes determining, based on a first sparsity value of the first bit plane, to apply the bitmasking operation to the first bit plane (of the set of bit planes), and determining, based on a second sparsity value of the second bit plane, not to apply the bitmasking operation to the second bit plane (of the set of bit planes).

[0091] In some aspects, method 600 further includes (e.g., after box 615) determining a magnitude threshold, wherein the magnitude threshold is a hyperparameter; generating a pruned set of bit planes based on pruning one or more words used to encode the set of bit planes based on the magnitude threshold; decoding the pruned set of bit planes to generate a pruned set of parameters; updating one or more parameters of the pruned set of parameters using training data; and encoding the pruned set of parameters based on applying the transformation operation, the bit plane transformation, and the bitmasking operation to generate a pruned and compressed set of parameters.

[0092] In some aspects, pruning the one or more words used to encode the set of bit planes includes: determining that the one or more words have a value less than the value threshold and setting each of the one or more words to a value of 0.

[0093] In some aspects, applying a bitmasking operation (at box 620) includes determining a pattern for a set of words used to encode the first bit plane for at least the first bit plane of the set of bit planes and compressing the first bit plane based on the pattern. Compressing the first bit plane may include identifying one or more words of the set of words that have values ​​matching the pattern, and replacing each of the one or more words with a corresponding mask bit indicating that the corresponding word has a value equal to the pattern.

[0094] Example processing system for data compression

[0095] In some respects, refer to Figures 1 to 6 The described workflows, techniques, and methods can be implemented on one or more devices or systems. Figure 7 An example processing system 700 is depicted, which is configured to perform various aspects of this disclosure, including, for example, regarding... Figures 1 to 6 The techniques and methods described. In some aspects, the processing system 700 may correspond to a compression system, such as... Figure 1 The compression system 110. For example, the processing system 700 may correspond to a system that compiles a machine learning model after training, trains a machine learning model, and / or uses the machine learning model for inference at runtime. Although depicted as a single system for clarity of concept, in some respects, as discussed above, the operations described below with respect to the processing system 700 can be distributed across any number of devices or systems.

[0096] The processing system 700 includes a central processing unit (CPU) 702, which in some examples may be a multi-core CPU. Instructions executed at the CPU 702 may be loaded, for example, from program memory associated with the CPU 702 or from a memory partition (e.g., a partition of memory 724).

[0097] The processing system 700 also includes additional processing components tailored for specific functions, such as a graphics processing unit (GPU) 704, a digital signal processor (DSP) 706, a neural processing unit (NPU) 708, a multimedia component 710 (e.g., a multimedia processing unit), and a wireless connectivity component 712.

[0098] NPUs (such as the NPU 708) are typically configured to implement dedicated circuitry for implementing control and arithmetic logic for executing machine learning algorithms, such as those for processing artificial neural networks (ANNs), deep neural networks (DNNs), random forests (RFs), etc. NPUs are sometimes alternatively referred to as neural signal processors (NSPs), tensor processing units (TPUs), neural network processors (NNPs), intelligent processing units (IPUs), vision processing units (VPUs), or graphics processing units.

[0099] NPUs such as the NPU 708 are configured to accelerate the execution of common machine learning tasks, such as image classification, machine translation, object detection, and various other predictive models. In some examples, multiple NPUs may be instantiated on a single chip, such as a system-on-a-chip (SoC), while in other examples, an NPU may be part of a dedicated neural network accelerator.

[0100] An NPU can be optimized for either training or inference, or in some cases configured to balance performance between the two. For an NPU capable of performing both training and inference, these two tasks can generally still be performed independently.

[0101] NPUs designed to accelerate training are typically configured to speed up the optimization of new models. This involves taking an existing dataset (usually labeled or sublabeled), iterating over the dataset, and subsequently tuning model parameters (such as weights and biases) to improve model performance—a highly computationally intensive operation. Generally, optimization based on incorrect predictions involves backtracking through the layers of the model and determining gradients to reduce prediction errors.

[0102] NPUs designed to accelerate inference are typically configured to operate on the full model. Therefore, such NPUs can be configured to take new data segments as input and rapidly process those segments through a trained model to generate model outputs (e.g., inference).

[0103] In some implementations, the NPU 708 is part of one or more of the CPU 702, GPU 704, and / or DSP 706.

[0104] In some examples, the wireless connectivity component 712 may include sub-components for, for example, third-generation (3G) connectivity, fourth-generation (4G) connectivity (e.g., 4G Long Term Evolution (LTE)), fifth-generation connectivity (e.g., 5G or New Radio (NR)), Wi-Fi connectivity, Bluetooth connectivity, and / or other wireless data transmission standards. The wireless connectivity component 712 is further coupled to one or more antennas 714.

[0105] The processing system 700 may also include one or more sensor processing units 716 associated with any type of sensor, one or more image signal processors (ISPs) 718 associated with any type of image sensor, and / or a navigation processor 720, which may include satellite-based positioning system components (e.g., GPS or GLONASS) and inertial positioning system components.

[0106] The processing system 700 may also include one or more input and / or output devices 722, such as screens, touch-sensitive surfaces (including touch-sensitive displays), physical buttons, speakers, microphones, etc.

[0107] In some examples, one or more processors in the processing system 700 may be based on the ARM or RISC-V instruction set.

[0108] The processing system 700 also includes a memory 724, which represents one or more static and / or dynamic memories, such as dynamic random access memory, flash-based static memory, etc. In this example, the memory 724 includes computer-executable components that can be executed by one or more of the aforementioned processors of the processing system 700.

[0109] Specifically, in this example, memory 724 includes a transformation component 724A, a bit plane component 724B, a mask component 724C, a trimming component 724D, and a fine-tuning component 724E. Memory 724 also includes model parameters 724F for one or more models (e.g., Figure 1 Input parameters 105 and / or Figure 4 The compressed parameter 140). Although not included in the illustrated example, in some respects, memory 724 may also include other data, such as training data (e.g., for training and / or fine-tuning the model). Although for the sake of conceptual clarity, Figure 7 The components are depicted as discrete components, but in various respects, the illustrated components (and other components not depicted) may be implemented together or separately.

[0110] The processing system 700 also includes a conversion circuit 726, a bit plane circuit 727, a masking circuit 728, a trimming circuit 729, and a fine-tuning circuit 730. The depicted circuits and other circuits not depicted can be configured to perform various aspects of the techniques described herein.

[0111] For example, conversion component 724A and / or conversion circuit 726 (which may correspond to Figure 1 The conversion component 115 can be used to convert input parameters from a first format (e.g., two's complement) to a second format (e.g., signed value), as discussed above. For example, conversion component 724A and / or conversion circuit 726 can convert parameters to another format that distributes bit sparsity in a way that promotes compression.

[0112] Bit plane component 724B and / or bit plane circuit 727 (which may correspond to...) Figure 1 The bit plane component 125 can be used to generate bit planes for the transformed (or input) parameters, as discussed above. For example, the bit plane component 724B and / or the bit plane circuit 727 can determine a defined, optimal, or desired bit plane word length and encode each bit plane based on the determined word length in order to maximize (or at least enhance) compressibility.

[0113] Masking component 724C and / or masking circuit 728 (which may correspond to Figure 1 The masking component 135 can be used to apply bitmasking operations as discussed above. For example, the masking component 724C and / or the masking circuit 728 can add mask bits to each bit-plane word (generated by the bit-plane component 724B and / or the bit-plane circuit 727) and remove any bit-plane word that has a value equal to a defined value (e.g., zero, or the pattern value of the word in the bit-plane).

[0114] The pruning component 724D and / or pruning circuit 729 can be used to prune bit-plane words, as discussed above. For example, the pruning component 724D and / or pruning circuit 729 can compare each bit-plane word with a defined threshold or range and prune bit-plane words that have a value less than the threshold (e.g., set to 0). As discussed above, this pruning of bit-plane words (pruning individual bits from multiple parameters) rather than pruning the entire parameter word can lead to higher model accuracy, such as compared to some conventional parameter pruning methods.

[0115] The fine-tuning component 724E and / or the fine-tuning circuit 730 can be used to perform model updates or fine-tuning after in-place plane pruning, as discussed above. For example, the fine-tuning component 724E and / or the fine-tuning circuit 730 can decode pruned parameters; update parameters using relative group training data; and re-encode such parameters (or allow other components to re-encode such parameters).

[0116] Although for the sake of clarity Figure 7 While depicted as separate components and circuits, the conversion circuit 726, bit plane circuit 727, mask circuit 728, trimming circuit 729, and fine-tuning circuit 730 may be implemented together or separately in other processing devices of the processing system 700, such as in the CPU 702, GPU 704, DSP 706, NPU 708, etc.

[0117] Generally speaking, the processing system 700 and / or its components can be configured to perform the methods described herein.

[0118] It is worth noting that, in other aspects, such as when the processing system 700 is a server computer, components of the processing system 700 may be omitted. For example, in other aspects, the multimedia component 710, the wireless connectivity component 712, the sensor processing unit 716, the ISP 718, and / or the navigation processor 720 may be omitted. Furthermore, various aspects of the processing system 700 may be distributed among multiple devices.

[0119] Example Terms

[0120] Specific implementation examples are described in the following numbered clauses:

[0121] Clause 1: A method comprising: accessing a set of parameters for a machine learning model, wherein the set of parameters is formatted according to a first encoding format; generating a converted set of parameters by applying a transformation operation to format the set of parameters according to a second encoding format; generating a set of bit planes by applying a bit plane transformation to the converted set of parameters; and generating a compressed set of parameters for the machine learning model by applying a bit masking operation to one or more of the bit planes in the set of bit planes.

[0122] Clause 2: The method according to Clause 1, wherein applying the bit plane transformation includes, for each corresponding bit plane of the set of bit planes, determining a corresponding word length for encoding the corresponding bit plane.

[0123] Clause 3: The method according to Clause 2, wherein determining the corresponding word length for encoding each corresponding bit plane includes adaptively selecting a first word length of the first bit plane of the set of bit planes, the adaptive selection including: encoding the first bit plane based on the first word length; determining a first sparsity value for the first bit plane encoded based on the first word length; encoding the first bit plane based on a second word length; determining a second sparsity value for the first bit plane encoded based on the second word length; and selecting the first word length of the first bit plane based on the first sparsity value and the second sparsity value.

[0124] Clause 4: The method according to any one of Clauses 1 to 3, wherein generating the compressed set of parameters comprises: determining, based on a first sparsity value of the first bit plane, to apply the bitmasking operation to the first bit plane of the set of bit planes; and determining, based on a second sparsity value of the second bit plane, not to apply the bitmasking operation to the second bit plane of the set of bit planes.

[0125] Clause 5: The method according to any one of Clauses 1 to 4, the method further comprising: generating a pruned set of bit planes based on pruning for encoding one or more words of the set of bit planes based on a magnitude threshold, wherein the magnitude threshold is a hyperparameter; decoding the pruned set of bit planes to generate a pruned set of parameters; updating one or more parameters of the pruned set of parameters using training data; and encoding the pruned set of parameters based on applying the transformation operation, the bit plane transformation, and the bitmasking operation to generate a pruned and compressed set of parameters.

[0126] Clause 6: The method according to Clause 5, wherein pruning the one or more words used to encode the set of bit planes comprises: determining that the one or more words have a magnitude value less than the magnitude threshold; and setting each of the one or more words to a value of zero.

[0127] Clause 7: The method according to any one of Clauses 1 to 6, wherein applying the bitmasking operation comprises, for at least the first bit plane of the set of bit planes: determining a pattern for encoding a set of words of the first bit plane; and compressing the first bit plane based on the pattern, the compression comprising: identifying one or more words of the set of words having a value matching the pattern; and replacing each of the one or more words with a corresponding mask bit indicating that the corresponding word has a value equal to the pattern.

[0128] Clause 8: The method according to any one of Clauses 1 to 7, wherein applying the conversion operation comprises: identifying one or more parameters in the set of parameters that have a value of zero; and encoding each of the one or more parameters without including a corresponding sign bit, wherein one or more other parameters having a non-zero value are encoded with a sign bit.

[0129] Clause 9: A processing system comprising: a memory including computer-executable instructions; and one or more processors configured to execute the computer-executable instructions and cause the processing system to perform a method according to any one of Clauses 1 to 8.

[0130] Clause 10: A processing system comprising components for performing the method according to any one of Clauses 1 to 8.

[0131] Clause 11: A non-transitory computer-readable medium comprising computer-executable instructions that, when executed by one or more processors of a processing system, cause the processing system to perform the method according to any one of Clauses 1 to 8.

[0132] Clause 12: A non-transitory computer-readable medium encoding logic that, when executed by a processing system, causes the processing system to perform the method according to any one of Clauses 1 to 8.

[0133] Clause 13: An apparatus comprising logic circuitry configured to perform the method according to any one of Clauses 1 to 8.

[0134] Clause 14: A computer program product embodied on a computer-readable storage medium, said computer-readable storage medium including code for performing the method according to any one of Clauses 1 to 8.

[0135] Additional Notes

[0136] The foregoing description is provided to enable any person skilled in the art to practice the various aspects described herein. The examples discussed herein do not limit the scope, applicability, or aspects set forth in the claims. Various modifications to these aspects will be apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects. For example, the function and arrangement of the elements discussed may be changed without departing from the scope of this disclosure. Various processes or components may be omitted, substituted, or added as appropriate in various examples. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Furthermore, features described with respect to some examples may be combined in some other examples. For example, any number of aspects set forth herein may be used to implement an apparatus or practice. Moreover, the scope of this disclosure is intended to cover such apparatuses or methods practiced using other structures, functionalities, or structures and functionalities that complement or replace the various aspects of this disclosure set forth herein. It should be understood that any aspect of this disclosure disclosed herein may be embodied by one or more elements of these claims.

[0137] As used herein, the term “exemplary” means “serving as an example, instance, or illustration.” Any aspect described as “exemplary” in this document is not necessarily to be construed as preferred or superior to other aspects.

[0138] As used in this article, the phrase “at least one of” in a list of entries refers to any combination of those entries, including a single member. For example, a, b, c, ab, ac, bc, and abc, as well as any combination with multiple identical elements (e.g., aa, aaa, aab, aac, abb, acc, bb, bbb, bbb, bbc, cc, and ccc, or any other ordering of a, b, and c).

[0139] As used herein, the term "determine" encompasses a wide variety of actions. For example, "determine" can include calculation, computation, processing, derivation, research, searching (e.g., looking in a table, database, or other data structure), ascertainment, and similar actions. Additionally, "determine" can include receiving (e.g., receiving information), accessing (e.g., accessing data in memory), etc. Furthermore, "determine" can include parsing, selecting, picking, building, etc.

[0140] The methods disclosed herein include one or more steps or actions for implementing the methods. The steps and / or actions of the methods may be interchanged without departing from the scope of the claims. In other words, unless a specific order of steps or actions is specified, the order and / or use of a particular step and / or action may be modified without departing from the scope of the claims. Furthermore, the various operations of the methods described above may be performed by any suitable component capable of performing the corresponding function. This component may include various hardware and / or software components and / or modules, including but not limited to circuits, application-specific integrated circuits (ASICs), or processors. Generally, where the operations illustrated in the figures are present, those operations may have corresponding components with similar numbering plus functional components.

[0141] The following claims are not intended to be limited to the aspects shown herein, but should be given the full scope consistent with the language of the claims. Within the claims, unless specifically stated otherwise, reference to the singular form of an element is not intended to mean “one and only one,” but rather “one or more.” Unless specifically stated otherwise, the term “some” refers to one or more. No element of any claim should be interpreted in accordance with 35 USC § 112(f) unless the element is expressly referred to by the phrase “for a component of,” or, in the case of a method claim, by the phrase “for a step of.” All structural and functional equivalents of the elements throughout the various aspects described herein that are known to a person of ordinary skill in the art, or will be known later, are expressly incorporated herein by reference and are intended to be covered by the claims. Furthermore, nothing disclosed herein is intended to be offered to the public, whether or not such disclosure is explicitly stated in the claims.

Claims

1. A processing system for data compression, the processing system comprising: The memory includes processor-executable instructions; and One or more processors, the one or more processors being configured to execute processor-executable instructions and enable the processing system: Access a set of parameters used for a machine learning model, wherein the set of parameters is formatted according to a first encoding format; A converted set of parameters is generated by applying a conversion operation to format the set of parameters according to a second encoding format. A set of bit planes is generated by applying a bit plane transformation to the already transformed set of parameters; as well as A compressed set of parameters for the machine learning model is generated by applying a bitmasking operation to one or more of the set of bit planes.

2. The processing system according to claim 1, wherein, In order to apply the bit-plane transformation, the one or more processors are configured to execute processor-executable instructions to cause the processing system to determine, for each corresponding bit-plane in the set of bit-planes, the corresponding word length for encoding the corresponding bit-plane.

3. The processing system according to claim 2, wherein, In order to determine the length of the corresponding word used to encode each corresponding bit plane, the one or more processors are configured to execute processor-executable instructions to enable the processing system to: The first bit plane of the set of bit planes is encoded based on the length of the first word; Determine a first sparsity value for the first bit plane encoded based on the length of the first word; The first bit plane is encoded based on the length of the second word; Determine a second sparsity value for the first bit plane encoded based on the second word length; as well as The first word length of the first bit plane is selected based on the first sparsity value and the second sparsity value.

4. The processing system according to claim 1, wherein, In order to generate the compressed set of parameters, the one or more processors are configured to execute processor-executable instructions to enable the processing system to: The bitmasking operation is applied to the first bit plane of the set of bit planes based on the first sparsity value of the first bit plane; as well as The bitmasking operation is not applied to the second bit plane of the set of bit planes based on the second sparsity value of the second bit plane.

5. The processing system of claim 1, wherein the one or more processors are configured to further execute processor-executable instructions to cause the processing system to: A pruned set of bit planes is generated by encoding one or more words of the set of bit planes based on a magnitude threshold, wherein the magnitude threshold is a hyperparameter. Decode the trimmed set of bit planes to generate a trimmed set of parameters; Use the training data to update one or more parameters of the pruned set of parameters; as well as The pruned set of parameters is encoded by applying the transformation operation, the bit plane transformation, and the bit mask operation to produce a pruned and compressed set of parameters.

6. The processing system according to claim 5, wherein, In order to prune the one or more words used to encode the set of bit planes, the one or more processors are configured to execute processor-executable instructions to make the processing system: Determine that the one or more words have a value less than the value threshold; as well as Set the value of each of the one or more words to zero.

7. The processing system according to claim 1, wherein, In order to apply the bitmasking operation, the one or more processors are configured to execute processor-executable instructions to cause the processing system to, for at least the first bit plane of the set of bit planes: Determine the pattern for a set of words used to encode the first bit plane; as well as The first bit plane is compressed based on the pattern, wherein, in order to compress the first bit plane based on the pattern, the one or more processors are configured to execute processor-executable instructions to cause the processing system to: Identify one or more words in the group of words that have values ​​matching the pattern; as well as Each corresponding word of the one or more words is replaced with a corresponding mask bit that indicates that the corresponding word has a value equal to the pattern.

8. The processing system according to claim 1, wherein, In order to apply the conversion operation, the one or more processors are configured to execute processor-executable instructions to enable the processing system to: Identify one or more parameters in the set of parameters that have a value of zero; and Each of the one or more parameters is encoded without including the corresponding sign bit, wherein one or more other parameters having a non-zero value are encoded with a sign bit.

9. A non-transitory computer-readable medium for data compression, the non-transitory computer-readable medium encoding logic that, when executed by a processing system, causes the processing system to: Access a set of parameters used for a machine learning model, wherein the set of parameters is formatted according to a first encoding format; A converted set of parameters is generated by applying a conversion operation to format the set of parameters according to a second encoding format. A set of bit planes is generated by applying a bit plane transformation to the already transformed set of parameters; as well as A compressed set of parameters for the machine learning model is generated by applying a bitmasking operation to one or more of the set of bit planes.

10. The non-transitory computer-readable medium according to claim 9, wherein, In order to apply the bit-plane transformation, the processing system is configured to execute the logic such that the processing system, for each corresponding bit-plane in the set of bit-planes, determines the corresponding word length for encoding the corresponding bit-plane.

11. The non-transitory computer-readable medium of claim 10, wherein, in order to determine the length of the corresponding word used to encode each corresponding bit plane, the processing system is configured to execute the logic such that the processing system: The first bit plane of the set of bit planes is encoded based on the length of the first word; Determine a first sparsity value for the first bit plane encoded based on the length of the first word; The first bit plane is encoded based on the length of the second word; Determine a second sparsity value for the first bit plane encoded based on the second word length; as well as The first word length of the first bit plane is selected based on the first sparsity value and the second sparsity value.

12. The non-transitory computer-readable medium according to claim 9, wherein, In order to generate the compressed set of parameters, the processing system is configured to execute the logic to make the processing system: The bitmasking operation is applied to the first bit plane of the set of bit planes based on the first sparsity value of the first bit plane; as well as The bitmasking operation is not applied to the second bit plane of the set of bit planes based on the second sparsity value of the second bit plane.

13. The non-transitory computer-readable medium according to claim 9, wherein, In order to apply the bitmasking operation, the processing system is configured to execute the logic such that the processing system, for at least the first bit plane of the set of bit planes: Determine the pattern for a set of words used to encode the first bit plane; as well as The first bit plane is compressed based on the pattern, wherein, in order to compress the first bit plane based on the pattern, the processing system is configured to execute the logic to make the processing system: Identify one or more words in the group of words that have values ​​matching the pattern; as well as Each corresponding word of the one or more words is replaced with a corresponding mask bit that indicates that the corresponding word has a value equal to the pattern.

14. The non-transitory computer-readable medium according to claim 9, wherein, In order to apply the conversion operation, the processing system is configured to execute the logic to make the processing system: Identify one or more parameters in the set of parameters that have a value of zero; and Each of the one or more parameters is encoded without including the corresponding sign bit, wherein one or more other parameters having a non-zero value are encoded with a sign bit.

15. An apparatus for data compression, the apparatus comprising logic circuitry configured to: Access a set of parameters used for a machine learning model, wherein the set of parameters is formatted according to a first encoding format; A converted set of parameters is generated by applying a conversion operation to format the set of parameters according to a second encoding format. A set of bit planes is generated by applying a bit plane transformation to the already transformed set of parameters; as well as A compressed set of parameters for the machine learning model is generated by applying a bitmasking operation to one or more of the set of bit planes.

16. The apparatus according to claim 15, wherein, In order to apply the bit-plane transformation, the logic circuit is configured to determine, for each corresponding bit-plane in the set of bit-planes, the corresponding word length for encoding the corresponding bit-plane.

17. The apparatus according to claim 16, wherein, To determine the length of the corresponding word used to encode each corresponding bit plane, the logic circuit is configured as follows: The first bit plane of the set of bit planes is encoded based on the length of the first word; Determine a first sparsity value for the first bit plane encoded based on the length of the first word; The first bit plane is encoded based on the length of the second word; Determine a second sparsity value for the first bit plane encoded based on the second word length; as well as The first word length of the first bit plane is selected based on the first sparsity value and the second sparsity value.

18. The apparatus according to claim 15, wherein, In order to generate the compressed set of parameters, the logic circuit is configured as follows: The bitmasking operation is applied to the first bit plane of the set of bit planes based on the first sparsity value of the first bit plane; as well as The bitmasking operation is not applied to the second bit plane of the set of bit planes based on the second sparsity value of the second bit plane.

19. The apparatus of claim 15, wherein the logic circuit is further configured to: A pruned set of bit planes is generated by encoding one or more words of the set of bit planes based on a magnitude threshold, wherein the magnitude threshold is a hyperparameter. Decode the trimmed set of bit planes to generate a trimmed set of parameters; Use the training data to update one or more parameters of the pruned set of parameters; as well as The pruned set of parameters is encoded by applying the transformation operation, the bit plane transformation, and the bit mask operation to produce a pruned and compressed set of parameters.

20. The apparatus according to claim 19, wherein, In order to trim the one or more words used to encode the set of bit planes, the logic circuit is configured as follows: Determine that the one or more words have a value less than the value threshold; as well as Set the value of each of the one or more words to zero.

21. The apparatus according to claim 15, wherein, In order to apply the bitmasking operation, the logic circuit is configured to target at least the first bit plane of the set of bit planes: Determine the pattern for a set of words used to encode the first bit plane; as well as The first bit plane is compressed based on the pattern, wherein, in order to compress the first bit plane based on the pattern, the logic circuit is configured as follows: Identify one or more words in the group of words that have values ​​matching the pattern; as well as Each corresponding word of the one or more words is replaced with a corresponding mask bit that indicates that the corresponding word has a value equal to the pattern.

22. The apparatus according to claim 15, wherein, In order to apply the conversion operation, the logic circuit is configured as follows: Identify one or more parameters in the set of parameters that have a value of zero; and Each of the one or more parameters is encoded without including the corresponding sign bit, wherein one or more other parameters having a non-zero value are encoded with a sign bit.

23. A processor implementation method for data compression, the method comprising: Access a set of parameters used for a machine learning model, wherein the set of parameters is formatted according to a first encoding format; A converted set of parameters is generated by applying a conversion operation to format the set of parameters according to a second encoding format; A set of bit planes is generated by applying a bit plane transformation to the already transformed set of parameters; as well as A compressed set of parameters for the machine learning model is generated by applying a bitmasking operation to one or more of the set of bit planes.

24. The processor implementation method of claim 23, wherein applying the bit plane transformation comprises determining, for each corresponding bit plane of the set of bit planes, a corresponding word length for encoding the corresponding bit plane.

25. The processor implementation method of claim 24, wherein determining the corresponding word length for encoding each corresponding bit plane includes adaptively selecting the first word length of the first bit plane of the set of bit planes, the adaptive selection including: The first bit plane is encoded using the first word length; Determine a first sparsity value for the first bit plane encoded using the first word length; The first bit plane is encoded using the second word length; Determine a second sparsity value for the first bit plane encoded using the second word length; as well as The first word length of the first bit plane is selected based on the first sparsity value and the second sparsity value.

26. The processor implementation method of claim 23, wherein generating the compressed set of parameters comprises: The bitmasking operation is applied to the first bit plane of the set of bit planes based on the first sparsity value of the first bit plane; as well as The bitmasking operation is not applied to the second bit plane of the set of bit planes based on the second sparsity value of the second bit plane.

27. The method implemented by the processor according to claim 23, the method further comprising: A pruned set of bit planes is generated by encoding one or more words of the set of bit planes based on a magnitude threshold, wherein the magnitude threshold is a hyperparameter. Decode the trimmed set of bit planes to generate a trimmed set of parameters; Use the training data to update one or more parameters of the pruned set of parameters; as well as The pruned set of parameters is encoded by applying the transformation operation, the bit plane transformation, and the bit mask operation to produce a pruned and compressed set of parameters.

28. The processor implementation method of claim 27, wherein pruning the one or more words used to encode the set of bit planes comprises: Determine that the one or more words have a value less than the value threshold; as well as Set the value of each of the one or more words to zero.

29. The processor implementation method of claim 23, wherein applying the bitmasking operation comprises, for at least the first bit plane of the set of bit planes: Determine the pattern for a set of words used to encode the first bit plane; as well as The first bit plane is compressed based on the pattern, the compression including: Identify one or more words in the group of words that have values ​​matching the pattern; as well as Each corresponding word of the one or more words is replaced with a corresponding mask bit that indicates that the corresponding word has a value equal to the pattern.

30. The processor implementation method of claim 23, wherein applying the conversion operation comprises: Identify one or more parameters in the set of parameters that have a value of zero; as well as Each of the one or more parameters is encoded without including the corresponding sign bit, wherein one or more other parameters having a non-zero value are encoded with a sign bit.

Images