Lossy recovery lossy significance compression

Abstract

A system and method for lossy compression and recovery of data are described. The original data is first truncated. The truncated data is then compressed. The compressed truncated data can then be stored and / or transmitted efficiently using fewer bits. To recover the data, the compressed data is then decompressed and the recovered bits are linked. The recovered bits are selected to compensate for the statistical bias introduced by the truncation.

Description

[0001] Cross-references to related applications

[0002] This application claims the benefit of U.S. nonprovisional patent application No. 16 / 220,540, filed December 14, 2018, the contents of which are incorporated herein by reference. Background Technology

[0003] Computer memory, caches, and links are designed to be lossless so that stored information can be accurately reproduced. However, in some applications such as machine learning, exact values ​​are not required. In fact, in many such cases, using exact values ​​leads to a performance degradation of the machine learning system without any other benefit. Attached Figure Description

[0004] A more detailed understanding can be obtained from the following description, given by way of example in conjunction with the accompanying drawings:

[0005] Figure 1 It is a block diagram of an example device in which one or more of the disclosed implementations can be implemented;

[0006] Figure 2A It is a block diagram based on an example compression unit;

[0007] Figure 2B This is a block diagram based on an example recovery unit;

[0008] Figure 2C This is an example flowchart of a technology used for compressing data;

[0009] Figure 2D This is an example flowchart of a technique used to recover compressed data;

[0010] Figure 3 This is an example of data compression implemented using only dictionary-based frequent value compression (FVC);

[0011] Figure 4 These are examples of data compression and recovery based on certain implementation methods;

[0012] Figure 5A This is a block diagram of a machine learning system that uses recovery units for evaluation;

[0013] Figure 5B This is a block diagram of a machine learning system that uses compression and recovery units for training;

[0014] Figure 6A Here is an example flowchart of machine learning techniques used to perform the evaluation; and

[0015] Figure 6B This is an example flowchart of machine learning techniques used to perform training. DETAILED DESCRIPTION

[0016] The present disclosure relates to techniques for lossy compression and recovery of data. According to the techniques, original data is truncated. The truncated data is then compressed. The compressed truncated data can then be efficiently stored and transmitted using fewer bits. To recover the data, the compressed data is decompressed. Recovery bits are then added to the decompressed data. In some implementations, the recovery bits are selected to compensate for statistical bias introduced by the truncation.

[0017] The disclosed systems and methods for lossy compression and recovery of data can be applied to central processing units (CPUs), graphics processing units (GPUs), accelerated processing units (APUs), field programmable gate arrays (FPGAs), or any other processing device that utilizes memory, cache, and interconnects. In some cases, the techniques for lossy compression and recovery of data can be used for memory compression, cache compression, register file compression, link compression, and other cases of data transmission and storage.

[0018] In many machine learning systems, workloads are affected by memory bottlenecks. Specifically, many machine learning workloads involve using artificial neural networks to generate one or more prediction scores based on one or more input values. Processing through artificial neural networks involves computing a transfer function for each neuron, where the inputs to each neuron are biased based on adjustable weights. In large neural network systems, the large number of neurons and thus the large number of weights results in a large amount of data being transferred between processing units and memory. Specifically, in the process of generating a prediction, a large amount of neuron weights can be transferred between memory and processing units. This fact can result in memory bandwidth becoming a bottleneck to the speed of generating such predictions.

[0019] An effective compression technique can be used to reduce the amount of data transferred between processing units and memory. The technique takes advantage of the fact that neuron weights typically have a "satisfactory" precision. Values more precise than that precision contribute little or nothing to the additional precision of the prediction model. Thus, the technique involves truncating the weight values to a certain precision and then compressing the result. A favorable technique for compression is a frequent value compression technique. In that technique, identical values in a data set are replaced by a key value in a dictionary. Truncating the weight values to a certain precision allows the dictionary to be constructed based on the higher significant bits of the weight values, which results in a higher likelihood that particular weight values can actually be replaced by a dictionary value. Recovery of the weight values involves decompressing the compressed values and then using the recovery bits in the least significant bits to recover the decompressed values. The particular recovery bits used can be fixed bits (such as the value 0), random bits, or can be selected according to any technically feasible technique. One benefit provided by random bits is that random bits reduce bias towards certain values that can be introduced by fixed bits. Fixed bits provide the benefit of ease of implementation. Further details are provided below.

[0020] Figure 1 is a block diagram of an example apparatus 100 in which one or more aspects of the present disclosure are implemented. The apparatus 100 includes, for example, a computer such as a server, a desktop or laptop computer, a gaming apparatus, a handheld apparatus, a set-top box, a television, a mobile phone, or a tablet computer. The apparatus 100 includes a processor 102, a memory 104, a storage 106, one or more input devices 108, and one or more output devices 110. The apparatus 100 optionally includes an input driver 112 and an output driver 114. It is understood that the apparatus 100 can include additional components not shown in FIG. 1 that do not affect the Figure 1 operation of the apparatus 100.

[0021] The processor 102 includes one or more of a central processing unit (CPU), a graphics processing unit (GPU), a CPU and GPU located on the same die, or one or more processor cores, where each processor core is a CPU or a GPU. The memory 104 is located on the same die as the processor 102 or is located separately from the processor 102. The memory 104 includes volatile or non-volatile memory, such as random access memory (RAM), dynamic RAM, or a cache.

[0022] The storage 106 includes a fixed or removable storage, such as a hard disk drive, a solid state drive, an optical disk, or a flash drive. The input devices 108 include one or more of a camera, a keyboard, a keypad, a touchscreen, a touchpad, a detector, a microphone, an accelerometer, a gyroscope, or a biometric scanner. The output devices 110 include one or more of a display, a speaker, a printer, a haptic feedback device, one or more lights, or an antenna.

[0023] Input driver 112 communicates with processor 102 and input device 108, and allows processor 102 to receive input from input device 108. Output driver 114 communicates with processor 102 and output device 110, and allows processor 102 to send output to output device 110.

[0024] Compression unit 105 and recovery unit 103 are shown in several different forms. Compression unit 105 receives data and outputs compressed data. Recovery unit 103 receives compressed data and outputs recovered data. In a first form, compression unit 105 and recovery unit 103 are software stored in memory 104 and executed on processor 102, as shown. In this form, compression unit 105 and / or recovery unit 103 can be stored in storage device 106 when not being executed. In a second form, compression unit 105 and recovery unit 103 are at least part of a hardware engine residing in output driver 114. In other forms, compression unit 105 and recovery unit 103 are a combination of software and hardware elements, wherein the hardware resides, for example, in output driver 114, and the software is executed, for example, on processor 102. Compression unit 105 stores compressed data in memory, such as memory 104, or any other memory, such as buffers stored in or associated with the hardware implementation of compression unit 105. Similarly, recovery unit 103 retrieves compressed data from memory, restores the data to a recovered state, and provides the restored data to other components within device 100. Although compression unit 105 and recovery unit 103 are described in some example implementations, it should be understood that the principles of compression and recovery can be used in any context where such compression and / or recovery are meaningful. Furthermore, in any particular implementation, compression, recovery, or both compression and recovery as described herein can be implemented.

[0025] In some cases, processor 102 uses compression unit 105 and / or recovery unit 103 to implement a set of instructions for loading and storing data. If processor 102 transmits lossy data, along with data to be stored lossily, to compression unit 105, compression unit 105 performs the techniques described herein for truncation and compression of data, and then outputs the data for storage. If processor 102 transmits lossy load instructions to recovery unit 103, specifying compressed data to be loaded lossily, recovery unit 103 extracts the compressed data, performs decompression and recovery techniques, and provides the decompressed and recovered data back to processor 102 for use.

[0026] In another example, compression and decompression are used for cache compression. In this example, when processor 102 executes a lossy load instruction (which targets one or more registers), compression unit 105 fetches (e.g., from backup memory), truncates and compresses cache lines, and stores the compressed cache lines in the cache. The cache then transfers a compressed version of the requested data to processor 102 for processing. The processor includes a recovery unit 103 that recovers data before placing it into its registers. When the processor executes a lossy store instruction, the values ​​in the registers are recompressed by compression unit 105, and these compressed values ​​are stored in the cache. When a cache write-back occurs, recovery unit 103 recovers the data and places it in backup memory.

[0027] In other cases, upon receiving a lossy load instruction, the cache reads the data from the cache, compresses it using compression unit 105, and sends the compressed data back to processor 102. Processor 102 then uses recovery unit 103 to recover the decompressed data.

[0028] Although it is stated that processor 102 can implement these instructions, any processing unit (including those described and not described in this disclosure) can implement and execute one or both of these instructions. Furthermore, in different implementations, compressed data may be used only in the cache, only in the backup memory, or a combination of both. The techniques described above for storing and using compressed data in the cache are merely examples, and it should be understood that compression unit 105 and recovery unit 103 can be used in any technically feasible manner to compress and recover data for use by the processor and storage devices in the cache or backup memory.

[0029] Figure 2A This is a block diagram of an example compression unit 105. Compression unit 105 includes a cutoff unit 202 coupled to a compression engine 204. In some cases, the cutoff unit 202 and the compression engine 204 are implemented as fixed-function circuitry. In other cases, the cutoff unit 202 and the compression engine 204 are implemented as software or firmware executed on a processor. In still other cases, the cutoff unit 202 and the compression engine 204 are implemented as a combination of fixed-function circuitry and software.

[0030] The truncation unit 202 receives raw data 405. In some cases, the raw data 405 is retrieved from memory 104, storage device 106, input driver 112, or cache line. The truncation unit 202 forms truncated data 410 by truncating bits of each fixed-size data segment (e.g., word) in the raw data 405. In some implementations, the size of each fixed-size data segment is the same as the size of a word in a computer system including the compression unit 105. In one example, this size is 32 bits. In other examples, the size of the fixed-size data segment processed by the compression unit 105 is different from the size of a word in a computer system including the compression unit 105. Any technically feasible size of the fixed-size data segment can be used.

[0031] The number of bits truncated by truncation unit 202 is set by parameter k. In some cases, parameter k is a software-defined parameter. If defined via software, it can be stored in a portion of memory 104. Alternatively, the parameter can be passed to truncation unit 202 as a parameter (either as a register parameter or as an immediate value) from an instruction (e.g., a lossy memory instruction) from processor 102 or output driver 114.

[0032] In one implementation, when compression unit 105 is used in a machine learning system such as machine learning system 500B, the machine learning system can pass a gradually decreasing value of k to compression unit 102 as the training process brings the neural network model closer to its final state. By gradually decreasing the value of k to achieve a gradually decreasing trade-off compression density for algorithm accuracy, processing speed can be improved in the early stages of training where high accuracy is not required. Subsequently, for example, after a certain number of training iterations, parameter k can be reduced to improve the accuracy of the predictive model. In other cases, parameter k is modified based on convergence criteria or other dynamic metrics associated with the machine learning algorithm.

[0033] In other cases, parameter k is determined based on the available resources of device 100. In other cases, k is a fixed number. In yet another case, parameter k is dynamically determined based on previously compressed data. For example, an initial k value can be used to compress the data. The compressed data is then analyzed to determine the effect of a larger k value. If the effect is below a predetermined threshold, a larger k value is used by compression unit 105.

[0034] The truncated data 410 is then received by the compression engine 204. The compression engine 204 performs a compression algorithm on the truncated data 410 to form compressed data 415. In some cases, the compression algorithm uses dictionary-based frequent value compression (FVC). In other cases, run-length compression, Huffman-based compression, or radix-incremental compression can be used. In alternative implementations, any technically feasible compression algorithm or combination of algorithms can be used. The compressed data 415 is then output to other components within the device 100.

[0035] Figure 2B This is a block diagram of an example recovery unit 103. Recovery unit 103 includes a decompression unit 206 coupled to recovery unit 208. In some cases, decompression unit 206 and recovery unit 208 are implemented as fixed-function circuits. In other cases, decompression unit 206 and recovery unit 208 are implemented by a processor executing software or firmware. In still other cases, decompression unit 206 and recovery unit 208 are implemented as a combination of fixed-function circuits and processor-implemented software.

[0036] Compressed data 415 is received by decompression unit 206. In some cases, compressed data 415 is retrieved from memory 104, storage device 106, input driver 112, or cache line. Decompression engine 206 performs a decompression algorithm on compressed data 415 to form decompressed truncated data 420. The type of decompression performed is determined based on the compression type used by compression unit 105. For example, when using FVC compression, decompressed truncated data is formed based on dictionary 435.

[0037] Then, recovery unit 208 receives the decompressed truncated data 420. Recovery unit 208 selects recovery bit 425. Then, recovery unit 208 links recovery bit 425 to the decompressed truncated data 420 to form recovered data 430. Then, the recovered data 430 is output to other components within device 100.

[0038] The number of recovery bits 425 is equal to the parameter k. In some cases, the truncated data 420 is chosen as the recovery bits 425 of a fixed value (such as all zeros). However, using a fixed value (such as all zeros) rounds the data toward negative infinity, thus introducing a bias. Higher values ​​(such as "FF") round the data toward positive infinity. In either case, the fixing technique causes a value bias. Introducing random values ​​helps mitigate this bias.

[0039] As a result, in other cases, recovery unit 208 uses a random recovery process to select recovery bit 425. In random recovery, statistical techniques are used to select recovery bit 425.

[0040] In one example, the random recovery process uses a pseudo-random number generator to generate recovery bit 425. In other implementations, recovery unit 208 implements the random recovery process by sampling values ​​from a probability distribution. In some cases, the probability distribution is parameterized. An example of a parameterized probability distribution that can be used is the Poisson distribution. In these cases, recovery unit 208 determines the parameters of the parameterized distribution by retrieving parameters from memory or dynamically determining values ​​based on previously recovered data. In other cases, recovery unit 208 utilizes a non-parameterized probability distribution. In some cases, the non-parameterized distribution is predefined and retrieved by recovery unit 208 from memory. In other cases, the non-parameterized distribution is dynamically determined by recovery unit 208 based on previously recovered data. For example, previously recovered data can be used to form Markov chains and Bayesian networks.

[0041] In some cases, recovery unit 208 selectively performs a random recovery process. For example, it may not be desirable to perform random recovery on zero values. Therefore, in this case, recovery unit 208 does not perform a random recovery process, but instead selects a recovery bit that is all zeros. More specifically, in some implementations, the uncompressed value is 0, and therefore the truncated value is also 0. When this value is encountered, instead of randomly selecting recovery bit 425 in this case, recovery unit 208 selects a value of 0 for recovery bit 425, making the result a value of 0. The reason for this is that if the initial data is 0, it may not be appropriate to recover the data to a value other than 0, which could happen if the random bits are recovered to the least significant bit.

[0042] Figure 2C This is a flowchart of process 200C implemented by compression unit 105. In step 211, raw data 405 is received. In some cases, raw data 405 is retrieved from memory 104, storage device 106, or input driver 112.

[0043] Optionally, in step 212, the number of k bits to be truncated is determined. In some cases, the parameter k is a software-defined parameter. When k is software-defined, this value is retrieved from memory 104 or passed to the truncation unit 202 via a parameter from processor 102 or output driver 114. In other cases, k is a fixed number.

[0044] In other cases, the parameter k is dynamically determined based on previously compressed data. For example, an initial k value can be used to compress the data. The compressed data is then analyzed to determine the impact of a larger k value. If the impact is below a predetermined threshold, a larger k value is determined as the output of step 212.

[0045] Then, in step 213, the original data 405 is truncated to form truncated data 410. The original data 405 is truncated by removing k bits from each fixed-size data segment (e.g., a word). In some implementations, the size of each fixed-size data segment is the same as the size of a word in a computer system including compression unit 105. In this example, this size is 32 bits. In other examples, the size of the fixed-size data segment processed by compression unit 105 is different from the size of a word in a computer system including compression unit 105. Any technically feasible size of the fixed-size data segment can be used.

[0046] Next, in step 214, the truncated data 410 is compressed to form compressed data 415. In many cases, step 214 is performed by compression engine 204. In some cases, the compression in step 330 is performed using dictionary-based frequent value compression (FVC). In alternative implementations, any technically feasible compression algorithm can be used.

[0047] These compression algorithms operate by analyzing data blocks and finding repeating values ​​that can be re-encoded more efficiently. By compressing truncated data 410, the compression algorithm is able to locate patterns in the data of spatially localized values ​​that are similar but not exactly identical. Traditionally, compression algorithms are not well-suited to such patterns because, although the values ​​are similar, they are not exactly the same (or sufficiently identical, depending on the specific compression algorithm). Therefore, compression algorithms become less efficient.

[0048] Then, in step 215, compressed data 415 is transmitted. In some cases, compressed data 415 is transmitted to memory 104, storage device 106, or output driver 114.

[0049] Figure 2D This is a flowchart of recovery process 200D. In step 221, compressed data 415 generated by process 200C is received. In some cases, compressed data 415 is retrieved from memory 104, storage device 106, or input driver 112.

[0050] In step 222, the compressed data 415 is decompressed to form decompressed truncated data 420. The type of decompression performed is determined based on the compression type used in step 214. For example, when using FVC compression, the decompressed truncated data is formed based on dictionary 435.

[0051] Next, in step 223, the recovery bits 425 are linked to the decompressed truncated data to form the recovered data 430. The number of recovery bits 425 is equal to the parameter k. In some cases, the recovery bits 425 are all zeros. In other cases, a random recovery process is used to determine the recovery bits. In random recovery, statistical techniques are used to select the recovery bits 425.

[0052] In some implementations, the random recovery process uses a pseudo-random number generator. In other implementations, the random recovery process involves sampling values ​​from a probability distribution. In some cases, the probability distribution is parameterized. For example, it is a Poisson distribution. In these cases, the parameters of the parameterized distribution can be predetermined or dynamically determined based on previously recovered data. In other cases, the probability distribution can be non-parameterized. In some cases, the non-parameterized distribution can be predefined, and in others, it is dynamically determined based on previously recovered data. For example, previously recovered data can be used to form Markov chains and Bayesian networks. By implementing the random recovery process, the statistical bias introduced by truncation in step 320 is reduced.

[0053] Then, in step 224, the recovered data 430 is transmitted. In some cases, the recovered data 430 is transmitted to the processor 102, memory 104, storage device 106, or output driver 114.

[0054] Figure 3 An example of using FVC alone is shown.

[0055] In FVC, data patterns are identified and stored in a dictionary (such as dictionary 435). Instead of storing the entire data pattern in memory, the indexes of the dictionary entries are stored in memory. More specifically, the FVC algorithm identifies frequently occurring patterns and creates keys for those patterns. The relationship between each pattern and each key is stored in the dictionary. By storing only the keys for the patterns in memory, fewer storage bits are required. For decompression, data is retrieved from memory, and the patterns stored in the dictionary are recovered using the dictionary keys. This recovery is performed for each key in the compressed data, such that in the recovered data, the keys are replaced with dictionary values, thus restoring the data to its uncompressed form.

[0056] exist Figure 3In the example of FVC shown, the original data 305 consists of 256 bits comprising eight 32-bit words. In this example, each original 32-bit word is replaced with a 2-bit code 310 if it can be compressed (i.e., if a dictionary value matching that 32-bit word exists). If the value of a word is not frequently used, it may remain uncompressed so that including those values ​​in the dictionary does not result in a net loss of the total stored data. The 2-bit code 310 indicates whether the value is uncompressed (00) or compressed, and specifies which of the maximum three dictionary entries 315 is used. In this example, only a few of the 32-bit words can be compressed due to the diversity of values ​​and the limited dictionary size. Therefore, this example shows that 208 bits are required when using FVC alone.

[0057] Figure 4 An example demonstrating the improved efficiency of process 200C is shown. In this example, raw data 405 is received according to step 211. In this example, raw data 405 consists of 256 bits representing eight 32-bit words. In step 212, parameter k is set to 2, and the least significant bit is truncated. Raw data 405 is then truncated according to step 213 to form truncated data 410. Truncated data 410 is then compressed according to step 214 to form compressed data 415. In this example, FVC compression is used in conjunction with dictionary 435 in step 214. By applying process 300C to raw data 405, only 40 bits are required compared to the 208 bits required when using FVC alone.

[0058] Figure 4 The recovery of compressed data 415 according to recovery process 200D is also illustrated. In step 221, compressed data 415 is received. Then, compressed data 415 is decompressed according to step 222 to form decompressed truncated data 420. In this example, decompression is performed using FVC and dictionary 435. The recovered bits 425 are then linked to the decompressed truncated data 420 according to step 335 to form the recovered data 430.

[0059] Figure 5A This is a block diagram of a machine learning system 500A that uses recovery unit 103 for evaluation. System 500A includes a weight database 502. The weights stored in database 502 assign relative importance to each of the multiple inputs received by the neuron nodes in evaluator 504.

[0060] Recovery unit 103 receives weights as input from database 502. Recovery unit 103 then executes process 200D to generate recovered data, which is transmitted to evaluator 504.

[0061] The evaluator 504 receives recovered weights from the recovery unit 103. The evaluator 504 consists of multiple interconnected neuron nodes. Each interconnected neuron receives multiple inputs from other neuron nodes or input sources 506. Each neuron then computes a separate output based on the weights received from the database 502 and the transfer function 508. The evaluator 504 outputs predicted data 510 based on the separate outputs of each of the multiple interconnected neuron nodes.

[0062] Figure 5B This is a block diagram of a machine learning system 500B trained using a compression unit 105 and a recovery unit 103. The machine learning system 500B includes a trainer 514 that determines weights to be stored in a database 502 based on training data 512. The trainer 514 determines the weights by comparing predicted data generated by an evaluator 504 with a predetermined output of the training data. The trainer 514 adjusts the weights based on a feedback mechanism.

[0063] Once determined by the trainer 514, the weights are stored in the database 502 using the compression unit 105. The compression unit 105 receives the weights as input. The compression unit 105 then executes process 200C to generate the data stored in the database 502.

[0064] In some cases, the value of k used by the compression unit 105 is dynamically changed during the training process. For example, in the early iterations of the training process, a higher level of noise in the weights can be tolerated. Therefore, a larger k value is used. In this case, the trainer 514 passes a smaller parameter to the compression unit 105. However, as training progresses and the weights begin to converge, the training process's tolerance for noise in the weights decreases. Therefore, a smaller k value is used. In this case, the trainer 514 passes a smaller k parameter to the compression unit 105.

[0065] Figure 6A This is a flowchart of evaluation process 600A. In step 610, the compressed weights are retrieved. Then, in step 620, a recovery process 200D is performed on the retrieved compressed weights to generate the recovered weights. Then, in step 630, the neural network generates an output based on the recovered weights.

[0066] Figure 6BThis is a flowchart of training process 600B. In step 605, compressed weights are retrieved. Then, in step 615, a recovery process 200D is performed on the retrieved compressed weights to generate recovered weights. Then, in step 625, the neural network generates output using the recovered weights. Then, in step 635, a training process is performed on the output. The result of the training process is the new weights. Then, in step 645, the new weights are compressed according to process 200C. Then, in step 655, the compressed new weights are stored.

[0067] It should be understood that many variations are possible based on the disclosure herein. Although features and elements are described above in specific combinations, each feature or element can be used alone without other features and elements, or in various combinations with or without other features and elements.

[0068] The provided method can be implemented in a general-purpose computer, processor, or processor core. Suitable processors include, for example, general-purpose processors, special-purpose processors, conventional processors, digital signal processors (DSPs), multiple microprocessors, one or more microprocessors associated with a DSP core, controllers, microcontrollers, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), any other type of integrated circuit (IC), and / or state machines. Such processors can be manufactured by configuring the manufacturing process using the results of processed hardware description language (HDL) instructions and other intermediate data, including netlists (such instructions can be stored on a computer-readable medium). The result of such processing can be a mask, which is then used in the semiconductor manufacturing process to manufacture processors that implement various aspects of the implementation scheme.

[0069] The methods or flowcharts provided herein can be implemented in a computer program, software, or firmware incorporated in a non-transitory computer-readable storage medium for execution by a general-purpose computer or processor. Examples of non-transitory computer-readable storage media include read-only memory (ROM), random access memory (RAM), registers, cache memory, semiconductor memory devices, magnetic media (such as internal hard disks and removable disks), magneto-optical media, and optical media such as CD-ROMs and digital versatile optical discs (DVDs).

Claims

1. A data storage and retrieval method comprising: receiving data; truncating a predetermined number of least significant bits of the data to form truncated data, wherein the predetermined number of bits is dynamically determined based on any one or a combination of: a software-defined parameter, a parameter passed in an instruction, a number of training iterations of a machine learning system, wherein fewer bits are truncated as the number of training iterations increases, available resources of a device, or a metric associated with previously compressed data; compressing the truncated data to form compressed data; and storing the compressed data in a memory or transmitting the compressed data to a receiving system.

2. The method of claim 1, further comprising: retrieving the compressed data; decompressing the compressed data to form decompressed truncated data; and linking a plurality of recovery bits to the decompressed truncated data to form recovered data, wherein a number of the recovery bits corresponds to the predetermined number of least significant bits of the data.

3. The method of claim 2, wherein the recovery bits are selected by sampling from a statistical distribution.

4. The method of claim 3, wherein the statistical distribution is a non-parametric distribution.

5. The method of claim 4, wherein the non-parametric distribution is determined based on previously recovered data.

6. The method of claim 3, wherein the statistical distribution is a parametric distribution.

7. The method of claim 6, wherein the parametric distribution is determined based on previously recovered data.

8. The method of claim 1, wherein the predetermined number of bits is determined based on previously compressed data or a desired level of noise.

9. The method of claim 1, further comprising: receiving the predetermined number of bits via a software-passed parameter.

10. A data storage and retrieval system comprising: a memory; and a processor communicatively coupled to the memory, wherein the processor: retrieves data from the memory, truncates a predetermined number of least significant bits of the data to form truncated data, wherein the predetermined number of bits is dynamically determined based on any one or a combination of: a software-defined parameter, a parameter passed in an instruction, a number of training iterations of a machine learning system, wherein fewer bits are truncated as the number of training iterations increases, available resources of a device, or a metric associated with previously compressed data; compresses the truncated data to form compressed data; and stores the compressed data in the memory or transmits the compressed data to a receiving system.

11. The system of claim 10, wherein the processor further: retrieves the compressed data; decompresses the compressed data to form decompressed truncated data; and links a plurality of recovery bits to the decompressed truncated data to form recovered data, wherein a number of the recovery bits corresponds to the predetermined number of least significant bits of the data.

12. The system of claim 11, wherein the recovery bits are selected by sampling from a statistical distribution.

13. The system of claim 12, wherein the statistical distribution is a non-parametric distribution.

14. The system of claim 13, wherein the non-parametric distribution is determined based on previously recovered data.

15. The system of claim 12, wherein the statistical distribution is a parametric distribution.

16. The system of claim 15, wherein the parametric distribution is determined based on previously recovered data.

17. The system of claim 10, wherein the predetermined number of bits is determined based on previously compressed data or an expected level of noise.

18. The system of claim 10, wherein the processor further: receives the predetermined number of bits via a parameter passed by software.

19. A non-transitory computer-readable storage medium storing instructions that, when executed by a processor, cause the processor to: retrieve data from a memory, truncate a predetermined number of least significant bits of the data to form truncated data, wherein the predetermined number of bits is dynamically determined based on any one or a combination of: a software-defined parameter, a parameter passed in an instruction, a number of training iterations of a machine learning system, wherein fewer bits are truncated as the number of training iterations increases, available resources of a device, or a metric associated with previously compressed data; compress the truncated data to form compressed data; and store the compressed data in the memory or transmit the compressed data to a receiving system.

20. The non-transitory computer-readable storage medium of claim 19, wherein the instructions further cause the processor to: retrieve the compressed data; decompress the compressed data to form decompressed truncated data; and link a number of recovery bits to the decompressed truncated data to form recovered data, wherein the number of recovery bits corresponds to the predetermined number of least significant bits of the data.

Images