Irreversible significant compression with irreversible restoration

Abstract

A system and method for lossy compression and decompression of data is described. First, raw data is truncated. Then, the truncated data is compressed. The compressed truncated data can then be efficiently stored and / or transmitted using fewer bits. To recover the data, the compressed data is decompressed and the recovered bits are concatenated. The recovered bits are selected to compensate for statistical biases introduced by the truncation.

Description

Technical Field

[0001] (Cross - Reference to Related Applications) This application claims the benefit of U.S. Patent Application No. 16 / 220,540, filed on December 14, 2018, the content of which is incorporated herein by reference.

Background Art

[0002] Computer memories, caches, and links are designed to be lossless in order to accurately reproduce the stored information. However, in some applications such as machine learning, exact values are not required. In fact, in many of these situations, using exact values provides no additional benefit and degrades the performance of the machine learning system.

[0003] A more detailed understanding can be obtained from the following description in conjunction with the accompanying drawings.

Brief Description of the Drawings

[0004] [Figure 1] It is a block diagram of an exemplary device in which one or more of the disclosed embodiments may be implemented. [Figure 2A] It is a block diagram of a compression unit according to an example. [Figure 2B] It is a block diagram of a restoration unit according to an example. [Figure 2C] It is an exemplary flowchart of a technique for compressing data. [Figure 2D] It is an exemplary flowchart of a technique for restoring compressed data. [Figure 3] It is a diagram showing an example of data compression implemented using only dictionary - based frequent value compression (FVC). [Figure 4] It is a diagram showing an example of compression and restoration of data according to a specific embodiment. [Figure 5A]This is a block diagram of a machine learning system that uses a recovery unit for evaluation. [Figure 5B] This is a block diagram of a machine learning system that uses compression and decompression units for training. [Figure 6A] This is an illustrative flowchart of the machine learning techniques used to perform the evaluation. [Figure 6B] This is an illustrative flowchart of the machine learning techniques used to perform training. [Modes for carrying out the invention]

[0005] This disclosure relates to techniques for lossy compression and data recovery. According to these techniques, raw data is truncated. The truncated data is then compressed. The compressed truncated data can then be efficiently stored and transmitted using fewer bits. To recover the data, the compressed data is decompressed. Recovery bits are then added to the decompressed data. In some embodiments, the recovery bits are selected to compensate for statistical bias introduced by the truncation.

[0006] The disclosed systems and methods for lossy compression and decompression of data may be applied to central processing units (CPUs), graphics processing units (GPUs), accelerator processing units (APUs), field-programmable gate arrays (FPGAs), or any other processing devices that utilize memory, caches, and interconnects. In some cases, the techniques for lossy compression and decompression of data may be used in memory compression, cache compression, register file compression, link compression, and other examples of data transmission and storage.

[0007] In many machine learning systems, the workload is bottlenecked by memory. Specifically, many machine learning workloads involve using artificial neural networks to generate one or more prediction scores based on one or more input values. Processing through artificial neural networks involves calculating the transfer function of each neuron, and the input to each neuron is biased based on adjustable weights. In large neural network systems, a large amount of data is transmitted between the processing unit and memory by a large number of neurons and thus weights. Specifically, in the process of generating predictions, the weights of many neurons may be transmitted between memory and the processing unit. This fact means that memory bandwidth can become a bottleneck in the speed at which such predictions are generated.

[0008] Effective compression techniques can reduce the amount of data transmitted between the processing unit and memory. This technique leverages the fact that neuron weights often have "sufficient" precision. Values ​​more precise than this provide little to no further precision to the predictive model. Therefore, this technique involves truncating weight values ​​to a specific precision and then compressing the results. A suitable compression technique is the frequent value compression technique. In this technique, identical values ​​in the dataset are replaced with key values ​​in a dictionary. By truncating weight values ​​to a specific precision, a dictionary can be constructed based on the most significant bit of the weight values, making it more likely that a particular weight value can actually be replaced with a dictionary value. Restoring weight values ​​involves decompressing the compressed values ​​and then restoring the decompressed values ​​using the least significant bit as a restoration bit. The specific restoration bit used may be a fixed bit (e.g., value 0) or a random bit, or may be selected according to technically feasible techniques. Random bits offer the advantage of reducing the bias towards specific values ​​that may be introduced with fixed bits. Fixed bits offer the advantage of being easy to implement. Further details are provided below.

[0009] Figure 1 is a block diagram of an exemplary device 100 in which one or more embodiments of the present disclosure are carried out. Device 100 includes, for example, a computer (e.g., a server, desktop, laptop computer, etc.), a gaming device, a handheld device, a set-top box, a television, a mobile phone, or a tablet computer. Device 100 includes a processor 102, memory 104, storage device 106, one or more input devices 108, and one or more output devices 110. Optionally, device 100 includes an input driver 112 and an output driver 114. It will be understood that device 100 may optionally include additional components not shown in Figure 1.

[0010] The processor 102 includes one or more of the following: a central processing unit (CPU), a graphics processing unit (GPU), a CPU and GPU located on the same die, and one or more processor cores, where each processor core is either a CPU or a GPU. The memory 104 is located on the same die as the processor 102 or separately from the processor 102. The memory 104 includes volatile or non-volatile memory (e.g., random access memory (RAM), dynamic RAM, cache, etc.).

[0011] The storage device 106 includes a fixed storage device or a removable storage device (e.g., a hard disk drive, solid-state drive, optical disk, or flash drive). The input device 108 includes one or more of the following: a camera, keyboard, keypad, touchscreen, touchpad, detector, microphone, accelerometer, gyroscope, or biometric scanner. The output device 110 includes one or more of the following: a display, speaker, printer, haptic feedback device, one or more lights, or antenna.

[0012] The input driver 112 communicates with the processor 102 and the input device 108, enabling the processor 102 to receive input from the input device 108. The output driver 114 communicates with the processor 102 and the output device 110, enabling the processor 102 to send output to the output device 110.

[0013] The compression unit 105 and the decompression unit 103 are shown in several different forms. The compression unit 105 receives data and outputs compressed data. The decompression unit 103 receives compressed data and outputs decompressed data. In the first form, the compression unit 105 and the decompression unit 103 are software stored in memory 104 and executed on the processor 102 as shown. In this form, if not being executed, the compression unit 105 and / or the decompression unit 103 may be stored in the storage device 106. In the second form, the compression unit 105 and the decompression unit 103 are at least part of a hardware engine residing in the output driver 114. In other forms, the compression unit 105 and the decompression unit 103 are a combination of software and hardware elements, where the hardware resides, for example, in the output driver 114, and the software is executed, for example, on the processor 102. The compression unit 105 stores the compressed data in memory such as memory 104, or in any other memory such as a buffer stored in or associated with the hardware implementation of the compression unit 105, or any other memory. Similarly, the decompression unit 103 retrieves the compressed data from memory, restores the data to its decompressed state, and provides the decompressed data to other components within device 100. Although the compression unit 105 and the decompression unit 103 are described in a particular exemplary mode of implementation, it should be understood that the principles of compression and decompression are applicable in any situation in which such compression and / or decompression makes sense. Furthermore, in any particular embodiment, compression, decompression, or both compression and decompression as described herein may be performed.

[0014] In some cases, the processor 102 implements a set of instructions for loading and storing data using the compression unit 105 and / or the decompression unit 103. If the processor 102 sends lossy store data to the compression unit 105 along with the data to be stored in a lossy manner, the compression unit 105 performs the techniques described herein to truncate and compress the data, and then outputs the data for storage. If the processor 102 sends a lossy load instruction to the decompression unit 103 specifying compressed data to be loaded in a lossy manner, the decompression unit 103 fetches the compressed data, performs decompression and decompression techniques, and returns the decompressed and restored data to the processor 102 for use.

[0015] In another example, compression and decompression are used for cache compression. In this example, when processor 102 executes a lossy load instruction (targeting one or more registers), compression unit 105 fetches, truncates, and compresses the cache lines (e.g., from auxiliary memory) and stores the compressed cache lines in the cache. The cache then sends the compressed version of the requested data to processor 102 for processing. The processor includes a restore unit 103 that restores the data before it is placed in the registers. When the processor executes a lossy store instruction, the values ​​in the registers are recompressed by compression unit 105, and these compressed values ​​are stored in the cache. When a cache write-back occurs, restore unit 103 restores the data and places it in auxiliary memory.

[0016] In another example, when the cache receives a lossy load instruction, it reads the data from the cache, compresses the data using the compression unit 105, and sends the compressed data back to the processor 102. The processor 102 then uses the decompression unit 103 to restore the decompressed data.

[0017] Although it is stated that processor 102 can implement these instructions, any processing unit, including those described and not described in this disclosure, can implement and execute any or both of these instructions. Furthermore, in different embodiments, compressed data may be used in a cache only, auxiliary memory only, or a combination of both cache and auxiliary memory. The above techniques for storing and using compressed data in a cache are merely examples, and it should be understood that the compression unit 105 and the decompression unit 103 can be used in any technically feasible way to compress and decompress data used by the processor and store it in a cache or auxiliary memory.

[0018] Figure 2A is a block diagram of an exemplary compression unit 105. The compression unit 105 includes a truncation unit 202 coupled to a compression engine 204. In some cases, the truncation unit 202 and the compression engine 204 are implemented as fixed-function circuits. In other examples, the truncation unit 202 and the compression engine 204 are implemented as software or firmware running on a processor. In yet another case, the truncation unit 202 and the compression engine 204 are implemented as a combination of fixed-function circuits and software.

[0019] Raw data 405 is received by the truncation unit 202. In some cases, the raw data 405 is obtained from the memory 104, the storage device 106, the input driver 112, or the cache line. The truncation unit 202 forms truncated data 410 by truncating the bits of each fixed-size data (e.g., word) in the raw data 405. In some embodiments, each fixed-size data is the same size as a word in the computer system that includes the compression unit 105. For example, this size is 32 bits. In other examples, the fixed-size data processed by the compression unit 105 is a size different from the size as a word in the computer system that includes the compression unit 105. Any technically feasible size can be used as the fixed-size data.

[0020] The number of bits truncated by the truncation unit 202 is set by the parameter k. In some cases, the parameter k is a software-defined argument. When defined by software, the parameter can be stored in a part of the memory 104. Alternatively, the parameter can be passed to the truncation unit 202 as an argument (e.g., by a register argument or an immediate value) to an instruction (e.g., an irreversible store instruction) from the processor 102 or the output driver 114.

[0021] In one embodiment, when the compression unit 105 is used in a machine learning system such as the machine learning system 500B, the machine learning system can pass a value that gradually decreases the value of k to the compression unit 105 as the training process approaches the final state of the neural network model. By gradually reducing the value of k and gradually trading off the compression density and the accuracy of the algorithm, the processing speed in the initial stage of training that does not require high accuracy can be improved. Later, for example, after training is repeated a certain number of times, the parameter k can be reduced to improve the accuracy of the prediction model. In other examples, the parameter k is changed based on a convergence criterion or other dynamic metric related to the machine learning algorithm.

[0022] In other examples, the parameter k is determined based on the available resources of device 100. In other examples, k is a constant. In still other examples, the parameter k is dynamically determined based on previously compressed data. For example, an initial value of k can be used to compress data. Next, the compressed data is analyzed to determine the impact of a larger k value. If the impact is below a predetermined threshold, the compression unit 105 uses the larger k value.

[0023] Next, the discarded data 410 is received by the compression engine 204. The compression engine 204 executes a compression algorithm on the discarded data 410 to form compressed data 415. In some cases, the compression algorithm uses dictionary-based frequent value compression (FVC). In other examples, run-length compression, Huffman-based compression, or base-delta compression can be used. In alternative embodiments, any technically feasible compression algorithm or combination of algorithms can be used. Next, the compressed data 415 is output to other components within the device 100.

[0024] FIG. 2B is a block diagram of an exemplary restoration unit 103. The restoration unit 103 includes a decompression unit 206 coupled to a restoration unit 208. In some cases, the decompression unit 206 and the restoration unit 208 are implemented as fixed-function circuits. In other examples, the decompression unit 206 and the restoration unit 208 are implemented by a processor that executes software or firmware. In still other cases, the decompression unit 206 and the restoration unit 208 are implemented as a combination of fixed-function circuits and processor-implemented software.

[0025] The compressed data 415 is received by the decompression unit 206. In some cases, the compressed data 415 is obtained from memory 104, storage device 106, input driver 112, or cache line. The decompression unit 206 executes a decompression algorithm on the compressed data 415 to form the decompressed truncated data 420. The type of decompression performed is determined based on the type of compression used by the compression unit 105. For example, if FVC compression is used, the decompressed truncated data is formed based on dictionary 435.

[0026] Next, the decompressed truncated data 420 is received by the recovery unit 208. The recovery unit 208 selects the recovery bit 425. Then, the recovery unit 208 concatenates the recovery bit 425 to the decompressed truncated data 420 to form the recovery data 430. The recovery data 430 is then output to other components within device 100.

[0027] The number of restore bits 425 is equal to the parameter k. In some cases, the truncated data 420 selects restore bits 425, which are all fixed values ​​such as zero. However, using fixed values ​​such as all zeros biases the data because it rounds towards negative infinity. Higher values ​​such as "FF" round the data towards positive infinity. In either case, the value shift is introduced by the fixed technique. Introducing random values ​​helps to mitigate this bias.

[0028] As a result, in other examples, the recovery unit 208 selects recovery bit 425 using a probabilistic recovery process. Probabilistic recovery uses statistical techniques to select recovery bit 425.

[0029] In one example, the probabilistic reconstruction process uses a pseudorandom number generator to generate reconstruction bits 425. In other embodiments, the reconstruction unit 208 performs the probabilistic reconstruction process by sampling values ​​from a probability distribution. In some cases, the probability distribution is parametric. An example of a usable parametric probability distribution is the Poisson distribution. In these cases, the reconstruction unit 208 determines the parameters of the parametric distribution by retrieving the parameters from memory or by dynamically determining the values ​​based on previously reconstructed data. In other examples, the reconstruction unit 208 utilizes a nonparametric probability distribution. In some cases, the nonparametric distribution is predefined and retrieved from memory by the reconstruction unit 208. In other examples, the nonparametric distribution is dynamically determined by the reconstruction unit 208 based on previously reconstructed data. For example, previously reconstructed data can be used to form Markov chains and Bayesian networks.

[0030] In some cases, the recovery unit 208 selectively performs a probabilistic recovery process. For example, it may not be desirable to perform a probabilistic recovery on zero values. In such cases, the recovery unit 208 does not perform a probabilistic recovery process and selects recovery bits that are all zero. More specifically, in some embodiments, the truncated value may also be zero because the uncompressed value is 0. In such cases, instead of probabilistically selecting recovery bit 425, the recovery unit 208 selects a value of 0 for recovery bit 425 so that the result is a value of 0. The reason for doing this is that if the original data is 0, it may not be appropriate to recover the data to a non-zero value. This occurs when a random bit is recovered to the least significant position.

[0031] Figure 2C is a flowchart of process 200C performed by the compression unit 105. In step 211, raw data 405 is received. In some cases, the raw data 405 is obtained from memory 104, storage device 106, or input driver 112.

[0032] Optionally, in step 212, the number of k bits to truncate is determined. In some cases, the parameter k is a software-defined argument. If k is software-defined, its value is obtained from memory 104 or passed to the truncation unit 202 via an argument from processor 102 or output driver 114. In other examples, k is a constant.

[0033] In yet another example, the parameter k is determined dynamically based on previously compressed data. For example, the data can be compressed using an initial value for k. The compressed data is then analyzed to determine the impact of a large k value. If the impact is below a predetermined threshold, a large k value is determined as the output of step 212.

[0034] Next, in step 213, the raw data 405 is truncated to form truncated data 410. The raw data 405 is truncated by truncating k bits of each fixed-size data (e.g., word) within the raw data 405. In some embodiments, each fixed-size data is the same size as a word in the computer system containing the compression unit 105. In the example, this size is 32 bits. In other examples, the fixed-size data processed by the compression unit 105 is a different size from the word in the computer system containing the compression unit 105. Any technically feasible size can be used for the fixed-size data.

[0035] Next, in step 214, the truncated data 410 is compressed to form compressed data 415. Often, step 214 is performed by the compression engine 204. In some cases, the compression in step 214 is performed using dictionary-based frequency compression (FVC). In alternative embodiments, any technically feasible compression algorithm can be used.

[0036] These compression algorithms work by analyzing blocks of data and finding repeating values ​​that can be re-encoded more efficiently. By performing compression on truncated data 410, the compression algorithm can find spatially localized patterns of values ​​within the data that have similar values ​​but are not exactly the same. Because the values ​​are similar but not exactly the same (or, depending on the compression algorithm, quite similar), conventional compression algorithms struggle with such patterns. Therefore, the compression algorithm is not very effective.

[0037] Next, in step 215, the compressed data 415 is transmitted. In some cases, the compressed data 415 is transmitted to the memory 104, the storage device 106, or the output driver 114.

[0038] Figure 2D is a flowchart of the recovery process 200D. In step 221, the compressed data 415 generated by process 200C is received. In some cases, the compressed data 415 is obtained from memory 104, storage device 106, or input driver 112.

[0039] In step 222, the compressed data 415 is decompressed to form decompressed truncated data 420. The type of decompression performed is determined based on the type of compression used in step 214. For example, if FVC compression is used, the decompressed truncated data is formed based on dictionary 435.

[0040] Next, in step 223, the recovery bits 425 are concatenated to the decompressed truncated data to form the recovery data 430. The number of recovery bits 425 is equal to the parameter k. In some cases, all recovery bits 425 are zero. In other examples, a stochastic recovery process is used to determine the recovery bits. In stochastic recovery, statistical techniques are used to select the recovery bits 425.

[0041] In some embodiments, the stochastic reconstruction process uses a pseudorandom number generator. In other embodiments, the stochastic reconstruction process involves sampling values ​​from a probability distribution. In some cases, the probability distribution is parametric. For example, the distribution is a Poisson distribution. In these cases, the parameters of the parametric distribution may be predetermined or dynamically determined based on previously reconstructed data. In some cases, the probability distribution may be nonparametric. In some examples, the nonparametric distribution may be predetermined, and in others, it may be dynamically determined based on previously reconstructed data. For example, previously reconstructed data can be used to form Markov chains and Bayesian networks. By performing the stochastic reconstruction process, the statistical bias introduced by the truncation in step 213 is reduced.

[0042] Next, in step 224, the recovery data 430 is sent. In some cases, the recovery data 430 is sent to the processor 102, memory 104, storage device 106, or output driver 114.

[0043] Figure 3 shows an example of using FVC alone.

[0044] In FVC, data patterns are identified and stored in a dictionary (e.g., dictionary 435). Instead of storing the entire data pattern in memory, an index to the dictionary entry is stored in memory. More specifically, the FVC algorithm identifies commonly occurring patterns and generates keys for those patterns. The relationship between each pattern and each key is stored in the dictionary. Storing only the keys for patterns in memory reduces the number of memory bits required. During decompression, the data is retrieved from memory, and the patterns stored in the dictionary are restored using the dictionary keys. This restoration is performed for each key in the compressed data, so in the restored data, the keys are replaced with dictionary values, and the data is restored to its uncompressed form.

[0045] In the FVC example shown in Figure 3, the raw data 305 consists of 256 bits, including eight 32-bit words. In this example, each of the original 32-bit words is replaced with a 2-bit code 310 if the word is compressible (i.e., if there is a dictionary value that matches the 32-bit word). If the value of a word is not used frequently enough, the word may be uncompressed, and as a result, even if their values ​​are included in the dictionary, they are not ultimately lost in the overall stored data. The 2-bit code 310 indicates whether the value is uncompressed (00) or compressed, and the 2-bit code specifies which of the up to three dictionary entries 315 is used. In this example, due to the diversity of values ​​and the limitations of dictionary size, only a very small number of 32-bit words can be compressed. Therefore, this example shows that 208 bits are required if only FVC is used.

[0046] Figure 4 shows an example demonstrating the improved efficiency of process 200C. In this example, raw data 405 is received according to step 211. In this example, raw data 405 consists of 256 bits representing eight 32-bit words. In step 212, the parameter k is set to 2 and the least significant bit is truncated. Next, raw data 405 is truncated according to step 213 to form truncated data 410. Next, truncated data 410 is compressed according to step 214 to form compressed data 415. In this example, FVC compression is used in conjunction with dictionary 435 in step 214. By applying process 300C to raw data 405, only 40 bits are required compared to 208 bits when using FVC alone.

[0047] Figure 4 further illustrates the recovery of compressed data 415 by the recovery process 200D. In step 221, the compressed data 415 is received. Next, according to step 222, the compressed data 415 is decompressed to form decompressed truncated data 420. In this example, decompression is performed using the FVC and dictionary 435. Next, according to step 335, the recovery bits 425 are concatenated to the decompressed truncated data 420 to form recovered data 430.

[0048] Figure 5A is a block diagram of a machine learning system 500A that utilizes a recovery unit 103 for evaluation. System 500A includes a weight database 502. The weights stored in database 502 assign relative importance to each of the multiple inputs received by neuron nodes in the evaluator 504.

[0049] The recovery unit 103 receives weights from the database 502 as input. Next, the recovery unit 103 executes process 200D to generate recovery data to be sent to the evaluator 504.

[0050] The evaluator 504 receives the weights restored from the restoration unit 103. The evaluator 504 is composed of multiple interconnected neuron nodes. Each interconnected neuron receives multiple inputs from other neuron nodes or input source 506. Next, each neuron node calculates an individual output based on the weights received from the database 502 and the transfer function 508. The evaluator 504 outputs the prediction data 510 based on the individual outputs of each of the multiple interconnected neuron nodes.

[0051] Figure 5B is a block diagram of a machine learning system 500B that utilizes a compression unit 105 and a decompression unit 103 for training. The machine learning system 500B includes a trainer 514 that determines weights to be stored in a database 502 based on training data 512. The trainer 514 determines the weights by comparing predicted data generated by the evaluator 504 with a predetermined output for the training data. The trainer 514 adjusts the weights based on a feedback mechanism.

[0052] Once the weights are determined by the relay 514, they are stored in the database 502 using the compression unit 105. The compression unit 105 receives the weights as input. The compression unit 105 then executes process 200C to generate the data to be stored in the database 502.

[0053] In some cases, the k value used by the compression unit 105 is dynamically changed by the training process. For example, in the early iterations of the training process, a high level of noise in the weights can be tolerated. Therefore, a larger k value is used. In this situation, the trainer 514 passes a smaller argument to the compression unit 105. However, as training progresses and the weights begin to converge, the training process becomes less tolerant of noise in the weights. Therefore, a smaller k value is used. In this situation, the trainer 514 passes a smaller k argument to the compression unit 105.

[0054] Figure 6A is a flowchart of the evaluation process 600A. In step 610, the compressed weights are obtained. Next, in step 620, the restoration process 200D is performed on the obtained and compressed weights to generate the restored weights. Then, in step 630, the output is generated by the neuronetwork based on the restored weights.

[0055] Figure 6B is a flowchart of the training process 600B. In step 605, the compressed weights are obtained. Next, in step 615, the restoration process 200D is performed on the obtained and compressed weights to generate the restored weights. Next, in step 625, the neural network generates an output using the restored weights. In step 635, the training process is performed on the output. The result of the training process becomes the new weights. Next, the new weights are compressed in step 645 according to process 200C. Next, the compressed new weights are stored in step 655.

[0056] It should be understood that many modifications are possible based on the disclosures herein. Although features and elements are described above in specific combinations, each feature or element may be used alone without other features and elements, or in various combinations with or without other features and elements.

[0057] The methods provided may be implemented in a general-purpose computer, processor, or processor core. Suitable processors include, by example, general-purpose processors, dedicated processors, conventional processors, digital signal processors (DSPs), multiple microprocessors, one or more microprocessors associated with a DSP core, controllers, microcontrollers, application-specific integrated circuits (ASICs), field-programmable gate array (FPGA) circuits, any other type of integrated circuit (IC), and / or state machines. Such processors may be manufactured by configuring a manufacturing process using the results of hardware description language (HDL) instructions and other intermediate data (instructions that can be stored in a computer-readable medium), including netlists, to be processed. The results of such a process may be maskwork, which is then used in a semiconductor manufacturing process to manufacture a processor that implements an aspect of the embodiment.

[0058] The methods or flowcharts provided herein may be implemented in computer programs, software, or firmware embedded in a non-temporary computer-readable storage medium for execution by a general-purpose computer or processor. Examples of non-temporary computer-readable storage media include magnetic media such as read-only memory (ROM), random access memory (RAM), registers, cache memory, semiconductor memory devices, internal hard disks, and removable disks, magneto-optical media, and optical media such as CD-ROM disks and digital multipurpose disks (DVDs).

Claims

1. A method used in machine learning, Receiving weight data containing one or more weight values ​​used by a machine learning model, The process involves truncating the number of bits in the weight data based on a truncation parameter that specifies the number of bits, thereby forming truncated weight data, wherein the truncation parameter is determined dynamically. The truncated weight data is compressed to form compressed weight data, This includes storing the compressed weight data in memory or transmitting the compressed weight data to a receiving system. method.

2. Obtaining the aforementioned compressed weight data, The compressed weight data is decompressed to form the decompressed truncated weight data, The process further includes concatenating the restored bits to the decompressed truncated weight data to form the restored weight data, The method according to claim 1.

3. The recovery bit is selected by sampling from a statistical distribution. The method according to claim 2.

4. The aforementioned statistical distribution is a nonparametric distribution. The method according to claim 3.

5. The aforementioned nonparametric distribution is determined based on previously recovered data. The method according to claim 4.

6. The aforementioned statistical distribution is a parametric distribution. The method according to claim 3.

7. The aforementioned parametric distribution is determined based on previously recovered data. The method according to claim 6.

8. The truncation parameter is dynamically determined based on previously compressed data or a desired noise level. The method according to claim 1.

9. It is a machine learning system, Memory and The memory comprises a processor that is communicatively coupled to the memory, The aforementioned processor, From the aforementioned memory, obtain weight data containing one or more weight values ​​used by the machine learning model, The process involves truncating the number of bits in the weight data based on a truncation parameter that specifies the number of bits, thereby forming truncated weight data, wherein the truncation parameter is determined dynamically. The truncated weight data is compressed to form compressed weight data, The compressed weight data is stored in the memory, or the compressed weight data is transmitted to the receiving system. It is configured to do, Machine learning system.

10. The aforementioned processor, Obtaining the aforementioned compressed weight data, The compressed weight data is decompressed to form the decompressed truncated weight data, The restored bits are concatenated to the decompressed truncated weight data to form the restored weight data, It is configured to do the following: The machine learning system according to claim 9.

11. The recovery bit is selected by sampling from a statistical distribution. A machine learning system according to claim 10.

12. The aforementioned statistical distribution is a nonparametric distribution. The machine learning system according to claim 11.

13. The aforementioned nonparametric distribution is determined based on previously recovered data. The machine learning system according to claim 12.

14. The aforementioned statistical distribution is a parametric distribution. The machine learning system according to claim 11.

15. The aforementioned parametric distribution is determined based on previously recovered data. The machine learning system according to claim 14.

16. The truncation parameter is dynamically determined based on previously compressed data or a desired noise level. The machine learning system according to claim 9.

17. A computer-readable storage medium for storing instructions, When the aforementioned instruction is executed by the processor, Receiving weight data containing one or more weight values ​​used by a machine learning model, The process involves truncating the number of bits in the weight data based on a truncation parameter that specifies the number of bits, thereby forming truncated weight data, wherein the truncation parameter is determined dynamically. The truncated weight data is compressed to form compressed weight data, The compressed weight data is stored in memory, or the compressed weight data is transmitted to the receiving system. The processor is made to execute a method used in a machine learning system, including, Computer-readable storage medium.

18. The aforementioned instruction is, Obtaining the aforementioned compressed weight data, The compressed weight data is decompressed to form the decompressed truncated weight data, The restored bits are concatenated to the decompressed truncated weight data to form the restored weight data, To cause the processor to execute the above, A computer-readable storage medium according to claim 17.

Images