Compression Techniques for Deep Neural Network Weights

Abstract

Various embodiments include methods and devices for compression and decompression of a weight data set. Some embodiments may include compressing the weight data by receiving a weight data set of binary numbers representing weight values, generating a frame payload including a compressed first frame of a first subset of weight values ​​in the weight data set, and generating a block of compressed weight data having the frame payload. Some embodiments may include recovering the weight data by retrieving the block of compressed weight data, the block of compressed weight data including a frame header associated with the frame payload, the frame header including a normalization factor indicator, and the frame payload including the compressed weight values, and generating a first recovered frame comprising recovered weight values ​​of the compressed weight values ​​of the frame payload.

Description

[Technical field]

[0001] Related Applications This application claims the benefit of priority to U.S. Non-provisional Application No. 17 / 220,620, filed April 1, 2021, the entire contents of which are incorporated herein by reference. [Background technology]

[0002] In deep neural network processing, a quantized neural network model may contain a large number of weights, which require a large amount of memory, system bandwidth, and power when the weights are used by a processor / hardware to implement the deep neural network processing. Summary of the Invention [Means for solving the problem]

[0003] Various disclosed aspects may include an apparatus and method for compression of weight data for a neural network. Various aspects may include receiving a weight data set of binary numbers representing weight values, generating a first frame payload including a compressed first frame of a first subset of weight values ​​in the weight data set, generating a first frame header associated with the first frame payload, the first frame header including a normalization factor indicator for the compressed first frame, and generating a block of compressed weight data having the first frame payload.

[0004] In some aspects, generating the first frame payload may include compressing the first frame of a first subset of weight values ​​in the weight data set by removing padding bits from each weight value of the first subset of weight values ​​according to a normalization factor for the first frame to generate a compressed first frame.

[0005] Some aspects may further include removing the offset value from each weight value of the first subset of weight values ​​for the first frame generating modified weight values ​​for the first frame, and compressing the first frame of the first subset of weight values ​​may include removing padding bits from the modified weight values ​​in the first frame according to the normalization factor for the first frame to generate a compressed first frame.

[0006] Some aspects may further include generating a global header for the block of compressed weight data, the global header including an offset value, and generating a first frame header associated with the first frame payload may include generating a first frame header, the first frame header including an offset indicator configured to indicate whether the offset value is removed from the first subset of weight values ​​for the first frame.

[0007] In some aspects, the padding bits may be sign extended bits.

[0008] Some aspects include identifying a first frame of a first subset of weight values ​​in the weight data set based on a pattern of padding bits in the weight values ​​that may be removed from the weight values ​​for lossless compression based on the selective search criteria; setting a first normalization factor for the first frame representing a number of padding bits to be removed from each weight value of the first subset of weight values; and identifying a second frame of a second subset of weight values ​​in the weight data set based on the pattern of padding bits in the weight values ​​that may be removed from the weight values ​​for lossless compression based on the selective search criteria. determining whether all bits of the second frame are zero-valued, and in response to determining that all of the bits of the second frame are zero-valued, setting a second normalization factor for the second frame to represent all of the bits of the second subset of weight values ​​for the second frame, compressing the second frame of the second subset of weight values ​​by removing all of the bits from the second subset of weight values ​​in accordance with the second normalization factor for the second frame, and generating a second frame header that is not associated with the frame payload.

[0009] Some aspects may further include identifying a first frame of a first subset of weight values ​​in the weight data set based on a pattern of padding bits in the weight values ​​that may be removed from the weight values ​​for lossless compression based on the selective search criteria, identifying a weight value among the first subset of weight values ​​for the first frame that has the highest number of significant bits, and setting a normalization factor for the first frame that represents a number of padding bits to be removed from each weight value among the first subset of weight values ​​based on the highest number of significant bits.

[0010] Some aspects may further include determining whether the first compression metric of the first frame payload exceeds a compression metric threshold, and generating the block of compressed weight data having the first frame payload may include generating the block of compressed weight data having the first frame payload in response to determining that the first compression metric of the first frame payload exceeds the compression metric threshold.

[0011] Some aspects may further include setting a compression metric threshold to a first compression metric of the first frame payload, generating a second frame payload including a compressed second frame of a second subset of the weight values ​​in the weight data set, determining whether the second compression metric of the second frame payload exceeds the compression metric threshold, in response to determining that the second compression metric exceeds the compression metric threshold, setting a compression metric threshold to the second compression metric of the second frame payload, generating a third frame payload including a compressed third frame of a third subset of the weight values ​​in the weight data set, and determining whether the third compression metric of the third frame payload exceeds the compression metric threshold, and may include generating a block of compressed weight data having the first frame payload in response to determining that the first compression metric of the first frame payload exceeds the compression metric threshold, and generating a block of compressed weight data having the second frame payload in response to determining that the third compression metric of the third frame payload does not exceed the compression metric threshold.

[0012] Various aspects may include apparatus and methods for recovery of weight data for a neural network. Various aspects may include retrieving a block of compressed weight data, the block of compressed weight data including a first frame header associated with a first frame payload, the first frame header including a first normalization factor indicator, the first frame payload including compressed weight values, and generating a first recovered frame comprising recovered weight values ​​of the compressed weight values ​​of the first frame payload.

[0013] In some aspects, the block of compressed weight data includes a global header having an offset value, and the first frame header includes an offset indicator configured to indicate whether an offset value is to be included for each recovered weight value generated from the first frame payload. Some aspects may further include parsing the global header for the offset value, parsing the first frame header for the offset indicator, and determining whether the offset indicator is set in the first frame header, and generating the first recovered frame may include including the offset value in each recovered weight value generated from the first frame payload associated with the first frame header in response to determining that the offset indicator is set in the first frame header.

[0014] In some aspects, the block of compressed weight data includes a second frame header not associated with the frame payload that includes a second normalization factor indicator. Some aspects may further include generating a second reconstructed frame including reconstructed weight values ​​having all zero-valued bits according to the second normalization factor indicator of the second frame header.

[0015] In some aspects, the block of compressed weight data includes a second frame header associated with the second frame payload, the second frame header including a second normalization factor indicator, and the second frame payload including the compressed weight value. Some aspects further include generating a second restored frame including a restored weight value of the compressed weight value of the second frame payload by adding padding bits to the compressed weight value of the second frame payload according to the second normalization factor indicator of the second frame header.

[0016] In some aspects, generating the first restored frame may include adding padding bits to compressed weight values ​​of the first frame payload in accordance with a first normalization factor indicator of the first frame header to generate restored weight values, wherein a value of the padding bits for a first compressed weight value among the compressed weight values ​​of the first frame payload is determined from most significant bits of the first compressed weight value.

[0017] In some aspects, the padding bits may be sign extension bits.

[0018] In some aspects, the first frame header includes a frame length configured to indicate a number of compressed weight values ​​in the first frame payload.

[0019] In some aspects, the block of compressed weight data includes a second frame header not associated with the frame payload that includes a frame length configured to indicate a number of compressed weight values. Some aspects may further include generating a second restored frame that includes a number of consecutive restored weight values ​​having all-zero valued bits that correspond to the frame length of the second frame header.

[0020] A further aspect includes a computing device having a compression or decompression processing device configured to perform the operations of any of the methods summarized above.

[0021] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate exemplary embodiments of the various embodiments and, together with the general description above and the detailed description below, serve to explain the features of the claims. [Brief description of the drawings]

[0022] [Figure 1] FIG. 1 is a component block diagram illustrating an exemplary computing device suitable for implementing various embodiments. [Diagram 2] FIG. 1 is a component block diagram illustrating an exemplary system-on-chip (SoC) suitable for implementing various embodiments. [Figure 3A] FIG. 2 is a data diagram illustrating an example of a framing of binary representation of decimal weight values ​​for compression suitable for implementing various embodiments. [Figure 3B] FIG. 2 is a data diagram illustrating an example of a framing of binary representation of decimal weight values ​​for compression suitable for implementing various embodiments. [Figure 4] FIG. 2 is a block diagram illustrating an example of a block of compressed weight data suitable for implementing various embodiments. [Diagram 5] FIG. 2 is a block diagram illustrating an example of a global header for a block of compressed weight data suitable for implementing various embodiments. [Figure 6] FIG. 2 is a block diagram illustrating an example frame header for a block of compressed weight data suitable for implementing various embodiments. [Figure 7] FIG. 2 is a data and component block diagram illustrating an example of decompression of compressed weight data suitable for implementing various embodiments. [Figure 8] FIG. 2 is a process flow diagram illustrating a method for compressing weight data, according to one embodiment. [Figure 9] FIG. 1 is a process flow diagram illustrating a method for setting weight data compression parameters, according to one embodiment. [Figure 10] FIG. 2 is a process flow diagram illustrating a method for compressing weight data, according to one embodiment. [Figure 11] FIG. 2 is a process flow diagram illustrating a method for decompressing compressed weight data, according to one embodiment. [Figure 12] FIG. 2 is a process flow diagram illustrating a method for compressing weight data, according to one embodiment. [Figure 13] FIG. 2 is a process flow diagram illustrating a method for decompressing compressed weight data, according to one embodiment. [Figure 14] FIG. 1 is a component block diagram illustrating an example mobile computing device suitable for implementing a weight data compression and / or recovery system in accordance with various embodiments. [Figure 15] FIG. 1 is a component block diagram illustrating an example mobile computing device suitable for implementing a weight data compression and / or recovery system in accordance with various embodiments. [Figure 16] FIG. 1 is a component block diagram illustrating an example server suitable for implementing a weight data compression and / or recovery system, in accordance with various embodiments. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0023] Various embodiments will now be described in detail with reference to the accompanying drawings. Wherever possible, the same reference numbers will be used throughout the drawings to refer to the same or like parts. References made to specific examples and implementations are for illustrative purposes only and do not limit the scope of the claims.

[0024] Various embodiments include methods for compression and decompression of weight data for deep neural networks, as well as computing devices that perform such methods. Some embodiments may include lossless compression techniques for weight data for deep neural networks. In some embodiments, compression may be based on identifying frames of weight data that exhibit patterns of repeated data, and removing the repeated data for compression. In some embodiments, global and / or frame header information regarding how the weight data is compressed may provide information for decompression of the compressed weight data.

[0025] Deep neural network processing often requires a large amount of data. This data may include a large amount of weight data, which imposes high memory, system bandwidth, and power costs on computer systems due to the processor / hardware used to perform deep neural network processing. The embodiments described herein present methods of lossless weight data compression and ultra-low-cost restoration, and devices for implementing the methods, which may reduce the memory, system bandwidth, and power costs of performing deep neural network processing that uses a large amount of weight data.

[0026] In some embodiments, compression of the weight data may be performed offline, and the compressed weight data may be stored in a memory for performing deep neural network processing. Offline compression of the weight data may enable a costly and time-consuming search for a suitable compression of the weight data, as measured by a compression metric of the compressed weight data compared to a compression metric threshold. In some embodiments, the compression metric threshold may be configured to identify the best compression of the weight data within the parameters for searching combinations of subsets of the weight data for compression. In some embodiments, sparsity in the weight data, for which consecutive weight values ​​are zero values, may be compressed in such a way that none of the weight data may be stored for consecutive zero values.

[0027] Not all of the weight data, such as padding data in weight values, may be useful in performing deep neural network processing. In some embodiments, compression of weight data may be performed by identifying a frame of weight data that exhibits a pattern of padding data that may be removed from the weight data without changing the weight values ​​of the frame. Removing the padding data may reduce the amount of data of the frame of weight data, and of the broader set of weight data. A compression parameter, such as a normalization factor configured to indicate the amount of padding data removed from each weight value in a frame, may be stored in association with the compressed weight data for use in restoring the compressed weight data. Removal of padding data from a frame for compression of the weight data of a frame may be accomplished by shifting the remaining bits of the weight data to overwrite the padding data that has been removed. The number of bits of weight data remaining after removal of the padding bits may be referred to herein as the compressed weight bit width.

[0028] In some embodiments, low-cost restoration may be achieved by using bitwise operations to denormalize the compressed weight data. The normalization factor may indicate to the restoration computing device the number of bits to shift each of the compressed weight values ​​to reintroduce padding bits into the compressed weight data. In some embodiments, bitwise arithmetic operations may be used to generate corresponding values ​​for the reintroduced padding data compared to the original padding data of the weight values. In some embodiments, the padding bits may be sign-extended bits.

[0029] The terms "computing device" and "mobile computing device" are used interchangeably herein to refer to any one or all of cellular telephones, smartphones, personal or mobile multimedia players, personal digital assistants (PDAs), laptop computers, tablet computers, convertible laptops / tablets (2-in-1 computers), smartbooks, ultrabooks, netbooks, palmtop computers, wireless email receivers, multimedia Internet-enabled cellular telephones, mobile game consoles, wireless game controllers, and similar personal electronic devices that include memory and programmable processors. The term "computing device" may further refer to stationary computing devices including personal computers, desktop computers, all-in-one computers, workstations, supercomputers, mainframe computers, embedded computers (such as in vehicles and other larger systems), servers, multimedia computers, and game consoles.

[0030] 1 illustrates a system including a computing device 100 suitable for use with various embodiments. Computing device 100 may include a SoC 102 having a processor 104, a memory 106, a communications interface 108, a memory interface 110, a peripheral device interface 120, and an artificial intelligence (AI) processor 124. Computing device 100 may further include a communications component 112, such as a wired or wireless modem, memory 114, an antenna 116 for establishing a wireless communications link, and / or peripheral devices 122. Processor 104 and AI processor 124 may include any of a variety of processing devices, e.g., several processor cores.

[0031] The term "system on chip" or "SoC" is used herein to generally refer to a set of interconnected electronic circuits including, but not limited to, a processing device, memory, and a communication interface. A processing device may include a variety of different types of processors 104 and / or processor cores, such as a general purpose processor, a central processing unit (CPU), a digital signal processor (DSP), a graphics processing unit (GPU), an accelerated processing unit (APU), a secure processing unit (SPU), a subsystem processor for a particular component of a computing device, such as an image processor for a camera subsystem or a display processor for a display, an auxiliary processor, a single-core processor, a multi-core processor, a controller, and / or a microcontroller. A processing device may further embody other hardware and combinations of hardware, such as a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), other programmable logic devices, discrete gate logic, transistor logic, performance monitoring hardware, watchdog hardware, and / or time references. An integrated circuit may be configured such that the components of the integrated circuit reside on a single piece of semiconductor material, such as silicon.

[0032] The SoC 102 may include one or more processors 104. The computing device 100 may include more than one SoC 102, thereby increasing the number of processors 104 and processor cores. The computing device 100 may also include processors 104 that are not associated with the SoC 102. The individual processors 104 may be multi-core processors. The processors 104 may each be configured for a specific purpose that may be the same or different than the other processors 104 of the computing device 100. One or more of the processors 104 and processor cores of the same or different configurations may be grouped together. A group of processors 104 or processor cores may be referred to as a multi-processor cluster.

[0033] The memory 106 of the SoC 102 may be a volatile or non-volatile memory configured to store data and processor-executable code for access by the processor 104 or by other components of the SoC 102, including the AI ​​processor 124. The computing device 100 and / or the SoC 102 may include one or more memories 106 configured for various purposes. The one or more memories 106 may include volatile memories, such as random access memory (RAM) or main memory, or cache memory. These memories 106 may be configured to temporarily hold limited amounts of data received from a data sensor or subsystem, data and / or processor-executable code instructions that are requested from the non-volatile memory and loaded into the memory 106 from the non-volatile memory in anticipation of future access based on various factors, and / or intermediate processing data and / or processor-executable code instructions generated by the processor 104 and / or the AI ​​processor 124 that are not stored in the non-volatile memory but are temporarily stored for quick future access. In some embodiments, any number and combination of memories 106 may include one-time programmable or read-only memory.

[0034] The memory 106 may be configured to at least temporarily store data and processor-executable code that is loaded into the memory 106 from another memory device, such as another memory 106 or memory 114, for access by one or more of the processors 104 or by other components of the SoC 102, including the AI ​​processor 124. The data or processor-executable code loaded into the memory 106 may be loaded in response to execution of a function by the processor 104 or by other components of the SoC 102, including the AI ​​processor 124. Loading data or processor-executable code into the memory 106 in response to execution of a function may result from a memory access request to the memory 106 that is unsuccessful or a "miss" due to the requested data or processor-executable code not being present in the memory 106. In response to the miss, a memory access request may be made to another memory 106 or memory 114 to load the requested data or processor-executable code into the memory 106 from the other memory 106 or memory 114. The loading of data or processor-executable code into memory 106 in response to the execution of a function may result from a memory access request to another memory 106 or memory 114, and the data or processor-executable code may be loaded into memory 106 for later access.

[0035] The memory interface 110 and the memory 114 may work in unison to enable the computing device 100 to store and retrieve data and processor-executable code on and from a volatile and / or non-volatile storage medium. The memory 114 may be configured much like an embodiment of the memory 106, where the memory 114 may store data or processor-executable code for access by one or more of the processors 104 or by other components of the SoC 102, including the AI ​​processor 124. In some embodiments, the memory 114 may be non-volatile and retain information after the computing device 100 is powered off. When power is applied again and the computing device 100 reboots, the information stored on the memory 114 may be available to the computing device 100. In some embodiments, the memory 114 may be volatile and not retain information after the computing device 100 is powered off. The memory interface 110 may control access to the memory 114 and enable the processor 104, or other components of the SoC 102, including the AI ​​processor 124, to read data from and write data to the memory 114.

[0036] The SoC 102 may also include an AI processor 124. The AI ​​processor 124 may be the processor 104, a portion of the processor 104, and / or an independent component of the SoC 102. The AI ​​processor 124 may be configured to execute a neural network for processing the activation values ​​and weight values ​​on the computing device 100. The computing device 100 may also include an AI processor 124 that is not associated with the SoC 102. Such an AI processor 124 may be an independent component of the computing device 100 and / or integrated into another SoC 102.

[0037] Some or all of the components of computing device 100 and / or SoC 102 may be arranged and / or combined differently while still performing the functionality of the various embodiments. Computing device 100 may not be limited to one of each of the components, and multiple instances of each component may be included in various configurations of computing device 100.

[0038] FIG. 2 illustrates an SoC 200 (e.g., SoC 102 in FIG. 1 ), which may be a component of a computing device (e.g., computing device 100 in FIG. 1 ), having an AI processor 206 (e.g., AI processor 124 in FIG. 1 ) and other components suitable for implementing an embodiment. With reference to FIGS. 1 and 2 , SoC 200 may include various components described above. For example, SoC 200 may include AI processor 206, processor 202 (e.g., processor 104 in FIG. 1 ), and processor memory (e.g., memory 106 in FIG. 1 ). In some embodiments, some such components, such as those described as part of SoC 200, may be located within a low power area 210 of SoC 200.

[0039] The blocks of compressed frames of the weight data set may be stored in a memory 208 remote from the SoC 200 (e.g., memory 114 in FIG. 1 ) and / or in a processor memory 204 on the SoC 200. In some embodiments, the blocks of compressed frames of the weight data set may be loaded from the memory 208 into the processor memory 204.

[0040] The AI ​​processor 206 may be configured to perform neural network processes, such as generating inferences, using the activation values ​​and weight values. The AI ​​processor 206 may retrieve weight values ​​for the neural network processes from the blocks of compressed frames of the weight data set. In some embodiments, the AI ​​processor 206 may receive the blocks of compressed frames of the weight data set from the memory 208. In some embodiments, the AI ​​processor 206 may receive the blocks of compressed frames of the weight data set from the processor memory 204.

[0041] The AI ​​processor 206 may be configured to decompress the blocks of compressed frames of the weight data set to retrieve the weight values. In some embodiments, the AI ​​processor 206 may be configured with software for decompressing the blocks of compressed frames of the weight data set. In some embodiments, the AI ​​processor 206 may be configured with circuitry and / or software for decompressing the blocks of compressed frames of the weight data set. Decompressing the blocks of compressed frames of the weight data set is described further herein.

[0042] The weight values ​​resulting from the decompression of the blocks of compressed frames of the weight data set may be used by the AI ​​processor 206 to perform a neural network process. In some embodiments, the AI ​​processor 206 may use a combination of weight values ​​retrieved from the blocks of compressed frames of the weight data set by the decompression and weight values ​​of the weight data set that were not compressed. In some embodiments, the results of the neural network process performed by the AI ​​processor 206 may be stored in the memory 204 and / or memory 208. In some embodiments, the results of the neural network process performed by the AI ​​processor 206 may be retrieved from the AI ​​processor 206, memory 204, and / or memory 208 for processing by the processor 202.

[0043] The description herein of the SoC 200 and its various components shown in FIG. 2 is intended to be exemplary only and in no way limiting. Some of the components of the illustrated exemplary SoC 200 may be variably configured, combined, and separated. Some of the components may be included in more or fewer numbers and may be arranged and connected differently within the SoC 200 or separate from the SoC 200. Similarly, numerous other components may be included in the SoC 200, such as other memories, processors, peripheral device subsystems, interfaces, and controllers.

[0044] 3A and 3B show an example framing of binary representations of decimal weight values ​​for compression suitable for implementing embodiments. With reference to FIGS. 1-3B, a weight data set 300 may include a representation of weight values. The example weight data set 300 in FIGS. 3A and 3B includes decimal and signed binary representations of integer weight values ​​(shown in FIGS. 3A and 3B for clarity).

[0045] For compression of weight values ​​from the weight data set 300, an analysis of the weight data set 300 may be performed to determine portions of the weight data set 300 that may be grouped into frames 302a, 302b, 302c, 302d, 302e, 304a, 304b, 304c, 304d. A computing device (e.g., computing device 100) may have a processor (e.g., processor 104, AI processor 124 in FIG. 1, processor 202, AI processor 206 in FIG. 2) configured to analyze the weight data set 300, identify various frames, including frames 302a, 302b, 302c, 302d, 302e, 304a, 304b, 304c, 304d, and / or other frames not shown, and select various combinations of frames to determine which frames to use to compress the weight data set 300. Analysis of the weights data set 300 may select combinations of frames for which the compression metric meets and / or exceeds a compression metric threshold. The compression metric and / or compression metric threshold may be based on a compression ratio, a compressed size, or the like.

[0046] In some embodiments, the compression metric threshold may be a predetermined value. A combination of frames that meets and / or exceeds the compression metric threshold may be used to compress the weight data set 300. In some embodiments, the compression metric threshold may be set to the best compression metric from an analysis of the compressed weight data set 300. For example, the compression metric of the first combination of frames may be set as the compression metric threshold, and the compression metric of any subsequent combination of frames that exceeds the compression metric threshold may be set as the compression metric threshold. Once no combination of frames can exceed the compression metric threshold, the compression metric threshold may be the best compression metric. The combination of frames associated with the best compression metric may be the combination of frames to be used to compress the weight data set 300.

[0047] In some embodiments, the identification of frames and / or the selection of frames may be implemented by a search algorithm. In some embodiments, the search algorithm may be configured to identify every frame and / or select every combination of frames. In some embodiments, the search algorithm may be an exhaustive search algorithm. In some embodiments, the search algorithm may be configured to identify frames and / or select combinations of frames based on selective search criteria that may limit the number of frames and / or the number of combinations of frames. In some embodiments, the search algorithm may be configured to identify frames and / or select combinations of frames based on selective search criteria that may prioritize some frames and / or some combinations of frames. For example, the selective search criteria may include a frame length (e.g., the number of weight values ​​in a frame), a range of frame lengths, a normalization factor (e.g., the number of bits of a weight value in a frame that may be removed without loss), a range of normalization factors, a weight value, a range of weight values, an offset value (e.g., a value by which a weight value of a frame may be changed), a range of offset values, etc.

[0048] The neural network may use weights quantized to a size. For example, the neural network may use weights quantized to 4 bits, 8 bits, 16 bits, etc. However, not all weight values ​​may use the full quantization size. Thus, the weight values ​​may include bits that represent the weight value, referred to herein as significant bits, which may include a sign bit, and bits that are not necessary to represent the weight value, referred to herein as padding bits. In some embodiments, the padding bits may be repeated bit values, such as repeating the most significant bits of the weight value and / or a bit value that is different from the most significant bit. In some embodiments, the padding bits may be sign-extended bits. Some or all of the padding bits may be removed from the weight value to compress the weight value without loss. Analysis of the weight data set 300 may identify a pattern in consecutive weight values ​​of padding bits that may be removed from the weight value without loss. For example, a pattern of padding bits that may be removed from the weight value without loss may be a shared number of some and / or all of the padding bits in consecutive weight values. A pattern of padding bits that can be removed from the weight values ​​without loss at consecutive weight values ​​may be identified based on the number of consecutive weight values, sometimes referred to as a frame length. The consecutive weight values ​​indicative of the pattern may be grouped as frames 302a, 302b, 302c, 302d, 302e, 304a, 304b, 304c, 304d. In some embodiments, identifying the weight values ​​indicative of the pattern may identify overlapping frames and selecting the frames may select non-overlapping frames. In some embodiments, the selected frames may be consecutive frames. In some embodiments, the selected frames may be non-consecutive frames.

[0049] In some embodiments, the processor may apply an offset to some and / or all of the weight values. In some embodiments, the processor may apply an offset to the weight values ​​of a frame. Applying an offset to a weight value may include modifying the weight value by the offset. For example, the offset may be removed from the weight value via subtraction from and / or addition to the weight value. Applying an offset to a weight value may generally reduce the number of bits, significant bits, in the weight value that may be used to represent the weight value. Similarly, applying an offset to a weight value may generally increase the number of padding bits in the weight value. The offset-applied weight values ​​may be grouped into frames based on a pattern of padding bits that may be removed from the weight value without loss.

[0050] To compress the selected frames, the processor may remove padding bits of the weight values ​​in the selected frames according to a pattern of padding bits that may be removed without loss from the weight values ​​of the selected frames. Each of the weight values ​​in the selected frames may have some or all of their padding bits removed. The number of padding bits removed from each weight value of the selected frames may be referred to herein as a normalization factor. The modification of the weight values ​​may leave only the significant bits, and / or the significant bits and less than all of the padding bits for the weight values ​​in the selected frames. In some embodiments, the padding bits removed from each of the weight values ​​in the selected frames may be based on the number of padding bits that may be removed from the weight value with the most significant bits in the selected frames. Thus, the number of padding bits that may be removed from the weight value with the most significant bits may also be removed from the other weight values ​​in the selected frames. Each selected frame may have its own normalization factor. In some embodiments, removing the number of padding bits may involve shifting the remaining bits to overwrite the padding bits that are being removed. The number of bits of the weight values ​​remaining after removal of the padding bits may be referred to herein as a compressed weight bit width.

[0051] The example shown in FIG. 3A illustrates a selection of frames 302a, 302b, 302c, 302d, 302e having variable frame lengths. Frame 302a has a frame length configured to represent a group of two weight values ​​(e.g., −13 and −33). Frame 302b has a frame length configured to represent a group of nine weight values ​​(e.g., −3, 2, −2, 4, 1, 0, −2, 9, and 10). Frame 302c has a frame length configured to represent a group of one weight value (e.g., −153). Frame 302d has a frame length configured to represent a group of three weight values ​​(e.g., 0, 1, and 0). Frame 302e has a frame length configured to represent a group of three weight values ​​(e.g., 0, 0, and 0).

[0052] The example shown in FIG. 3A further illustrates a selection of frames 302a, 302b, 302c, 302d, 302e having variable normalization factors. Frame 302a has a normalization factor configured to represent the removal of 8 padding bits from each of the weight values ​​in frame 302a. Frame 302b has a normalization factor configured to represent the removal of 11 padding bits from each of the weight values ​​in frame 302b. Frame 302c has a normalization factor configured to represent the removal of 6 padding bits from each of the weight values ​​in frame 302c. Frame 302d has a normalization factor configured to represent the removal of 14 padding bits from each of the weight values ​​in frame 302d. In some embodiments, frame 302e has a normalization factor configured to represent the removal of 15 padding bits from each of the weight values ​​in frame 302e. In some embodiments, frame 302e has a normalization factor configured to represent the removal of all bits from each of the weight values ​​in frame 302e. As described further herein, frames such as frame 302e having only a weight value of 0 may be represented within a compressed block by a header without a frame payload, allowing all of the frame's bits to be removed during compression.

[0053] The example shown in FIG. 3B illustrates the selection of frames 304a, 304b, 304c, 304d having uniform frame lengths. In this example, the uniform frame lengths are eight weight values. However, the uniform frame lengths can be any number of weight values. In some embodiments, the uniform frame lengths can be a power-of-two number of weight values ​​(e.g., 2, 4, 8, 16, 32, 64, 128, etc.). Frame 304a has a frame length configured to represent a group of eight weight values ​​(e.g., −13, −33, −3, 2, −2, 4, 1, and 0). Frame 304b has a frame length configured to represent a group of eight weight values ​​(e.g., −2, 9, 10, −153, 0, 1, 0, and 33). Frame 304c has a frame length configured to represent a group of eight weight values ​​(e.g., 2, −26, 0, 0, 0, −9, −81, and 0). Frame 304d has a frame length configured to represent a group of eight weight values ​​(eg, -3, -12, 1, -125, -1, 5, 0, and -154).

[0054] The example shown in FIG. 3B further illustrates the selection of frames 304a, 304b, 304c, 304d with variable normalization factors. Frame 304a has a normalization factor configured to represent the removal of eight padding bits from each of the weight values ​​in frame 304a. Frame 304b has a normalization factor configured to represent the removal of six padding bits from each of the weight values ​​in frame 304b. Frame 304c has a normalization factor configured to represent the removal of seven padding bits from the weight values ​​in frame 304c. Frame 304d has a normalization factor configured to represent the removal of six padding bits from each of the weight values ​​in frame 304d.

[0055] The examples in Figures 3A and 3B are intended to be illustrative and do not limit the scope of the claims or this specification. The weight data set 300 may include any number of weight values, and the weight values ​​may be of any size and / or format. A processor of a computing device analyzing the weight data set 300 may identify and select frames of any number and / or length and having any normalization factor.

[0056] FIG. 4 illustrates an example of a block of compressed weight data suitable for implementing embodiments. With reference to FIGS. 1-4, the block of compressed weight data 400 may include a global header 402, any number of frame headers 404a, 404b, 404c, 404d, and any number of frame payloads 406a, 406b, 406c. A compression computing device (e.g., computing device 100) may have a processor (e.g., processor 104, AI processor 124 in FIG. 1, processor 202, AI processor 206 in FIG. 2) configured to compress a weight data set (e.g., weight data set 300 in FIG. 3A and FIG. 3B). Compression of the weight data set may generate a block of compressed weight data 400. A restoration computing device (e.g., computing device 100) may have a processor (e.g., processor 104, AI processor 124 in FIG. 1, processor 202, AI processor 206 in FIG. 2) configured to restore the block of compressed weight data 400. The block 400 of compressed weight data may be stored in memory of the compression computing device and / or the decompression computing device (eg, memories 106, 114 in FIG. 1, processor memory 204, memory 208 in FIG. 2).

[0057] As described further herein, the global header 402 may include parameters that may be applied for restoration of any combination and / or all of the frame payloads 406a, 406b, 406c. As described further herein, the frame headers 404a, 404b, 404c may include parameters that may be applied for restoration of the associated frame payloads 406a, 406b, 406c, and the frame header 404d may include parameters that may be applied for restoration without the associated frame payload. The frame payloads 406a, 406b, 406c may include compressed weight values ​​of the associated frames (e.g., frames 302a, 302b, 302c, 302d, 302e, 304a, 304b, 304c, 304d in FIG. 3A and FIG. 3B). In some embodiments, the block of compressed weight data 400 may be ordered such that associated frame headers 404a, 404b, 404c and frame payloads 406a, 406b, 406c are paired. In some embodiments, the block of compressed weight data 400 may be ordered such that the frame headers 404a, 404b, 404c, 404d and frame payloads 406a, 406b, 406c are ordered in the order of the frames in the weight data set.

[0058] FIG. 5 illustrates an example of a global header for a block of compressed weight data (e.g., block of compressed weight data 400 in FIG. 4) suitable for implementing embodiments. With reference to FIGS. 1-5, the global header 500 (e.g., global header 402 in FIG. 4) may be of any size and may include parameters that may be applied for decompression of any combination and / or all of the frame payloads of the compressed weight data (e.g., frame payloads 406a, 406b, 406c in FIG. 4). A compression computing device (e.g., computing device 100) may have a processor (e.g., processor 104, AI processor 124 in FIG. 1, processor 202, AI processor 206 in FIG. 2) configured to compress a weight data set (e.g., weight data set 300 in FIGS. 3A and 3B). Compression of the weight data set may generate the global header 500. A decompression computing device (e.g., computing device 100) may have a processor (e.g., processor 104, AI processor 124 in FIG. 1, processor 202, AI processor 206 in FIG. 2) configured to decompress blocks of compressed weight data using global header 500. Global header 500 may be stored in memory (e.g., memory 106, 114 in FIG. 1, processor memory 204, memory 208 in FIG. 2) of the compression computing device and / or the decompression computing device. In some embodiments, global header 500 may be any bit width. For example, global header 500 may be 9 bytes.

[0059] The global header 500 may include any number of offsets 502, 504. In some embodiments, the offsets 502, 504 may be values ​​applied to weight values ​​of any number and / or combination of frames (e.g., frames 302a, 302b, 302c, 302d, 302e, 304a, 304b, 304c, 304d in FIG. 3A and FIG. 3B) for compression of the weight data set. In some embodiments, the default value of the offsets 502, 504 may be set to 0. The offsets 502, 504 may be applied to weight values ​​for compression of the weight data set to modify the weight values ​​by the offsets. For example, the offsets may be removed from the weight values ​​via subtraction from and / or addition to the weight values. Applying an offset to a weight value may generally reduce the number of bits, significant bits, in the weight value that may be used to represent the weight value. Similarly, applying an offset to a weight value may generally increase the number of padding bits in the weight value. The offset value may be used in decompressing a block of compressed weight data to restore a weight value that was modified by the offset during compression to its original weight value. For example, the offset may be added to the modified weight value via subtraction from and / or addition to the modified weight value. In some embodiments, the offset 502, 504 may have the same bit width as the weight value. For example, the offset 502, 504 may have a bit width of a number of bits that is a power of 2, such as 2 bits, 4 bits, 8 bits, 16 bits, 32 bits, 64 bits, 128 bits, etc.

[0060] In some embodiments, the global header 500 may include any number of reserved bits 506 that may be configured to provide parameters for decompressing the block of compressed weight data. In some embodiments, the number of reserved bits 506 may be 1 bit, 2 bits, etc.

[0061] In some embodiments, the global header 500 may include a compressed sign value 508 configured to indicate whether the frame payload of the block of compressed weight data includes signed compressed weight data. The compressed sign value 508 may be generated during compression of the weight data set. For example, the compressed sign value 508 may be configured to indicate that the frame payload of the block of compressed weight data includes signed compressed weight data when at least one weight value of the weight data set is a signed weight value. In another example, the compressed sign value 508 may be configured to indicate that the frame payload of the block of compressed weight data includes signed compressed weight data when at least one weight value of at least one frame of the weight data set is a signed weight value. In another example, the compressed sign value 508 may be configured to indicate that the frame payload of the block of compressed weight data does not include signed compressed weight data when none of the weight values ​​of the weight data set are signed weight values. In another example, the compressed sign value 508 may be configured to indicate that the frame payload of a block of compressed weight data does not contain signed compressed weight data when none of the weight values ​​in any frame of the weight data set are signed weight values.

[0062] The compressed sign value 508 may be used during decompression of the block of compressed weight data to determine whether the decompressed weight values ​​are signed. For example, the compressed sign value 508 may be configured to represent that the frame payload of the block of compressed weight data includes signed compressed weight data, and thus the decompressed weight values ​​are signed. Decompressing the signed weight values ​​may include copying the value of a most significant bit of each compressed weight value, such as a sign bit, and using the value of the most significant bit as a sign extension bit to decompress each respective compressed weight value. In another example, the compressed sign value 508 may be configured to represent that the frame payload of the block of compressed weight data does not include signed compressed weight data, and thus the decompressed weight values ​​are unsigned. Decompressing the unsigned weight values ​​may include adding padding bits to each compressed weight value. The padding bits may be any combination or pattern of bit values. For example, the padding bits may all be the same bit value, such as all "0"s or all "1"s. In some embodiments, the padding bits may be the bit value of the most significant bit of the compressed weight value, or the inverse of that bit value. In some embodiments, the compressed code value 508 may be of any bit width. For example, the compressed code value 508 may be 1 bit.

[0063] In some embodiments, the global header 500 may include a frame header size 510 configured to represent a size of the frame headers (e.g., frame headers 404a, 404b, 404c, 404d in FIG. 4) in the block of compressed weight data. The frame header size 510 may be generated during compression of the weight data set. For example, the frame header size 510 may be generated based on a uniform size for all of the frame headers in the block of compressed weight data. In some embodiments, the frame header size 510 may be a preconfigured value. The frame header size 510 may be used during decompression of the block of compressed weight data to determine where the frame headers and / or frame payloads are located within the block of compressed weight data. For example, the uniform size of the frame headers may be used to identify that data within the block of compressed weight data that is the size of a frame header may be a frame header and that data within the block of compressed weight data that is not the size of a frame header may be a frame payload. In some embodiments, the frame header size 510 may be any bit width. For example, the frame header size 510 may be 2 bits.

[0064] In some embodiments, the global header 500 may include an uncompressed width 512 configured to represent a bit width of the uncompressed weight data. For example, the uncompressed width 512 may be configured to represent that the uncompressed weights may have a bit width of a number of bits that is a power of two, such as 2 bits, 4 bits, 8 bits, 16 bits, 32 bits, 64 bits, 128 bits, etc. The uncompressed width 512 may be generated during compression of the weight data set. For example, the uncompressed width 512 may be generated based on the bit width of the weight values ​​in the weight data set. In another example, the uncompressed width 512 may be a preconfigured value. The uncompressed width 512 may be used during decompression of a block of compressed weight data to determine the bit width of the decompressed weight values. In some embodiments, the uncompressed width 512 may be any bit width. For example, the uncompressed width 512 may be 3 bits.

[0065] In some embodiments, the global header 500 may include an uncompressed buffer size 514 configured to represent a bit-width of a buffer configured to store the uncompressed weight data. The uncompressed buffer size 514 may be generated during compression of the weight data set. For example, the uncompressed buffer size 514 may be generated based on a buffer size for the weight data set. In another example, the uncompressed buffer size 514 may be a preconfigured value. The uncompressed buffer size 514 may be used during decompression of a block of compressed weight data to determine a buffer size for the decompressed weight values. In some embodiments, the uncompressed buffer size 514 may be any bit-width. For example, the uncompressed buffer size 514 may be 32 bits.

[0066] FIG. 6 illustrates an example of a frame header for a block of compressed weight data (e.g., block of compressed weight data 400 in FIG. 4) suitable for implementing embodiments. With reference to FIGS. 1-6, the frame header 600 (e.g., frame headers 404a, 404b, 404c, 404d in FIG. 4) may be of any size and may include parameters that may be applied for decompression of any combination and / or all of the frame payloads of compressed weight data (e.g., frame payloads 406a, 406b, 406c in FIG. 4). A compression computing device (e.g., computing device 100) may have a processor (e.g., processor 104, AI processor 124 in FIG. 1, processor 202, AI processor 206 in FIG. 2) configured to compress a weight data set (e.g., weight data set 300 in FIGS. 3A and 3B). Compression of the weight data set may generate the frame header 600. A decompression computing device (e.g., computing device 100) may have a processor (e.g., processor 104, AI processor 124 in FIG. 1, processor 202, AI processor 206 in FIG. 2) configured to decompress blocks of compressed weight data using the frame header 600. The frame header 600 may be stored in a memory (e.g., memory 106, 114 in FIG. 1, processor memory 204, memory 208 in FIG. 2) of the compression computing device and / or the decompression computing device.

[0067] In some embodiments, the frame header 600 may be associated with a frame in the weight data set (e.g., frames 302a, 302b, 302c, 302d, 302e, 304a, 304b, 304c, 304d in FIGS. 3A and 3B). In some embodiments, the frame header 600 may be associated with a frame payload of a block of compressed weight data. In some embodiments, the frame header 600 may be included in a block of compressed weight data without association with a frame payload. For example, the frame header 600 may be included in a block of compressed weight data without association with a frame payload for the frame header 600 associated with a frame of weight data containing all-zero weight values. In some embodiments, the frame header 600 may be any bit width. For example, the frame header 600 may be 8 bits.

[0068] The frame header 600 may include a frame length 602 configured to represent the number of weight values ​​included in an associated frame of the weight data set. In some embodiments, the frame length 602 may similarly be configured to represent the number of compressed weight values ​​included in an associated frame payload of a block of compressed weight data. In some embodiments, the frame length 602 may similarly be configured to represent the number of compressed weight values ​​without an associated frame payload of a block of compressed weight data. For example, the frame length 602 may be configured to represent the number of compressed consecutive zero weight values. The frame length 602 may be generated during compression of the weight data set. For example, the frame length 602 may be generated based on the number of weight values ​​in an associated frame of the weight data set. In another example, the frame length 602 may be generated based on the number of compressed weight values ​​in an associated frame payload. In some embodiments, the frame length 602 may be a pre-configured value. For example, the frame length 602 may be a preconfigured power of a power of two value, such as 2 bits, 4 bits, 8 bits, 16 bits, 32 bits, 64 bits, 128 bits, etc. In another example, the frame length 602 may be a value divisible by 16 for 8-bit weight values. In another example, the frame length 602 may be a value divisible by 8 for 16-bit weight values. The frame length 602 may be used during decompression of a block of compressed weight data to determine the number of weight values ​​to recover from an associated frame payload. In some embodiments, the frame length 602 may be used during decompression of a block of compressed weight data to determine the number of weight values ​​to recover for a frame header 600 that is not associated with a frame payload. For example, the frame length 602 may be used to determine the number of consecutive zero weight values ​​to recover. The frame length 602 may be any bit width. For example, the frame length 602 may be 4 bits for 8-bit weight values. In another example, the frame length 602 may be 3 bits for 16-bit weight values.

[0069] The frame header 600 may include a normalization factor 604 configured to represent a number of padding bits removed from weight values ​​included in an associated frame of the weight data set. In some embodiments, the normalization factor 604 may similarly be configured to represent a number of padding bits to add to compressed weight values ​​included in an associated frame payload of the block of compressed weight data. In some embodiments, the normalization factor 604 may similarly be configured to represent a number of bits removed from zero weight values ​​included in an associated frame of the weight data set. In some embodiments, the normalization factor 604 may similarly be configured to represent a number of bits to add for compressed weight values ​​without an associated frame payload of the block of compressed weight data. For example, the normalization factor 604 may be configured to represent a number of bits for zero weight values. The normalization factor 604 may be generated during compression of the weight data set. For example, the normalization factor 604 may be generated based on a number of bits removed from weight values ​​in an associated frame of the weight data set to generate an associated frame payload. The normalization factor 604 may be used during decompression of a block of compressed weight data to determine the number of padding bits to add to the compressed weight values ​​to recover from the associated frame payload. In some embodiments, the normalization factor 604 may be used during decompression of a block of compressed weight data to determine the number of bits for zero weight values ​​to recover for the frame header 600 that are not associated with a frame payload. The normalization factor 604 may be any bit width. For example, the normalization factor 604 may be 3 bits for an 8-bit weight value. In another example, the normalization factor 604 may be 4 bits for a 16-bit weight value.

[0070] In some embodiments, the normalization factor 604 may be replaced in the frame header 600 by a compressed weight bit-width of the remaining bits of the weight values ​​included in the associated frame of the weight data set after removal of padding bits in the example shown in Figure 6. During decompression, the normalization factor 604 may be determined for a frame using the compressed weight bit-width of the associated frame and an uncompressed width (e.g., uncompressed width 512 in Figure 5) configured to represent the bit-width of the uncompressed weight data set. For example, the compressed weight bit-width of the associated frame may be subtracted from the uncompressed width to determine the normalization factor 604.

[0071] In some embodiments, the frame header 600 may include a normalization factor indicator, which may be the normalization factor 604. In some embodiments, the frame header 600 may include a normalization factor indicator, which in the example shown in FIG. 6 may be a compressed weight bit width replaced with the normalization factor 604.

[0072] The frame header 600 may include an offset indicator 606 configured to indicate whether and / or which offset has been applied to the weight values ​​included in the associated frame of the weight data set (e.g., offsets 502, 504 in FIG. 5). In some embodiments, the offset indicator 606 may be similarly configured to indicate whether and / or which offset has been applied to the compressed weight values ​​included in the associated frame payload of the block of compressed weight data. In some embodiments, the offset indicator 606 may be similarly configured to indicate whether and / or which offset will be applied to recover the compressed weight values ​​included in the associated frame payload of the block of compressed weight data. The offset indicator 606 may be a value associated with an offset included in a global header (e.g., global header 500 in FIG. 5) of the block of compressed weight data. For example, the offset indicator 606 may be a bit flag for which a first value is associated with a first offset in the global header and a second value is associated with a second offset in the global header. The offset indicator 606 may indicate which of the offsets in the global header were used in compressing the weight values ​​of the weight data set and which of the offsets in the global header are to be used in decompressing the compressed weight values. The offset indicator 606 may be any bit width. For example, the offset indicator 606 may be 1 bit.

[0073] The examples in Figures 4-6 are intended to be illustrative and not limiting of the claims or the scope of this specification. The blocks of compressed weight data 400 in Figure 4, the global header 500 in Figure 5, and the frame headers in Figure 6 may be variably included and / or excluded, sized differently, and / or ordered differently.

[0074] FIG. 7 illustrates an example of decompression of a block of compressed weight data (e.g., block of compressed weight data 400 in FIG. 4) suitable for implementing embodiments. With reference to FIGS. 1-7, the block of compressed weight data 700 may be decompressed to generate a block of decompressed weight data 720. A decompression computing device (e.g., computing device 100) may have a processor (e.g., processor 104, AI processor 124 in FIG. 1, processor 202, AI processor 206 in FIG. 2) configured to decompress the block of compressed weight data 700. The processor may be configured with circuitry and / or software for implementing a decompressor 710. In some embodiments, software configured to implement the decompressor 710 may be stored in a memory of the decompression computing device (e.g., memory 106, 114 in FIG. 1, processor memory 204, memory 208 in FIG. 2). The decompression computing device may use the global header 704 (e.g., global header 402 in FIG. 4, global header 500 in FIG. 5), frame headers 706a, 706b, 706c, 706d, 706e (e.g., frame headers 404a, 404b, 404c, 404d in FIG. 4, frame header 600 in FIG. 6), and / or frame payloads 702a, 702b, 702c, 702d (e.g., frame payloads 406a, 406b, 406c in FIG. 4) of the compressed weight data block 700 to decompress the compressed weight data block 700. In some embodiments, the compressed weight data block 700 may be stored in a memory of the decompression computing device (e.g., memory 106, 114 in FIG. 1, processor memory 204, memory 208 in FIG. 2).

[0075] The decompressor 710 may include a header parser 712 and a block denormalizer 714. The decompressor 710 may retrieve the block of compressed weight data 700. For example, the decompressor 710 may retrieve the block of compressed weight data 700 from a memory of a decompression computing device in which the block of compressed weight data 700 may be stored.

[0076] The header parser 712 may parse the global header 704 to extract parameters for restoring the block of compressed weight data 700. The header parser 712 may also parse the frame headers 706a, 706b, 706c, 706d to extract parameters for restoring the associated frame payloads 702a, 702b, 702c, 702d. In some embodiments, the header parser 712 may also parse the frame header 706e to extract parameters for restoring a weight value of 0 without an associated frame payload.

[0077] The block denormalizer 714 may use parameters retrieved from the global header 704 and the frame headers 706a, 706b, 706c, 706d, 706e to recover the compressed block of weight data 700. More specifically, the block denormalizer 714 may use parameters retrieved from the global header 704 and the frame headers 706a, 706b, 706c, 706d to recover the associated frame payloads 702a, 702b, 702c, 702d. Additionally, the block denormalizer 714 may use parameters retrieved from the global header 704 and the frame headers 706e to recover a weight value of 0 without an associated frame payload. In some embodiments, the block denormalizer 714 may recover the weight values ​​of the frame payloads 702a, 702b, 702c, 702d by shifting each of the compressed weight values ​​by the number of bits of the normalization factor (e.g., normalization factor 604 in FIG. 6) from the associated frame header 706a, 706b, 706c, 706d. In some embodiments, the block denormalizer 714 may use bitwise arithmetic operations to adjust the value of the padding bits added. For example, the block denormalizer 714 may use bitwise arithmetic operations to adjust the value of the padding bits added to match the most significant bits of the compressed weight values. As a further example, the block denormalizer 714 may use bitwise arithmetic operations to adjust the value of the sign extension bits added to match the most significant bits, such as the sign bit, of the compressed weight values. The recovered weight values ​​may be grouped as a block 720 of recovered weight data.

[0078] For example, the block denormalizer 714 may use parameters retrieved from the global header 704 and the frame header 706a to recover the associated frame payload 702a. In this example, the block denormalizer 714 may retrieve from the frame header 706a a frame length (e.g., frame length 602 in FIG. 6) configured to represent a group of two weight values, a normalization factor configured to represent eight padding bits, and an offset indicator (e.g., offset indicator 606 in FIG. 6) configured to indicate that no offset is to be used to recover the frame payload 702a. The block denormalizer 714 may apply the normalization factor to the two compressed weight values ​​of the frame payload 702a and add eight padding bits to each of the compressed weight values. In this example, the block denormalizer 714 may copy the value of the most significant bit of each compressed weight value and use the value of the most significant bit as a padding bit to recover each respective compressed weight value. As a further example, the block denormalizer 714 may copy the value of the most significant bit of each compressed weight value, such as a sign bit, and use the value of the most significant bit as a sign extension bit to restore each respective compressed weight value. The full bit-width restored weight values ​​with padding bits and significant bits may be used as part of the block 720 of restored weight data.

[0079] In a further example, the block denormalizer 714 may use parameters retrieved from the global header 704 and the frame header 706b to recover the associated frame payload 702b. In this example, the block denormalizer 714 may retrieve from the frame header 706b a frame length configured to represent a group of nine weight values, a normalization factor configured to represent eleven padding bits, and an offset indicator configured to indicate that no offset is to be used to recover the frame payload 702b. The block denormalizer 714 may apply the normalization factor to the nine compressed weight values ​​of the frame payload 702b and add eleven padding bits to each of the compressed weight values. In this example, the block denormalizer 714 may copy the value of the most significant bit of each compressed weight value and use the value of the most significant bit as a padding bit to recover each respective compressed weight value. As a further example, the block denormalizer 714 may copy the value of the most significant bit of each compressed weight value, such as a sign bit, and use the value of the most significant bit as a sign extension bit to restore each respective compressed weight value. The full bit-width restored weight values ​​with padding bits and significant bits may be used as part of the block 720 of restored weight data.

[0080] In a further example, the block denormalizer 714 may use parameters retrieved from the global header 704 and the frame header 706c to recover the associated frame payload 702c. In this example, the block denormalizer 714 may retrieve from the frame header 706c a frame length configured to represent one group of weight values, a normalization factor configured to represent six padding bits, and an offset indicator configured to indicate that no offset is to be used to recover the frame payload 702c. The block denormalizer 714 may apply the normalization factor to the compressed weight values ​​of one of the frame payloads 702c and add six padding bits to each of the compressed weight values. In this example, the block denormalizer 714 may copy the value of the most significant bit of the compressed weight value and use the value of the most significant bit as the padding bit to recover the compressed weight value. As a further example, the block denormalizer 714 may copy the value of the most significant bit, such as the sign bit, of the compressed weight value and use the value of the most significant bit as a sign extension bit to restore the compressed weight value. The full bit-width restored weight value with padding bits and significant bits may be used as part of the block 720 of restored weight data.

[0081] In a further example, the block denormalizer 714 may use parameters retrieved from the global header 704 and the frame header 706d to recover the associated frame payload 702d. In this example, the block denormalizer 714 may retrieve from the frame header 706d a frame length configured to represent a group of three weight values, a normalization factor configured to represent 14 padding bits, and an offset indicator configured to indicate that no offset is to be used to recover the frame payload 702d. The block denormalizer 714 may apply the normalization factor to the three compressed weight values ​​of the frame payload 702d and add 14 padding bits to each of the compressed weight values. In this example, the block denormalizer 714 may copy the value of the most significant bit of each compressed weight value and use the value of the most significant bit as a padding bit to recover each respective compressed weight value. As a further example, the block denormalizer 714 may copy the value of the most significant bit of each compressed weight value, such as a sign bit, and use the value of the most significant bit as a sign extension bit to restore each respective compressed weight value. The full bit-width restored weight values ​​with padding bits and significant bits may be used as part of the block 720 of restored weight data.

[0082] In a further example, the block denormalizer 714 may use parameters retrieved from the global header 704 and the frame header 706e to recover the zero weight values ​​without the associated frame payload. In this example, the block denormalizer 714 may retrieve from the frame header 706e a frame length configured to represent a group of three weight values, a normalization factor configured to represent 16 padding bits, and an offset indicator configured to indicate that no offset is to be used to recover the zero weight values. The block denormalizer 714 may apply the normalization factor to the three compressed weight values ​​for the zero weight values ​​and add 16 bits for each of the compressed zero weight values. In this example, the block denormalizer 714 may add 16 zero value bits to recover each respective compressed zero weight value. The full bit-width recovered weight values ​​with padding bits and significant bits may be used as part of the block 720 of recovered weight data.

[0083] In some embodiments, the weight values ​​of the weight data set may be modified by an offset value (e.g., offsets 502, 504 in FIG. 5) as part of compressing the weight values. In the case of such modified weight values, the frame header for the associated frame payload having the compressed weight value derived from the modified weight value may include an offset indicator configured to indicate that the offset will be used to recover the frame payload. Following application of the frame header normalization factor to the compressed weight of the frame payload, the block denormalizer 714 may modify the full bit-width weight value of the frame payload by the offset value. The full bit-width recovered weight value with padding bits and significant bits may be used as part of the block 720 of recovered weight data.

[0084] The example in FIG. 7 is intended to be illustrative and does not limit the scope of the claims or this specification. The block of compressed weight data 700 and the block of restored weight data 720 may include any number of weight values, and the weight values ​​may be of any size and / or format. The processor of the restoration computing device that restores the compressed weight data 700 and generates the restored weight data 720 may do so using any number and combination of global headers, frame headers, and frame payloads. Similarly, the processor of the restoration computing device that restores the compressed weight data 700 and generates the restored weight data 720 may do so using any number, combination, and value offsets, frame lengths, normalization factors, and compressed weight values.

[0085] FIG. 8 illustrates a method 800 for compressing weight data, according to one embodiment. With reference to FIGS. 1-8, the method 800 may be implemented in a computing device (e.g., computing device 100 in FIG. 1), in hardware, in software running on a processor, or in a combination of a processor and dedicated hardware configured with software (e.g., processor 104, memory 106, 114, AI processor 124 in FIG. 1, processor 202, processor memory 204, AI processor 206, memory 208 in FIG. 2). To encompass alternative configurations permitted in various embodiments, hardware implementing the method 800 is referred to herein as a "compression processing device."

[0086] At block 802, a compression processing device may receive a weights data set (e.g., weights data set 300 in FIGS. 3A and 3B). In some embodiments, the weights data set may be retrieved from a memory by the compression processing device. In some embodiments, the compression processing device that retrieves the weights data set at block 802 may be a processor or an AI processor.

[0087] At block 804, the compression processing device may identify frames in the weight data set (e.g., frames 302a, 302b, 302c, 302d, 302e, 304a, 304b, 304c, 304d in FIGS. 3A and 3B) that may be removed from the weight values ​​for lossless compression based on selective search criteria. Identification of the frames may be implemented by a search algorithm. In some embodiments, the search algorithm may be configured to identify every frame. In some embodiments, the search algorithm may be an exhaustive search algorithm. In some embodiments, the search algorithm may be configured to identify frames based on selective search criteria that may limit the number of frames. In some embodiments, the search algorithm may be configured to identify frames based on selective search criteria that may prioritize some frames and / or some combinations of frames. For example, the selective search criteria may include a frame length, a range of frame lengths, a normalization factor representing the number of padding bits that may be removed from the weight values ​​of the frames, a range of normalization factors, a weight value, a range of weight values, an offset value, a range of offset values, etc. The neural network may use weights quantized to a size. For example, the neural network may use weights quantized to 4 bits, 8 bits, 16 bits, etc. However, not all weight values ​​may use the full quantization size. Thus, the weight values ​​may include significant bits, which may include a sign bit, and padding bits. In some embodiments, the padding bits may be repeated bit values, such as repeating the most significant bits of the weight value and / or a bit value different from the most significant bits. In some embodiments, the padding bits may be sign extension bits.

[0088] The analysis of the weight data set may identify a pattern in consecutive weight values ​​of padding bits that may be removed from the weight values ​​without loss. For example, the pattern of padding bits that may be removed from the weight values ​​without loss may be a shared number of some and / or all of the padding bits in consecutive weight values. The pattern of padding bits that may be removed from the weight values ​​without loss in consecutive weight values ​​may be identified based on the number of consecutive weight values. The consecutive weight values ​​that exhibit a pattern may be grouped as a frame. In some embodiments, the identification of weight values ​​that exhibit a pattern identifies overlapping frames. In some embodiments, the compression processing device that identifies frames in the weight data set in block 804 may be a processor or an AI processor.

[0089] In block 806, the compression processing device may select a combination of frames. The selection of frames may be implemented by a search algorithm. In some embodiments, the search algorithm may be configured to select every combination of frames. In some embodiments, the search algorithm may be an exhaustive search algorithm. In some embodiments, the search algorithm may be configured to select the combination of frames based on selective search criteria that may limit the number of combinations of frames. In some embodiments, the search algorithm may be configured to select the combination of frames based on selective search criteria that may prioritize some frames and / or some combinations of frames. For example, the selective search criteria may include a frame length, a range of frame lengths, a normalization factor, a range of normalization factor, a weight value, a range of weight values, an offset value, a range of offset values, etc. In some embodiments, the selection of frames may select non-overlapping frames. In some embodiments, the selected frames may be consecutive frames. In some embodiments, the selected frames may be non-consecutive frames. In some embodiments, the compression processing device that selects the combination of frames in block 806 may be a processor or an AI processor.

[0090] The compression processing device may compress the selected combination of frames at block 808. Compression of the selected combination of frames is further described herein in method 900 with reference to Figure 9 and method 1000 with reference to Figure 10. In some embodiments, the compression processing device that compresses the selected frames at block 808 may be a processor or an AI processor.

[0091] At block 810, the compression processing device may calculate a compression metric for the selected combination for the frame. The compression metric may be based on a compression ratio, a compressed size, etc. The compression processing device that calculates the compression metric for the selected combination for the frame at block 810 may be a processor or an AI processor.

[0092] In decision block 812, the compression processing device may determine whether the compression metric meets and / or exceeds a compression metric threshold. The compression metric threshold may be based on a compression ratio, a compressed size, etc. In some embodiments, the compression metric threshold may be a predetermined value. A combination of frames that meets and / or exceeds the compression metric threshold may be used to compress the weight data set. In some embodiments, the compression metric threshold may be set to the best compression metric from an analysis of the compression metrics of multiple sets of selected combinations of frames. For example, the compression metric of the first selected combination of frames may be set as the compression metric threshold, and the compression metric of any subsequent selected combination of frames that exceeds the compression metric threshold may be set as the compression metric threshold. Once no selected combination of frames can exceed the compression metric threshold, the compression metric threshold may be the best compression metric. The combination of frames associated with the best compression metric may be the combination of frames to be used to compress the weight data set, as described further herein. The compression processing device that determines whether the compression metric meets and / or exceeds the compression metric threshold in decision block 812 may be a processor or an AI processor.

[0093] In response to a determination that the compression metric does not meet and / or exceed the compression metric threshold (i.e., decision block 812="No"), the compression processing device may select a combination of frames in block 806 as previously described herein.

[0094] In response to a determination that the compression metric meets and / or exceeds the compression metric threshold (i.e., decision block 812="Yes"), the compression processing device may determine whether there are any remaining combinations for the frame in optional decision block 814. As explained above, the best compression metric may be based on a comparison of each of the frame combinations to the compression metric threshold. Thus, when a frame combination has not yet been compared to the compression metric threshold, the remaining combination of frames may be compared to the compression metric threshold. In some embodiments, the compression processing device may determine whether there are any remaining combinations for the frame from a list, table, array, queue, stack, etc. of frame combinations. The compression processing device that determines whether there are any remaining combinations for the frame in optional decision block 814 may be a processor or an AI processor.

[0095] In response to a determination that there are remaining combinations for the frame (ie, optional decision block 814="Yes"), the compression processing device may select a combination of the frame at block 806 as previously described herein.

[0096] In response to a determination that the compression metric meets and / or exceeds a compression metric threshold (i.e., decision block 812="Yes") or in response to a determination that there are no remaining combinations for the frames (i.e., optional decision block 814="No"), the compression processing device may store the compressed selected combination of frames in block 816. The compressed selected combination of frames may be stored in a memory configured for persistent storage. The compression processing device that stores the compressed selected combination of frames in block 816 may be a processor or an AI processor.

[0097] FIG. 9 illustrates a method 900 for setting weight data compression parameters, according to one embodiment. With reference to FIGS. 1-9, the method 900 may be implemented in a computing device (e.g., computing device 100 in FIG. 1), in hardware, in software running on a processor, or in a combination of a processor and dedicated hardware configured with software (e.g., processor 104, memory 106, 114, AI processor 124 in FIG. 1, processor 202, processor memory 204, AI processor 206, memory 208 in FIG. 2). To encompass alternative configurations permitted in various embodiments, the hardware implementing the method 900 is referred to herein as a "compression processing device." In some embodiments, the method 900 may further describe block 808 of the method 800 described with reference to FIG. 8.

[0098] In optional block 902, the compression processing device may determine a frame length of the selected frame (e.g., frames 302a, 302b, 302c, 302d, 302e, 304a, 304b, 304c, 304d in FIG. 3A and FIG. 3B). In some embodiments, the frame length may be based on a number of consecutive weight values ​​that indicate a pattern of padding bits that can be removed from the weight values ​​without loss, used in identifying a frame in the weight data set (e.g., weight data set 300 in FIG. 3A and FIG. 3B), as described for block 804 in method 800 described with reference to FIG. 8. In some embodiments, the frame length may be predetermined during identifying a frame in the weight data set, as described for block 804 in method 800 described with reference to FIG. 8, and optional block 902 may not need to be performed. In optional block 902, the compression processing device that determines the frame length of the selected frame may be a processor or an AI processor.

[0099] In optional block 904, the compression processing device may remove an offset value from each weight value in the frame. Use of the offset value may be optional and block 904 may not be performed in embodiments in which an offset value is not used. In some embodiments, the offset value may not be used for compression of the selected frame as described for block 806 in the method 800 described with reference to FIG. 8. In some embodiments, the offset value may be used for compression of the selected frame having at least some weight values ​​for which the number of significant bits may be reduced by removing the offset value from the weight value. In some embodiments, the removal of the offset value from the weight value of the frame may depend on whether the removal of the offset value from the weight value results in a modified weight value with fewer significant bits or more padding bits than the weight value of the frame having the most significant bits or the fewest padding bits. In some embodiments, the offset value may be predetermined. In some embodiments, the offset value may be determined based on a value that may reduce the highest number of significant bits or increase the lowest number of padding bits for the weight of the frame. In some embodiments, the offset value may be removed by adding or subtracting the offset value and the weight value in the frame. In optional block 904, the compression processing device that removes the offset value from each weight value in the frame may be a processor or an AI processor.

[0100] In block 906, the compression processing device may identify the weight value with the most significant bits or the least padding bits in the frame. The compression processing device may analyze the bits of the weight value to determine significant bits representing the value of the weight value, including signed values, and compare which of the weight values ​​has the most significant bits. Similarly, the compression processing device may analyze the bits of the weight value to determine padding bits of the value of the weight value, and compare which of the weight values ​​has the least padding bits. The compression processing device that identifies the weight value with the most significant bits or the least padding bits in the frame in block 906 may be a processor or an AI processor.

[0101] At block 908, the compression processing device may set a normalization factor for the frame using the weight value with the most significant bits or the fewest padding bits in the frame. The normalization factor may represent a number of padding bits that may be removed from each frame's weight value. Setting the normalization factor may include determining the number of padding bits of the weight value with the most significant bits or the fewest padding bits in the frame and setting the normalization factor to the number of padding bits. In some embodiments, multiple weight values ​​may have the most significant bits or the fewest padding bits, and the compression processing device may select one of the weight values ​​to set the normalization factor. In some embodiments, the compression processing device may determine whether all of the bits of the frame are zero values. In other words, the compression processing device may determine whether all of the weight values ​​in the frame are zero values. The compression processing device may analyze each bit of the weight values ​​in the frame to determine their values ​​and determine whether each bit has a zero value. In response to determining that all of the bits of the frame are zero values, the compression processing device may set the normalization factor to the full bit width of the weight values ​​of the frame. In other words, the compression processing device may treat all of the bits of the weight value as padding bits and set the normalization factor to all of the bits of the weight value. The compression processing device that sets the normalization factor for the frame in block 908 using the weight value with the most significant bits or the fewest padding bits in the frame may be a processor or an AI processor.

[0102] FIG. 10 illustrates a method 1000 for compressing weight data according to one embodiment. With reference to FIGS. 1-10, the method 1000 may be implemented in a computing device (e.g., computing device 100 in FIG. 1), in hardware, in software running on a processor, or in a combination of a processor and dedicated hardware configured with software (e.g., processor 104, memory 106, 114, AI processor 124 in FIG. 1, processor 202, processor memory 204, AI processor 206, memory 208 in FIG. 2). To encompass alternative configurations allowed in various embodiments, the hardware implementing the method 1000 is referred to herein as a "compression processing device." In some embodiments, the method 1000 may be implemented in a continuation of the method 900 described herein with reference to FIG. 9.

[0103] In optional decision block 1002, the compression processing device may determine whether a normalization factor for a frame (e.g., frames 302a, 302b, 302c, 302d, 302e, 304a, 304b, 304c, 304d in FIGS. 3A and 3B) in the weight data set (e.g., weight data set 300 in FIGS. 3A and 3B) is the full bit-width of the frame's weight values. The normalization factor for the frame may be a normalization factor set for the frame in block 908 of the method 900 described herein with reference to FIG. 9. In some embodiments, the compression processing device may determine the bit-width of the frame's weight values ​​by analyzing the frame's weight values ​​and determining the bit-width of the weight values. In some embodiments, the bit-width of the frame's weight values ​​may be a preconfigured value. For example, the bit-width of the frame's weight values ​​may be a power of two value, such as 2 bits, 4 bits, 8 bits, 16 bits, 32 bits, 64 bits, 128 bits, etc. In optional decision block 1002, the compression processing device that determines whether the normalization coefficients for the frame are the full bit width of the weighting values ​​of the frame may be a processor or an AI processor.

[0104] In block 1004, the compression processing device may remove padding bits from the weight values ​​of the frame according to the normalization factor. The normalization factor may be configured to represent a number of padding bits to be removed from the weight values ​​of the frame. The number of padding bits to be removed from the frame, represented by the normalization factor, may be read by the compression processing device, and the compression processing device may remove that number of padding bits from each of the weight values ​​of the frame. In some embodiments, removing the padding bits may involve shifting the remaining bits of the weight values ​​of the frame to overwrite the padding bits being removed. Following removal of the padding bits, the remaining bits of the weight values ​​may be referred to as the compressed weight bit width. The compression processing device that removes padding bits from the weight values ​​of the frame according to the normalization factor in block 1004 may be a processor or an AI processor. In some embodiments, the compression processing device may remove padding bits from the weight values ​​of the frame according to the normalization factor in block 1004 in response to a determination that the normalization factor for the frame is not the full bit width of the weight values ​​of the frame (i.e., any decision block 1002="No")

[0105] At block 1006, the compression processing device may generate a frame payload for the frame (e.g., frame payloads 406a, 406b, 406c in FIG. 4, frame payloads 702a, 702b, 702c, 702d in FIG. 7) and a frame header (e.g., frame headers 404a, 404b, 404c in FIG. 4, frame header 600 in FIG. 6, frame headers 706a, 706b, 706c, 706d in FIG. 7). Generating a frame payload for the frame may include storing remaining bits of the frame's weight value. The stored remaining bits of the frame's weight value may be a compressed weight value for the frame. Generating a frame header for the frame may include storing compression parameters for the frame payload in association with the frame payload that may be used for decompression of the compressed weight value. The compression parameters may include a frame length (e.g., frame length 602 in FIG. 6), a normalization factor indicator (e.g., normalization factor 604 in FIG. 6, compressed weight bit width described with reference to FIG. 6), and / or an offset indicator (e.g., offset indicator 606 in FIG. 6). The frame length may be the frame length determined for the frame at block 902 of method 900 as described herein with reference to FIG. 9. The normalization factor indicator may be the normalization factor set for the frame at block 908 of method 900 described herein with reference to FIG. 9. The normalization factor indicator may be the compressed weight bit width resulting from removal of padding bits at block 1004. The offset indicator may be a value based on whether and / or which offset factor is applied to the weight values ​​of the frame at any block 904 in method 900 described herein with reference to FIG. 9. The frame header and the frame payload may be stored in association with one another in memory. In block 1006, a compression processing device that generates a frame payload and a frame header for the frame may be a processor or an AI processor.

[0106] In response to determining that the normalization coefficients for the frame are the full bit-width of the weight values ​​in the frame (i.e., optional decision block 1002="Yes"), the compression processing device may remove all of the bits from the weight values ​​in the frame in optional block 1012. The compression processing device that removes all of the bits from the weight values ​​in the frame in optional block 1012 may be a processor or an AI processor.

[0107] In optional block 1014, the compression processing device may generate a frame header for the frame (e.g., frame header 404d in FIG. 4, frame header 600 in FIG. 6, frame header 706e in FIG. 7). Generating the frame header in block 1014 may be performed in a similar manner to generating the frame header in block 1006. For a frame for which all of the bits are removed from the weight value, no frame payload may be generated. In optional block 1014, the compression processing device generating the frame header for the frame may be a processor or an AI processor.

[0108] Following generating a frame payload and frame header for the frame at block 1006 or generating a frame header for the frame at optional block 1014, the compression processing device may determine whether there are any remaining frames at decision block 1008. The compression processing device may determine whether there are any remaining frames from a list, table, array, queue, stack, etc. of frames. The compression processing device that determines whether there are any remaining frames at decision block 1008 may be a processor or an AI processor.

[0109] In response to a determination that there are no remaining frames (i.e., decision block 1008="No"), the compression processing device may generate a global header (e.g., global header 402 in FIG. 4, global header 500 in FIG. 5, global header 704 in FIG. 7) and a block of compressed weight data (e.g., block of compressed weight data 400 in FIG. 4, block of compressed weight data 700 in FIG. 7) in block 1010. Generating a global header for the block of compressed weight data may include storing compression parameters and weight data set parameters for the frame header and frame payload in association with the frame header and frame payload. The compression parameters may include an offset (e.g., offsets 502, 504 in FIG. 5). The offset may be a value of an offset applied to the weight values ​​of the frame in block 904 of the method 900 described herein with reference to FIG. 9. In some embodiments, the global header may include any number of reserved bits (e.g., reserved bits 506 in FIG. 5) that may be configured to provide parameters for restoring the block of compressed weight data.

[0110] In some embodiments, the global header may include a compressed sign value (e.g., compressed sign value 508 in FIG. 5) configured to indicate whether the frame payload of the block of compressed weight data contains signed compressed weight data. The compressed sign value may be generated during compression of the weight data set.

[0111] In some embodiments, the global header may include a frame header size (e.g., frame header size 510 in FIG. 5) configured to represent a size of the frame headers in the block of compressed weight data. The frame header size may be generated during compression of the weight data set. For example, the frame header size may be generated based on a uniform size for all of the frame headers in the block of compressed weight data. In some embodiments, the frame header size may be a preconfigured value.

[0112] In some embodiments, the global header may include an uncompressed width (e.g., uncompressed width 512 in FIG. 5 ) configured to represent a bit width of the uncompressed weight data set. For example, the uncompressed width may be configured to represent that the uncompressed weights may have a bit width of a number of bits that is a power of two, such as 2 bits, 4 bits, 8 bits, 16 bits, 32 bits, 64 bits, 128 bits, etc. The uncompressed width may be generated during compression of the weight data set. For example, the uncompressed width may be generated based on a bit width of the weight values ​​in the weight data set. In another example, the uncompressed width may be a preconfigured value. In some embodiments, the global header may include an uncompressed buffer size (e.g., uncompressed buffer size 514 in FIG. 5 ) configured to represent a bit width of a buffer configured to store the uncompressed weight data set. The uncompressed buffer size may be generated during compression of the weight data set. For example, the uncompressed buffer size may be generated based on a buffer size for the weight data set. In another example, the uncompressed buffer size may be a preconfigured value.

[0113] Generating the block of compressed weight data may include storing the global header, the frame header, and the frame payload in association with one another in memory. The compression processing device that generates the global header and the block of compressed weight data in block 1010 may be a processor or an AI processor.

[0114] In response to a determination that there is a remaining frame (i.e., decision block 1008="Yes"), the compression processing device may determine a frame length of the selected frame in any block 902 of the method 900 described herein with reference to FIG. 9, remove an offset value from each weight value in the frame in any block 904 of the method 900 described herein with reference to FIG. 9, or identify a weight value in the frame with the most significant bits or the fewest padding bits in the frame in block 906 of the method 900 described herein with reference to FIG. 9.

[0115] FIG. 11 illustrates a method 1100 for recovering weight data according to one embodiment. With reference to FIGS. 1-11, the method 1100 may be implemented in a computing device (e.g., computing device 100 in FIG. 1), in hardware, in software running on a processor, or in a combination of a processor and dedicated hardware configured with software (e.g., processor 104, memory 106, 114, AI processor 124 in FIG. 1, processor 202, processor memory 204, AI processor 206, memory 208 in FIG. 2, recoverer 710, header parser 712, block denormalizer 714 in FIG. 7). To encompass alternative configurations permitted in various embodiments, the hardware implementing the method 1100 is referred to herein as a "recovery processing device."

[0116] At block 1102, a decompression processing device may retrieve a block of compressed weight data (e.g., block of compressed weight data 400 in FIG. 4, block of compressed weight data 700 in FIG. 7). The block of compressed weight data may be retrieved from a memory of a computing device, such as a processor memory on a SoC (e.g., SoC 102 in FIG. 1, SoC 200 in FIG. 2) and / or a memory separate from the SoC. The decompression processing device that retrieves the block of compressed weight data at block 1102 may be a processor or an AI processor.

[0117] At block 1104, the decompression processing device may parse a global header of the block of compressed weight data (e.g., global header 402 in FIG. 4, global header 500 in FIG. 5, global header 704 in FIG. 7). The global header may be parsed to retrieve parameters for decompressing the block of compressed weight data. In some embodiments, the parameters for decompressing the block of compressed weight data retrieved from the global header may include an offset (e.g., offsets 502, 504 in FIG. 5). In some embodiments, the parameters for decompressing the block of compressed weight data retrieved from the global header may include a compressed sign value (e.g., compressed sign value 508 in FIG. 5) configured to indicate whether the frame payload of the block of compressed weight data (e.g., frame payloads 406a, 406b, 406c in FIG. 4 and frame payloads 702a, 702b, 702c, 702d in FIG. 7) contains signed compressed weight data. In some embodiments, parameters for recovering the block of compressed weight data retrieved from the global header may include a frame header size (e.g., frame header size 510 in FIG. 5) configured to represent a size of a frame header in the block of compressed weight data (e.g., frame headers 404a, 404b, 404c, 404d in FIG. 4, frame header 600 in FIG. 6, frame headers 706a, 706b, 706c, 706d, 706e in FIG. 7). In some embodiments, parameters for recovering the block of compressed weight data retrieved from the global header may include an uncompressed width (e.g., uncompressed width 512 in FIG. 5) configured to represent a bit width of the uncompressed weight data set. For example, the uncompressed width may be configured to represent that the uncompressed weights may have a bit width of a number of bits that is a power of two, such as 2 bits, 4 bits, 8 bits, 16 bits, 32 bits, 64 bits, 128 bits, etc.In some embodiments, the global header may include an uncompressed buffer size (e.g., uncompressed buffer size 514 in FIG. 5) configured to represent a bit-width of a buffer configured to store the uncompressed weight data set. The decompression processing device that parses the global header of the block of compressed weight data at block 1104 may be a processor or an AI processor.

[0118] At block 1106, the decompression processing device may parse frame headers of the blocks of compressed weight data. The frame headers may be parsed to retrieve parameters for decompressing the blocks of compressed weight data. The individual frame headers may be parsed to retrieve parameters for decompressing the associated frame payload. In some embodiments, the individual frame headers may be parsed to retrieve parameters for decompressing a weight value of zero without an associated frame payload. In some embodiments, the parameters for decompressing the blocks of compressed weight data retrieved from the frame headers may include a frame length (e.g., frame length 602 in FIG. 6), a normalization factor (e.g., normalization factor 604 in FIG. 6), and / or an offset indicator (e.g., offset indicator 606 in FIG. 6). The frame length may be configured to represent the number of weight values ​​included in an associated frame of the weight data set. In some embodiments, the frame length 602 may similarly be configured to represent the number of compressed weight values ​​included in an associated frame payload of the blocks of compressed weight data. In some embodiments, the frame length 602 may be similarly configured to represent the number of compressed weight values ​​without the associated frame payload of the block of compressed weight data. The normalization factor may be configured to represent the number of padding bits to add to the compressed weight values ​​included in the associated frame payload. In some embodiments, the normalization factor may be configured to represent the number of bits to add for the compressed weight values ​​without the associated frame payload. The offset indicator 606 may be configured to represent whether and / or which offset is to be applied to recover the compressed weight values ​​included in the associated frame payload. In block 1106, the recovery processing device that parses the frame header of the block of compressed weight data may be a processor or an AI processor.

[0119] In decision block 1108, the decompression processing device may determine whether the frame payload of the block of compressed weight data is signed. The compressed weight data of the frame payload may include signed and / or unsigned weight values. Having at least one signed weight value may cause the frame payload to be signed. In some embodiments, having at least one signed weight value in the compressed weight data may cause all frame payloads to be signed. Determining whether the frame payload is signed may be based on the compressed sign value retrieved from the global header in block 1104. The decompression processing device that determines whether the frame payload of the block of compressed weight data is signed in decision block 1108 may be a processor or an AI processor.

[0120] In response to determining that the frame payload of a block of compressed weight data is not signed (i.e., decision block 1108="No"), the decompression processing device may add padding bits to the bits in the frame payload according to a normalization factor at block 1110. The normalization factor may be used to indicate to the decompression processing device how many padding bits to add to the bits of each compressed weight value of the frame payload such that the bit width of each decompressed weight value totals the bit width of the original uncompressed weight value of the weight data set. In some embodiments, the decompression processing device may add padding bits to the bits in the frame payload by shifting each of the compressed weight values ​​by the number of bits of the normalization factor. In some embodiments, the decompression processing device may use bitwise arithmetic operations to adjust the value of the padding bits added. In the case of unsigned compressed weight data, what bit values ​​may be used for the padding bits may be preconfigured based on the most significant bits of each compressed weight value or based on the resource cost of the value of the padding bits. In block 1110, the reconstruction processing device that adds padding bits to bits in the frame payload according to the normalization factor may be a processor or an AI processor.

[0121] In response to determining that the frame payload of the block of compressed weight data is signed (i.e., decision block 1108="Yes"), the decompression processing device may add padding bits to the bits in the frame payload according to the normalization factor and the sign bit in block 1120. The normalization factor may be used in a similar manner as described in block 1110. In some embodiments, the decompression processing device may use bitwise arithmetic operations to adjust the value of the padding bits added. In the case of signed compressed weight data, what bit value may be used for the padding bits may be based on the value of the sign bit of each signed compressed weight value. In some embodiments, the padding bits may be sign extension bits. The decompression processing device that adds padding bits to the bits in the frame payload according to the normalization factor and the sign bit in block 1120 may be a processor or an AI processor.

[0122] Following adding padding bits to bits in the frame payload according to the normalization factor at block 1110 or adding padding bits to bits in the frame payload according to the normalization factor and the sign bit at block 1120, the reconstruction processing device may determine whether an offset indicator is set for the frame payload at optional decision block 1112. Whether an offset indicator is set for the frame payload may be determined from the output of parsing the associated frame header of the frame payload, and in particular parsing the offset indicator, at block 1106. The reconstruction processing device that determines whether an offset indicator is set for the frame payload at optional decision block 1112 may be a processor or an AI processor.

[0123] In response to a determination that the offset indicator is set for the frame payload (i.e., optional decision block 1112="Yes"), the reconstruction processing device may include an offset value for each recovered weight value in the frame payload in optional block 1114. The offset value may be determined from the output of parsing the global header in block 1104 and the offset. In some embodiments, the global header may include multiple offsets and the offset indicator of the associated frame header may be configured to indicate to the reconstruction processing device which offset from the global header to use. The reconstruction processing device may include the offset value in each recovered weight value in the frame payload. For example, the reconstruction processing device may add or subtract the offset value and each recovered weight value in the frame payload. The reconstruction processing device that includes the offset value in optional block 1114 for each recovered weight value in the frame payload may be a processor or an AI processor.

[0124] In decision block 1116, the reconstruction processing device may determine whether there is a remaining frame payload. The reconstruction processing device may determine whether there is a remaining frame payload from a list, table, array, queue, stack, etc. of frame payloads. The reconstruction processing device that determines whether there is a remaining frame payload in decision block 1116 may be a processor or an AI processor. In some embodiments, the reconstruction processing device may determine whether there is a remaining frame payload in decision block 1116 following adding padding bits to bits in the frame payload according to a normalization factor in block 1110 or adding padding bits to bits in the frame payload according to a normalization factor and a sign bit in block 1120. In some embodiments, the reconstruction processing device may determine whether there is a remaining frame payload in decision block 1116 in response to a determination that the offset indicator is not set for the frame payload (i.e., optional decision block 1112="No") or following including an offset value for each reconstructed weight value in the frame payload in optional block 1114.

[0125] In response to a determination that there is no remaining frame payload (i.e., decision block 1116="No"), the reconstruction processing device may generate a block of reconstructed weight data (e.g., block 720 of reconstructed weight data in FIG. 7) at block 1118. The reconstruction processing device may output and store in memory the reconstructed weight values ​​resulting from adding padding bits at block 1110. In some embodiments, the reconstruction processing device may output and store in memory the reconstructed weight values ​​resulting from adding padding bits at block 1120. In some embodiments, the reconstruction processing device may output and store in memory the reconstructed weight values ​​resulting from including an offset to the reconstructed weight values ​​at any block 1114. The reconstruction processing device that generates the block of reconstructed weight data at block 1118 may be a processor or an AI processor.

[0126] In response to determining that there is remaining frame payload (ie, decision block 1116 = "Yes"), the decompression processing device may parse the frame header of the block of compressed weight data at block 1106.

[0127] FIG. 12 illustrates a method 1200 for compressing weight data according to one embodiment. With reference to FIGS. 1-12, the method 1200 may be implemented in a computing device (e.g., computing device 100 in FIG. 1), in hardware, in software running on a processor, or in a combination of a processor and dedicated hardware configured with software (e.g., processor 104, memory 106, 114, AI processor 124 in FIG. 1, processor 202, processor memory 204, AI processor 206, memory 208 in FIG. 2). To encompass alternative configurations permitted in various embodiments, hardware implementing the method 1200 is referred to herein as a "compression processing device."

[0128] In block 1202, the compression processing device may identify frames (e.g., frames 302a, 302b, 302c, 302d, 302e, 304a, 304b, 304c, 304d in FIG. 3A and FIG. 3B) in a weight data set (e.g., weight data set 300 in FIG. 3A and FIG. 3B) based on a pattern of padding bits in the weight values ​​of the weight data set that may be removed from the weight values ​​for lossless compression based on selective search criteria. Block 1202 may be performed in a manner similar to the operations in block 804 of method 800 as described with reference to FIG. 8.

[0129] The compression processing device may select a combination of frames at block 1204. Block 1204 may be implemented in a manner similar to the operations at block 806 of method 800 as described with reference to FIG.

[0130] In block 1206, the compression processing device may remove padding bits from weight values ​​in frames of the selected combination of frames according to a normalization factor for each frame (e.g., normalization factor 604 in FIG. 6) to generate a frame payload for each of the selected frames (e.g., frame payloads 406a, 406b, 406c in FIG. 4 and frame payloads 702a, 702b, 702c, 702d in FIG. 7). The normalization factors may each represent a number of padding bits that may be removed from the weight values ​​of each frame. Block 1206 may be implemented in a manner similar to the operations in block 808 of method 800 as described with reference to FIG. 8 and / or block 1004 and / or optional block 1012 of method 1000 as described with reference to FIG. 10.

[0131] At block 1208, the compression processing device may determine whether the compression metric of the frame payload exceeds a compression metric threshold. Block 1208 may be implemented in a manner similar to the operation at decision block 812 of method 800 as described with reference to FIG.

[0132] At block 1210, the compression processing device may generate a block of compressed weight data having the frame payload (e.g., block 400 of compressed weight data in FIG. 4) in response to determining that the compression metric of the frame payload exceeds the compression metric threshold. Block 1210 may be implemented in a manner similar to the operations at block 1010 of method 1000 as described with reference to FIG.

[0133] FIG. 13 illustrates a method 1300 for recovering weight data according to one embodiment. With reference to FIGS. 1-13, the method 1300 may be implemented in a computing device (e.g., computing device 100 in FIG. 1), in hardware, in software running on a processor, or in a combination of a processor and dedicated hardware configured with software (e.g., processor 104, memory 106, 114, AI processor 124 in FIG. 1, processor 202, processor memory 204, AI processor 206, memory 208 in FIG. 2, recoverer 710, header parser 712, block denormalizer 714 in FIG. 7). To encompass alternative configurations allowed in various embodiments, the hardware implementing the method 1300 is referred to herein as a "recovery processing device."

[0134] At block 1302, the decompression processing device may retrieve a block of compressed weight data (e.g., block of compressed weight data 400 in FIG. 4, block of compressed weight data 700 in FIG. 7). The block of compressed weight data may include a frame header (e.g., frame headers 404a, 404b, 404c, 404d in FIG. 4, frame header 600 in FIG. 6, and frame headers 706a, 706b, 706c, 706d, 706e in FIG. 7) associated with a frame payload (e.g., frame payloads 406a, 406b, 406c in FIG. 4, and frame payloads 702a, 702b, 702c, 702d in FIG. 7). The frame header may include a normalization factor (e.g., normalization factor 604 in FIG. 6) representing the number of padding bits removed from the weight value generating the associated frame payload. The frame payload includes the compressed weight value. Block 1302 may be implemented in a manner similar to the operations in block 1102 of method 1100 as described with reference to FIG.

[0135] At block 1304, the reconstruction processing device may parse the frame header for the normalization coefficients. Block 1304 may be implemented in a manner similar to the operation at block 1106 of method 1100 as described with reference to FIG.

[0136] In block 1306, the reconstruction processing device may add padding bits to the compressed weight values ​​of the frame payload according to the normalization factor of the associated frame header to generate reconstructed weight values. Block 1306 may be implemented in a manner similar to the operations in blocks 1110 and / or 1120 of method 1100 as described with reference to FIG.

[0137] Weight data compression and / or decompression systems according to various embodiments (including but not limited to those described above with reference to FIGS. 1-13) may be implemented in a wide variety of computing systems, including mobile computing devices, one example of a mobile computing device suitable for use with various embodiments is shown in FIG. 14. The mobile computing device 1400 may include a processor 1402 coupled to a touch screen controller 1404 and an internal memory 1406. The processor 1402 may be one or more multi-core integrated circuits, general purpose or designated for specific processing tasks. The internal memory 1406 may be volatile or non-volatile memory, and may be secure and / or encrypted or non-secure and / or non-encrypted memory, or any combination thereof. Examples of memory types that may be utilized include, but are not limited to, DDR, LPDDR, GDDR, WIDEIO, RAM, SRAM, DRAM, P-RAM, R-RAM, M-RAM, STT-RAM, and embedded DRAM. The touchscreen controller 1404 and the processor 1402 may also be coupled to a touchscreen panel 1412, such as a resistive-sensing touchscreen, a capacitive-sensing touchscreen, an infrared-sensing touchscreen, etc. Additionally, the display of the mobile computing device 1400 need not have touchscreen capabilities.

[0138] The mobile computing device 1400 may have one or more wireless signal transceivers 1408 (e.g., Peanut, Bluetooth, ZigBee, Wi-Fi, RF radio) for transmitting and receiving communications and an antenna 1410 coupled to each other and / or to the processor 1402. The transceivers 1408 and antennas 1410 may be used with the circuits described above to implement various wireless transmission protocol stacks and interfaces. The mobile computing device 1400 may include a cellular network wireless modem chip 1416 that enables communication over a cellular network and is coupled to the processor.

[0139] The mobile computing device 1400 may include a peripheral device connection interface 1418 coupled to the processor 1402. The peripheral device connection interface 1418 may be solely configured to accept one type of connection or may be configured to accept various types of common or proprietary physical and communication connections, such as Universal Serial Bus (USB), FireWire, Thunderbolt, or PCIe. The peripheral device connection interface 1418 may also be coupled to a similarly configured peripheral device connection port (not shown).

[0140] The mobile computing device 1400 may also include a speaker 1414 for providing audio output. The mobile computing device 1400 may also include a housing constructed of plastic, metal, or a combination of materials for enclosing all or a portion of the components described herein. The mobile computing device 1400 may include a power source 1422 coupled to the processor 1402, such as a disposable battery or a rechargeable battery. The rechargeable battery may also be coupled to a peripheral device connection port to receive charging current from a source external to the mobile computing device 1400. The mobile computing device 1400 may also include a physical button 1424 for receiving user input. The mobile computing device 1400 may also include a power button 1426 for turning the mobile computing device 1400 on and off.

[0141] Weight data compression and / or recovery systems according to various embodiments (including but not limited to those described above with reference to FIGS. 1-13) may be implemented in a wide variety of computing systems, including a laptop computer 1500, an example of which is shown in FIG. 15. Many laptop computers include a touchpad touch surface 1517 that serves as the computer's pointing device and may therefore receive drag, scroll, and flick gestures similar to those implemented on the computing devices described above equipped with touch screen displays. The laptop computer 1500 typically includes a processor 1502 coupled to a volatile memory 1512 and a large capacity non-volatile memory, such as a flash memory disk drive 1513. In addition, the computer 1500 may have one or more antennas 1508 for transmitting and receiving electromagnetic radiation, which may be connected to a wireless data link and / or a cellular telephone transceiver 1516 coupled to the processor 1502. The computer 1500 may also include a floppy disk drive 1514 and a compact disk (CD) drive 1515 coupled to the processor 1502. In a notebook configuration, the computer housing includes a touchpad 1517, a keyboard 1518, and a display 1519, all coupled to the processor 1502. Other configurations of computing devices may include a computer mouse or trackball coupled to the processor (e.g., via a USB input), as is well known, and may also be used with various embodiments.

[0142] Weight data compression and / or decompression systems according to various embodiments (including but not limited to those described above with reference to FIGS. 1-13) may also be implemented in fixed computing systems, such as any of a variety of commercially available servers. An exemplary server 1600 is shown in FIG. 16. Such a server 1600 typically includes one or more multi-core processor assemblies 1601 coupled to volatile memory 1602 and large capacity non-volatile memory, such as a disk drive 1604. As shown in FIG. 16, multi-core processor assemblies 1601 may be added to the server 1600 by inserting them into a rack of assemblies. The server 1600 may also include a floppy disk drive, compact disk (CD) or digital versatile disk (DVD) disk drive 1606 coupled to the processor 1601. The server 1600 may also include a network access port 1603 coupled to the multi-core processor assembly 1601 for establishing a network interface connection with a network 1605, such as a local area network, the Internet, a public switched telephone network, and / or a cellular data network (e.g., CDMA, TDMA, GSM, PCS, 3G, 4G, LTE, 5G, or any other type of cellular data network) coupled to other broadcast system computers and servers.

[0143] Example implementations are described in the following paragraphs. Some of the following implementation examples are described with respect to example methods, but further example implementations may include the example methods described in the following paragraphs implemented by a computing device including a compression processing device configured to perform operations of the example methods, the example methods described in the following paragraphs implemented by a computing device including a decompression processing device configured to perform operations of the example methods, the example methods described in the following paragraphs implemented by a computing device including means for performing the functions of the example methods, and the example methods described in the following paragraphs implemented as a non-transitory processor-readable storage medium having stored thereon processor-executable instructions configured to cause a processor of a computing device to perform operations of the example methods.

[0144] Example 1. A method executed on a processor of a computing device, the method comprising: receiving a weight data set of binary numbers representing weight values; generating a first frame payload comprising a compressed first frame of a first subset of weight values ​​in the weight data set; generating a first frame header associated with the first frame payload, the first frame header comprising a normalization factor indicator for the compressed first frame; and generating a block of compressed weight data comprising the first frame payload.

[0145] Example 2. The method of Example 1, wherein generating the first frame payload includes compressing the first frame of a first subset of weight values ​​in the weight data set by removing padding bits from each weight value in the first subset of weight values ​​according to a normalization factor for the first frame to generate a compressed first frame.

[0146] Example 3. The method of Example 2, wherein the method further includes removing the offset value from each weight value of the first subset of weight values ​​for the first frame to generate modified weight values ​​for the first frame, and wherein compressing the first frame of the first subset of weight values ​​includes removing padding bits from the modified weight values ​​in the first frame in accordance with a normalization factor for the first frame to generate a compressed first frame.

[0147] Example 4. The method of Example 3, wherein the method further includes generating a global header for the block of compressed weight data, the global header including an offset value, and wherein generating a first frame header associated with the first frame payload includes generating a first frame header, the first frame header including an offset indicator configured to indicate whether the offset value is removed from the first subset of weight values ​​for the first frame.

[0148] Example 5. The method of example 2 through example 4, wherein the padding bits are sign extension bits.

[0149] Example 6. Identifying a first frame of a first subset of weight values ​​in the weight data set based on a pattern of padding bits in the weight values ​​that may be removed from the weight values ​​for lossless compression based on selective search criteria; setting a first normalization factor for the first frame representing a number of padding bits to be removed from each weight value of the first subset of weight values; identifying a second frame of a second subset of weight values ​​in the weight data set based on a pattern of padding bits in the weight values ​​that may be removed from the weight values ​​for lossless compression based on selective search criteria; The method of any of Examples 1-5, further comprising: determining whether all bits of the second frame are zero-valued; in response to determining that all of the bits of the second frame are zero-valued, setting a second normalization factor for the second frame to represent all of the bits of the second subset of weight values ​​for the second frame; compressing the second frame of the second subset of weight values ​​by removing all of the bits from the second subset of weight values ​​according to the second normalization factor for the second frame; and generating a second frame header not associated with the frame payload.

[0150] Example 7. The method of any of Examples 1-6, further comprising: identifying a first frame of a first subset of weight values ​​in the weight data set based on a pattern of padding bits in the weight values ​​that can be removed from the weight values ​​for lossless compression based on the selective search criteria; identifying a weight value in the first subset of weight values ​​for the first frame that has a highest number of significant bits; and setting a normalization factor for the first frame representing a number of padding bits to be removed from each weight value in the first subset of weight values ​​based on the highest number of significant bits.

[0151] Example 8. The method of any of Examples 1-7, wherein the method further includes determining whether a first compression metric of the first frame payload exceeds a compression metric threshold, and wherein generating the block of compressed weight data having the first frame payload includes generating the block of compressed weight data having the first frame payload in response to determining that the first compression metric of the first frame payload exceeds a compression metric threshold.

[0152] The method of Example 8, wherein the method further comprises: setting a compression metric threshold to a first compression metric of the first frame payload; generating a second frame payload comprising a compressed second frame of a second subset of the weight values ​​in the weight data set; determining whether the second compression metric of the second frame payload exceeds the compression metric threshold; in response to a determination that the second compression metric exceeds the compression metric threshold, setting a compression metric threshold to the second compression metric of the second frame payload; generating a third frame payload comprising a compressed third frame of a third subset of the weight values ​​in the weight data set; and determining whether a third compression metric of the third frame payload exceeds the compression metric threshold; and wherein in response to a determination that the first compression metric of the first frame payload exceeds the compression metric threshold, generating a block of compressed weight data having the first frame payload comprises a block of compressed weight data having the second frame payload in response to a determination that the third compression metric of the third frame payload does not exceed the compression metric threshold.

[0153] Example 10. A method executed in a processor of a computing device, comprising: extracting a block of compressed weight data, the block of compressed weight data including a first frame header associated with a first frame payload, the first frame header including a first normalization factor indicator, and the first frame payload including compressed weight values; and generating a first decompressed frame including decompressed weight values ​​of the compressed weight values ​​of the first frame payload.

[0154] Example 11. The method of Example 10, wherein the block of compressed weight data includes a global header having an offset value, and the first frame header includes an offset indicator configured to indicate whether an offset value will be included for each recovered weight value generated from the first frame payload, and the method further includes the steps of parsing the global header for the offset value, parsing the first frame header for the offset indicator, and determining whether the offset indicator is set in the first frame header, and generating a first recovered frame includes the step of including the offset value in each recovered weight value generated from the first frame payload associated with the first frame header in response to a determination that the offset indicator is set in the first frame header.

[0155] Example 12. The method of any of Examples 10 or 11, wherein the block of compressed weight data includes a second frame header not associated with the frame payload that includes a second normalization factor indicator, and the method further includes generating a second restored frame that includes restored weight values ​​having all zero-valued bits according to the second normalization factor indicator of the second frame header.

[0156] Example 13. The method of any of Examples 10-12, wherein the block of compressed weight data includes a second frame header associated with a second frame payload, the second frame header including a second normalization factor indicator, and the second frame payload including a compressed weight value, and the method further includes generating a second restored frame including a restored weight value of the compressed weight value of the second frame payload by adding padding bits to the compressed weight value of the second frame payload in accordance with the second normalization factor indicator of the second frame header.

[0157] Example 14. The method of any of Examples 10-13, wherein the step of generating a first restored frame includes the step of adding padding bits to compressed weight values ​​of the first frame payload according to a first normalization factor indicator of the first frame header to generate restored weight values, and a value of the padding bits for a first compressed weight value among the compressed weight values ​​of the first frame payload is determined from a most significant bit of the first compressed weight value.

[0158] Example 15. The method of any of examples 13 or 14, wherein the padding bits are sign extension bits.

[0159] Example 16. The method of any of Examples 10-15, wherein the first frame header includes a frame length configured to indicate a number of compressed weight values ​​in the first frame payload.

[0160] Example 17. The method of any of Examples 10-16, wherein the block of compressed weight data includes a second frame header not associated with the frame payload that includes a frame length configured to indicate a number of compressed weight values, and the method further includes generating a second restored frame that includes a number of consecutive restored weight values ​​having all-zero valued bits that correspond to the frame length of the second frame header.

[0161] Computer program code or "program code" for execution on a programmable processor to perform operations of various embodiments may be written in a high level programming language such as C, C++, C#, Smalltalk, Java, JavaScript, Visual Basic, Structured Query Language (e.g., Transact-SQL), Perl, or a variety of other programming languages. As used in this application, program code or programs stored on a computer-readable storage medium may refer to machine language code (such as object code) whose format is understandable by a processor.

[0162] The above method descriptions and process flow diagrams are provided merely as illustrative examples and do not require or imply that the operations of the various embodiments must be performed in the order presented. As will be appreciated by one of ordinary skill in the art, the order of operations in the above-described embodiments may be performed in any order. Words such as "then," "then," and "next" do not limit the order of operations; these words are merely used to guide the reader through the method description. Additionally, any reference to a claim element in the singular, for example using the articles "a," "an," or "the," should not be construed as limiting the element to the singular.

[0163] The various exemplary logic blocks, modules, circuits, and algorithmic operations described with respect to the various embodiments may be implemented as electronic hardware, computer software, or a combination of both. To clearly illustrate this interchangeability of hardware and software, the various exemplary components, blocks, modules, circuits, and operations have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the particular application and design constraints imposed on the overall system. Those skilled in the art may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the claims.

[0164] The hardware used to implement the various example logic, logic blocks, modules, and circuits described in connection with the embodiments disclosed herein may be implemented or performed using general purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A general purpose processor may be a microprocessor, but alternatively, the processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of computing devices, e.g., a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration. Alternatively, some operations or methods may be performed by circuitry specific to a given function.

[0165] In one or more embodiments, the described functions may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored as one or more instructions or codes on a non-transitory computer-readable medium or a non-transitory processor-readable medium. The operations of the methods or algorithms disclosed herein may be embodied in a processor-executable software module that may reside on a non-transitory computer-readable storage medium or a non-transitory processor-readable storage medium. A non-transitory computer-readable storage medium or a non-transitory processor-readable storage medium may be any storage medium that may be accessed by a computer or a processor. By way of example and not limitation, such a non-transitory computer-readable medium or a non-transitory processor-readable medium may include RAM, ROM, EEPROM, FLASH memory, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that may be used to store desired program code in the form of instructions or data structures and that may be accessed by a computer. As used herein, disk and disc include compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk, and Blu-ray disc, where disks typically reproduce data magnetically and discs reproduce data optically using lasers. Combinations of the above are also included within the scope of non-transitory computer-readable media and non-transitory processor-readable media. Additionally, operations of a method or algorithm may reside as one or any combination or set of code and / or instructions on a non-transitory processor-readable medium and / or a non-transitory computer-readable medium, which may be incorporated into a computer program product.

[0166] The above description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the claims. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other embodiments and implementations without departing from the scope of the claims. Thus, the present disclosure is not intended to be limited to the embodiments and implementations described herein, but should be accorded the widest scope consistent with the following claims and the principles and novel features disclosed herein. [Explanation of symbols]

[0167] 100 computing devices 102, 200 SoC 104, 202, 1402, 1502 processors 106, 114, 208 Memory 108 Communication Interface 110 Memory Interface 112 Communication Components 116, 1410, 1508 Antenna 120 Peripheral Device Interface 122 Peripheral Devices 124 Artificial Intelligence (AI) Processor, AI Processor 204 Processor memory, memory 206 AI Processor 210 Low Power Area 300 weights dataset, compressed weights dataset 302a, 302b, 302c, 302d, 302e, 304a, 304b, 304c, 304d Frames 400 blocks of compressed weight data 402, 500, 704 Global Headers 404a, 404b, 404c, 404d, 600, 706a, 706b, 706c, 706d, 706e Frame Header 406a, 406b, 406c, 702a, 702b, 702c, 702d Frame Payload 502, 504 offset 506 Reserved Bits 508 Compressed Code Value 510 Frame Header Size 512 uncompressed width 514 Uncompressed Buffer Size 602 Frame Length 604 Normalization Factor 606 Offset Indicator 700 blocks of compressed weight data, compressed weight data 710 Restorer 720 Block of Recovered Weight Data, Recovered Weight Data 712 Header Parser 714 Block Denormalizer 1400 Mobile Computing Devices 1404 Touch Screen Controller 1406 Internal Memory 1408 Radio signal transceiver, transceiver 1412 Touch Screen Panel 1414 Speaker 1416 Cellular Network Wireless Modem Chip 1418 Peripheral device connection interface 1422 Power supply 1424 Physical Buttons 1426 Power Button 1500 Laptop Computer, Computer 1512, 1602 Volatile Memory 1513, 1604 Disk Drive 1514 Floppy Disk Drive 1515 Compact Disc (CD) Drive 1516 Wireless Data Link and / or Cellular Telephone Transceivers 1517 Touchpad touch surface, touchpad 1518 Keyboard 1519 Display 1600 Server 1601 Multi-core processor assembly, processor 1603 Network Access Port 1605 Network 1606 Floppy disk drive, compact disk (CD) or digital versatile disk (DVD) disk drive

Claims

1. 1. A method executed in a processor of a computing device, comprising: receiving a binary weight data set representing weight values ​​for a neural network; generating a first frame payload comprising a compressed first frame of a first subset of the weight values ​​in the weight data set; generating a first frame header associated with the first frame payload, the first frame header including a normalization factor indicator for the compressed first frame, the normalization factor indicator indicating a number of padding bits to be removed from the weight values; generating a block of compressed weight data comprising the first frame payload; The method includes:

2. 2. The method of claim 1 , wherein generating the first frame payload comprises compressing a first frame of the first subset of weight values ​​in the weight data set by removing padding bits from each weight value of the first subset of weight values ​​according to a normalization factor for the first frame to generate the compressed first frame.

3. the method further comprising removing an offset value from each weight value in the first subset of weight values ​​for the first frame generating modified weight values ​​for the first frame; 3. The method of claim 2, wherein compressing the first frame of the first subset of weight values ​​comprises removing the padding bits from the modified weight values ​​in the first frame according to the normalization factor for the first frame to generate the compressed first frame.

4. the method further comprising generating a global header for the block of compressed weight data, the global header including the offset value; 4. The method of claim 3, wherein generating the first frame header associated with the first frame payload comprises generating the first frame header, the first frame header including an offset indicator configured to indicate whether the offset value is removed from the first subset of weight values ​​for the first frame.

5. identifying a first frame of the first subset of weight values ​​in the weight data set based on a pattern of padding bits in the weight values ​​that can be removed from the weight values ​​for lossless compression based on selective search criteria; establishing a first normalization factor for the first frame representing a number of padding bits to be removed from each weight value in the first subset of weight values; identifying a second frame of a second subset of the weight values ​​in the weight data set based on a pattern of padding bits in the weight values ​​that can be removed from the weight values ​​for lossless compression based on selective search criteria; determining whether all bits of the second frame have a value of zero; in response to determining that all of the bits of the second frame have a zero value, setting a second normalization factor for the second frame to represent all of the bits of the second subset of the weight values ​​for the second frame; compressing the second frame of the second subset of weight values ​​by removing all of the bits from the second subset of weight values ​​according to the second normalization factor for the second frame; generating a second frame header that is not associated with the frame payload; The method of claim 1, further comprising:

6. identifying a first frame of the first subset of weight values ​​in the weight data set based on a pattern of padding bits in the weight values ​​that can be removed from the weight values ​​for lossless compression based on selective search criteria; identifying a weight value from the first subset of weight values ​​for the first frame that has a highest number of significant bits; setting a normalization factor for the first frame representing a number of padding bits to be removed from each weight value in the first subset of weight values ​​based on the highest number of significant bits; The method of claim 1, further comprising:

7. 2. The method of claim 1 , further comprising determining whether a first compression metric of the first frame payload exceeds a compression metric threshold, and wherein generating the block of compressed weight data having the first frame payload comprises generating the block of compressed weight data having the first frame payload in response to determining that the first compression metric of the first frame payload exceeds the compression metric threshold.

8. The method further comprising: setting the compression metric threshold to the first compression metric of the first frame payload; generating a second frame payload comprising a compressed second frame of a second subset of the weight values ​​in the weight data set; determining whether a second compression metric of the second frame payload exceeds the compression metric threshold; in response to determining that the second compression metric exceeds the compression metric threshold, setting the compression metric threshold to the second compression metric of the second frame payload; generating a third frame payload comprising a compressed third frame of a third subset of the weight values ​​in the weight data set; determining whether a third compression metric of the third frame payload exceeds the compression metric threshold; Further comprising:

8. The method of claim 7, wherein generating the block of compressed weight data having the first frame payload in response to a determination that the first compression metric of the first frame payload exceeds the compression metric threshold comprises generating the block of compressed weight data having the second frame payload in response to a determination that the third compression metric of the third frame payload does not exceed the compression metric threshold.

9. 1. A computing device comprising: a compression processing device configured to perform operations, the operations comprising: receiving a binary weight data set representing weight values ​​for the neural network; generating a first frame payload comprising a compressed first frame of a first subset of the weight values ​​in the weight data set; generating a first frame header associated with the first frame payload, the first frame header including a normalization factor indicator for the compressed first frame, the normalization factor indicator indicating a number of padding bits to be removed from the weight value; and generating a block of compressed weight data comprising the first frame payload; a computing device comprising:

10. 1. A method executed in a processor of a computing device, comprising: retrieving a block of compressed weight data, the block of compressed weight data including a first frame header associated with a first frame payload, the first frame header including a first normalization factor indicator, the first frame payload including compressed weight values ​​for a neural network, the first normalization factor indicator indicating a number of padding bits to be removed from weight values; generating a first reconstructed frame comprising reconstructed weight values ​​of the compressed weight values ​​of the first frame payload; The method includes:

11. the block of compressed weight data includes a global header having an offset value; the first frame header includes an offset indicator configured to indicate, for each recovered weight value generated from the first frame payload, whether the offset value is to be included; The method further comprising: parsing the global header for the offset value; parsing the first frame header for the offset indicator; determining whether the offset indicator is set in the first frame header; Further comprising:

11. The method of claim 10, wherein generating the first recovered frame comprises, in response to determining that the offset indicator is set in the first frame header, including the offset value in each recovered weight value generated from the first frame payload associated with the first frame header.

12. the block of compressed weight data includes a second frame header not associated with a frame payload that includes a second normalization factor indicator, and the method further comprises: generating a second reconstructed frame according to the second normalization factor indicator of the second frame header, the second reconstructed frame comprising reconstructed weight values ​​having all zero-valued bits; 11. The method of claim 10, further comprising:

13. 11. The method of claim 10, wherein the block of compressed weight data includes a second frame header associated with a second frame payload, the second frame header including a second normalization factor indicator, and the second frame payload includes compressed weight values, the method further comprising generating a second restored frame comprising restored weight values ​​of the compressed weight values ​​of the second frame payload by adding padding bits to the compressed weight values ​​of the second frame payload in accordance with the second normalization factor indicator of the second frame header.

14. 11. The method of claim 10, wherein generating the first restored frame comprises adding padding bits to the compressed weight values ​​of the first frame payload according to the first normalization factor indicator of the first frame header to generate restored weight values, wherein values ​​of the padding bits for a first compressed weight value among the compressed weight values ​​of the first frame payload are determined from most significant bits of the first compressed weight value, and the padding bits are sign extension bits.

15. 1. A computing device comprising: a restoration processing device configured to perform operations, said operations comprising: retrieving a block of compressed weight data, the block of compressed weight data including a first frame header associated with a first frame payload, the first frame header including a first normalization factor indicator, the first frame payload including compressed weight values ​​for a neural network, the first normalization factor indicator indicating a number of padding bits to be removed from weight values; generating a first reconstructed frame comprising reconstructed weight values ​​of the compressed weight values ​​of the first frame payload; a computing device comprising:

Images