Compression techniques for deep neural network weights

Abstract

Various embodiments include methods and devices for compression and decompression of a weight data set. Some embodiments may include compressing the weight data by receiving a weight data set of binary numbers representing weight values, generating a frame payload including a compressed first frame of a first subset of weight values ​​in the weight data set, and generating a block of compressed weight data having the frame payload. Some embodiments may include recovering the weight data by retrieving the block of compressed weight data, the block of compressed weight data including a frame header associated with the frame payload, the frame header including a normalization factor indicator, and the frame payload including the compressed weight values, and generating a first recovered frame comprising recovered weight values ​​of the compressed weight values ​​of the frame payload.

Description

Technical Field

[0001] Related Applications This application claims the benefit of priority of U.S. Non-Provisional Application No. 17 / 220,620, filed Apr. 1, 2021, the entire content of which is incorporated herein by reference.

Background Art

[0002] In deep neural network processing, a quantized neural network model may include a large number of weights. The large number of weights requires a large amount of memory, system bandwidth, and power when the weights are used by a processor / hardware to perform deep neural network processing.

Summary of the Invention

Means for Solving the Problems

[0003] The various aspects disclosed may include an apparatus and method for compression of weight data for a neural network. The various aspects may include receiving a binary weight data set representing weight values, generating a first frame payload including a compressed first frame of a first subset of the weight values in the weight data set, generating a first frame header associated with the first frame payload, wherein the first frame header includes a normalization factor indicator for the compressed first frame, and generating a block of compressed weight data having the first frame payload.

[0004] In some aspects, generating the first frame payload may include compressing a first frame of a first subset of the weight values in the weight data set by removing padding bits from each of the weight values of the first subset of the weight values according to a normalization factor for the first frame to generate the compressed first frame.

[0005] Some embodiments may further include removing an offset value from each weight value of a first subset of the weight values ​​of the first frame to generate a modified weight value of the first frame, and compressing the first frame of the first subset of weight values ​​may include removing a padding bit from the modified weight values ​​in the first frame according to a normalization coefficient for the first frame to generate a compressed first frame.

[0006] Some embodiments may further include generating a global header for a block of compressed weight data, wherein the global header includes an offset value, and generating a first frame header associated with a first frame payload may include generating a first frame header, wherein the first frame header includes an offset indicator configured to indicate whether the offset value is removed from a first subset of the weight values ​​of the first frame.

[0007] In some embodiments, the padding bit may be a sign-extended bit.

[0008] Several embodiments include: identifying a first frame of a first subset of weight values ​​in a weight dataset based on a selective search criterion, based on a pattern of padding bits in the weight values ​​that can be removed from the weight values ​​for lossless compression; setting a first normalization coefficient for the first frame, which represents the number of padding bits to be removed from each weight value in the first subset of weight values; and identifying a second frame of a second subset of weight values ​​in a weight dataset based on a selective search criterion, based on a pattern of padding bits in the weight values ​​that can be removed from the weight values ​​for lossless compression. This may further include identifying, determining whether all bits of the second frame are zero, setting a second normalization coefficient for the second frame to represent all bits of the second subset of weight values ​​in response to the determination that all bits of the second frame are zero, compressing the second frame of the second subset of weight values ​​by removing all bits from the second subset of weight values ​​according to the second normalization coefficient for the second frame, and generating a second frame header that is not associated with the frame payload.

[0009] Some embodiments may further include identifying a first frame of a first subset of weight values ​​in a weight dataset based on a selective search criterion, based on a pattern of padding bits in the weight values ​​which can be removed from the weight values ​​for lossless compression; identifying a weight value in the first subset of weight values ​​of the first frame which has the highest number of effective bits; and setting a normalization coefficient for the first frame which, based on the highest number of effective bits, represents the number of padding bits to be removed from each weight value in the first subset of weight values.

[0010] Some embodiments may further include determining whether a first compression metric of a first frame payload exceeds a compression metric threshold, and generating a block of compressed weight data having the first frame payload may include generating a block of compressed weight data having the first frame payload in response to the determination that the first compression metric of the first frame payload exceeds a compression metric threshold.

[0011] Some embodiments may further include setting a compression metric threshold to a first compression metric of a first frame payload; generating a second frame payload containing a compressed second frame of a second subset of weight values ​​in a weight dataset; determining whether the second compression metric of the second frame payload exceeds the compression metric threshold; setting the compression metric threshold to a second compression metric of the second frame payload in response to the determination that the second compression metric exceeds the compression metric threshold; generating a third frame payload containing a compressed third frame of a third subset of weight values ​​in a weight dataset; and determining whether the third compression metric of the third frame payload exceeds the compression metric threshold; generating a block of compressed weight data containing the first frame payload in response to the determination that the third compression metric of the third frame payload does not exceed the compression metric threshold; and generating a block of compressed weight data containing the second frame payload in response to the determination that the third compression metric of the third frame payload does not exceed the compression metric threshold.

[0012] Various embodiments may include apparatus and methods for recovering weight data for a neural network. Various embodiments may include taking a block of compressed weight data, wherein the block of compressed weight data includes a first frame header associated with a first frame payload, the first frame header includes a first normalization coefficient indicator, the first frame payload includes compressed weight values, and generating a first recovered frame comprising the recovered weight values ​​of the compressed weight values ​​of the first frame payload.

[0013] In some embodiments, a block of compressed weight data includes an offset indicator, wherein a global header having an offset value is included, and a first frame header is configured to indicate whether the offset value will be included for each restored weight value generated from the first frame payload. In some embodiments, this may further include parsing the global header for the offset value, parsing the first frame header for the offset indicator, and determining whether the offset indicator is set in the first frame header, and generating the first restored frame may include including the offset value in each restored weight value generated from the first frame payload associated with the first frame header in response to the determination that the offset indicator is set in the first frame header.

[0014] In some embodiments, the compressed weight data block includes a second frame header, which is not associated with the frame payload, and includes a second normalization coefficient indicator. In some embodiments, this may further include generating a second restored frame, which includes restored weight values ​​having all zero bits, according to the second normalization coefficient indicator of the second frame header.

[0015] In some embodiments, a block of compressed weight data includes a second frame header associated with a second frame payload, the second frame header includes a second normalization coefficient indicator, and the second frame payload includes compressed weight values. In some embodiments, the invention further includes generating a second restored frame, which includes restored weight values ​​of the compressed weight values ​​of the second frame payload, by adding padding bits to the compressed weight values ​​of the second frame payload according to the second normalization coefficient indicator of the second frame header.

[0016] In some embodiments, generating a first restored frame may include adding padding bits to the compressed weight values ​​of the first frame payload according to a first normalization coefficient indicator of the first frame header in order to generate restored weight values, wherein the value of the padding bits for the first compressed weight values ​​of the compressed weight values ​​of the first frame payload is determined from the most significant bit of the first compressed weight values.

[0017] In some embodiments, the padding bits may be sign extension bits.

[0018] In some embodiments, the first frame header includes a frame length configured to indicate the number of compressed weight values ​​of the first frame payload.

[0019] In some embodiments, the compressed weight data includes a second frame header, which is not associated with the frame payload, and includes a frame length configured to indicate the number of compressed weight values. In some embodiments, this may further include generating a second restored frame, which includes several consecutive restored weight values, each having a bit of all zero values, corresponding to the frame length of the second frame header.

[0020] A further aspect includes a computing device having a compression processing device or a restoration processing device configured to perform any of the operations of the methods summarized above.

[0021] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate exemplary embodiments of various embodiments, and together with the general description above and the detailed description of the embodiments for implementing the invention below, serve to explain the features of the claims.

Brief Description of the Drawings

[0022] [Figure 1] It is a component block diagram showing an exemplary computing device suitable for implementing various embodiments. [Figure 2] It is a component block diagram showing an exemplary system on chip (SoC) suitable for implementing various embodiments. [Figure 3A] It is a data diagram showing an example of framing of the binary representation of decimal weight values for compression suitable for implementing various embodiments. [Figure 3B] It is a data diagram showing an example of framing of the binary representation of decimal weight values for compression suitable for implementing various embodiments. [Figure 4] It is a block diagram showing an example of a block of compressed weight data suitable for implementing various embodiments. [Figure 5] It is a block diagram showing an example of a global header of a block of compressed weight data suitable for implementing various embodiments. [Figure 6] It is a block diagram showing an example of a frame header of a block of compressed weight data suitable for implementing various embodiments. [Figure 7] It is a data and component block diagram showing an example of restoration of compressed weight data suitable for implementing various embodiments. [Figure 8] It is a process flow diagram showing a method for compressing weight data according to an embodiment. [Figure 9] A process flow diagram showing a method for setting weight data compression parameters according to one embodiment. [Figure 10] A process flow diagram showing a method for compressing weight data according to one embodiment. [Figure 11] A process flow diagram showing a method for restoring compressed weight data according to one embodiment. [Figure 12] A process flow diagram showing a method for compressing weight data according to one embodiment. [Figure 13] A process flow diagram showing a method for restoring compressed weight data according to one embodiment. [Figure 14] A component block diagram showing an exemplary mobile computing device suitable for implementing a weight data compression and / or restoration system according to various embodiments. [Figure 15] A component block diagram showing an exemplary mobile computing device suitable for implementing a weight data compression and / or restoration system according to various embodiments. [Figure 16] A component block diagram showing an exemplary server suitable for implementing a weight data compression and / or restoration system according to various embodiments.

Best Mode for Carrying Out the Invention

[0023] Various embodiments will be described in detail with reference to the accompanying drawings. As far as possible, the same reference numbers are used throughout the drawings to refer to the same or similar parts. References made to specific examples and implementations are for illustrative purposes only and do not limit the scope of the claims.

[0024] Various embodiments include methods for compressing and restoring weight data for deep neural networks, as well as computing devices for performing such methods. Some embodiments may include lossless compression techniques for weight data for deep neural networks. In some embodiments, compression may be based on identifying frames of weight data that exhibit repeating data patterns, and removing the repeating data for compression. In some embodiments, global and / or frame header information regarding how the weight data is compressed may provide information for restoring the compressed weight data.

[0025] Deep neural network processing often requires large amounts of data. This data may include large amounts of weight data, which imposes high memory, system bandwidth, and power costs on the computer system due to the processor / hardware used to perform the deep neural network processing. Embodiments described herein present methods for lossless weight data compression and ultra-low-cost decompression, as well as devices for implementing these methods, thereby reducing the memory, system bandwidth, and power costs of performing deep neural network processing that uses large amounts of weight data.

[0026] In some embodiments, weight data compression may be performed offline, and the compressed weight data may be stored in memory for performing deep neural network processing. Offline compression of weight data may allow for a costly and time-consuming search for appropriate compression of the weight data, such as measured by a compression metric of the compressed weight data compared to a compression metric threshold. In some embodiments, the compression metric threshold may be configured to identify the best compression of the weight data within parameters for searching for a combination of subsets of the weight data for compression. In some embodiments, sparsity in the weight data, such as consecutive weight values ​​being zero, may be compressed in such a way that none of the weight data are stored due to consecutive zero values.

[0027] Not all weight data, including padding data in weight values, is useful when performing deep neural network processing. In some embodiments, weight data compression can be performed by identifying frames of weight data that exhibit patterns of padding data that can be removed from the weight data without changing the weight values ​​of the frame. By removing padding data, the amount of data in the weight data frame and in the broader set of weight data can be reduced. Compression parameters, such as normalization coefficients configured to indicate the amount of padding data removed from each weight value in the frame, can be stored in relation to the compressed weight data for use when restoring the compressed weight data. Removal of padding data from a frame for weight data compression can be achieved by shifting the remaining bits of the weight data to overwrite the removed padding data. The number of bits of weight data remaining after the removal of padding bits is sometimes referred to herein as the compressed weight bit width.

[0028] In some embodiments, low-cost reconstruction can be achieved by using bitwise operations to denormalize the compressed weight data. A normalization coefficient may indicate to the reconstruction computing device the number of bits to shift each of the compressed weight values ​​in order to reintroduce padding bits into the compressed weight data. In some embodiments, bitwise arithmetic operations may be used to generate corresponding values ​​for the reintroduced padding data by comparing the weight values ​​with the original padding data. In some embodiments, the padding bits may be sign extension bits.

[0029] The terms “computing device” and “mobile computing device” are used interchangeably herein to refer to any or all of the following personal electronic devices, including cellular phones, smartphones, personal or mobile multimedia players, personal digital assistants (PDAs), laptop computers, tablet computers, convertible laptops / tablets (2-in-1 computers), smartbooks, ultrabooks, netbooks, palmtop computers, wireless email receivers, multimedia internet-enabled cellular phones, mobile game consoles, wireless game controllers, and similar personal electronic devices including memory and programmable processors. The term “computing device” may further refer to stationary computing devices, including personal computers, desktop computers, all-in-one computers, workstations, supercomputers, mainframe computers, embedded computers (such as in vehicles and other larger systems), servers, multimedia computers, and game consoles.

[0030] Figure 1 shows a system including a computing device 100 suitable for use with various embodiments. The computing device 100 may include a SoC 102 having a processor 104, memory 106, a communication interface 108, a memory interface 110, a peripheral device interface 120, and an artificial intelligence (AI) processor 124. The computing device 100 may further include a communication component 112 such as a wired or wireless modem, memory 114, an antenna 116 for establishing a wireless communication link, and / or peripheral devices 122. The processor 104 and the AI ​​processor 124 may include any of various processing devices, for example, several processor cores.

[0031] The term “system on a chip” or “SoC” is used herein to refer to a set of interconnected electronic circuits, including, but not limited to, processing devices, memory, and communication interfaces. Processing devices may include various different types of processors and / or processor cores, such as general-purpose processors, central processing units (CPUs), digital signal processors (DSPs), graphics processing units (GPUs), accelerated processing units (APUs), secure processing units (SPUs), subsystem processors, auxiliary processors, single-core processors, multi-core processors, controllers, and / or microcontrollers, which are specific components of computing devices. Processing devices may further embody other hardware and combinations of hardware, such as field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), other programmable logic devices, discrete gate logic, transistor logic, performance monitoring hardware, watchdog hardware, and / or time-based criteria. Integrated circuits may be configured such that the components of the integrated circuit reside on a single semiconductor material such as silicon.

[0032] An SoC 102 may contain one or more processors 104. A computing device 100 may contain two or more SoCs 102, thereby increasing the number of processors 104 and processor cores. A computing device 100 may also contain processors 104 not associated with an SoC 102. Each individual processor 104 may be a multi-core processor. Each processor 104 may be configured for a specific purpose, which may be the same as or different from other processors 104 in the computing device 100. One or more processors 104 and processor cores with the same or different configurations may be grouped together. A group of processors 104 or processor cores may be called a multiprocessor cluster.

[0033] The memory 106 of the SoC 102 may be volatile or non-volatile memory configured to store data and processor-executable code for access by the processor 104 or by other components of the SoC 102, including the AI ​​processor 124. The computing device 100 and / or SoC 102 may include one or more memories 106 configured for various purposes. One or more memories 106 may include volatile memory, such as random access memory (RAM) or main memory, or cache memory. These memories 106 may be configured to temporarily hold a limited amount of data received from a data sensor or subsystem, data and / or processor-executable code instructions that are requested for non-volatile memory and loaded from non-volatile memory into memory 106 in anticipation of future access based on various factors, and / or intermediate processing data and / or processor-executable code instructions generated by the processor 104 and / or AI processor 124 that are not stored in non-volatile memory but are temporarily stored for quick future access. In some embodiments, any number and combination of memories 106 may include one-time programmable or read-only memory.

[0034] Memory 106 may be configured to at least temporarily store data and processor executable code that are loaded into memory 106 from another memory device, such as another memory 106 or memory 114, for access by one or more of the processors 104, or by other components of the SoC 102, including the AI ​​processor 124. The data or processor executable code loaded into memory 106 may be loaded in response to the execution of a function by the processor 104, or by other components of the SoC 102, including the AI ​​processor 124. Loading data or processor executable code into memory 106 in response to the execution of a function may result from a memory access request to memory 106 that fails or "misses" because the requested data or processor executable code is not in memory 106. In response to a miss, a memory access request to another memory 106 or memory 114 may be made to load the requested data or processor executable code into memory 106 from that other memory 106 or memory 114. Loading data or processor-executable code into memory 106 in response to the execution of a function may result from a memory access request to another memory 106 or memory 114, and the data or processor-executable code may be loaded into memory 106 for later access.

[0035] The memory interface 110 and memory 114 may work in harmony to enable the computing device 100 to store data and processor executable code on a volatile and / or non-volatile storage medium, and to retrieve data and processor executable code from the volatile and / or non-volatile storage medium. Memory 114 may be configured much like one embodiment of memory 106, where memory 114 can store data or processor executable code for access by one or more of the processors 104, or by other components of the SoC 102, including the AI ​​processor 124. In some embodiments, memory 114 is non-volatile and may retain information after the computing device 100 is powered off. When power is restored and the computing device 100 restarts, the information stored on memory 114 may become available to the computing device 100. In some embodiments, memory 114 may be volatile and may not retain information after the computing device 100 is powered off. The memory interface 110 controls access to the memory 114 and can enable other components of the SoC 102, including the processor 104 or the AI ​​processor 124, to read data from and write data to the memory 114.

[0036] SoC102 may also include an AI processor 124. The AI ​​processor 124 may be a processor 104, a part of processor 104, and / or an independent component of SoC102. The AI ​​processor 124 may be configured to run a neural network for processing activation and weight values ​​on computing device 100. Computing device 100 may also include an AI processor 124 that is not associated with SoC102. Such an AI processor 124 may be an independent component of computing device 100 and / or integrated with other SoC102.

[0037] Some or all of the components of computing device 100 and / or SoC 102 may be arranged and / or combined in different ways, while still performing the functions of various embodiments. Computing device 100 is not limited to one of each of its components, and multiple instances of each component may be included in various configurations of computing device 100.

[0038] Figure 2 shows an SoC 200 (e.g., SoC 102 in Figure 1) which may be a component of a computing device (e.g., computing device 100 in Figure 1) having an AI processor 206 (e.g., AI processor 124 in Figure 1) and other components suitable for implementing one embodiment. Referring to Figures 1 and 2, the SoC 200 may include the various components described above. For example, the SoC 200 may include the AI ​​processor 206, a processor 202 (e.g., processor 104 in Figure 1), and processor memory (e.g., memory 106 in Figure 1). In some embodiments, some such components, such as those described as part of the SoC 200, may be located within the low-power area 210 of the SoC 200.

[0039] Blocks of compressed frames of the weight dataset can be stored in memory 208, which is separate from the SoC200 (for example, memory 114 in Figure 1), and / or in processor memory 204 on the SoC200. In some embodiments, blocks of compressed frames of the weight dataset can be loaded from memory 208 into processor memory 204.

[0040] The AI ​​processor 206 may be configured to perform neural network processes, such as generating inferences, using activation values ​​and weight values. The AI ​​processor 206 may retrieve weight values ​​for the neural network process from a block of compressed frames of the weight dataset. In some embodiments, the AI ​​processor 206 may receive a block of compressed frames of the weight dataset from memory 208. In some embodiments, the AI ​​processor 206 may receive a block of compressed frames of the weight dataset from processor memory 204.

[0041] The AI ​​processor 206 may be configured to reconstruct compressed frame blocks of the weight dataset in order to extract weight values. In some embodiments, the AI ​​processor 206 may consist of software for reconstructing compressed frame blocks of the weight dataset. In some embodiments, the AI ​​processor 206 may consist of circuitry and / or software for reconstructing compressed frame blocks of the weight dataset. Reconstructing compressed frame blocks of the weight dataset will be further described herein.

[0042] The weight values ​​resulting from the reconstruction of compressed frame blocks of the weight dataset can be used by the AI ​​processor 206 to perform a neural network process. In some embodiments, the AI ​​processor 206 may use a combination of the weight values ​​extracted from the compressed frame blocks of the weight dataset by reconstruction and the weight values ​​from the uncompressed weight dataset. In some embodiments, the results of the neural network process performed by the AI ​​processor 206 can be stored in memory 204 and / or memory 208. In some embodiments, the results of the neural network process performed by the AI ​​processor 206 can be retrieved from the AI ​​processor 206, memory 204, and / or memory 208 for processing by processor 202.

[0043] The description herein of the SoC200 and its various components shown in Figure 2 is intended to be illustrative only and not limiting. Some of the components of the exemplary SoC200 shown may be variably configured, combined, and separated. Some of the components may be included in more or fewer numbers and may be arranged and connected differently within the SoC200 or separately from the SoC200. Similarly, a number of other components may be included in the SoC200, such as other memory, processors, peripheral device subsystems, interfaces, and controllers.

[0044] Figures 3A and 3B show examples of framing binary representations of decimal weight values ​​for compression, which are suitable for implementing the embodiment. Referring to Figures 1-3B, the weight dataset 300 may include representations of weight values. The example weight dataset 300 in Figures 3A and 3B includes decimal and signed binary representations of integer weight values ​​(shown in Figures 3A and 3B for clarity).

[0045] For the purpose of compressing weight values ​​from the weight dataset 300, the weight dataset 300 is analyzed so that portions of the weight dataset 300 can be grouped into frames 302a, 302b, 302c, 302d, 302e, 304a, 304b, 304c, and 304d. A computing device (e.g., computing device 100) may have a processor (e.g., processor 104 in Figure 1, AI processor 124, processor 202 in Figure 2, AI processor 206 in Figure 2) configured to analyze the weight dataset 300, identify various frames including frames 302a, 302b, 302c, 302d, 302e, 304a, 304b, 304c, 304d, and / or other frames not shown, and select various combinations of frames to determine which frames to use to compress the weight dataset 300. Analysis of the weight dataset 300 may select combinations of frames where the compression metric satisfies and / or exceeds the compression metric threshold. The compression metric and / or compression metric threshold may be based on the compression ratio, compression size, etc.

[0046] In some embodiments, the compression metric threshold may be a predetermined value. Combinations of frames that satisfy and / or exceed the compression metric threshold may be used to compress the weight dataset 300. In some embodiments, the compression metric threshold may be set to the best compression metric from an analysis of the compressed weight dataset 300. For example, the compression metric of the first combination of frames may be set as the compression metric threshold, and the compression metric of any subsequent combination of frames that exceeds the compression metric threshold may be set as the compression metric threshold. When no combination of frames can exceed the compression metric threshold, the compression metric threshold may be the best compression metric. The combination of frames associated with the best compression metric may be the combination of frames that should be used to compress the weight dataset 300.

[0047] In some embodiments, frame identification and / or frame selection may be implemented by a search algorithm. In some embodiments, the search algorithm may be configured to identify any frame and / or select any combination of frames. In some embodiments, the search algorithm may be a brute-force search algorithm. In some embodiments, the search algorithm may be configured to identify frames and / or select combinations of frames based on selective search criteria that may limit the number of frames and / or combinations of frames. In some embodiments, the search algorithm may be configured to identify frames and / or select combinations of frames based on selective search criteria that may prioritize certain frames and / or certain combinations of frames. For example, selective search criteria may include frame length (e.g., the number of weight values ​​in a frame), a range of frame lengths, a normalization factor (e.g., the number of bits in a frame's weight values ​​that can be removed without loss), a range of normalization factors, weight values, a range of weight values, an offset value (e.g., a value that may change the frame's weight values ​​by that amount), a range of offset values, and so on.

[0048] A neural network may use weights quantized to a certain size. For example, a neural network may use weights quantized to 4 bits, 8 bits, 16 bits, etc. However, not all weight values ​​may use the full quantization size. Thus, a weight value may include bits that represent the weight value, which may include a sign bit, referred herein as a significant bit, and bits that are not necessary to represent the weight value, referred herein as padding bits. In some embodiments, padding bits may be repeating bit values, such as repeating the most significant bit of the weight value and / or a different bit value from the most significant bit. In some embodiments, padding bits may be sign extension bits. Some or all of the padding bits may be removed from the weight value to compress the weight value without loss. Analysis of the weight dataset 300 may identify patterns in consecutive weight values ​​of padding bits that can be removed from the weight value without loss. For example, a pattern of padding bits that can be removed from the weight value without loss may be a shared number of some and / or all of the padding bits in consecutive weight values. A pattern of padding bits that can be removed from a weight value without loss in a sequence of weight values ​​can be identified based on the number of consecutive weight values, sometimes called the frame length. Consecutive weight values ​​exhibiting a pattern can be grouped as frames 302a, 302b, 302c, 302d, 302e, 304a, 304b, 304c, and 304d. In some embodiments, the identification of patterned weight values ​​may identify overlapping frames, while frame selection may select non-overlapping frames. In some embodiments, the selected frames may be consecutive frames. In some embodiments, the selected frames may be discontinuous frames.

[0049] In some embodiments, the processor may apply an offset to some and / or all of the weight values. In some embodiments, the processor may apply an offset to the weight values ​​of a frame. Applying an offset to a weight value may involve modifying the weight value by the amount of the offset. For example, the offset may be removed from the weight value by subtraction from and / or addition to the weight value. Applying an offset to a weight value may generally reduce the number of bits in the weight value that can be used to represent the weight value, the number of effective bits. Similarly, applying an offset to a weight value may generally increase the number of padding bits in the weight value. The offset weight values ​​may be grouped into frames based on a pattern of padding bits that can be removed from the weight value without loss.

[0050] To compress the selected frames, the processor may remove padding bits from the weight values ​​within the selected frames according to a pattern of padding bits that can be removed without loss from the weight values ​​of the selected frames. Each of the weight values ​​within the selected frames may have some or all of its padding bits removed. The number of padding bits removed from each weight value within the selected frames may be referred to herein as the normalization factor. The modification of the weight values ​​may leave only the valid bits, and / or fewer than the valid bits and all of the padding bits for the weight values ​​within the selected frames. In some embodiments, the number of padding bits removed from each of the weight values ​​within the selected frames may be based on the number of padding bits that can be removed from the weight value with the most valid bits within the selected frames. Thus, that number of padding bits that can be removed from the weight value with the most valid bits may also be removed from the other weight values ​​within the selected frames. Each selected frame may have its own normalization factor. In some embodiments, removing that number of padding bits may involve shifting the remaining bits to overwrite the padding bits being removed. The number of bits remaining in the weight values ​​after the removal of padding bits may be referred to herein as the compressed weight bit width.

[0051] The example shown in Figure 3A illustrates the selection of frames 302a, 302b, 302c, 302d, and 302e, which have variable frame lengths. Frame 302a has a frame length configured to represent a group of two weight values ​​(e.g., -13 and -33). Frame 302b has a frame length configured to represent a group of nine weight values ​​(e.g., -3, 2, -2, 4, 1, 0, -2, 9, and 10). Frame 302c has a frame length configured to represent a group of one weight value (e.g., -153). Frame 302d has a frame length configured to represent a group of three weight values ​​(e.g., 0, 1, and 0). Frame 302e has a frame length configured to represent a group of three weight values ​​(e.g., 0, 0, and 0).

[0052] The example shown in Figure 3A further illustrates the selection of frames 302a, 302b, 302c, 302d, and 302e having variable normalization coefficients. Frame 302a has a normalization coefficient configured to represent removing eight padding bits from each of the weight values ​​in frame 302a. Frame 302b has a normalization coefficient configured to represent removing eleven padding bits from each of the weight values ​​in frame 302b. Frame 302c has a normalization coefficient configured to represent removing six padding bits from the weight values ​​in frame 302c. Frame 302d has a normalization coefficient configured to represent removing fourteen padding bits from each of the weight values ​​in frame 302d. In some embodiments, frame 302e has a normalization coefficient configured to represent removing fifteen padding bits from each of the weight values ​​in frame 302e. In some embodiments, frame 302e has a normalization coefficient configured to represent removing all bits from each of the weight values ​​in frame 302e. As will be further explained herein, frames such as frame 302e, which have only a weight value of 0, may be represented within a compressed block by the header without a frame payload, allowing all bits of the frame to be removed during compression.

[0053] The example shown in Figure 3B illustrates the selection of frames 304a, 304b, 304c, and 304d having uniform frame lengths. In this example, the uniform frame length is eight weight values. However, the uniform frame length can be any number of weight values. In some embodiments, the uniform frame length can be a number of weight values ​​that are powers of two (e.g., 2, 4, 8, 16, 32, 64, 128, etc.). Frame 304a has a frame length configured to represent a group of eight weight values ​​(e.g., -13, -33, -3, 2, -2, 4, 1, and 0). Frame 304b has a frame length configured to represent a group of eight weight values ​​(e.g., -2, 9, 10, -153, 0, 1, 0, and 33). Frame 304c has a frame length configured to represent a group of eight weight values ​​(e.g., 2, -26, 0, 0, 0, -9, -81, and 0). Frame 304d has a frame length configured to represent a group of eight weight values ​​(for example, -3, -12, 1, -125, -1, 5, 0, and -154).

[0054] The example shown in Figure 3B further illustrates the selection of frames 304a, 304b, 304c, and 304d, which have variable normalization coefficients. Frame 304a has normalization coefficients configured to represent the removal of eight padding bits from each of the weight values ​​in frame 304a. Frame 304b has normalization coefficients configured to represent the removal of six padding bits from each of the weight values ​​in frame 304b. Frame 304c has normalization coefficients configured to represent the removal of seven padding bits from the weight values ​​in frame 304c. Frame 304d has normalization coefficients configured to represent the removal of six padding bits from each of the weight values ​​in frame 304d.

[0055] The examples in Figures 3A and 3B are intended to be illustrative and do not limit the claims or the scope of this specification. The weight dataset 300 may contain any number of weight values, which may be of any size and / or format. The processor of a computing device analyzing the weight dataset 300 may identify and select frames of any number and / or length, and having any normalization coefficient.

[0056] Figure 4 shows an example of a compressed weight data block suitable for implementing an embodiment. Referring to Figures 1 to 4, the compressed weight data block 400 may include a global header 402, any number of frame headers 404a, 404b, 404c, 404d, and any number of frame payloads 406a, 406b, 406c. A compression computing device (e.g., computing device 100) may have a processor (e.g., processor 104, AI processor 124 in Figure 1, processor 202, AI processor 206 in Figure 2) configured to compress a weight dataset (e.g., weight dataset 300 in Figures 3A and 3B). Compression of the weight dataset may produce a compressed weight data block 400. A restoration computing device (e.g., computing device 100) may have a processor (e.g., processor 104, AI processor 124 in Figure 1, processor 202, AI processor 206 in Figure 2) configured to restore the compressed weight data block 400. The compressed weight data block 400 may be stored in the memory of the compression computing device and / or the decompression computing device (for example, memories 106 and 114 in Figure 1, and processor memory 204 and memory 208 in Figure 2).

[0057] As further described herein, the global header 402 may include parameters that can be applied for the restoration of any combination and / or all of the frame payloads 406a, 406b, and 406c. As further described herein, the frame headers 404a, 404b, and 404c may include parameters that can be applied for the restoration of the associated frame payloads 406a, 406b, and 406c, and the frame header 404d may include parameters that can be applied for restoration without an associated frame payload. The frame payloads 406a, 406b, and 406c may include compressed weight values ​​of the associated frames (for example, frames 302a, 302b, 302c, 302d, 302e, 304a, 304b, 304c, and 304d in Figures 3A and 3B). In some embodiments, the compressed weight data blocks 400 may be ordered such that the associated frame headers 404a, 404b, 404c and frame payloads 406a, 406b, 406c are paired. In some embodiments, the compressed weight data blocks 400 may be ordered such that the frame headers 404a, 404b, 404c, 404d and frame payloads 406a, 406b, 406c are ordered in the order of frames in the weight dataset.

[0058] Figure 5 shows an example of a global header for a compressed weight data block (e.g., compressed weight data block 400 in Figure 4) suitable for implementing an embodiment. Referring to Figures 1 to 5, the global header 500 (e.g., global header 402 in Figure 4) is of arbitrary size and may contain parameters that can be applied for any combination and / or all restoration of the compressed weight data frame payloads (e.g., frame payloads 406a, 406b, 406c in Figure 4). The compression computing device (e.g., computing device 100) may have a processor (e.g., processor 104, AI processor 124 in Figure 1, processor 202, AI processor 206 in Figure 2) configured to compress the weight dataset (e.g., weight dataset 300 in Figures 3A and 3B). Compression of the weight dataset may generate the global header 500. A recovery computing device (e.g., computing device 100) may have processors (e.g., processor 104, AI processor 124 in Figure 1, processor 202, AI processor 206 in Figure 2) configured to recover blocks of compressed weight data using a global header 500. The global header 500 may be stored in the memory of the compression computing device and / or recovery computing device (e.g., memories 106, 114 in Figure 1, processor memory 204, memory 208 in Figure 2). In some embodiments, the global header 500 may be of any bit width. For example, the global header 500 may be 9 bytes.

[0059] The global header 500 may contain any number of offsets 502, 504. In some embodiments, offsets 502, 504 may be values ​​applied to the weight values ​​of any number and / or combinations of frames (e.g., frames 302a, 302b, 302c, 302d, 302e, 304a, 304b, 304c, 304d in Figures 3A and 3B) for the purpose of compressing the weight dataset. In some embodiments, the default value of offsets 502, 504 may be set to 0. Offsets 502, 504 may be applied to the weight values ​​for the purpose of compressing the weight dataset in order to modify the weight values ​​by the offsets only. For example, the offsets may be removed from the weight values ​​by subtraction from and / or addition to the weight values. Applying offsets to weight values ​​can generally reduce the number of bits in the weight value that can be used to represent the weight value, the number of effective bits. Similarly, applying offsets to weight values ​​can generally increase the number of padding bits in the weight value. The offset value may be used in restoring a block of compressed weight data to return the weight values, which have been modified by the offset during compression, back to their original values. For example, the offset may be added to the modified weight values ​​by subtraction from and / or addition to the modified weight values. In some embodiments, the offsets 502, 504 may have the same bit width as the weight values. For example, the offsets 502, 504 may have a bit width of a number of bits that is a power of 2, such as 2 bits, 4 bits, 8 bits, 16 bits, 32 bits, 64 bits, 128 bits, etc.

[0060] In some embodiments, the global header 500 may include any number of reserved bits 506 that can be configured to provide parameters for restoring blocks of compressed weight data. In some embodiments, the number of reserved bits 506 may be 1 bit, 2 bits, and so on.

[0061] In some embodiments, the global header 500 may include a compressed sign value 508 configured to indicate whether the frame payload of a block of compressed weight data contains signed compressed weight data. The compressed sign value 508 may be generated during the compression of the weight dataset. For example, the compressed sign value 508 may be configured to indicate that the frame payload of a block of compressed weight data contains signed compressed weight data when at least one weight value in the weight dataset is a signed weight value. In another example, the compressed sign value 508 may be configured to indicate that the frame payload of a block of compressed weight data contains signed compressed weight data when at least one weight value in at least one frame of the weight dataset is a signed weight value. In yet another example, the compressed sign value 508 may be configured to indicate that the frame payload of a block of compressed weight data does not contain signed compressed weight data when none of the weight values ​​in the weight dataset are signed weight values. In another example, the compressed sign value 508 may be configured to indicate that the frame payload of a block of compressed weight data does not contain signed compressed weight data, when none of the weight values ​​in any frame of the weight dataset are signed weight values.

[0062] The compressed sign value 508 may be used during the restoration of a block of compressed weight data to determine whether the restored weight values ​​are signed or not. For example, the compressed sign value 508 may be configured to indicate that the frame payload of the block of compressed weight data contains signed compressed weight data, and therefore the restored weight values ​​are signed. Restoring signed weight values ​​may involve copying the values ​​of the most significant bits of each compressed weight value, such as the sign bit, and using the values ​​of the most significant bits as sign extension bits to restore each respective compressed weight value. In another example, the compressed sign value 508 may be configured to indicate that the frame payload of the block of compressed weight data does not contain signed compressed weight data, and therefore the restored weight values ​​are unsigned. Restoring unsigned weight values ​​may involve adding padding bits to each compressed weight value. The padding bits can be any combination or pattern of bit values. For example, the padding bits could all be the same bit value, such as all "0" or all "1". In some embodiments, the padding bit may be the bit value of the most significant bit of the compressed weight value, or the opposite bit value. In some embodiments, the compressed sign value 508 may be of any bit width. For example, the compressed sign value 508 may be 1 bit.

[0063] In some embodiments, the global header 500 may include a frame header size 510 configured to represent the size of the frame headers in the compressed weight data blocks (e.g., frame headers 404a, 404b, 404c, 404d in Figure 4). The frame header size 510 may be generated during the compression of the weight dataset. For example, the frame header size 510 may be generated based on a uniform size for all frame headers in the compressed weight data blocks. In some embodiments, the frame header size 510 may be a pre-configured value. The frame header size 510 may be used during the restoration of the compressed weight data blocks to determine where the frame headers and / or frame payloads are located within the compressed weight data blocks. For example, a uniform size for frame headers may be used to identify data in the compressed weight data blocks that are the size of the frame headers and that are not the size of the frame headers and that that are the frame payloads. In some embodiments, the frame header size 510 may be of any bit width. For example, the frame header size 510 may be 2 bits.

[0064] In some embodiments, the global header 500 may include an uncompressed width 512 configured to represent the bit width of the uncompressed weight data. For example, the uncompressed width 512 may be configured to indicate that the uncompressed weights may have a bit width of a power of 2, such as 2 bits, 4 bits, 8 bits, 16 bits, 32 bits, 64 bits, or 128 bits. The uncompressed width 512 may be generated during the compression of the weight dataset. For example, the uncompressed width 512 may be generated based on the bit width of the weight values ​​in the weight dataset. In another example, the uncompressed width 512 may be a pre-configured value. The uncompressed width 512 may be used during the restoration of a block of compressed weight data to determine the bit width of the restored weight values. In some embodiments, the uncompressed width 512 may be any bit width. For example, the uncompressed width 512 may be 3 bits.

[0065] In some embodiments, the global header 500 may include an uncompressed buffer size 514 configured to represent the bit width of a buffer configured to store uncompressed weight data. The uncompressed buffer size 514 may be generated during the compression of the weight dataset. For example, the uncompressed buffer size 514 may be generated based on the buffer size for the weight dataset. In another example, the uncompressed buffer size 514 may be a pre-configured value. The uncompressed buffer size 514 may be used during the restoration of a block of compressed weight data to determine the buffer size for the restored weight values. In some embodiments, the uncompressed buffer size 514 may be of any bit width. For example, the uncompressed buffer size 514 may be 32 bits.

[0066] Figure 6 shows an example of a frame header for a compressed weight data block (e.g., compressed weight data block 400 in Figure 4) suitable for implementing an embodiment. Referring to Figures 1 to 6, the frame header 600 (e.g., frame headers 404a, 404b, 404c, 404d in Figure 4) can be of any size and may contain parameters that can be applied for any combination and / or all restoration of the compressed weight data frame payload (e.g., frame payloads 406a, 406b, 406c in Figure 4). The compression computing device (e.g., computing device 100) may have a processor (e.g., processor 104, AI processor 124 in Figure 1, processor 202, AI processor 206 in Figure 2) configured to compress the weight dataset (e.g., weight dataset 300 in Figures 3A and 3B). Compression of the weight dataset may generate the frame header 600. A reconstructive computing device (e.g., computing device 100) may have processors (e.g., processor 104, AI processor 124 in Figure 1, processor 202, AI processor 206 in Figure 2) configured to reconstruct blocks of compressed weight data using a frame header 600. The frame header 600 may be stored in the memory of the compression computing device and / or the reconstructive computing device (e.g., memories 106, 114 in Figure 1, processor memory 204, memory 208 in Figure 2).

[0067] In some embodiments, the frame header 600 may be associated with a frame in the weight dataset (for example, frames 302a, 302b, 302c, 302d, 302e, 304a, 304b, 304c, 304d in Figures 3A and 3B). In some embodiments, the frame header 600 may be associated with the frame payload of a block of compressed weight data. In some embodiments, the frame header 600 may be included in a block of compressed weight data without association with a frame payload. For example, the frame header 600 may be included in a block of compressed weight data without association with a frame payload for a frame header 600 associated with a frame of weight data containing all zero weight values. In some embodiments, the frame header 600 may be of any bit width. For example, the frame header 600 may be 8 bits.

[0068] The frame header 600 may include a frame length 602 configured to represent the number of weight values ​​contained in the associated frame of the weight dataset. In some embodiments, the frame length 602 may similarly be configured to represent the number of compressed weight values ​​contained in the associated frame payload of the compressed weight data block. In some embodiments, the frame length 602 may similarly be configured to represent the number of compressed weight values ​​without the associated frame payload of the compressed weight data block. For example, the frame length 602 may be configured to represent the number of compressed, consecutive zero weight values. The frame length 602 may be generated during the compression of the weight dataset. For example, the frame length 602 may be generated based on the number of weight values ​​in the associated frame of the weight dataset. In another example, the frame length 602 may be generated based on the number of compressed weight values ​​in the associated frame payload. In some embodiments, the frame length 602 may be a pre-configured value. For example, the frame length 602 can be a pre-configured power of 2, such as 2 bits, 4 bits, 8 bits, 16 bits, 32 bits, 64 bits, or 128 bits. In another example, the frame length 602 can be a value divisible by 16 for an 8-bit weight value. In yet another example, the frame length 602 can be a value divisible by 8 for a 16-bit weight value. The frame length 602 can be used during the reconstruction of a block of compressed weight data to determine the number of weight values ​​to be reconstructed from an associated frame payload. In some embodiments, the frame length 602 can be used during the reconstruction of a block of compressed weight data to determine the number of weight values ​​to be reconstructed for a frame header 600 not associated with a frame payload. For example, the frame length 602 can be used to determine the number of consecutive zero weight values ​​to be reconstructed. The frame length 602 can be any bit width. For example, the frame length 602 can be 4 bits for an 8-bit weight value. In yet another example, the frame length 602 can be 3 bits for a 16-bit weight value.

[0069] The frame header 600 may include a normalization coefficient 604 configured to represent the number of padding bits removed from the weight values ​​contained in the associated frame of the weight dataset. In some embodiments, the normalization coefficient 604 may similarly be configured to represent the number of padding bits to be added to the compressed weight values ​​contained in the associated frame payload of the compressed weight data block. In some embodiments, the normalization coefficient 604 may be configured to represent the number of bits removed from the weight values ​​of zero contained in the associated frame of the weight dataset. In some embodiments, the normalization coefficient 604 may similarly be configured to represent the number of bits to be added for the compressed weight values ​​without the associated frame payload of the compressed weight data block. For example, the normalization coefficient 604 may be configured to represent the number of bits for the weight values ​​of zero. The normalization coefficient 604 may be generated during the compression of the weight dataset. For example, the normalization coefficient 604 may be generated based on the number of bits removed from the weight values ​​in the associated frame of the weight dataset in order to generate the associated frame payload. The normalization factor 604 may be used during the reconstruction of a block of compressed weight data to determine the number of padding bits to add to the compressed weight values ​​in order to reconstruct them from the associated frame payload. In some embodiments, the normalization factor 604 may be used during the reconstruction of a block of compressed weight data to determine the number of bits for the weight values ​​of zero to reconstruct for frame headers 600 that are not associated with a frame payload. The normalization factor 604 can be of any bit width. For example, the normalization factor 604 may be 3 bits for an 8-bit weight value. In another example, the normalization factor 604 may be 4 bits for a 16-bit weight value.

[0070] In some embodiments, the normalization coefficient 604 may be replaced in the frame header 600 by the compressed weight bit width of the remaining bits of the weight values ​​contained in the associated frame of the weight dataset after the removal of padding bits, as shown in the example in Figure 6. During reconstruction, the normalization coefficient 604 may be determined for the frame using the compressed weight bit width of the associated frame and an uncompressed width (e.g., the uncompressed width 512 in Figure 5) configured to represent the bit width of the uncompressed weight dataset. For example, the compressed weight bit width of the associated frame may be subtracted from the uncompressed width to determine the normalization coefficient 604.

[0071] In some embodiments, the frame header 600 may include a normalization coefficient indicator, which may be a normalization coefficient 604. In some embodiments, the frame header 600 may include a normalization coefficient indicator, which may be a compressed weight bit width substituted for the normalization coefficient 604, as shown in the example in Figure 6.

[0072] The frame header 600 may include an offset indicator 606 configured to indicate whether an offset (for example, offsets 502, 504 in Figure 5) was applied to a weight value contained in the associated frame of the weight dataset, and / or which offset was applied. In some embodiments, the offset indicator 606 may similarly be configured to indicate whether an offset was applied to a compressed weight value as a compressed weight value contained in the associated frame payload of a block of compressed weight data, and / or which offset was applied. In some embodiments, the offset indicator 606 may similarly be configured to indicate whether an offset will be applied to restore a compressed weight value contained in the associated frame payload of a block of compressed weight data, and / or which offset will be applied. The offset indicator 606 may be a value associated with an offset contained in the global header of the block of compressed weight data (for example, global header 500 in Figure 5). For example, the offset indicator 606 may be a bit flag for which a first value is associated with a first offset in the global header, and a second value is associated with a second offset in the global header. The offset indicator 606 may indicate which of the offsets in the global header were used to compress the weight values ​​in the weight dataset, and which of the offsets in the global header will be used to restore the compressed weight values. The offset indicator 606 can be of any bit width. For example, the offset indicator 606 can be 1 bit.

[0073] The examples in Figures 4 to 6 are intended to be illustrative and do not limit the claims or the scope of this specification. The compressed weight data block 400 in Figure 4, the global header 500 in Figure 5, and the frame header in Figure 6 may be variably included and / or excluded, sized differently, and / or ordered differently.

[0074] Figure 7 shows an example of restoring a compressed weight data block (e.g., compressed weight data block 400 in Figure 4) suitable for implementing an embodiment. Referring to Figures 1 to 7, the compressed weight data block 700 can be restored to generate the restored weight data block 720. The restoration computing device (e.g., computing device 100) may have a processor (e.g., processor 104, AI processor 124 in Figure 1, processor 202, AI processor 206 in Figure 2) configured to restore the compressed weight data block 700. The processor may consist of circuitry for implementing the restorer 710 and / or software for that purpose. In some embodiments, the software configured for implementing the restorer 710 may be stored in the memory of the restoration computing device (e.g., memories 106, 114 in Figure 1, processor memory 204, memory 208 in Figure 2). The recovery computing device may recover the compressed weight data block 700 using the global header 704 (e.g., global header 402 in Figure 4, global header 500 in Figure 5), frame headers 706a, 706b, 706c, 706d, 706e (e.g., frame headers 404a, 404b, 404c, 404d in Figure 4, frame header 600 in Figure 6), and / or frame payloads 702a, 702b, 702c, 702d (e.g., frame payloads 406a, 406b, 406c in Figure 4). In some embodiments, the compressed weight data block 700 may be stored in the memory of the recovery computing device (e.g., memories 106, 114 in Figure 1, processor memory 204, memory 208 in Figure 2).

[0075] The restorer 710 may include a header parser 712 and a block denormalizer 714. The restorer 710 may retrieve a block 700 of the compressed weight data. For example, the restorer 710 may retrieve a block 700 of the compressed weight data from the memory of a recovery computing device in which the block 700 of the compressed weight data is stored.

[0076] The header parser 712 may parse the global header 704 to extract parameters for reconstructing the compressed weight data block 700. The header parser 712 may also parse the frame headers 706a, 706b, 706c, and 706d to extract parameters for reconstructing the associated frame payloads 702a, 702b, 702c, and 702d. In some embodiments, the header parser 712 may also parse the frame header 706e to extract parameters for reconstructing weight values ​​of 0 without the associated frame payload.

[0077] The block denormalizer 714 may use parameters extracted from the global header 704 and frame headers 706a, 706b, 706c, 706d, and 706e to reconstruct the compressed weight data block 700. More specifically, the block denormalizer 714 may use parameters extracted from the global header 704 and frame headers 706a, 706b, 706c, and 706d to reconstruct the associated frame payloads 702a, 702b, 702c, and 702d. Furthermore, the block denormalizer 714 may use parameters extracted from the global header 704 and frame header 706e to reconstruct weight values ​​of 0 without the associated frame payloads. In some embodiments, the block denormalizer 714 may restore the weight values ​​of the frame payloads 702a, 702b, 702c, and 702d by shifting each of the compressed weight values ​​by the number of bits of the normalization coefficient (e.g., normalization coefficient 604 in Figure 6) from the associated frame headers 706a, 706b, 706c, and 706d. In some embodiments, the block denormalizer 714 may use bitwise arithmetic to adjust the value of the added padding bits. For example, the block denormalizer 714 may use bitwise arithmetic to adjust the value of the added padding bits to match the most significant bit of the compressed weight value. As a further example, the block denormalizer 714 may use bitwise arithmetic to adjust the value of the added sign extension bits to match the most significant bit of the compressed weight value, such as the sign bit. The restored weight values ​​may be grouped as a block 720 of restored weight data.

[0078] For example, the block denormalizer 714 may use parameters extracted from the global header 704 and frame header 706a to reconstruct the associated frame payload 702a. In this example, the block denormalizer 714 may extract from the frame header 706a a frame length (e.g., frame length 602 in Figure 6) configured to represent a group of two weight values, a normalization coefficient configured to represent eight padding bits, and an offset indicator (e.g., offset indicator 606 in Figure 6) configured to indicate that there is no offset which will be used to reconstruct the frame payload 702a. The block denormalizer 714 may apply the normalization coefficient to the two compressed weight values ​​of the frame payload 702a and add eight padding bits to each of the compressed weight values. In this example, the block denormalizer 714 may copy the value of the most significant bit of each compressed weight value and use the value of the most significant bit as a padding bit to reconstruct each respective compressed weight value. As a further example, the block denormalizer 714 may copy the value of the most significant bit of each compressed weight value, such as the sign bit, and use the value of the most significant bit as the sign extension bit to restore each respective compressed weight value. The restored weight values ​​with full bit width, including padding bits and significant bits, may be used as part of block 720 of the restored weight data.

[0079] In a further example, the block denormalizer 714 may use parameters extracted from the global header 704 and frame header 706b to reconstruct the associated frame payload 702b. In this example, the block denormalizer 714 may extract from the frame header 706b a frame length configured to represent a group of nine weight values, a normalization coefficient configured to represent eleven padding bits, and an offset indicator configured to indicate that there is no offset which will be used to reconstruct the frame payload 702b. The block denormalizer 714 may apply the normalization coefficient to the nine compressed weight values ​​of the frame payload 702b and add eleven padding bits to each of the compressed weight values. In this example, the block denormalizer 714 may copy the value of the most significant bit of each compressed weight value and use the value of the most significant bit as a padding bit to reconstruct each respective compressed weight value. As a further example, the block denormalizer 714 may copy the value of the most significant bit of each compressed weight value, such as the sign bit, and use the value of the most significant bit as the sign extension bit to restore each respective compressed weight value. The restored weight values ​​with full bit width, including padding bits and significant bits, may be used as part of block 720 of the restored weight data.

[0080] In a further example, the block denormalizer 714 may use parameters extracted from the global header 704 and frame header 706c to reconstruct the associated frame payload 702c. In this example, the block denormalizer 714 may extract from the frame header 706c a frame length configured to represent a group of weight values, a normalization coefficient configured to represent six padding bits, and an offset indicator configured to indicate that there is no offset which will be used to reconstruct the frame payload 702c. The block denormalizer 714 may apply the normalization coefficient to one of the compressed weight values ​​in the frame payload 702c and add six padding bits to each of the compressed weight values. In this example, the block denormalizer 714 may copy the value of the most significant bit of the compressed weight value and use the value of the most significant bit as the padding bit to reconstruct the compressed weight value. As a further example, the block denormalizer 714 may recover the compressed weight values ​​by copying the values ​​of the most significant bits, such as the sign bit, of the compressed weight values ​​and using the values ​​of the most significant bits as sign extension bits. The recovered weight values ​​with full bit width, including padding bits and significant bits, may be used as part of block 720 of the recovered weight data.

[0081] In a further example, the block denormalizer 714 may use parameters extracted from the global header 704 and frame header 706d to reconstruct the associated frame payload 702d. In this example, the block denormalizer 714 may extract from the frame header 706d a frame length configured to represent a group of three weight values, a normalization coefficient configured to represent 14 padding bits, and an offset indicator configured to indicate that there is no offset which will be used to reconstruct the frame payload 702d. The block denormalizer 714 may apply the normalization coefficient to the three compressed weight values ​​of the frame payload 702d and add 14 padding bits to each of the compressed weight values. In this example, the block denormalizer 714 may copy the value of the most significant bit of each compressed weight value and use the value of the most significant bit as a padding bit to reconstruct each respective compressed weight value. As a further example, the block denormalizer 714 may copy the value of the most significant bit of each compressed weight value, such as the sign bit, and use the value of the most significant bit as the sign extension bit to restore each respective compressed weight value. The restored weight values ​​with full bit width, including padding bits and significant bits, may be used as part of block 720 of the restored weight data.

[0082] In a further example, the block denormalizer 714 may use parameters extracted from the global header 704 and frame header 706e to reconstruct the weight values ​​of zero without the associated frame payload. In this example, the block denormalizer 714 may extract from the frame header 706e a frame length configured to represent a group of three weight values, a normalization coefficient configured to represent 16 padding bits, and an offset indicator configured to indicate that there is no offset which will be used to reconstruct the weight values ​​of zero. The block denormalizer 714 may apply the normalization coefficient to the three compressed weight values ​​for the weight values ​​of zero and add 16 bits for each of the compressed weight values ​​of zero. In this example, the block denormalizer 714 may add 16 bits of the zero value to reconstruct each respective compressed weight value of zero. The fully bit-width reconstructed weight values, with padding bits and significant bits, may be used as part of a block 720 of the reconstructed weight data.

[0083] In some embodiments, the weight values ​​of a weight dataset may be modified by offset values ​​(e.g., offsets 502, 504 in Figure 5) as part of compressing the weight values. In the case of such modified weight values, the frame header for the associated frame payload, having the compressed weight values ​​derived from the modified weight values, may include an offset indicator configured to indicate that the offsets will be used to restore the frame payload. Following the application of the frame header's normalization coefficient to the compressed weights of the frame payload, the block denormalizer 714 may modify the full bit-width weight values ​​of the frame payload by the offset values. The fully bit-width restored weight values, with padding bits and significant bits, may be used as part of the restored weight data block 720.

[0084] The example in Figure 7 is intended to be illustrative and does not limit the claims or the scope of this specification. The compressed weight data block 700 and the restored weight data block 720 may contain any number of weight values, the weight values ​​may be of any size and / or format. The processor of the restoration computing device that restores the compressed weight data 700 and generates the restored weight data 720 may do so using any number and combination of global headers, frame headers, and frame payloads. Similarly, the processor of the restoration computing device that restores the compressed weight data 700 and generates the restored weight data 720 may do so using any number, combinations, and value offsets, frame lengths, normalization coefficients, and compressed weight values.

[0085] Figure 8 shows a method 800 for compressing weight data according to one embodiment. Referring to Figures 1 to 8, the method 800 may be implemented in a computing device (e.g., computing device 100 in Figure 1), in hardware, in software running on a processor, or in a combination of a software-configured processor and dedicated hardware (e.g., processor 104, memory 106, 114, AI processor 124 in Figure 1, processor 202, processor memory 204, AI processor 206, memory 208 in Figure 2). To encompass alternative configurations made possible in various embodiments, the hardware implementing the method 800 is referred to herein as a “compression processing device.”

[0086] In block 802, the compression device may receive a weight dataset (for example, the weight dataset 300 in Figures 3A and 3B). In some embodiments, the weight dataset may be retrieved from memory by the compression device. In some embodiments, the compression device retrieving the weight dataset in block 802 may be a processor or an AI processor.

[0087] In block 804, the compression device may identify frames in the weight dataset (e.g., frames 302a, 302b, 302c, 302d, 302e, 304a, 304b, 304c, 304d in Figures 3A and 3B) that can be removed from the weight values ​​for lossless compression, based on selective search criteria. Frame identification may be implemented by a search algorithm. In some embodiments, the search algorithm may be configured to identify any frame. In some embodiments, the search algorithm may be a brute-force search algorithm. In some embodiments, the search algorithm may be configured to identify frames based on selective search criteria that may limit the number of frames. In some embodiments, the search algorithm may be configured to identify frames based on selective search criteria that may prioritize certain frames and / or certain combinations of frames. For example, selective search criteria may include frame length, a range of frame lengths, a normalization coefficient representing the number of padding bits that can be removed from the frame's weight values, a range of normalization coefficients, weight values, a range of weight values, an offset value, a range of offset values, and so on. A neural network can use weights quantized to a certain size. For example, a neural network can use weights quantized to 4 bits, 8 bits, 16 bits, etc. However, not all weight values ​​may use the full quantization size. Therefore, a weight value may include significant bits, which may include a sign bit, and padding bits. In some embodiments, the padding bits may be repeating bit values, such as repeating the most significant bit of the weight value and / or a different bit value from the most significant bit. In some embodiments, the padding bits may be sign extension bits.

[0088] Analysis of the weight dataset can identify patterns in consecutive weight values ​​of padding bits that can be removed from the weight values ​​without loss. For example, a pattern of padding bits that can be removed from weight values ​​without loss may be a shared number of some and / or all of the padding bits in consecutive weight values. Patterns of padding bits that can be removed from weight values ​​without loss in consecutive weight values ​​may be identified based on the number of consecutive weight values. Consecutive weight values ​​exhibiting a pattern may be grouped as frames. In some embodiments, the identification of weight values ​​exhibiting a pattern identifies overlapping frames. In some embodiments, the compression processing device that identifies frames in the weight dataset in block 804 may be a processor or an AI processor.

[0089] In block 806, the compression device may select a combination of frames. Frame selection may be implemented by a search algorithm. In some embodiments, the search algorithm may be configured to select any combination of frames. In some embodiments, the search algorithm may be a brute-force search algorithm. In some embodiments, the search algorithm may be configured to select a combination of frames based on selective search criteria that may limit the number of frame combinations. In some embodiments, the search algorithm may be configured to select a combination of frames based on selective search criteria that may prioritize some frames and / or some combinations of frames. For example, selective search criteria may include frame length, frame length range, normalization coefficient, normalization coefficient range, weight value, weight value range, offset value, offset value range, etc. In some embodiments, frame selection may select non-overlapping frames. In some embodiments, the selected frames may be consecutive frames. In some embodiments, the selected frames may be discontinuous frames. In some embodiments, the compression device that selects a combination of frames in block 806 may be a processor or an AI processor.

[0090] In block 808, the compression device may compress a selected combination of frames. Compression of a selected combination of frames is further described herein in Method 900 with reference to Figure 9 and in Method 1000 with reference to Figure 10. In some embodiments, the compression device that compresses the selected frames in block 808 may be a processor or an AI processor.

[0091] In block 810, the compression processing device may calculate a compression metric for a selected combination of frames. The compression metric may be based on the compression ratio, compression size, etc. The compression processing device that calculates the compression metric for a selected combination of frames in block 810 may be a processor or an AI processor.

[0092] In decision block 812, the compression processing device may determine whether the compression metric satisfies and / or exceeds a compression metric threshold. The compression metric threshold may be based on the compression ratio, compression size, etc. In some embodiments, the compression metric threshold may be a predetermined value. Combinations of frames that satisfy and / or exceed the compression metric threshold may be used to compress the weight dataset. In some embodiments, the compression metric threshold may be set to the best compression metric from an analysis of the compression metrics of multiple sets of selected combinations of frames. For example, the compression metric of the first selected combination of frames may be set as the compression metric threshold, and the compression metric of any selected combination of frames that exceeds the compression metric threshold may be set as the compression metric threshold. When no selected combination of frames can exceed the compression metric threshold, the compression metric threshold may be the best compression metric. The combination of frames associated with the best compression metric may be the combination of frames that should be used to compress the weight dataset, as further described herein. In decision block 812, the compression processing device that determines whether the compression metric satisfies and / or exceeds the compression metric threshold may be a processor or an AI processor.

[0093] In response to a determination that the compression metric does not meet and / or exceeds the compression metric threshold (i.e., determination block 812 = "No"), the compression processing device may select a combination of frames in block 806, as previously described herein.

[0094] In response to a determination that the compression metric satisfies and / or exceeds the compression metric threshold (i.e., decision block 812 = "Yes"), the compression processing device may, in any decision block 814, determine whether there are any remaining combinations for the frames. As described above, the best compression metric can be obtained based on a comparison of each frame combination with the compression metric threshold. Therefore, when a frame combination has not yet been compared with the compression metric threshold, the remaining frame combinations can be compared with the compression metric threshold. In some embodiments, the compression processing device may determine whether there are any remaining combinations for the frames from a list, table, array, queue, stack, etc. of frame combinations. The compression processing device that determines whether there are any remaining combinations for the frames in any decision block 814 may be a processor or an AI processor.

[0095] In response to the determination that there are remaining combinations for frames (i.e., any decision block 814 = "Yes"), the compression device may select a combination of frames in block 806 as previously described herein.

[0096] In response to a decision that the compression metric satisfies and / or exceeds the compression metric threshold (i.e., decision block 812 = "Yes"), or in response to a decision that there are no remaining combinations for the frame (i.e., any decision block 814 = "No"), the compression processing device may store the compressed selected combination of frames in block 816. The compressed selected combination of frames may be stored in memory configured for persistent storage. The compression processing device that stores the compressed selected combination of frames in block 816 may be a processor or an AI processor.

[0097] Figure 9 shows a method 900 for setting weight data compression parameters according to one embodiment. Referring to Figures 1 to 9, method 900 may be implemented in a computing device (e.g., computing device 100 in Figure 1), in hardware, in software running on a processor, or in a combination of a software-configured processor and dedicated hardware (e.g., processor 104, memory 106, 114, AI processor 124 in Figure 1, processor 202, processor memory 204, AI processor 206, memory 208 in Figure 2). To encompass alternative configurations made possible in various embodiments, the hardware implementing method 900 is referred to herein as a “compression processing device”. In some embodiments, method 900 may be a further description of block 808 of method 800, which was described with reference to Figure 8.

[0098] In any block 902, the compression device may determine the frame length of the selected frames (e.g., frames 302a, 302b, 302c, 302d, 302e, 304a, 304b, 304c, 304d in Figures 3A and 3B). In some embodiments, the frame length may be based on a number of consecutive weight values ​​that represent a pattern of padding bits that can be removed from the weight values ​​without loss, used in identifying frames in a weight dataset (e.g., weight dataset 300 in Figures 3A and 3B), as described for block 804 in method 800 described with reference to Figure 8. In some embodiments, the frame length may be predetermined during the identification of frames in the weight dataset, as described for block 804 in method 800 described with reference to Figure 8, and it may not be necessary to perform any block 902. In any block 902, the compression device that determines the frame length of the selected frames may be a processor or an AI processor.

[0099] In any block 904, the compression device may remove an offset value from each weight value in the frame. The use of an offset value may be optional, and block 904 may not be implemented in embodiments where an offset value is not used. In some embodiments, an offset value may not be used for the compression of selected frames, as described for block 806 in method 800 described with reference to Figure 8. In some embodiments, an offset value may be used for the compression of selected frames having at least some weight values, where removing the offset value from the weight value can reduce the number of effective bits for that weight value. In some embodiments, the removal of an offset value from the weight value of a frame may depend on whether removing the offset value from the weight value results in a modified weight value with fewer effective bits or more padding bits than the weight value of the frame with the most effective bits or the fewest padding bits. In some embodiments, the offset value may be predetermined. In some embodiments, the offset value may be determined based on a value that can reduce the maximum number of effective bits for the frame weight or increase the minimum number of padding bits. In some embodiments, the offset value may be removed by adding or subtracting the offset value and weight value in the frame. In any block 904, the compression device that removes the offset value from each weight value in the frame may be a processor or an AI processor.

[0100] In block 906, the compression device may identify the weight value with the most significant bits or the fewest padding bits in the frame. The compression device may analyze the bits of the weight value, including the signed value, to determine the significant bits representing the weight value's value and compare which weight value has the most significant bits. Similarly, the compression device may analyze the bits of the weight value, to determine the padding bits representing the weight value's value and compare which weight value has the fewest padding bits. In block 906, the compression device that identifies the weight value with the most significant bits or the fewest padding bits in the frame may be a processor or an AI processor.

[0101] In block 908, the compression device may set a normalization factor for a frame using the weight value with the most significant bits or the fewest padding bits in the frame. The normalization factor may represent the number of padding bits that can be removed from the weight value of each frame. Setting the normalization factor may involve determining the number of padding bits for the weight value with the most significant bits or the fewest padding bits in the frame, and setting the normalization factor to the number of padding bits. In some embodiments, multiple weight values ​​may have the most significant bits or the fewest padding bits, and the compression device may select any of those weight values ​​to set the normalization factor. In some embodiments, the compression device may determine whether all the bits of a frame are zero. In other words, the compression device may determine whether all the weight values ​​in a frame are zero. The compression device may analyze each bit of the weight values ​​in a frame to determine their values ​​and determine whether each bit has a zero value. In response to the determination that all the bits of a frame are zero, the compression device may set the normalization factor to the full bit width of the weight value of the frame. In other words, a compression device may treat all the bits of a weight value as padding bits and set a normalization coefficient for all the bits of a weight value. In block 908, a compression device that sets a normalization coefficient for a frame using the weight value with the most active bits or the fewest padding bits in the frame could be a processor or an AI processor.

[0102] Figure 10 shows a method 1000 for compressing weight data according to one embodiment. Referring to Figures 1 to 10, method 1000 can be implemented in a computing device (e.g., computing device 100 in Figure 1), in hardware, in software running on a processor, or in a combination of a software-configured processor and dedicated hardware (e.g., processor 104, memory 106, 114, AI processor 124 in Figure 1, processor 202, processor memory 204, AI processor 206, memory 208 in Figure 2). To encompass alternative configurations made possible in various embodiments, the hardware implementing method 1000 is referred to herein as a “compression processing device”. In some embodiments, method 1000 can be implemented as a continuation of method 900 described herein with reference to Figure 9.

[0103] In any decision block 1002, the compression device may determine whether the normalization coefficient for a frame (e.g., frames 302a, 302b, 302c, 302d, 302e, 304a, 304b, 304c, 304d in Figures 3A and 3B) in the weight dataset (e.g., weight dataset 300 in Figures 3A and 3B) is the full bit width of the frame's weight values. The normalization coefficient for a frame may be the normalization coefficient set for the frame in block 908 of method 900 described herein with reference to Figure 9. In some embodiments, the compression device may determine the bit width of the frame's weight values ​​by analyzing the frame's weight values ​​and determining the bit width of the weight values. In some embodiments, the bit width of the frame's weight values ​​may be a pre-configured value. For example, the bit width of the frame's weight values ​​may be a power of 2, such as 2 bits, 4 bits, 8 bits, 16 bits, 32 bits, 64 bits, 128 bits, etc. In any decision block 1002, the compression processing device that determines whether the normalization coefficient for a frame is the full bit width of the frame's weight values ​​may be a processor or an AI processor.

[0104] In block 1004, the compression device may remove padding bits from the frame weight values ​​according to a normalization coefficient. The normalization coefficient may be configured to represent the number of padding bits to be removed from the frame weight values. The number of padding bits to be removed from the frame, as represented by the normalization coefficient, may be read by the compression device, and the compression device may remove that number of padding bits from each of the frame weight values. In some embodiments, removing padding bits may involve shifting the remaining bits of the frame weight value to override the removed padding bits. Following the removal of padding bits, the remaining bits of the weight value may be called the compressed weight bit width. In block 1004, the compression device that removes padding bits from the frame weight values ​​according to a normalization coefficient may be a processor or an AI processor. In some embodiments, in response to a determination that the normalization coefficient for a frame is not the full bit width of the frame weight value (i.e., any determination block 1002 = "No"), the compression device may remove padding bits from the frame weight values ​​according to a normalization coefficient in block 1004.

[0105] In block 1006, the compression device may generate frame payloads for frames (e.g., frame payloads 406a, 406b, 406c in Figure 4, and frame payloads 702a, 702b, 702c, 702d in Figure 7) and frame headers (e.g., frame headers 404a, 404b, 404c in Figure 4, frame header 600 in Figure 6, and frame headers 706a, 706b, 706c, 706d in Figure 7). Generating frame payloads for frames may include storing the remaining bits of the frame's weight value. The stored remaining bits of the frame's weight value may be the compressed weight value of the frame. Generating frame headers for frames may include storing compression parameters for the frame payload, in relation to that frame payload, which can be used to restore the compressed weight value. The compression parameters may include the frame length (e.g., frame length 602 in Figure 6), a normalization factor indicator (e.g., normalization factor 604 in Figure 6, the compressed weight bit width as described with reference to Figure 6), and / or an offset indicator (e.g., offset indicator 606 in Figure 6). The frame length may be the frame length determined for the frame in block 902 of Method 900, as described herein with reference to Figure 9. The normalization factor indicator may be the normalization factor set for the frame in block 908 of Method 900, as described herein with reference to Figure 9. The normalization factor indicator may be the compressed weight bit width resulting from the removal of padding bits in block 1004. The offset indicator may be a value based on whether and / or which offset factor is applied to the weight values ​​of the frame in any block 904 of Method 900, as described herein with reference to Figure 9. The frame header and frame payload may be stored in memory in relation to each other. In block 1006, the compression processing device that generates the frame payload and frame header for the frame may be a processor or an AI processor.

[0106] In response to the determination that the normalization coefficient for a frame is the full bit width of the frame's weight values ​​(i.e., any decision block 1002 = "Yes"), the compression device may, in any block 1012, remove all bits from the weight values ​​in the frame. The compression device that removes all bits from the weight values ​​in the frame in any block 1012 may be a processor or an AI processor.

[0107] In any block 1014, the compression device may generate frame headers for the frames (e.g., frame header 404d in Figure 4, frame header 600 in Figure 6, and frame header 706e in Figure 7). Generating frame headers in block 1014 may be done in a similar manner to generating frame headers in block 1006. For frames where all the bits for that purpose are removed from the weight values, no frame payload may be generated. In any block 1014, the compression device that generates frame headers for the frames may be a processor or an AI processor.

[0108] Following the generation of a frame payload and frame header for a frame in block 1006, or the generation of a frame header for a frame in any block 1014, the compression device may determine in decision block 1008 whether or not there are remaining frames. The compression device may determine whether or not there are remaining frames from a list, table, array, queue, stack, etc. The compression device that determines whether or not there are remaining frames in decision block 1008 may be a processor or an AI processor.

[0109] In response to the determination that there are no remaining frames (i.e., determination block 1008 = "No"), the compression processing device may generate a global header (e.g., global header 402 in Figure 4, global header 500 in Figure 5, global header 704 in Figure 7) and a block of compressed weight data (e.g., block 400 of compressed weight data in Figure 4, block 700 of compressed weight data in Figure 7) in block 1010. Generating a global header for the block of compressed weight data may include storing compression parameters and weight dataset parameters for the frame header and frame payload in relation to the frame header and frame payload. The compression parameters may include offsets (e.g., offsets 502, 504 in Figure 5). The offsets may be the values ​​of the offsets applied to the weight values ​​of the frame in block 904 of method 900 as described herein with reference to Figure 9. In some embodiments, the global header may include any number of reserved bits (e.g., reserved bit 506 in Figure 5) which may be configured to provide parameters for restoring the block of compressed weight data.

[0110] In some embodiments, the global header may include a compressed sign value (e.g., compressed sign value 508 in Figure 5) configured to indicate whether the frame payload of a block of compressed weight data contains signed compressed weight data. The compressed sign value may be generated during the compression of the weight dataset.

[0111] In some embodiments, the global header may include a frame header size (e.g., frame header size 510 in Figure 5) configured to represent the size of the frame header in a block of compressed weight data. The frame header size may be generated during the compression of the weight dataset. For example, the frame header size may be generated based on a uniform size for all frame headers in a block of compressed weight data. In some embodiments, the frame header size may be a pre-configured value.

[0112] In some embodiments, the global header may include an uncompressed width (e.g., uncompressed width 512 in Figure 5) configured to represent the bit width of an uncompressed weight dataset. For example, the uncompressed width may be configured to indicate that an uncompressed weight may have a bit width of a power of 2, such as 2 bits, 4 bits, 8 bits, 16 bits, 32 bits, 64 bits, or 128 bits. The uncompressed width may be generated during the compression of the weight dataset. For example, the uncompressed width may be generated based on the bit width of the weight values ​​in the weight dataset. In another example, the uncompressed width may be a pre-configured value. In some embodiments, the global header may include an uncompressed buffer size (e.g., uncompressed buffer size 514 in Figure 5) configured to represent the bit width of a buffer configured to store an uncompressed weight dataset. The uncompressed buffer size may be generated during the compression of the weight dataset. For example, the uncompressed buffer size may be generated based on the buffer size for the weight dataset. In another example, the uncompressed buffer size may be a pre-configured value.

[0113] Generating a block of compressed weight data may involve storing the global header, frame header, and frame payload in memory in relation to each other. In block 1010, the compression processing device that generates the global header and the block of compressed weight data may be a processor or an AI processor.

[0114] In response to the determination that there are remaining frames (i.e., determination block 1008 = "Yes"), the compression device may, in any block 902 of the method 900 described herein with reference to Figure 9, determine the frame length of the selected frame;, in any block 904 of the method 900 described herein with reference to Figure 9, remove the offset value from each weight value in the frame; or, in block 906 of the method 900 described herein with reference to Figure 9, identify the weight value with the most effective bits or the fewest padding bits in the frame.

[0115] Figure 11 shows a method 1100 for recovering weight data according to one embodiment. Referring to Figures 1 to 11, the method 1100 may be implemented in a computing device (e.g., computing device 100 in Figure 1), in hardware, in software running on a processor, or in a combination of a software-configured processor and dedicated hardware (e.g., processor 104, memory 106, 114, AI processor 124 in Figure 1, processor 202, processor memory 204, AI processor 206, memory 208 in Figure 2, recoverer 710, header parser 712, block denormalizer 714 in Figure 7). To encompass alternative configurations made possible in various embodiments, the hardware implementing the method 1100 is referred to herein as a “recovery processing device”.

[0116] In block 1102, the decompression device may retrieve a block of compressed weight data (for example, block 400 of compressed weight data in Figure 4, block 700 of compressed weight data in Figure 7). The block of compressed weight data may be retrieved from the memory of the computing device, such as the processor memory on the SoC (for example, SoC102 in Figure 1, SoC200 in Figure 2), and / or memory separate from the SoC. In block 1102, the decompression device that retrieves the block of compressed weight data may be a processor or an AI processor.

[0117] In block 1104, the decompression device may parse the global header of the compressed weight data block (e.g., global header 402 in Figure 4, global header 500 in Figure 5, and global header 704 in Figure 7). The global header may be parsed to extract parameters for decompressing the compressed weight data block. In some embodiments, the parameters for decompressing the compressed weight data block extracted from the global header may include offsets (e.g., offsets 502, 504 in Figure 5). In some embodiments, the parameters for decompressing the compressed weight data block extracted from the global header may include compressed sign values ​​(e.g., compressed sign value 508 in Figure 5) configured to indicate whether the frame payload of the compressed weight data block (e.g., frame payloads 406a, 406b, 406c in Figure 4, and frame payloads 702a, 702b, 702c, 702d in Figure 7) contains signed compressed weight data. In some embodiments, the parameters for reconstructing the compressed weight data blocks extracted from the global header may include a frame header size (e.g., frame header size 510 in Figure 5) configured to represent the size of the frame headers in the compressed weight data blocks (e.g., frame headers 404a, 404b, 404c, 404d in Figure 4, frame header 600 in Figure 6, and frame headers 706a, 706b, 706c, 706d, 706e in Figure 7). In some embodiments, the parameters for reconstructing the compressed weight data blocks extracted from the global header may include an uncompressed width (e.g., uncompressed width 512 in Figure 5) configured to represent the bit width of the uncompressed weight dataset. For example, the uncompressed width may be configured to represent that the uncompressed weights may have a bit width of a power of 2, such as 2 bits, 4 bits, 8 bits, 16 bits, 32 bits, 64 bits, 128 bits, etc.In some embodiments, the global header may include an uncompressed buffer size (for example, an uncompressed buffer size 514 in Figure 5) configured to represent the bit width of the buffer configured to store the uncompressed weight dataset. In block 1104, the decompression processing device that parses the global header of the block of compressed weight data may be a processor or an AI processor.

[0118] In block 1106, the decompression device may parse the frame header of a block of compressed weight data. The frame header may be parsed to extract parameters for decompressing the block of compressed weight data. Individual frame headers may be parsed to extract parameters for decompressing the associated frame payload. In some embodiments, individual frame headers may be parsed to extract parameters for decompressing weight values ​​of 0 without the associated frame payload. In some embodiments, the parameters for decompressing a block of compressed weight data extracted from the frame header may include the frame length (e.g., frame length 602 in Figure 6), a normalization factor (e.g., normalization factor 604 in Figure 6), and / or an offset indicator (e.g., offset indicator 606 in Figure 6). The frame length may be configured to represent the number of weight values ​​contained in the associated frame of the weight dataset. In some embodiments, the frame length 602 may similarly be configured to represent the number of compressed weight values ​​contained in the associated frame payload of the block of compressed weight data. In some embodiments, the frame length 602 may similarly be configured to represent the number of compressed weight values ​​without the associated frame payload of the compressed weight data block. The normalization coefficient may be configured to represent the number of padding bits to be added to the compressed weight values, which were included in the associated frame payload. In some embodiments, the normalization coefficient may be configured to represent the number of bits to be added for the compressed weight values, which were included in the associated frame payload. The offset indicator 606 may be configured to indicate whether and / or which offset will be applied to restore the compressed weight values, which were included in the associated frame payload. In block 1106, the restoration processing device that parses the frame header of the compressed weight data block may be a processor or an AI processor.

[0119] In decision block 1108, the decompression device may determine whether the frame payload of the compressed weight data block is signed. The compressed weight data of the frame payload may include signed and / or unsigned weight values. Having at least one signed weight value may make the frame payload signed. In some embodiments, having at least one signed weight value in the compressed weight data may make all frame payloads signed. Determining whether the frame payload is signed may be based on the compressed signed value extracted from the global header in block 1104. In decision block 1108, the decompression device that determines whether the frame payload of the compressed weight data block is signed may be a processor or an AI processor.

[0120] In response to a determination that the frame payload of a block of compressed weight data is unsigned (i.e., determination block 1108 = "No"), the decompression device may, in block 1110, add padding bits to the bits in the frame payload according to a normalization coefficient. The normalization coefficient may be used to indicate to the decompression device how many padding bits to add to each compressed weight value bit in the frame payload so that the bit width of each restored weight value totals the bit width of the original uncompressed weight values ​​in the weight dataset. In some embodiments, the decompression device may add padding bits to the bits in the frame payload by shifting each of the compressed weight values ​​by the number of bits of the normalization coefficient. In some embodiments, the decompression device may use bitwise arithmetic to adjust the value of the added padding bits. For unsigned compressed weight data, which bit values ​​can be used for the padding bits may be preconfigured based on the most significant bit of each compressed weight value or based on the resource cost of the padding bit values. In block 1110, the restoration device that adds padding bits to the bits in the frame payload according to the normalization coefficient may be a processor or an AI processor.

[0121] In response to the determination that the frame payload of a block of compressed weight data is signed (i.e., determination block 1108 = "Yes"), the decompression device may, in block 1120, add padding bits to the bits in the frame payload according to a normalization coefficient and a sign bit. The normalization coefficient may be used in a similar manner as described in block 1110. In some embodiments, the decompression device may use bitwise arithmetic operations to adjust the value of the added padding bits. For signed compressed weight data, what bit value may be used for the padding bits may be based on the sign bit value of each signed compressed weight value. In some embodiments, the padding bits may be sign extension bits. The decompression device that adds padding bits to the bits in the frame payload according to a normalization coefficient and a sign bit in block 1120 may be a processor or an AI processor.

[0122] In block 1110, following the addition of padding bits to the bits in the frame payload according to a normalization coefficient, or in block 1120, following the addition of padding bits to the bits in the frame payload according to a normalization coefficient and a sign bit, the restoration device may determine in an optional decision block 1112 whether or not an offset indicator is set for the frame payload. Whether or not an offset indicator is set for the frame payload may be determined from the output of parsing the associated frame header of the frame payload in block 1106, and more specifically, parsing the offset indicator. The restoration device that determines in an optional decision block 1112 whether or not an offset indicator is set for the frame payload may be a processor or an AI processor.

[0123] In response to the decision that the offset indicator is set for the frame payload (i.e., any decision block 1112 = "Yes"), the restoration device may, in any block 1114, include an offset value for each restored weight value in the frame payload. The offset value may be determined from the output obtained by parsing the global header in block 1104 and the offsets. In some embodiments, the global header may include multiple offsets, and the offset indicator of the associated frame header may be configured to indicate to the restoration device which offset from the global header to use. The restoration device may include an offset value for each restored weight value in the frame payload. For example, the restoration device may add or subtract the offset value and each restored weight value in the frame payload. The restoration device that includes an offset value for each restored weight value in the frame payload in any block 1114 may be a processor or an AI processor.

[0124] In decision block 1116, the recovery device may determine whether there is a remaining frame payload. The recovery device may determine whether there is a remaining frame payload from a list, table, array, queue, stack, etc. of frame payloads. The recovery device that determines whether there is a remaining frame payload in decision block 1116 may be a processor or an AI processor. In some embodiments, the recovery device may determine whether there is a remaining frame payload in decision block 1116, following the addition of padding bits to the bits in the frame payload according to a normalization coefficient in block 1110, or the addition of padding bits to the bits in the frame payload according to a normalization coefficient and a sign bit in block 1120. In some embodiments, the recovery device may determine whether there is a remaining frame payload in decision block 1116, following the determination that no offset indicator is set for the frame payload (i.e., any decision block 1112 = "No"), or following the inclusion of an offset value for each recovered weight value in the frame payload in any block 1114.

[0125] In response to a decision that there is no remaining frame payload (i.e., decision block 1116 = "No"), the regeneration device may generate a block of regenerated weight data (for example, block 720 of regenerated weight data in Figure 7) in block 1118. The regeneration device may output and store in memory the regenerated weight values ​​resulting from adding padding bits in block 1110. In some embodiments, the regeneration device may output and store in memory the regenerated weight values ​​resulting from adding padding bits in block 1120. In some embodiments, the regeneration device may output and store in memory the regenerated weight values ​​resulting from including an offset in the regenerated weight values ​​in any block 1114. The regeneration device that generates the block of regenerated weight data in block 1118 may be a processor or an AI processor.

[0126] In response to the determination that there is a remaining frame payload (i.e., decision block 1116 = "Yes"), the decompression device may, in block 1106, parse the frame header of the compressed weight data block.

[0127] Figure 12 shows a method 1200 for compressing weight data according to one embodiment. Referring to Figures 1 to 12, the method 1200 may be implemented in a computing device (e.g., computing device 100 in Figure 1), in hardware, in software running on a processor, or in a combination of a software-configured processor and dedicated hardware (e.g., processor 104, memory 106, 114, AI processor 124 in Figure 1, processor 202, processor memory 204, AI processor 206, memory 208 in Figure 2). To encompass alternative configurations made possible in various embodiments, the hardware implementing the method 1200 is referred to herein as a “compression processing device”.

[0128] In block 1202, the compression device may identify frames (e.g., frames 302a, 302b, 302c, 302d, 302e, 304a, 304b, 304c, 304d in Figures 3A and 3B) in the weight dataset (e.g., weight dataset 300 in Figures 3A and 3B) based on a selective search criterion and a pattern of padding bits in the weight values ​​of the weight dataset that can be removed from the weight values ​​for lossless compression. Block 1202 may be implemented in a manner similar to the operation in block 804 of method 800 as described with reference to Figure 8.

[0129] In block 1204, the compression device may select a combination of frames. Block 1204 may be carried out in a manner similar to the operation in block 806 of method 800, as described with reference to Figure 8.

[0130] In block 1206, the compression device may remove padding bits from the weight values ​​within a frame of a selected combination of frames according to a normalization coefficient for each frame (e.g., normalization coefficient 604 in Figure 6) in order to generate a frame payload for each of the selected frames (e.g., frame payloads 406a, 406b, 406c in Figure 4, and frame payloads 702a, 702b, 702c, 702d in Figure 7). The normalization coefficient may each represent the number of padding bits that can be removed from the weight value of each frame. Block 1206 may be performed in a manner similar to the operation in block 808 of method 800 as described with reference to Figure 8, and / or in block 1004 and / or any block 1012 of method 1000 as described with reference to Figure 10.

[0131] In block 1208, the compression device may determine whether the compression metric of the frame payload exceeds the compression metric threshold. Block 1208 may be performed in a manner similar to the operation in determination block 812 of method 800, as described with reference to Figure 8.

[0132] In block 1210, the compression processing device may, in response to a determination that the compression metric of the frame payload exceeds a compression metric threshold, generate a block of compressed weight data containing the frame payload (for example, block 400 of compressed weight data in Figure 4). Block 1210 may be carried out in a manner similar to the operation in block 1010 of method 1000 as described with reference to Figure 10.

[0133] Figure 13 shows a method 1300 for recovering weight data according to one embodiment. Referring to Figures 1 to 13, the method 1300 may be implemented in a computing device (e.g., computing device 100 in Figure 1), in hardware, in software running on a processor, or in a combination of a software-configured processor and dedicated hardware (e.g., processor 104, memory 106, 114, AI processor 124 in Figure 1, processor 202, processor memory 204, AI processor 206, memory 208 in Figure 2, recoverer 710, header parser 712, block denormalizer 714 in Figure 7). To encompass alternative configurations made possible in various embodiments, the hardware implementing the method 1300 is referred to herein as a “recovery processing device”.

[0134] In block 1302, the decompression device may extract a block of compressed weight data (e.g., block 400 of compressed weight data in Figure 4, block 700 of compressed weight data in Figure 7). The block of compressed weight data may include frame headers associated with the frame payload (e.g., frame payloads 406a, 406b, 406c in Figure 4, and frame payloads 702a, 702b, 702c, 702d in Figure 7) (e.g., frame headers 404a, 404b, 404c, 404d in Figure 4, frame header 600 in Figure 6, and frame headers 706a, 706b, 706c, 706d, 706e in Figure 7). The frame header may include a normalization factor (e.g., normalization factor 604 in Figure 6) representing the number of padding bits removed from the weight values ​​that generate the associated frame payload. The frame payload contains the compressed weight values. Block 1302 may be carried out in a manner similar to the operation in block 1102 of method 1100 as described with reference to Figure 11.

[0135] In block 1304, the recovery device may parse the frame header for normalization coefficients. Block 1304 may be performed in a manner similar to the operation in block 1106 of method 1100, as described with reference to Figure 11.

[0136] In block 1306, the recovery device may add padding bits to the compressed weight values ​​of the frame payload according to the normalization coefficient of the associated frame header in order to generate the recovered weight values. Block 1306 may be carried out in a manner similar to the operation in blocks 1110 and / or 1120 of method 1100 as described with reference to Figure 11.

[0137] Weighted data compression and / or decompression systems in various embodiments (including, but not limited to, the embodiments described above with reference to Figures 1 to 13) may be implemented in a wide variety of computing systems, including mobile computing devices, and an example of a mobile computing device suitable for use with various embodiments is shown in Figure 14. The mobile computing device 1400 may include a processor 1402 coupled to a touchscreen controller 1404 and internal memory 1406. The processor 1402 may be one or more multi-core integrated circuits designated for general-purpose or specific processing tasks. The internal memory 1406 may be volatile memory or non-volatile memory, and may also be secure memory and / or encrypted memory or non-secure memory and / or unencrypted memory, or any combination thereof. Examples of memory types that may be utilized include, but are not limited to, DDR, LPDDR, GDDR, WIDEIO, RAM, SRAM, DRAM, P-RAM, R-RAM, M-RAM, STT-RAM, and embedded DRAM. The touchscreen controller 1404 and processor 1402 may also be coupled to the touchscreen panel 1412, such as a resistive touchscreen, a capacitive touchscreen, or an infrared touchscreen. Additionally, the display of the mobile computing device 1400 does not need to have touchscreen functionality.

[0138] The mobile computing device 1400 may have one or more radio signal transceivers 1408 (e.g., Peanut, Bluetooth, ZigBee, Wi-Fi, RF radio) for sending and receiving communications, coupled to each other and / or to the processor 1402, and an antenna 1410. The transceivers 1408 and antenna 1410 may be used together with the aforementioned circuitry to implement various wireless transmission protocol stacks and interfaces. The mobile computing device 1400 may include a cellular network wireless modem chip 1416 that enables communication over a cellular network and is coupled to the processor.

[0139] The mobile computing device 1400 may include a peripheral device connection interface 1418 coupled to the processor 1402. The peripheral device connection interface 1418 may be configured to accept one type of connection on its own, or it may be configured to accept various types of common or proprietary physical and communication connections, such as Universal Serial Bus (USB), FireWire, Thunderbolt, or PCIe. The peripheral device connection interface 1418 may also be coupled to a similarly configured peripheral device connection port (not shown).

[0140] The mobile computing device 1400 may also include a speaker 1414 for providing audio output. The mobile computing device 1400 may also include a housing made of plastic, metal, or a combination of materials for housing all or some of the components described herein. The mobile computing device 1400 may include a power supply 1422 coupled to the processor 1402, such as a disposable battery or a rechargeable battery. The rechargeable battery may also be coupled to a peripheral device connection port to receive charging current from an external source to the mobile computing device 1400. The mobile computing device 1400 may also include a physical button 1424 for receiving user input. The mobile computing device 1400 may also include a power button 1426 for turning the mobile computing device 1400 on and off.

[0141] Weight data compression and / or decompression systems according to various embodiments (including, but not limited to, the embodiments described above with reference to Figures 1 to 13) may be implemented in a wide variety of computing systems, including a laptop computer 1500, one example of which is shown in Figure 15. Many laptop computers include a touch surface 1517 of a touchpad that acts as the computer's pointing device and can therefore receive drag, scroll, and flick gestures similar to those implemented on the aforementioned computing devices equipped with touchscreen displays. The laptop computer 1500 typically includes a processor 1502 coupled with volatile memory 1512 and large-capacity non-volatile memory such as a disk drive 1513 of flash memory. In addition, the computer 1500 may have one or more antennas 1508 for sending and receiving electromagnetic radiation, which may be connected to a wireless data link and / or cellular telephone transceiver 1516 coupled to the processor 1502. The computer 1500 may also include a floppy disk drive 1514 and a compact disk (CD) drive 1515 coupled to the processor 1502. In a notebook configuration, the computer housing includes a touchpad 1517, a keyboard 1518, and a display 1519, all coupled to the processor 1502. Other configurations of the computing device, as is well known, may include a computer mouse or trackball coupled to the processor (for example, via a USB input), which may also be used in various embodiments.

[0142] Weighted data compression and / or decompression systems in various embodiments (including, but not limited to, the embodiments described above with reference to Figures 1 to 13) can also be implemented in a fixed computing system, such as one of a variety of commercially available servers. An exemplary server 1600 is shown in Figure 16. Such a server 1600 typically includes one or more multicore processor assemblies 1601 coupled with volatile memory 1602 and large-capacity non-volatile memory such as a disk drive 1604. As shown in Figure 16, the multicore processor assemblies 1601 can be added to the server 1600 by inserting them into a rack of assemblies. The server 1600 may also include a floppy disk drive, compact disk (CD), or digital versatile disk (DVD) disk drive 1606 coupled to the processor 1601. Server 1600 may also include a network access port 1603 coupled to a multicore processor assembly 1601 for establishing network interface connections with network 1605, such as local area networks, the Internet, public switched telephone networks, and / or cellular data networks (e.g., CDMA, TDMA, GSM, PCS, 3G, 4G, LTE, 5G, or any other type of cellular data network) coupled to other broadcast system computers and servers.

[0143] Implementation examples are described in the following paragraphs. While some of the following implementation examples are described in terms of exemplary methods, further exemplary implementations may include exemplary methods described in the following paragraphs, implemented by a computing device including a compression device configured to perform the operations of the exemplary method; exemplary methods described in the following paragraphs, implemented by a computing device including a decompression device configured to perform the operations of the exemplary method; exemplary methods described in the following paragraphs, implemented by a computing device including means for performing the functions of the exemplary method; and exemplary methods described in the following paragraphs, implemented as a non-temporary processor-readable storage medium storing processor-executable instructions configured to cause the processor of a computing device to perform the operations of the exemplary method.

[0144] Example 1. A method to be performed on the processor of a computing device, comprising: receiving a binary weight dataset representing weight values; generating a first frame payload containing a compressed first frame of a first subset of weight values ​​in the weight dataset; generating a first frame header associated with the first frame payload, wherein the first frame header contains a normalization coefficient indicator for the compressed first frame; and generating a block of compressed weight data having the first frame payload.

[0145] Example 2. The method of Example 1, wherein the step of generating a first frame payload includes the step of compressing a first frame of a first subset of weight values ​​in a weight dataset by removing padding bits from each weight value in the first subset of weight values ​​according to a normalization coefficient for the first frame, in order to generate a compressed first frame.

[0146] Example 3. The method of Example 2, further comprising the step of removing an offset value from each weight value of a first subset of weight values ​​of a first frame to generate modified weight values ​​of a first frame, and the step of compressing a first frame of the first subset of weight values ​​comprising the step of removing a padding bit from the modified weight values ​​in the first frame according to a normalization coefficient for the first frame to generate a compressed first frame.

[0147] Example 4. The method of Example 3, further comprising the step of generating a global header for a block of compressed weight data, wherein the global header includes an offset value, and the step of generating a first frame header associated with a first frame payload, wherein the first frame header includes an offset indicator configured to indicate whether the offset value is removed from a first subset of the weight values ​​of the first frame.

[0148] Example 5. The methods of Examples 2-4, where the padding bits are sign extension bits.

[0149] Example 6. Based on a selective search criterion, the steps include identifying a first frame of a first subset of weight values ​​in a weight dataset based on a pattern of padding bits in the weight values ​​that can be removed from the weight values ​​for lossless compression; setting a first normalization coefficient for the first frame that represents the number of padding bits to be removed from each weight value in the first subset of weight values; and identifying a second frame of a second subset of weight values ​​in a weight dataset based on a selective search criterion, based on a pattern of padding bits in the weight values ​​that can be removed from the weight values ​​for lossless compression. A method from Examples 1 to 5, further comprising: determining whether all bits of a second frame are zero; setting a second normalization coefficient for the second frame to represent all bits of a second subset of the weight values ​​of the second frame in response to the determination that all bits of the second frame are zero; compressing the second frame of the second subset of weight values ​​by removing all bits from the second subset of weight values ​​according to the second normalization coefficient for the second frame; and generating a second frame header not associated with a frame payload.

[0150] Example 7. Any method from Examples 1 to 6, further comprising: identifying a first frame of a first subset of weight values ​​in a weight dataset based on a selective search criterion, based on a pattern of padding bits in the weight values ​​that can be removed from the weight values ​​for lossless compression; identifying a weight value in the first subset of weight values ​​of the first frame having the highest number of effective bits; and setting a normalization coefficient for the first frame that represents the number of padding bits to be removed from each weight value in the first subset of weight values, based on the highest number of effective bits.

[0151] Example 8. Any method from Examples 1 to 7, further comprising the step of determining whether a first compression metric of a first frame payload exceeds a compression metric threshold, and the step of generating a block of compressed weight data having the first frame payload, in response to the determination that the first compression metric of the first frame payload exceeds a compression metric threshold.

[0152] Example 9. The method of Example 8, further comprising the steps of: setting a compression metric threshold to a first compression metric of a first frame payload; generating a second frame payload containing a second compressed frame of a second subset of weight values ​​in a weight dataset; determining whether the second compression metric of the second frame payload exceeds a compression metric threshold; setting the compression metric threshold to a second compression metric of the second frame payload in response to the determination that the second compression metric exceeds a compression metric threshold; generating a third frame payload containing a third compressed frame of a third subset of weight values ​​in a weight dataset; and determining whether the third compression metric of the third frame payload exceeds a compression metric threshold, wherein the method further comprises the steps of: generating a block of compressed weight data having the first frame payload in response to the determination that the first compression metric of the first frame payload exceeds a compression metric threshold; and generating a block of compressed weight data having the second frame payload in response to the determination that the third compression metric of the third frame payload does not exceed a compression metric threshold.

[0153] Example 10. A method performed in the processor of a computing device, comprising the steps of: extracting a block of compressed weight data, wherein the block of compressed weight data includes a first frame header associated with a first frame payload, the first frame header includes a first normalization coefficient indicator, and the first frame payload includes compressed weight values; and generating a first restored frame, which includes restored weight values ​​of the compressed weight values ​​of the first frame payload.

[0154] Example 11. The method of Example 10, wherein a block of compressed weight data includes an offset indicator, the global header having an offset value, and the first frame header is configured to indicate whether or not the offset value will be included for each restored weight value generated from the first frame payload, the method further comprising the steps of parsing the global header for the offset value, parsing the first frame header for the offset indicator, and determining whether or not the offset indicator is set in the first frame header, and the step of generating a first restored frame comprising the step of including the offset value in each restored weight value generated from the first frame payload associated with the first frame header, in response to the determination that the offset indicator is set in the first frame header.

[0155] Example 12. A method from either Example 10 or Example 11, wherein a block of compressed weight data includes a second frame header not associated with the frame payload, which includes a second normalization coefficient indicator, and the method further includes the step of generating a second restored frame, which includes restored weight values ​​having all zero bits according to the second normalization coefficient indicator of the second frame header.

[0156] Example 13. Any method from Examples 10 to 12, wherein a block of compressed weight data includes a second frame header associated with a second frame payload, the second frame header includes a second normalization coefficient indicator, the second frame payload includes compressed weight values, and the method further includes the step of generating a second restored frame, which includes restored weight values ​​of the compressed weight values ​​of the second frame payload by adding padding bits to the compressed weight values ​​of the second frame payload according to the second normalization coefficient indicator of the second frame header.

[0157] Example 14. Any method from Examples 10 to 13, wherein the step of generating a first restored frame includes adding padding bits to the compressed weight values ​​of the first frame payload according to a first normalization coefficient indicator of the first frame header in order to generate restored weight values, the value of the padding bits for the first compressed weight values ​​of the compressed weight values ​​of the first frame payload is determined from the most significant bit of the first compressed weight values.

[0158] Example 15. Either the method in Example 13 or Example 14, where the padding bit is a sign extension bit.

[0159] Example 16. Any method from Examples 10 to 15, wherein the first frame header includes a frame length configured to indicate the number of compressed weight values ​​of the first frame payload.

[0160] Example 17. Any method from Examples 10 to 16, wherein the compressed weight data block includes a second frame header not associated with the frame payload, which includes a frame length configured to indicate the number of compressed weight values, and the method further includes the step of generating a second restored frame which includes several consecutive restored weight values ​​having all zero bits corresponding to the frame length of the second frame header.

[0161] Computer program code for execution on a programmable processor to perform the operations of various embodiments may be written in high-level programming languages ​​such as C, C++, C#, Smalltalk, Java, JavaScript, Visual Basic, structured query languages ​​(e.g., Transact-SQL), Perl, or various other programming languages. As used in this application, program code or programs stored on a computer-readable storage medium may refer to machine language code (such as object code) whose format is understandable by a processor.

[0162] The above description of the method and process flow diagram are provided merely as illustrative examples and do not require or imply that the operations of the various embodiments must be performed in the order presented. As will be understood by those skilled in the art, the order of operations in the above embodiments may be performed in any order. Words such as “then,” “next,” and “then” do not limit the order of operations, and these words are simply used to guide the reader through the description of the method. Furthermore, any reference to a claim element in the singular form, for example using the articles “a,” “an,” or “the,” should not be interpreted as limiting that element to the singular form.

[0163] The various exemplary logic blocks, modules, circuits, and algorithmic operations described in relation to various embodiments may be implemented as electronic hardware, computer software, or a combination of both. To clearly demonstrate this hardware- and software compatibility, the various exemplary components, blocks, modules, circuits, and operations have generally been described above in terms of their functions. Whether such functions are implemented as hardware or software depends on the specific application and the design constraints imposed on the overall system. A person skilled in the art may implement the described functions in various ways for each specific application, but such implementation decisions should not be construed as causing a departure from the claims.

[0164] The hardware used to implement the various exemplary logics, logic blocks, modules, and circuits described in relation to the embodiments disclosed herein may be implemented or run using general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs) or other programmable logic devices, discrete gate logic or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A general-purpose processor may be a microprocessor, but alternatively, the processor may be any conventional processor, controller, microcontroller, or state machine. The processor may also be implemented as a combination of computing devices, for example, a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors working with a DSP core, or any other such configuration. Alternatively, some operations or methods may be performed by circuits specific to a given function.

[0165] In one or more embodiments, the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored as one or more instructions or codes on a non-temporary computer-readable medium or a non-temporary processor-readable medium. The operation of the methods or algorithms disclosed herein may be embodied in a processor-executable software module that may reside on a non-temporary computer-readable storage medium or a non-temporary processor-readable storage medium. The non-temporary computer-readable storage medium or a non-temporary processor-readable storage medium may be any storage medium accessible by a computer or processor. Such a non-temporary computer-readable medium or a non-temporary processor-readable medium may include, but are not limited to, RAM, ROM, EEPROM, FLASH memory, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that may be used to store desired program code in the form of instructions or data structures and may be accessed by a computer. The terms "disk" and "disc" as used herein include compact discs (CDs), laser discs, optical discs, digital multipurpose discs (DVDs), floppy disks, and Blu-ray® discs, where a disk typically reproduces data magnetically, and a disc reproduces data optically using a laser. The above combinations also fall within the scope of non-temporary computer-readable media and non-temporary processor-readable media. Additionally, the operation of a method or algorithm may exist as one or any combination or set of code and / or instructions on non-temporary processor-readable media and / or non-temporary computer-readable media that can be incorporated into a computer program product.

[0166] The foregoing description of the embodiments disclosed is provided to enable any person skilled in the art to construct or use the claims. Various modifications to these embodiments will be readily apparent to a person skilled in the art, and the general principles defined herein may be applied to other embodiments and implementations without departing from the claims. Accordingly, this disclosure is not intended to be limited to the embodiments and implementations described herein, but should be given the broadest scope that is consistent with the following claims and the principles and novel features disclosed herein. [Explanation of symbols]

[0167] 100 Computing Devices 102, 200 SoCs 104, 202, 1402, 1502 processors 106, 114, 208 memory 108 Communication Interfaces 110 Memory Interface 112 Communication Components 116, 1410, 1508 antennas 120 Peripheral Device Interfaces 122 Peripheral Devices 124 Artificial Intelligence (AI) Processor, AI Processor 204 processor memory, memory 206 AI Processors 210 Low-power areas 300 weight dataset, compressed weight dataset 302a, 302b, 302c, 302d, 302e, 304a, 304b, 304c, 304d Frame 400 blocks of compressed weight data 402, 500, 704 global headers 404a, 404b, 404c, 404d, 600, 706a, 706b, 706c, 706d, 706e frame headers 406a, 406b, 406c, 702a, 702b, 702c, 702d Frame Payload 502, 504 offset 506 reserved bits 508 Compressed sign value 510 Frame Header Size 512 uncompressed width 514 Uncompressed buffer size 602 frame length 604 Normalization coefficient 606 Offset Indicator 700 compressed weight data blocks, compressed weight data 710 Restorer 720 Reconstructed weight data blocks, Reconstructed weight data 712 Header Parser 714 Block Denormalizer 1400 mobile computing devices 1404 Touchscreen Controller 1406 internal memory 1408 Wireless signal transceiver, transceiver 1412 Touchscreen Panel 1414 Speaker 1416 Cellular Network Wireless Modem Chip 1418 Peripheral device connection interface 1422 Power supply 1424 Physical Buttons 1426 Power button 1500 Laptop Computers, Computers 1512, 1602 Volatile memory 1513, 1604 Disk Drives 1514 Floppy disk drive 1515 Compact Disc (CD) Drive 1516 Wireless Data Link and / or Cellular Telephone Transceiver 1517 Touch surface of the touchpad, touchpad 1518 keyboard 1519 Display 1600 Servers 1601 Multicore Processor Assembly, Processor 1603 Network Access Port 1605 Network 1606 Floppy disk drive, compact disc (CD) or digital multipurpose disc (DVD) drive

Claims

1. A method performed on the processor of a computing device, The steps include receiving a binary weight dataset representing weight values ​​for a neural network, A step of generating a first frame payload comprising a compressed first frame of a first subset of the weight values ​​in the weight dataset, A step of generating a first frame header associated with the first frame payload, wherein the first frame header includes a normalization factor indicator for the compressed first frame, the normalization factor indicator indicates the number of padding bits removed from the weight values, and the first frame header includes a frame length configured to represent the number of weight values ​​contained in the first frame payload. The steps of generating a block of compressed weight data having the first frame payload and A method that includes this.

2. The method according to claim 1, wherein the step of generating the first frame payload includes compressing the first frame of the first subset of weight values ​​in the weight dataset by removing padding bits from each weight value of the first subset of weight values ​​according to a normalization coefficient for the first frame, in order to generate the compressed first frame.

3. The method further includes the step of removing an offset value from each weight value of the first subset of the weight values ​​of the first frame that generates the modified weight values ​​of the first frame, The method according to claim 2, wherein the step of compressing the first frame of the first subset of the weight values ​​includes the step of removing the padding bits from the modified weight values ​​in the first frame according to the normalization coefficient for the first frame in order to produce the compressed first frame.

4. The method further includes the step of generating a global header for the block of compressed weight data, wherein the global header includes the offset value, The method according to claim 3, wherein the step of generating the first frame header associated with the first frame payload includes a step of generating the first frame header, wherein the first frame header includes an offset indicator configured to indicate whether the offset value is removed from the first subset of the weight values ​​of the first frame.

5. Steps include identifying a first frame of the first subset of the weight values ​​in the weight dataset based on a selective search criterion, based on a pattern of padding bits in the weight values ​​that can be removed from the weight values ​​for lossless compression, The steps include setting a first normalization coefficient for the first frame, which represents the number of padding bits to be removed from each weight value in the first subset of the weight values, Steps include identifying a second frame of a second subset of the weight values ​​in the weight dataset based on a selective search criterion, based on a pattern of padding bits in the weight values ​​that can be removed from the weight values ​​for lossless compression, The steps include determining whether all bits of the second frame are zero, In response to the determination that all of the bits in the second frame are 0, The steps of setting a second normalization coefficient for the second frame in order to represent all of the bits of the second subset of the weight values ​​of the second frame, The steps of compressing the second frame of the second subset of weight values ​​by removing all of the bits from the second subset of weight values ​​according to the second normalization coefficient for the second frame, The steps include generating a second frame header that is not associated with the frame payload, and The method according to claim 1, further comprising:

6. Steps include identifying a first frame of the first subset of the weight values ​​in the weight dataset based on a selective search criterion, based on a pattern of padding bits in the weight values ​​that can be removed from the weight values ​​for lossless compression, Steps include identifying weight values ​​from a first subset of the weight values ​​of the first frame having the highest number of valid bits, The steps of: setting a normalization coefficient for the first frame, which represents the number of padding bits to be removed from each weight value in the first subset of the weight values, based on the maximum number of effective bits; The method according to claim 1, further comprising:

7. The method according to claim 1, further comprising the step of determining whether a first compression metric of the first frame payload exceeds a compression metric threshold, wherein the step of generating the block of compressed weight data having the first frame payload includes the step of generating the block of compressed weight data having the first frame payload in response to the determination that the first compression metric of the first frame payload exceeds the compression metric threshold.

8. The method described above is The steps include setting the compression metric threshold to the first compression metric of the first frame payload, The steps include generating a second frame payload comprising a compressed second frame of a second subset of the weight values ​​in the weight dataset, A step of determining whether the second compression metric of the second frame payload exceeds the compression metric threshold, Steps of setting the compression metric threshold to the second compression metric of the second frame payload in response to a determination that the second compression metric exceeds the compression metric threshold, The steps include generating a third frame payload comprising a compressed third frame of a third subset of the weight values ​​in the weight dataset, The steps include determining whether the third compression metric of the third frame payload exceeds the compression metric threshold, and It further includes, The method according to claim 7, wherein the step of generating the block of compressed weight data having the first frame payload in response to a determination that the first compression metric of the first frame payload exceeds the compression metric threshold includes the step of generating the block of compressed weight data having the second frame payload in response to a determination that the third compression metric of the third frame payload does not exceed the compression metric threshold.

9. A computing device, A compression processing device configured to perform an operation, wherein the operation is Receiving a binary weight dataset representing weight values ​​for a neural network. To generate a first frame payload comprising a compressed first frame of a first subset of the weight values ​​in the weight dataset, Generating a first frame header associated with the first frame payload, wherein the first frame header includes a normalization factor indicator for the compressed first frame, the normalization factor indicator indicates the number of padding bits removed from the weight values, and the first frame header includes a frame length configured to represent the number of weight values ​​contained in the first frame payload, and To generate a block of compressed weight data having the first frame payload described above. Computing devices, including [this].

10. A method performed on the processor of a computing device, Steps of extracting a block of compressed weight data, wherein the block of compressed weight data includes a first frame header associated with a first frame payload, the first frame header includes a first normalization coefficient indicator, the first frame payload includes compressed weight values ​​for a neural network, the first normalization coefficient indicator indicates the number of padding bits removed from the weight values, and the first frame header includes a frame length configured to represent the number of weight values ​​contained in the first frame payload. A step of generating a first restored frame comprising the restored weight values ​​of the compressed weight values ​​of the first frame payload, and A method that includes this.

11. The block of compressed weight data includes a global header having an offset value, The first frame header includes an offset indicator configured to indicate whether the offset value will be included for each restored weight value generated from the first frame payload, The method described above is The steps include: parsing the global header with respect to the offset value; With respect to the offset indicator, the steps include parsing the first frame header, The steps include determining whether or not the offset indicator is set in the first frame header, and It further includes, The method according to claim 10, wherein the step of generating the first restored frame includes the step of including the offset value in each restored weight value generated from the first frame payload associated with the first frame header, in response to a decision that the offset indicator is set in the first frame header.

12. The compressed weight data block includes a second frame header, which is not associated with the frame payload, and the method includes a second normalization coefficient indicator. A step of generating a second restored frame having restored weight values ​​with all bits having a value of 0, according to the second normalization coefficient indicator of the second frame header. The method according to claim 10, further comprising:

13. The method according to claim 10, wherein the block of compressed weight data includes a second frame header associated with a second frame payload, the second frame header includes a second normalization coefficient indicator, the second frame payload includes compressed weight values, and the method further comprises the step of generating a second restored frame comprising restored weight values ​​of the compressed weight values ​​of the second frame payload by adding padding bits to the compressed weight values ​​of the second frame payload according to the second normalization coefficient indicator of the second frame header.

14. The method according to claim 10, wherein the step of generating the first restored frame includes adding padding bits to the compressed weight values ​​of the first frame payload in accordance with the first normalization coefficient indicator of the first frame header in order to generate restored weight values, wherein the value of the padding bits for the first compressed weight values ​​of the compressed weight values ​​of the first frame payload is determined from the most significant bit of the first compressed weight values, and the padding bits are sign extension bits.

15. A computing device, A recovery processing device configured to perform an operation, wherein the operation is Extracting a block of compressed weight data, wherein the block of compressed weight data includes a first frame header associated with a first frame payload, the first frame header includes a first normalization coefficient indicator, the first frame payload includes compressed weight values ​​for a neural network, the first normalization coefficient indicator indicates the number of padding bits removed from the weight values, and the first frame header includes a frame length configured to represent the number of weight values ​​contained in the first frame payload, and To generate a first restored frame comprising the restored weight values ​​of the compressed weight values ​​of the first frame payload. Computing devices, including [this].

Images