Decoder for decoding codeworks of tunstall codes
Through the Tunstall encoding method and decoder, multiple symbols are encoded into fixed bit lengths, solving the problem of inefficiency of existing decoders, and achieving efficient deep neural network compression and decoding, significantly reducing memory capacity and energy consumption, and improving The speed of reasoning.
Patent Information
- Application Number
- CN202280091000.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2021-12-06
- Filing Date
- 2022-12-06
- Publication Date
- 2024-11-08
AI Technical Summary
The existing decoder has low decoding efficiency, especially for devices with limited resources, resulting in high memory capacity requirements and energy consumption. The traditional F2V encoding method has high decoding complexity, so it is impossible to decode multiple symbols in one clock cycle, which affects the inference. speed.
Using the Tunstall encoding method, multiple symbols are encoded into fixed bit lengths through the Tunstall decoder to realize parallel decoding, reducing the decoding complexity to O(n), and improving decoding efficiency.
It significantly reduces the memory capacity requirements and energy consumption of the device, improves the inference speed, and realizes efficient deep neural network compression and decoding, which is 10 times faster than traditional methods.
Smart Images

Figure CN118923044A_ABST
Abstract
Description
CROSS-REFERENCE TO RELATED APPLICATIONS
[0001] This application claims priority to Singapore patent application number 10202113561S filed on December 6, 2021, the entire contents of which are incorporated herein by reference. Technical Field
[0002] The present invention generally relates to a decoder for decoding codewords of a Tunstall code, which decoder may be referred to as a Tunstall decoder. Background Art
[0003] In the era of artificial intelligence of things (AIoT), more and more edge devices require ultra-low power consumption, while the chip area is limited. At the same time, certain on-chip data, such as the weights of neural networks (such as deep learning neural networks), also require larger memories (usually occupying larger areas) to accommodate more complex and larger neural network models. Therefore, in this case, data can be compressed on the sender or producer side and decompressed on the receiver or consumer side to save chip area and circuit power consumption.
[0004] For example, deep learning has been widely used in many artificial intelligence (AI) fields. Among them, deep neural networks (DNNs) provide the highest accuracy for various applications. To achieve higher accuracy, DNNs may require more parameters, which in turn increases energy consumption and memory requirements for storing weights, especially for inference. These challenges become more prominent for resource-limited devices (such as but not limited to battery-powered edge platforms).
[0005] To address these challenges, existing research has explored pruning, quantization, and entropy coding to compress deep neural networks. Generally speaking, pruning removes weights and corresponding calculations that have minimal or no impact on accuracy. In this regard, disclosed techniques include channel, filter, connection, and layer pruning. However, these techniques may require manual intervention on certain network layers to achieve better results. For example, DNN hardware accelerators called efficient inference engine (EIE) and Eyeriss-v2 have been disclosed, which use sparse representation to compress pruned weights. However, the implementation of sparse representation is complex, requires specific hardware support, and may result in longer execution time.
[0006] Quantization is a method that can be used to compress DNNs by reducing the number of bits per variable for weights and activations. For example, state-of-the-art quantization methods can compress weights from 8 bits to 4 bits without losing accuracy. However, further reducing the number of bits, such as to 2 bits, results in a significant drop in accuracy.
[0007] The focus of entropy coding is to exploit the peak distribution of quantized values to encode the quantized values of weights (or activations) into a more compact representation, thereby reducing the number of bits per variable. In practice, if the number of bits per weight can be reduced from 4 bits to 2 bits, for example, the memory capacity requirements and the corresponding memory access energy can be significantly reduced, while the inference speed can be significantly accelerated. Unlike pruning or quantization, entropy coding does not rely on training data to learn the codec, and quantized values can be encoded and decoded without retraining. Because encoding and decoding are lossless, the use of entropy coding does not result in a loss of accuracy.
[0008] Although some previous studies have adopted entropy coding, these studies have adopted fixed-to-variable (F2V) coding methods, such as arithmetic coding and Huffman coding. For example, Huffman coding is used to compress quantization weights in deep compression and coreset-based compression. However, the decoding stage of the F2V coding method is costly because the encoded string requires inefficient bit-by-bit processing. In addition, because the codeword length in the F2V coding method is variable and cannot be indexed, it is difficult to develop a parallel implementation of the decoding stage. Therefore, it is impossible to decode multiple symbols in one clock cycle. Therefore, the decoding computational complexity of the F2V coding method is very high. For example, given the number of codewords (quantization values) n and the inverse of the compression ratio k, the decoding complexity is as high as O(n·k). Unlike the encoding stage, which can be performed offline, the decoding stage may require online processing. Therefore, in real-time applications, if the decoding efficiency is not high, it will greatly reduce the inference speed.
[0009] Therefore, there is a need to provide a decoder that attempts to overcome or at least improve one or more deficiencies in conventional decoders, and more specifically, to improve decoding efficiency, for example, to reduce memory capacity requirements and energy consumption of devices (especially resource-limited devices). The present invention was developed in this context. Summary of the invention
[0010] According to a first aspect of the present invention, there is provided a decoder for decoding a codeword of a Tunstall code, the decoder comprising: a sub-decoder for receiving an input codeword of a Tunstall code input to the decoder and outputting a decoded symbol of the input codeword; a symbol memory for receiving and storing the decoded symbol of the input codeword from the sub-decoder; and a controller for controlling the symbol memory to output one or more decoded symbols stored in the symbol memory, wherein the sub-decoder comprises: a node memory for storing a plurality of codewords of the Tunstall code respectively assigned to a plurality of nodes for a plurality of nodes of a Tunstall tree of a Tunstall code corresponding to a first layer of the Tunstall tree; and a comparator for comparing the input codeword with a plurality of codewords assigned to a plurality of nodes corresponding to the first layer of the Tunstall tree received from the node memory, and based on the comparison, generating a decoded symbol of the input codeword relative to the first layer of the Tunstall tree.
[0011] According to a second aspect of the present invention, there is provided a method for operating a decoder for decoding codewords of a Tunstall code according to the above-mentioned first aspect of the present invention, the method comprising: a sub-decoder of the decoder receives an input codeword of a Tunstall code input to the decoder; the sub-decoder outputs a decoded symbol of the input codeword; a symbol memory of the decoder receives and stores the decoded symbol of the input codeword from the sub-decoder; and a controller of the decoder controls the symbol memory to output one or more decoded symbols stored in the symbol memory, wherein the method further comprises a comparator of the sub-decoder comparing the input codeword with a plurality of codewords received from a node memory of the sub-decoder and assigned to a plurality of nodes corresponding to a first layer of a Tunstall tree, and the comparator generates a decoded symbol of the input codeword relative to the first layer of the Tunstall tree based on the comparison.
[0012] According to a third aspect of the present invention, there is provided a decoder for decoding a codeword of a Tunstall code, the decoder comprising: a symbol memory comprising a plurality of memory entries, each memory entry storing one or more decoded symbols of a codeword of the Tunstall code corresponding to the memory entry; and a controller for receiving an input codeword of the Tunstall code input to the decoder, and controlling the symbol memory to output one or more decoded symbols stored in a memory entry of the plurality of memory entries corresponding to the input codeword.
[0013] According to a fourth aspect of the present invention, there is provided a method of operating a decoder for decoding a codeword of a Tunstall code according to the third aspect of the present invention, the method comprising: a controller of the decoder receiving an input codeword of the Tunstall code; and the controller of the decoder controlling a symbol memory of the decoder to output one or more decoded symbols corresponding to the input codeword stored in a plurality of memory entries. BRIEF DESCRIPTION OF THE DRAWINGS
[0014] For those skilled in the art, embodiments of the present invention will be readily understood and apparent from the following written description which is given by way of example only in conjunction with the accompanying drawings, in which: Figure 1 depicts a schematic block diagram of a decoder for decoding a codeword of a Tunstall code according to various first embodiments of the present invention; Figure 2 depicts a schematic block diagram of a decoder for decoding a codeword of a Tunstall code according to various second embodiments of the present invention; Figure 3 depicts a schematic flow chart of a method of operating a decoder for decoding a codeword of a Tunstall code according to various first embodiments of the present invention; Figure 4 depicts a schematic flow chart of a method of operating a decoder for decoding codewords of a Tunstall code according to various second embodiments of the present invention; FIG. 5A to FIG. 5C Depicts an example construction of a Tunstall tree; Figure 6 Depicted showing FIG. 5A to FIG. 5C Table of codebooks for an example 3-bit Tunstall tree shown in (Table I); Figure 7 depicts a schematic diagram illustrating a first (or memory-oriented (MO)) decoding method according to various first example embodiments of the present invention; Figure 8 An example algorithm (Algorithm 1) for implementing the MO decoding method according to each first example embodiment of the present invention is shown; Fig. 9 depicts a schematic diagram of an MO decoder for decoding codewords of a Tunstall code according to first exemplary embodiments of the present invention; Fig.10 depicts a schematic diagram showing an example data format of decoded symbols stored in a symbol memory for a plurality of codewords according to various first example embodiments of the present invention; Fig.11ADepicted is an example method or algorithm (Algorithm 2) for updating decoded symbols of corresponding codewords stored in on-chip memory according to various first example embodiments of the present invention; Fig. 11B depicts a schematic flow chart illustrating an example method for updating an example codebook according to first exemplary embodiments of the present invention; Fig.12 depicts a schematic diagram of an example Tunstall tree illustrating a second (or logic-oriented (LO)) decoding method according to various second example embodiments of the present invention; Fig.13 Depicting an example method or algorithm (Algorithm 3) for determining the rank of a selected node of a layer of a Tunstall tree according to various second example embodiments of the present invention; Fig.14 depicts a schematic flow chart of a LO decoding method according to various second exemplary embodiments of the present invention; Fig.15 An example algorithm (Algorithm 4) for implementing the MO decoding method according to each second example embodiment of the present invention is shown; Fig.16 depicts an example data format of node-associated information according to various second example embodiments of the present invention; Fig.17 depicts a schematic diagram illustrating an example LO decoder having a first example configuration or architecture according to various second example embodiments of the present invention; Fig.18 depicts a schematic diagram showing an example architecture of a sub-decoder of a LO decoder according to various second example embodiments of the present invention; Fig.19 depicts a table (Table II) showing resource utilization and decoding cycles of a MO decoder with different memory entries and memory types; Fig. 20 depicts a table (Table III) showing resource utilization and decoding cycles of LO decoders with different numbers of sub-decoders and memory banks; Fig.21 Depicts a schematic diagram of a deep learning accelerator system according to various example embodiments of the present invention; Fig. 22 depicts a table summarizing the capacity of main memory and the estimated memory access power in different methods (Table IV); Fig.23 depicts a table (Table V) showing memory access cycles in different methods; Fig.24Depicted is a table (Table VI) showing the performance of the above LO Tunstall decoder and MO Tunstall decoder for ResNet-50; Fig.25A and Fig.25B Two example Tunstall and Huffman codewords stored in two 32-bit main memory cells are shown, respectively; and Fig.26 A table showing the decoding times for the Tunstall and Huffman software is depicted (Table VII). DETAILED DESCRIPTION
[0015] Various embodiments of the present invention provide a decoder for decoding codewords of a Tunstall code, which may be referred to as a Tunstall decoder.
[0016] In particular, embodiments of the present invention advantageously provide a variable-to-fixed (V2F) entropy coding method, namely Tunstall coding, to compress data. In embodiments, Tunstall coding is used to compress the quantized weights of a neural network. For example, embodiments find that Tunstall coding can achieve very high compression ratios on various deep networks. In addition, the decoding of Tunstall coding according to embodiments of the present invention is much more efficient than F2V decoding. For example, because Tunstall coding encodes multiple symbols into a fixed bit length, multiple bits can be processed simultaneously (i.e., in parallel) in the decoding stage. Therefore, embodiments find that the decoding of Tunstall coding is much faster. In addition, embodiments point out that the decoding complexity in Tunstall coding can be only O(n), which is much lower than the complexity of the F2V coding method (as described in the background technology, up to O(n·k)). Therefore, embodiments of the present invention advantageously provide a V2F entropy coding method, namely Tunstall coding, to improve decoding efficiency, such as for deep network compression, and for hardware implementation of Tunstall decoding. Specifically, the various embodiments of the present invention advantageously provide a decoder, more specifically, a Tunstall decoder, which attempts to overcome or at least improve one or more deficiencies in conventional decoders, and more specifically, improves decoding efficiency, for example, by reducing memory capacity requirements and energy consumption of devices (particularly resource-limited devices).
[0017] Figure 1A schematic block diagram of a decoder 100 for decoding a codeword of a Tunstall code according to various first embodiments of the present invention is depicted. The decoder 100 includes: a sub-decoder 104 (or a sub-decoder circuit or module) for receiving an input codeword of a Tunstall code input to the decoder 100 and outputting a decoded symbol of the input codeword; a symbol memory 106 for receiving and storing the decoded symbol of the input codeword from the sub-decoder 104; and a controller 108 (or a controller circuit or module) for controlling the symbol memory 106 to output one or more decoded symbols stored in the symbol memory 106. The sub-decoder 104 includes: a node memory 112, which is used to store (or has stored) a plurality of codewords of the Tunstall code respectively assigned to the plurality of nodes for the plurality of nodes of the Tunstall tree of the Tunstall code corresponding to the first layer of the Tunstall tree; and a comparator 114 (or a comparator circuit or module), which is used to compare the input codeword with the plurality of codewords assigned to the plurality of nodes corresponding to the first layer of the Tunstall tree received from the node memory 112, and based on the comparison, generate a decoding symbol of the input codeword relative to the first layer of the Tunstall tree.
[0018] In each first embodiment, the above-mentioned generation of decoding symbols of input code words relative to the first layer of the Tunstall tree includes: based on comparison, determining one of the multiple nodes corresponding to the first layer of the Tunstall tree as a determined node of the first layer of the Tunstall tree; and based on the determined node, generating decoding symbols of the input code words relative to the first layer of the Tunstall tree.
[0019] In each first embodiment, the subdecoder 104 is also used to: determine whether the determined node of the first layer of the Tunstall tree has an associated grandchild node, according to the Tunstall tree, the grandchild node is two layers below the determined node; and based on determining (i.e., if it is determined) that the determined node of the first layer of the Tunstall tree has an associated grandchild node, obtain the addresses (e.g., starting addresses) of multiple nodes of the Tunstall tree corresponding to the next layer (the immediately next layer or subsequent layer) of the Tunstall tree relative to the first layer of the Tunstall tree.
[0020] In each first embodiment, the sub-decoder 104 also includes a selector (or a selector circuit or module), which is communicatively coupled to the comparator 114 and the node memory 113, and the selector is used to perform the above-mentioned determination of whether the node of the first layer of the Tunstall tree has an associated grandchild node, and the above-mentioned acquisition of the addresses of multiple nodes of the Tunstall tree corresponding to the next layer of the Tunstall tree.
[0021] In each first embodiment, according to a first example configuration or architecture, the decoder 100 further includes one or more additional sub-decoders (or additional sub-decoder circuits or modules). For example, the above-mentioned sub-decoder 104 and the one or more additional sub-decoders can thus form a series of pipeline stages for decoding an input codeword. In this regard, the sub-decoder 104 is a first sub-decoder associated with a first layer of the Tunstall tree (e.g., a layer immediately following the root of the Tunstall tree), and the one or more additional sub-decoders are respectively associated with one or more additional layers of the Tunstall tree.
[0022] In each first embodiment, according to a first example configuration or architecture, each of the one or more additional sub-decoders is used to receive an input codeword and output a decoded symbol of the input codeword relative to a layer of a Tunstall tree associated with the additional sub-decoder, and includes: a node memory, which is used to store (or has stored) a plurality of codewords of the Tunstall code respectively assigned to a plurality of nodes for a plurality of nodes of a Tunstall tree of a Tunstall code corresponding to the layer of the Tunstall tree associated with the additional sub-decoder; and a comparator, which is used to compare the input codeword with a plurality of codewords assigned to a plurality of nodes corresponding to the layer of the Tunstall tree associated with the additional sub-decoder and received from the node memory, and based on the comparison, generate a decoded symbol of the input codeword relative to the layer of the Tunstall tree associated with the additional sub-decoder.
[0023] In each first embodiment, according to the first example configuration or architecture, the above-mentioned generation of decoding symbols of input codewords of the layer of the Tunstall tree associated with the additional sub-decoder includes: based on the comparison, determining one of the multiple nodes corresponding to the layer of the Tunstall tree as a determined node of the layer of the Tunstall tree; and based on the determined node, generating decoding symbols of the input codewords of the layer of the Tunstall tree associated with the additional sub-decoder.
[0024] In each first embodiment, according to a first example configuration or architecture, the first sub-decoder 104 is used to output the addresses of multiple nodes corresponding to the next layer of the Tunstall tree relative to the first layer of the Tunstall tree to an additional sub-decoder in one or more additional sub-decoders associated with the next layer of the Tunstall tree based on determining (i.e., if it is determined) that the determined node of the first layer of the Tunstall tree associated with the first sub-decoder 104 has an associated grandchild node, and the comparator of the additional sub-decoder compares the input codeword with the multiple codewords assigned to the multiple nodes corresponding to the next layer of the Tunstall tree received from the node memory of the additional sub-decoder based on the received addresses of the multiple nodes, and based on the comparison, generates a decoded symbol of the input codeword relative to the next layer of the Tunstall tree associated with the additional sub-decoder. In this regard, the additional sub-decoder is also used to output the decoded symbol of the input codeword relative to the next layer of the Tunstall tree associated with the additional sub-decoder to the symbol memory 106.
[0025] In each first embodiment, according to the first example configuration or architecture, each of the one or more additional sub-decoders is used to output the addresses of multiple nodes corresponding to the next layer of the Tunstall tree relative to the layer of the Tunstall tree to another additional sub-decoder associated with the next layer of the Tunstall tree based on determining that the determined node of the layer of the Tunstall tree associated with the additional sub-decoder has an associated grandchild node, and the comparator of the other additional sub-decoder compares the input codeword with the multiple codewords assigned to the multiple nodes corresponding to the next layer of the Tunstall tree received from the node memory of the other additional sub-decoder based on the received addresses of the multiple nodes, and based on the comparison, generates a decoded symbol of the input codeword relative to the next layer of the Tunstall tree. In this regard, the other additional sub-decoder is also used to output the decoded symbol of the input codeword relative to the next layer of the Tunstall tree to the symbol memory.
[0026] Therefore, in the same or similar manner as sub-decoder 104, each additional sub-decoder may also include a selector (or a selector circuit or module) that is communicatively coupled to the comparator and the node memory of the additional sub-decoder and is used to determine whether a determined node of a layer of the Tunstall tree associated with the additional sub-decoder has an associated grandchild node, and is used to obtain the above-mentioned addresses of multiple nodes corresponding to the next layer of the Tunstall tree relative to the layer of the Tunstall tree.
[0027] In each first embodiment, according to a first example configuration or architecture, the first sub-decoder 104 and each of the one or more additional sub-decoders are used to determine (e.g., by a selector of the sub-decoder) whether the determination node of the layer of the Tunstall tree associated with the sub-decoder has an associated child node based on determining that the determination node of the layer of the Tunstall tree associated with the sub-decoder does not have an associated grandchild node, wherein the child node is one layer below the determination node according to the Tunstall tree. In this regard, the decoder 100 also includes a subtractor (or a subtractor circuit or module) that is used to generate a decoded symbol relative to the input codeword of the next layer of the layer Tunstall tree relative to the Tunstall tree based on the input codeword and the codeword assigned to the determination node of the layer of the Tunstall tree associated with the sub-decoder based on the determination that the determination node of the layer of the Tunstall tree associated with the sub-decoder has an associated child node.
[0028] In each first embodiment, according to the second example configuration or architecture, the node memory 112 of the subdecoder 104 is used to store (or has stored therein) a plurality of codewords of the Tunstall code respectively assigned to a plurality of nodes corresponding to the layer of the Tunstall tree for each of the plurality of layers of the Tunstall tree, including the first layer of the Tunstall tree described above, and for a plurality of nodes of the Tunstall tree corresponding to the layer of the Tunstall tree. That is, the node memory 112 is used to store (or has stored therein) a plurality of codewords assigned to a plurality of nodes for a plurality of layers of the Tunstall tree.
[0029] In each first embodiment, according to the second example configuration or architecture, the comparator 114 is used to determine (for example, determined by the selector of the subdecoder 104) that the determination node of the layer of the Tunstall tree has an associated grandchild node based on the subdecoder 104: based on the addresses of the multiple nodes of the Tunstall tree corresponding to the next layer of the Tunstall tree, compare the input codeword with the multiple codewords of the multiple nodes of the Tunstall tree corresponding to the next layer of the Tunstall tree (the next next layer or the subsequent layer) received from the node memory 112; based on the comparison, determine one of the multiple nodes corresponding to the next layer of the Tunstall tree as the determination node of the next layer of the Tunstall tree; and based on the determination node of the next layer of the Tunstall tree, generate a decoded symbol of the input codeword relative to the next layer of the Tunstall tree. In this regard, the subdecoder 104 is also used to output the decoded symbol of the input codeword relative to the next layer of the Tunstall tree to the symbol memory 106.
[0030] In each first embodiment, according to the second example configuration or architecture, the subdecoder 104 is further configured to determine whether the determination node of the layer of the Tunstall tree has an associated child node (e.g., by a selector of the subdecoder 104) based on determining that the determination node of the layer of the Tunstall tree does not have an associated grandchild node, wherein the child node is one layer below the determination node according to the Tunstall tree. In this regard, the decoder 100 further includes a subtractor configured to generate a decoded symbol of an input codeword of a next layer of the Tunstall tree relative to the layer of the Tunstall tree based on the determination by the subdecoder 104 that the determination node of the layer of the Tunstall tree has an associated child node and based on the input codeword and the codeword assigned to the determination node of the layer of the Tunstall tree.
[0031] In each first embodiment, the symbol memory 106 includes a plurality of memory groups, each memory group is used to store one or more decoded symbols of a corresponding codeword. In this regard, the controller 108 is used to control the symbol memory 106 to output one or more decoded symbols of a corresponding codeword stored in the memory group for each of the plurality of memory groups.
[0032] In each first embodiment, the input codeword corresponds to the compressed quantized weights of the neural network. In this regard, one or more decoded symbols of the input codeword stored in the symbol memory 106 and output from the decoder 100 correspond to the uncompressed quantized weights of the neural network.
[0033] In each first embodiment, decoder 100 is an integrated circuit. In each first embodiment, controller 108 and sub-decoder 104 (if any, and each additional sub-decoder) are communicatively coupled to symbol memory 106. In each first embodiment, controller 108 is also communicatively coupled to sub-decoder 104 (if any, and each additional sub-decoder).
[0034] Figure 2 A schematic block diagram of a decoder 200 for decoding a codeword of a Tunstall code according to each second embodiment of the present invention is depicted. The decoder 200 includes: a symbol memory 206, including a plurality of memory entries, each memory entry storing one or more decoded symbols of a codeword of a Tunstall code corresponding to the memory entry; and a controller 208 (or a controller circuit or module) for receiving an input codeword of a Tunstall code input to the decoder, and controlling the symbol memory to output one or more decoded symbols stored in one of the plurality of memory entries corresponding to the input codeword. It will be understood by those skilled in the art that, unless otherwise specified or required by the context, one or more decoded symbols of a codeword refers to all decoded symbols of the codeword, wherein the number of decoded symbols of a codeword may be one or more.
[0035] In each second embodiment, the control symbol memory 206 outputs one or more decoded symbols stored in one of the plurality of memory entries corresponding to the input codeword as an address of one of the plurality of memory entries based on the input codeword.
[0036] In each second embodiment, the symbol memory 206 is an on-chip memory. In this regard, the controller 208 is communicatively coupled to the symbol memory 206 and the off-chip memory, wherein a plurality of memory entries of the symbol memory 206 respectively store one or more decoded symbols of a first plurality of codewords of the Tunstall code, and the off-chip memory stores one or more decoded symbols of the codeword for each of the second plurality of codewords of the Tunstall code. In this regard, the controller 208 is used to determine whether the one or more decoded symbols of the input codeword are located in the symbol memory 206 or in the off-chip memory, and control the determined one of the symbol memory 206 and the off-chip memory to output the one or more decoded symbols of the input codeword.
[0037] In each second embodiment, a plurality of memory entries of the symbol memory are respectively updated with one or more decoded symbols of a plurality of remapped codewords, the plurality of remapped codewords being located in a plurality of codebook entries of an updated codebook, replacing a plurality of codewords having the highest frequency counts in a previous plurality of codebook entries. Thus, the first plurality of codewords corresponds to the plurality of remapped codewords.
[0038] In various second embodiments, each of the plurality of memory entries of the symbol memory 106 further stores a symbol number parameter indicating the number of decoded symbols of the corresponding codeword stored in the memory entry.
[0039] In each second embodiment, the input codeword corresponds to compressed quantized weights of the neural network. In this regard, the one or more decoded symbols stored in each of the plurality of memory entries of the symbol memory and the one or more decoded symbols of the input codeword output from the decoder correspond to uncompressed quantized weights of the neural network.
[0040] In various second embodiments, decoder 200 is an integrated circuit. In various second embodiments, controller 208 is communicatively coupled to symbol memory 206.
[0041] Figure 3A schematic flow chart of a method 300 of operating a decoder 100 for decoding a codeword of a Tunstall code according to various first embodiments of the present invention is depicted. The method 300 includes: the sub-decoder 104 of the decoder 100 receives (at 302) an input codeword of a Tunstall code input to the decoder 100; the sub-decoder 104 outputs (at 304) a decoded symbol of the input codeword; the symbol memory 106 of the decoder 100 receives (at 306) and stores the decoded symbol of the input codeword from the sub-decoder 104; and the controller 108 of the decoder 100 controls (at 308) the symbol memory 106 to output one or more decoded symbols stored in the symbol memory 106. In addition, with respect to the above-mentioned (at 304) output of the decoded symbol of the input codeword, the method 300 also includes the comparator 114 of the sub-decoder 104 comparing the input codeword with a plurality of codewords assigned to a plurality of nodes corresponding to the first layer of the Tunstall tree and received from the node memory 112 of the sub-decoder 104, and the comparator 114 generating a decoded symbol of the input codeword relative to the first layer of the Tunstall tree based on the comparison.
[0042] Those skilled in the art will appreciate that the various operations of the method 300 correspond to the various functions or operations performed by the decoder 100 described herein according to various embodiments (or exemplary embodiments), and therefore, for the sake of clarity and brevity, there is no need to repeat the method 300 of operating the decoder 100. In other words, the various embodiments described herein in the context of the functions or operations of the decoder 100 are also valid for the corresponding method 300 of operating the decoder 100.
[0043] Figure 4 A schematic flow chart of a method 400 of operating a decoder 200 for decoding a codeword of a Tunstall code according to various second embodiments of the present invention is depicted. The method 400 includes: the controller 208 of the decoder 200 receives (at 402) an input codeword of a Tunstall code; and the controller 208 of the decoder 200 controls (at 404) the symbol memory 206 of the decoder 200 to output one or more decoded symbols stored in one of a plurality of memory entries corresponding to the input codeword.
[0044] Similarly, those skilled in the art will appreciate that the various operations of the method 400 correspond to the various functions or operations performed by the decoder 200 described herein according to various embodiments (or exemplary embodiments), and therefore, for the sake of clarity and brevity, there is no need to repeat the method 400 of operating the decoder 200. In other words, the various embodiments described herein in the context of the functions or operations of the decoder 200 are also valid for the corresponding method 400 of operating the decoder 200.
[0045] As described above, decoder 100 and decoder 200 may be implemented or realized as an integrated circuit, such as but not limited to a field-programmable gate array (FPGA).
[0046] The memory or computer-readable storage medium (e.g., symbol memory 106 / 206 and node memory 112) used in each embodiment can be a volatile memory, such as a dynamic random access memory (DRAM) or a non-volatile memory, such as a programmable read only memory (PROM), an erasable PROM (EPROM), an electrically erasable EPROM (EEPROM), or a flash memory, such as a floating gate memory, a charge trap memory, a magnetoresistive random access memory (MRAM), or a phase change random access memory (PCRAM). The controller (or processor) used in each embodiment can be any controller with processing capabilities for performing the various functions or operations described herein.
[0047] A "circuit" may be understood as any type of logic implementation entity, which may be a dedicated circuit or a processor that executes software stored in a memory, firmware, or any combination thereof. Therefore, in various embodiments, a "circuit" may be a hardwired logic circuit or a programmable logic circuit, such as a programmable processor, such as a microprocessor (e.g., a complex instruction set computer (CISC) processor or a reduced instruction set computer (RISC) processor). A "circuit" may also be a processor that executes software, such as any type of computer program, such as a computer program using a virtual machine code (e.g., Java). According to various embodiments, any other type of implementation of the corresponding function may also be understood as a "circuit".
[0048] Some parts of the present disclosure are presented in the form of functions or symbolic representations of data operations in algorithms and computer memories, either explicitly or implicitly. These algorithm descriptions and functions or symbolic representations are the means by which the technical personnel in the field of data processing are used to most effectively convey the essence of their work to other technical personnel in the field. Here, an algorithm is generally considered to be a series of self-consistent steps that can achieve a desired result. These steps require physical operations to physical quantities, such as being able to store, transmit, combine, compare, and otherwise operate electrical signals, magnetic signals, or optical signals.
[0049] According to different implementations, the functional components or modules described herein can be implemented as hardware components or modules. More specifically, in the hardware sense, a module is a functional hardware unit designed to be used in conjunction with other components or modules. For example, a module can be implemented using discrete electronic components or can form part of a complete electronic circuit, such as an application specific integrated circuit (ASIC). There are many other possibilities.
[0050] Those skilled in the art will appreciate that the terms used herein are only for the purpose of describing various embodiments and are not intended to limit the present invention. Unless the context clearly indicates otherwise, the singular forms "one", "a kind" and "the" used herein also include plural forms. It should be further understood that the terms "include" and / or "comprise" used in this specification specify the presence of the features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.
[0051] Unless otherwise specified or the context requires otherwise, the use of names such as "first", "second", etc. in this document to refer to elements or features does not limit the number or order of such elements or features. For example, such names may be used herein as a convenient way to distinguish between two or more elements or instances of elements. Therefore, referring to a first element and a second element does not necessarily mean that only two elements can be used, or that the first element must be before the second element. In addition, a phrase referring to "at least one" of a list of items refers to any single item therein or any combination of two or more items therein.
[0052] In order to make the present invention easy to understand and put into practice, the following will only describe various exemplary embodiments of the present invention in an exemplary and non-limiting manner. However, it will be appreciated by those skilled in the art that the present invention can be embodied in a variety of different forms or configurations and should not be construed as being limited to the exemplary embodiments described below. On the contrary, these exemplary embodiments are provided to make this disclosure comprehensive and complete and to fully convey the scope of the present invention to those skilled in the art.
[0053] In particular, for a better understanding of the present invention, unless otherwise specified or the context requires otherwise, without limitation or loss of generality, the following will describe various exemplary embodiments of the present invention using codewords corresponding to compressed quantized weights of a neural network. However, those skilled in the art will appreciate that the present invention is not limited to codewords being neural network quantized weights, and the present invention may be or represent any type of data as long as it is desired to compress or reduce the size of the data.
[0054] Data compression is the process of converting an input data stream into another output data stream of smaller size. In order to retain the information in the input data, each exemplary embodiment of the present invention uses entropy coding. In general, entropy coding is a lossless data compression scheme that is independent of the specific characteristics of the medium. Specifically, each exemplary embodiment of the present invention uses a variable to fixed (V2F) entropy method called Tunstall coding to encode one or more symbols into a fixed width codeword. During decoding, for example, multiple symbols can be decoded simultaneously (i.e., in parallel), and therefore the decoding speed can be significantly improved.
[0055] As an example practical application, each example embodiment provides an efficient Tunstall decoder for deep neural network compression. For example, in edge applications, power and area-efficient deep neural network (DNN) designs are very important. Through compression or quantization, compact DNNs can significantly reduce memory accesses, thereby realizing such designs. Lossless entropy coding can further significantly reduce the size of the neural network. In this regard, each example embodiment provides hardware support for such entropy coding modules to fully benefit from the reduced memory requirements. In particular, each example embodiment uses Tunstall coding to compress the quantized weights of the neural network. In this regard, each example embodiment finds that Tunstall coding can achieve a very high compression ratio. In addition, the decoding of Tunstall coding according to each embodiment of the present invention can achieve very fast decoding speeds on various deep networks. Each example embodiment provides two hardware accelerated decoding techniques to provide streamlined decoding capabilities. In this regard, various example embodiments synthesize these designs on field programmable gate arrays (FPGAs). As will be discussed below, experimental results show that decoders according to various example embodiments of the present invention can achieve up to 10 times the decoding speed compared to traditional or state-of-the-art decoders.
[0056] Therefore, the exemplary embodiments of the present invention advantageously provide a V2F entropy coding method, namely Tunstall coding, to compress the quantized weights of a neural network. For example, the exemplary embodiments found that Tunstall coding can achieve very high compression ratios on various deep networks. In addition, the decoding of Tunstall coding according to the exemplary embodiments of the present invention is much more efficient than F2V decoding. For example, unlike the F2V coding method, the V2F coding method (or more specifically, Tunstall coding) encodes multiple symbols into a fixed number of bits. Therefore, at the decoding level, multiple bits can be processed simultaneously and multiple symbols can be decoded in each clock cycle, thereby speeding up decoding. For example, because the coded string can be divided into fixed-length bit blocks based on the length of the codeword, the coded string can also be decoded in parallel. In addition, the decoding complexity in Tunstall coding is only O(n), which is much lower than the complexity of the F2V coding method. Therefore, the exemplary embodiments use the V2F Tunstall coding method to achieve high compression ratio and low decoding complexity.
[0057] The exemplary embodiments of the present invention further develop a hardware implementation for the decoding stage in Chisel and propose two hardware accelerated decoding techniques, for example, for use in deep neural networks. In this regard, the first exemplary embodiments of the present invention provide a first decoding method (and a corresponding first decoder), which can be referred to as a memory-oriented (MO) decoding method (and a corresponding memory-oriented (MO) decoder) or a dictionary-based decoding method (and a corresponding dictionary-based decoder). The first decoding method or the first decoder corresponds to the above reference Figure 2 The decoder 200 for decoding a codeword of a Tunstall code according to each second embodiment of the present invention is described. Each second exemplary embodiment of the present invention provides a second decoding method (and a corresponding second decoder), which can be referred to as a logic-oriented (LO) decoding method (and a corresponding LO decoder) or a Tunstall tree-based decoding method (and a corresponding Tunstall tree-based decoder). The second decoding method or the second decoder corresponds to the above reference Figure 1 The decoder 100 for decoding the codeword of Tunstall code according to each first embodiment of the present invention is described. In each exemplary embodiment, the MO decoder and the LO decoder are synthesized or implemented on an FPGA. Therefore, each exemplary embodiment of the present invention is advantageously: Adopting V2F entropy coding method, namely Tunstall coding, to solve the problem of low decoding efficiency in deep neural network compression; and Provides two hardware implementations for the Tunstall decoding method, namely the MO and LO decoder architectures.
[0058] As will be discussed later below, experimental results show that by using a decoder according to various example embodiments of the present invention, for example, the memory usage of the inference level of deep networks (i.e., ResNet-50 and MobileNetv2) can be reduced by 18 times and 2.3 times compared to a full-precision 32-bit network and a quantized 4-bit network, respectively, while achieving up to 10 times speedup over the F2V decoder. Tunstall Coding Overview Construction and encoding of Tunstall trees
[0059] As an illustrative example, FIG. 5A to FIG. 5C An example construction of a Tunstall tree is depicted. In this example, the probabilities of the three symbols "A", "B", and "C" are 0.7, 0.18, and 0.12, respectively. Therefore, a 3-bit Tunstall tree can be constructed to encode the symbols. Specifically, the construction of the Tunstall tree is performed in an iterative manner. At the beginning, the Tunstall tree has only one root with a probability of 1.0. For each iteration, the leaf node with the highest probability is selected and converted into a new subtree with K child nodes, where K is the total number of symbols (e.g., K=3 in this example). For example, in Figure 5B In the second iteration shown, the one with the maximum probability of 0.7 (as Figure 5A The leaf node associated with "A" is converted into a new subtree having child nodes associated with "AA", "AB", and "AC", respectively. Figure 5C In the third iteration shown, the leaf node associated with "AA" with the maximum probability of 0.49 is selected and converted into a new subtree with child nodes associated with "AAA", "AAB", and "AAC", respectively. To complete the Tunstall tree, iterations continue until the total number of leaf nodes reaches 2 C (i.e., equal to or as close as possible to 2 C but no more than 2 C ), where C is the fixed length of the Tunstall codeword (eg, in this example, C=3).
[0060] Figure 6 Depicted showing FIG. 5A to FIG. 5C000101010) (i.e., codewords "000", "101", and "010"), where the symbol "AAA" is encoded as the codeword "000", the symbol "B" is encoded as the codeword "101", and the symbol "AAC" is encoded as the codeword "010". Therefore, it can be seen that one or more symbols are encoded into a fixed-length codeword (e.g., a 3-bit codeword in this example). Therefore, the Tunstall code consists of a set of fixed-size codewords (e.g., a set of 3-bit Tunstall codes in this example), each of which encodes a string of variable-size input symbols. decoding
[0061] Because the codewords of Tunstall encoding have a fixed length, at the decoding level, each example embodiment can process the codeword at a time and efficiently decode the codeword, for example, by referring to a codebook. For example, if the fixed length of the codeword is 8 bits, 8 bits can be processed per cycle. Therefore, the decoding method according to each example embodiment can be advantageously byte-oriented and therefore much faster than the decoding of the F2V encoding method that requires bit-by-bit processing of the encoded string. Memory-Oriented (MO) Decoding Decoding method or algorithm
[0062] Figure 7 A schematic diagram illustrating a first or MO decoding method 700 according to various first example embodiments of the present invention is depicted. The MO decoding method 700 includes: a controller receiving an input codeword 702 of a Tunstall code; and the controller controlling a symbol memory to output a decoded symbol stored in one of a plurality of memory entries 706 corresponding to the input codeword 702. In various first example embodiments, based on the input codeword 702 used as an address of one of the plurality of memory entries, the symbol memory is controlled to output a decoded symbol stored in one of the plurality of memory entries. In other words, the input codeword 702 can be used as an index to locate or retrieve a decoded symbol (e.g., an uncompressed weight of a neural network) stored in a memory entry corresponding to the input codeword 702. For example, if the input codeword 702 is "010", based on the input codeword used as an address or index, a decoded symbol can be read out from a third memory entry ("MEM 2") corresponding to the address or index. Therefore, the MO decoding method 700 utilizes a codebook (also referred to as a dictionary) of the Tunstall code. In various example embodiments, a dedicated memory segment 706 located on-chip may be provided for storing a codebook (or a portion of the codebook) for the Tunstall code. Figure 7As shown, a predetermined number (N) of memory entries of a dedicated memory segment 706 may be provided to store corresponding N codebook entries. In this regard, when an input codeword 702 is received, the corresponding memory entry in which the corresponding decoded symbol (e.g., the uncompressed weights of the neural network) is stored may be read. As an illustrative example, Figure 8 An example algorithm (Algorithm 1) for implementing the MO decoding method 700 according to each first example embodiment of the present invention is shown. Hardware Implementation
[0063] Fig. 9 A schematic diagram of an MO decoder 900 for decoding a codeword of a Tunstall code according to various first exemplary embodiments of the present invention is depicted, corresponding to the Figure 7 The MO decoding method 700 according to each first exemplary embodiment of the present invention is described. As shown in the figure, the MO decoder 900 (which may also be referred to as a decoder based on a tightly coupled memory) includes: a symbol memory 906, including (for example, corresponding to Figure 7 ) multiple memory entries of the dedicated memory segment 706 shown in , each memory entry stores a decoded symbol of a codeword of the Tunstall code corresponding to the memory entry; and a controller 908, which is used to receive an input codeword 902 of the Tunstall code of the decoder 900 and control the symbol memory 906 to output a decoded symbol stored in one of the multiple memory entries corresponding to the input codeword 902.
[0064] In each first exemplary embodiment, the symbol memory 906 is an on-chip memory. In this regard, the controller 908 is communicatively coupled to the symbol memory 906 and the off-chip memory 907. A plurality of memory entries of the symbol memory 906 respectively store decoded symbols of a first plurality of codewords of the Tunstall code. The off-chip memory 907 stores decoded symbols of the codeword for each of the second plurality of codewords of the Tunstall code. In this regard, the controller 908 is used to determine whether the decoded symbol of the input codeword 902 is located in the symbol memory 906 or in the off-chip memory 907, and to control the determined one of the symbol memory 906 and the off-chip memory 907 to output the decoded symbol of the input codeword 902. In this regard, each first exemplary embodiment points out that the entire dictionary (or codebook) of the Tunstall code may be too large to be completely stored on-chip (decoded symbols of all codewords of the codebook), and on the other hand, the delay may be too large to read the decoded symbols of all codewords off-chip. In order to solve this problem, various exemplary first embodiments advantageously provide the above-mentioned on-chip memory 906 and off-chip memory 907 configuration or architecture.
[0065] Therefore, if Fig. 9As shown, according to each first exemplary embodiment, the MO decoder 900 may include two main components, namely, an on-chip memory 906 having a predetermined number (M) of memory entries, the memory storing part (or all) of the dictionary of the Tunstall code, and an MO controller 908. Therefore, if the decoded symbol corresponding to the input codeword 902 is stored in one of the plurality of memory entries of the on-chip memory 906, the decoded symbol (or uncompressed data) corresponding to the input codeword 902 can be directly read by consuming only a small amount of power. For example, in this way, the codeword can be decoded every cycle or every two cycles on the chip, depending on the type of the on-chip memory 906.
[0066] Fig.10 Depicted is a schematic diagram showing an example data format of decoded symbols stored in the symbol memory 906 for a plurality of codewords according to various first example embodiments of the present invention. Fig.10 , each row shows the decoded symbols stored for the corresponding codeword and may correspond to a memory word. In the example data format, the width of each decoded symbol (or uncompressed symbol or data) stored in the symbol memory 906 is 5 bits, the number of decoded symbols for a codeword ranges from 1 to 6, and the width of the corresponding codeword is 10 bits.
[0067] For example, Fig.10 One decoded symbol corresponding to the first example codeword stored in the corresponding memory entry is shown; two decoded symbols corresponding to the second example codeword stored in the corresponding memory entry; three decoded symbols corresponding to the third example codeword stored in the corresponding memory entry; four decoded symbols corresponding to the fourth example codeword stored in the corresponding memory entry; and six decoded symbols corresponding to the fifth example codeword stored in the corresponding memory entry. In each first example embodiment, each of the multiple memory entries of symbol memory 906 also stores a symbol number parameter (which may also be referred to as a valid number parameter in this article), which indicates the number of decoded symbols of the corresponding codeword stored in the memory entry. Therefore, the valid number parameter under the valid number field is used to indicate the number of valid decoded symbols in the memory entry (or memory word) of the corresponding codeword. For example, in Fig.10In the example shown, for the corresponding first example codeword, the valid number parameter "3'b001" indicates that one valid decoding symbol of the corresponding first example codeword is stored in the corresponding memory entry (for example, based on the valid number parameter, the receiver will ignore the other five symbols in the current data read from the memory entry). As another example, for the corresponding third example codeword, the valid number parameter "3'b011" indicates that three valid decoding symbols of the corresponding third example codeword are stored in the corresponding memory entry (for example, based on the valid number parameter, the receiver will ignore the other three symbols in the current data read from the memory entry). As another example, for the corresponding fifth example codeword, the valid number parameter "3'b101" indicates that six valid decoding symbols of the corresponding fifth example codeword are stored in the corresponding memory entry.
[0068] Therefore, in Fig.10 In the example data format shown, empty slots indicate that the receiver should ignore any values or data in the current data read from those empty slots. Fig.10 In the example data format shown, the width of each memory word of the corresponding codeword can be 33 bits, wherein every 5 bits between the 3rd bit and the 33rd bit represent a decoded symbol (or uncompressed symbol) of the corresponding codeword. Fig.10 The example data formats shown in are example sizes, such as the number of symbols and the data width, and data formats of various sizes may be employed as desired or appropriate, such as based on the size of a codebook.
[0069] In each first exemplary embodiment, as described above, the controller 908 may be used to decide whether to read the decoded symbols corresponding to the input codeword 902 from on-chip or off-chip. For example, in the case where the number of entries in the codebook exceeds (e.g., far exceeds) the available memory entries in the on-chip memory 906, the controller 908 may be configured in this manner. For example, if the decoded symbols of the input codeword 902 are stored in the on-chip memory 906, the controller 908 may enable the on-chip memory 906, that is, control the on-chip memory 906 to output the decoded symbols corresponding to the input codeword 902. Otherwise, the controller 908 may disable the on-chip memory 906 for the input codeword 902, and request the decoded symbols from the off-chip memory 907 and wait for a response from the off-chip memory 907.
[0070] In each first exemplary embodiment, a plurality of memory entries of a symbol memory (i.e., an on-chip memory) 906 may be updated with decoded symbols of a plurality of remapped codewords, respectively. In this regard, the plurality of remapped codewords are located in a plurality of codebook entries of an updated codebook, replacing a plurality of codewords having the highest frequency counts in the plurality of codebook entries previously mentioned. In this regard, for example, as described above, the entire dictionary (or codebook) of the Tunstall code may be too large to be fully stored on-chip (decoded symbols of all codewords), and on the other hand, the delay may be too large to read decoded symbols of all codewords off-chip. In this regard, according to each first exemplary embodiment, based on a trade-off between the size of the on-chip memory 906 and the read delay, the on-chip memory 906 may be used to store a predetermined number (M) of the highest frequency codewords, for example, to improve the hit rate of the decoded symbols of the corresponding codewords stored in the on-chip memory 906. Therefore, the first plurality of codewords mentioned above correspond to the plurality of remapped codewords mentioned above (now considered to be the highest frequency codewords after remapping).
[0071] As an illustrative example, Fig.11A An example method or algorithm (Algorithm 2) for updating an example codebook (which may also be referred to as a dictionary) of a Tunstall code according to various first example embodiments of the present invention is depicted. For example, the update process can be performed in an offline manner as part of data preparation for optimizing the generation of a Tunstall tree and a codebook. In Algorithm 2, keys (D) represents a set of codewords of the codebook, and values (D) represents a frequency count of a codeword. As an illustrative example, Fig. 11B A schematic flow chart illustrating an example method for updating an example codebook according to various first example embodiments of the present invention, including sorting and remapping a codebook of Tunstall codes. For example, first, the frequencies of codewords of the original or initial codebook relative to the compressed data set may be counted (e.g., Fig. 11B The codewords of the codebook (and the corresponding decoded / uncompressed symbols) may then be sorted based on the frequency counts of the codewords (e.g., from highest to lowest frequency count) (e.g., as shown in the left codebook in FIG. 1 ). Fig. 11B Then, while maintaining the position of the decoded / uncompressed symbols, all codewords of the classification codebook are remapped according to the ranking (or order) of the codewords in the classification codebook to form an updated codebook (e.g., as shown in Fig. 11B11C ). For example, in the original or initial codebook, the codeword "010" has the highest frequency count of 9999. In the sorted codebook, the codeword "010" and the corresponding decoded / uncompressed symbol are sorted in the first entry of the codebook based on the highest frequency count of the codeword. Then, while maintaining the position of the decoded / uncompressed symbols, all codewords of the sorted codebook are remapped according to the ranking (or order) of the codewords to form an updated codebook as shown in the right codebook in FIG. 11C . For example, this causes the codeword "010" to move to the third entry of the codebook (because the codeword "010" has the third highest ranking), and the codeword "000" to move to the first entry of the codebook (because the codeword "000" has the highest ranking). Therefore, as shown in the figure, for example, the codeword "010" that was previously located in the first entry of the codebook due to its highest frequency count has been replaced by the codeword "000" (the remapped codeword) that is located in the first entry of the codebook. In addition, as can be seen in Fig. 11B As seen in the right codebook of the remapped codeword "000", the decoded / uncompressed symbol is now remapped to "AAC" (i.e., the decoded / uncompressed symbol of the previous codeword "010"). Therefore, the updated codebook (e.g., Fig. 11B A predetermined number of highest frequency codewords (i.e., the highest remapped codewords, now considered to be the highest frequency codewords after remapping) of the right codebook in the on-chip memory 906 can be respectively allocated to multiple memory entries of the on-chip memory 906, and the corresponding decoded symbols are also transmitted to the above-mentioned multiple memory entries.
[0072] In each first exemplary embodiment, the number of decoded symbols corresponding to a codeword may range from 2 to N, where N corresponds to the degree of the Tunstall tree (eg, FIG. 5A to FIG. 5C In the example Tunstall tree shown in FIG. , N=3).
[0073] In each first exemplary embodiment, if one decoded symbol (or uncompressed symbol) per cycle is desired or required, the MO decoder 900 may also include a multi-write port first in first out (FIFO) buffer for converting parallel decoded symbols into sequential decoded symbols. Logic-Oriented (LO) Decoding Decoding method or algorithm
[0074] Fig.12 Depicted is a schematic diagram illustrating an example Tunstall tree of a second or LO decoding method according to various second example embodiments of the present invention. Fig.12 As shown, the LO decoding method can be based on FIG. 5A to FIG. 5C The reverse of the encoding process shown in . Fig.12As shown, the Tunstall tree has multiple layers, and for each of the multiple layers, it includes multiple nodes corresponding to the layers of the Tunstall tree. In addition, as shown in the figure, the multiple nodes corresponding to the layers of the Tunstall tree have been respectively assigned multiple codewords of the Tunstall code. For example, for Fig.12 In the example Tunstall tree shown in , there is a root node and a plurality of layers after or below the root node (e.g., layers 1 to 3 in the example), and a plurality of nodes corresponding to each respective layer. Fig.12 As shown, the edge or branch of a node (upstream edge or branch) can represent a decoded symbol (or uncompressed symbol) associated with the node. In this regard, the multiple decoded symbols associated with multiple nodes of a layer of the Tunstall tree can be indexed accordingly, for example, according to the order of the decoded symbols of the layer (e.g., "0" indexes "A", "1" indexes "B", "2" indexes "C").
[0075] As an illustrative example, reference will now be made to Fig.12 The exemplary Tunstall tree shown in , describes the decoding of the input codeword "001" according to the LO decoding method of each second exemplary embodiment. First, the input codeword is compared with the three child nodes of the root node (codewords "000", "101", and "110"). In this regard, because the input codeword is between the codewords "000" and "101", the node assigned with the codeword "000" (which can be simply referred to as node "000") is selected (or determined), for example, as the determined node of this layer (layer 1) of the Tunstall tree. Because the rank (or order) of the codeword assigned to the selected node "000" at layer 1 is "0", (for example, because the index of the decoding symbol "A" at layer 1 is "0"), the input codeword is "A" (the first decoding symbol of the codeword) relative to the decoding symbol of layer 1. For example, at layer 1, the rank (or order) of codeword "000" is "0", the rank of codeword "101" is "1", and the rank of codeword "110" is "2". Repeating the process, node "000" of layer 2 is selected (or determined), and the second decoded symbol is "A" (which is the second decoded symbol of the codeword). Finally, as will be explained below, the decoded symbol of the input codeword with respect to layer 3 is "B" (which is the third and final decoded symbol of the codeword). Therefore, the input codeword "001" is decoded into the symbol sequence "AAB". Therefore, in each second exemplary embodiment, from Fig.12 As can be seen from the example Tunstall tree in Figure 1, the decoded symbols (uncompressed symbols) of a layer of the Tunstall tree can be derived based on the ranking of the selected nodes of that layer.
[0076] In various second example embodiments, the number of decoded symbols for a corresponding codeword may range from 2 to N, where N corresponds to the degree of the Tunstall tree.
[0077] Fig.13 An example method or algorithm (Algorithm 3) for determining the ranking (or order) of codewords associated with a selected node (or determined node) of a layer of a Tunstall tree according to each second example embodiment of the present invention is depicted, wherein c represents the input codeword being decoded, and nc represents the codewords assigned to multiple nodes at the layer. First, the input codeword (c) is compared with multiple codewords (nc) assigned to multiple nodes corresponding to the layer of the Tunstall tree. For example, for each of the multiple codewords assigned to the multiple nodes, the input codeword is compared with the codeword assigned to the node as to whether the input codeword is greater than or equal to the codeword assigned to the node (for example, if so, "1" is assigned, otherwise "0" is assigned). For example, the result of the comparison is represented by "compResult" in Algorithm 3. Then, based on the comparison, one of the multiple nodes can be determined as the determined node of the layer of the Tunstall tree. For example, this can be performed based on the "onehot" function shown in Algorithm 3, in which compResult is exclusive or (XOR) with an array consisting of "0" and the most significant bit (msb) to the second least significant bit (lsb) of compResult. The function is used to find the first "1" in compResult (for example, the onehot of "001" is "001", the onehot of "011" is "010", and the onehot of "111" is "100"). The onehot result can then be converted to an integer (r) (for example, "001" is converted to "0", "010" is converted to "1", and "100" is converted to "2") using, for example, a lookup table in a circuit, and such an integer represents the rank (or order) of the codeword associated with the selected node (or determined node) of the layer of the Tunstall tree. In this regard, the decoded symbol relative to the layer of the Tunstall tree can therefore be obtained (or retrieved) based on an index corresponding to (or equal to) the rank of the codeword associated with the selected node.
[0078] Fig.14A schematic flow chart of an LO decoding method 1400 according to each second example embodiment of the present invention is depicted. The LO decoding method 1400 includes: (at 1402) receiving an input codeword for decoding; (at 1404) reading codewords assigned to multiple nodes at a first level of a Tunstall tree; (at 1406) generating decoded symbols relative to the first level of the Tunstall tree, including determining a ranking (or order) of codewords associated with a selected node at the first level of the Tunstall tree; and (at 1408) determining whether the selected node at the first level of the Tunstall tree has an associated grandchild node (i.e., a node two levels below the selected node of the Tunstall tree). Fig.14 As shown, method 1400 performs an iterative loop including steps 1404, 1406, and 1408 for each layer of the Tunstall tree having a selected node determined to have an associated grandchild node, wherein each iteration starts from a next layer of the Tunstall tree relative to the current layer of the Tunstall tree. For example, method 1400 loops back to read (at 1404) the codewords assigned to multiple nodes of the next layer of the Tunstall tree (now becoming the current layer), and (at 1406) generate decoded symbols relative to the current layer of the Tunstall tree, including determining the ranking of the codeword associated with the selected node of the current layer of the Tunstall tree. If at 1408, the selected node of the current layer of the Tunstall tree does not have an associated grandchild node, then method 1400 includes (at 1410) determining whether the selected node of the current layer of the Tunstall tree has an associated child node (i.e., a node one layer below the selected node of the Tunstall tree). If so, the method 1400 includes determining a decoded symbol relative to the next level (relative to the current level) of the Tunstall tree by subtracting the codeword assigned to the selected node from the input codeword. For example, if the input codeword is "010" and the codeword assigned to the selected node is "000", then "010" minus "000" is "010" in binary or "2" in octal. In this regard, reference is made to Fig.12 In the example shown in , the decoded symbol at layer 3 with index "2" is "C", so "C" is the decoded symbol of the input codeword relative to layer 3. If not, then method 1400 returns (at 1414) the decoded symbol of the input codeword and ends. That is, the decoding of the input codeword has been completed.
[0079] As an illustrative example, Fig.15An example algorithm (Algorithm 4) for implementing the MO decoding method 700 according to each second example embodiment of the present invention is shown, wherein c represents the input codeword being decoded, nc represents an array of codewords assigned to multiple nodes in the layer of the Tunstall tree; ns represents an array of Boolean values indicating whether each of the multiple nodes in the layer of the Tunstall tree has a child node; na represents an array of node addresses in the layer of the Tunstall tree; and ng represents an array of Boolean values indicating whether each of the multiple nodes in the layer of the Tunstall tree has a grandchild node. In each second example embodiment, each node (n) has associated information including the assigned codeword (represented by nc), whether the node has a child node (represented by ns), and whether the node has a grandchild node (represented by na). As an illustrative example, Fig.16 An example data format for such information associated with a node(n) is depicted. Hardware Implementation
[0080] According to various second exemplary embodiments, there is provided an LO decoder for decoding a codeword of a Tunstall code, the decoder comprising one or more sub-decoders, a symbol memory, and a controller.
[0081] As a first example configuration or architecture, the LO decoder can be configured to have N pipeline stages. In this regard, the first N-1 pipeline stages can each include a subdecoder, and the last pipeline stage can include a subtractor. Each subdecoder can be used to decode a corresponding layer of the Tunstall tree to generate a corresponding decoded symbol relative to the corresponding layer of the Tunstall tree. For example, in the case where the Tunstall tree depth is greater than the number of subdecoders in the LO decoder, the last subdecoder in the N-1 pipeline stages can iteratively decode multiple layers of the Tunstall tree to generate corresponding decoded symbols for each of the multiple layers of the Tunstall tree. As an example, assuming that the Tunstall tree has 8 layers and the LO decoder has a pipeline including five subdecoders, then the last subdecoder in the pipeline can be used to iteratively decode 3 layers (i.e., layers 5 to 7) of the Tunstall tree when needed, and the last layer is decoded with a subtractor when needed.
[0082] As a second example configuration or architecture, the LO decoder can be configured to have only one sub-decoder, rather than the multiple sub-decoders in the first example described above. In this regard, the sub-decoder can be used to iteratively decode multiple layers of the Tunstall tree to produce corresponding decoded symbols relative to each of the multiple layers of the Tunstall tree. In this regard, a state machine can be provided to control the sub-decoder to iteratively decode each of the multiple layers of the Tunstall tree. Similar to the first example, the LO decoder can also include a subtractor.
[0083] Fig.17 A schematic diagram illustrating an example LO decoder 1700 having the above-described first example configuration or architecture according to various second example embodiments of the present invention is depicted. The LO decoder 1700 includes a sub-decoder 1704 for receiving an input codeword 1702 of a Tunstall code input to the LO decoder 1700 and outputting a decoded symbol 1705 of the input codeword 1702; a symbol memory 1706 for receiving and storing the decoded symbol 1705 of the input codeword 1702 from the sub-decoder 1704; and a controller 1708 for controlling the symbol memory 1706 to output one or more decoded symbols stored in the symbol memory 1706.
[0084] Fig.18 A schematic diagram showing an example architecture of a sub-decoder 1704 according to various second example embodiments of the present invention is depicted. The sub-decoder 1704 includes: a node memory 1802 for storing a plurality of codewords of the Tunstall code respectively assigned to a plurality of nodes for a plurality of nodes of the Tunstall tree of the Tunstall code corresponding to a first layer of the Tunstall tree; and a comparator 1806 for comparing an input codeword 1702 with a plurality of codewords assigned to a plurality of nodes corresponding to the first layer of the Tunstall tree received from the node memory 1802, and based on the comparison, generating a decoded symbol 1705 of the input codeword 1702 relative to the first layer of the Tunstall tree.
[0085] In each second example embodiment, the above-mentioned generation of the decoding symbol 1705 of the input codeword 1702 relative to the first layer of the Tunstall tree includes: based on the comparison, determining one of the multiple nodes corresponding to the first layer of the Tunstall tree as the determined node of the first layer of the Tunstall tree; and based on the determined node, generating the decoding symbol 1705 of the input codeword 1702 relative to the first layer of the Tunstall tree.
[0086] In each second exemplary embodiment, the subdecoder 1704 is further used to: determine whether the determined node of the first level of the Tunstall tree has an associated grandchild node, according to the Tunstall tree, the grandchild node is two levels below the determined node; and based on determining (i.e., if it is determined) that the determined node of the first level of the Tunstall tree has an associated grandchild node, obtain addresses 1808 (e.g., starting addresses) of a plurality of nodes of the Tunstall tree corresponding to a next level of the Tunstall tree relative to the first level of the Tunstall tree. For example, if Fig.16 As shown, the example data format of the node information associated with the node includes the starting address of the child nodes of the node (i.e., bits [10:13] of the example data format). In this regard, the above-mentioned address 1808 of the plurality of nodes corresponding to the next level of the Tunstall tree is the starting address of the child nodes of the determined node, and thus the above-mentioned address can be obtained from the node memory 1802.
[0087] In each second example embodiment, according to the first example configuration or architecture, the subdecoder 1704 also includes a selector 1807, which is communicatively coupled to the comparator 1806 and the node memory 1802, and the selector determines whether the determined node of the first layer of the Tunstall tree has an associated grandchild node, and obtains the addresses of multiple nodes of the Tunstall tree corresponding to the next layer of the Tunstall tree as mentioned above. For example, the selector 1807 can receive the decoded symbol 1705 from the comparator 1806 (based on the input address input to the subdecoder 1704), and receive the node information associated with the multiple nodes corresponding to the first layer of the Tunstall tree from the node memory 1802. For example, the selector 1807 may include a multiplexer, which is used to receive the decoded symbol 1705 as an input selection signal for selecting the node information associated with the determined node (of the multiple nodes) of the first layer of the Tunstall tree. In this regard, as will be referred to below Fig.16 As described, for example, the most significant bit of the node information indicates whether the node has a grandchild node (e.g., "1" indicates yes or true, and "0" indicates no or false). If the node has a grandchild node, bits [10:13] of the node information represent the starting address of the child node in the node memory 1802, and therefore, if it is determined that the determined node of the first level of the Tunstall tree has a grandchild node, the selector 1807 can obtain and output the starting address.
[0088] In each second example embodiment, according to the first example configuration or architecture, the LO decoder 1700 may also include one or more additional sub-decoders (e.g., Fig.17In the example shown, four additional sub-decoders 1704a, 1704b, 1704c, 1704d are shown. In this regard, sub-decoder 1704 is a first sub-decoder and is associated with a first layer of the Tunstall tree, and one or more additional sub-decoders 1704a, 1704b, 1704c, 1704d are associated with one or more additional layers of the Tunstall tree, respectively. In the same or similar manner as sub-decoder 1704, each of the one or more additional sub-decoders 1704a, 1704b, 1704c, 1704d can be used to receive an input codeword 1702 and output a decoded symbol 1710 (e.g., a decoded symbol 1710) relative to the input codeword 1702 of the layer of the Tunstall tree to which the additional sub-decoder is associated. Fig.17 The additional sub-decoders 1704a, 1704b, 1704c, 1704d in the example shown in FIG. 1 are used to output decoded symbols 1705a, 1705b, 1705c, 1705d, respectively. Fig.18 In a manner similar to or identical to the sub-decoder 1704 described above, each additional sub-decoder may further include: a node memory for storing a plurality of codewords of the Tunstall code respectively assigned to a plurality of nodes of the Tunstall tree of the Tunstall code corresponding to the layer of the Tunstall tree associated with the additional sub-decoder; and a comparator for comparing the input codeword with the plurality of codewords assigned to the plurality of nodes corresponding to the layer of the Tunstall tree associated with the additional sub-decoder received from the node memory, and generating a decoded symbol of the input codeword relative to the layer of the Tunstall tree associated with the additional sub-decoder based on the comparison. That is, the codeword may be decoded by comparing the input codeword with the codeword of the layer of the Tunstall tree associated with the additional sub-decoder. Fig.18 Each additional sub-decoder is configured in the same or similar manner as the sub-decoder 1704 described according to the second exemplary embodiment.
[0089] Therefore, in a manner identical or similar to the sub-decoder 1704, the above-mentioned generation of decoding symbols of the input codeword 1702 of the layer of the Tunstall tree associated with the additional sub-decoder includes: based on the comparison, determining one of the multiple nodes corresponding to the layer of the Tunstall tree as a determined node of the layer of the Tunstall tree; and based on the determined node, generating decoding symbols of the input codeword 1702 of the layer of the Tunstall tree associated with the additional sub-decoder.
[0090] In each second example embodiment, according to the first example configuration or architecture, the first sub-decoder 1704 is used to output the addresses 1808 of multiple nodes corresponding to the next layer of the Tunstall tree relative to the first layer of the Tunstall tree to the additional sub-decoder 1704a in one or more additional sub-decoders 1704a, 1704b, 1704c, 1704d associated with the next layer of the Tunstall tree based on determining (i.e., if it is determined) that the determined node of the first layer of the Tunstall tree associated with the first sub-decoder 1704 has an associated grandchild node, so that the comparator of the additional sub-decoder 1704a compares the input codeword 1702 with the multiple codewords assigned to the multiple nodes corresponding to the next layer of the Tunstall tree received from the node memory of the additional sub-decoder 1704a based on the received addresses 1808 of the multiple nodes, and based on the comparison, generates a decoding symbol 1705a of the input codeword 1702 relative to the next layer of the Tunstall tree associated with the additional sub-decoder 1704a. In this regard, the additional sub-decoder 1704a is further configured to output, to the symbol memory 1706, decoded symbols 1705a of the input codeword 1702 of the next layer of the Tunstall tree associated with the additional sub-decoder 1704a.
[0091] In each second example embodiment, according to the first example configuration or architecture, in the same or similar manner as the sub-decoder 1704, each of the one or more additional sub-decoders 1704a, 1704b, 1704c, 1704d is used to output addresses of a plurality of nodes corresponding to a next layer of the Tunstall tree relative to the layer of the Tunstall tree to another additional sub-decoder (e.g., an additional sub-decoder) associated with the next layer of the Tunstall tree based on determining (i.e., if it is determined) that the determined node of the layer of the Tunstall tree associated with the additional sub-decoder has an associated grandchild node. The additional sub-decoder 1704a may output the address corresponding to the next layer to the additional sub-decoder 1704b, and the additional sub-decoder 1704b may output the address corresponding to the next layer to the additional sub-decoder 1704c, and so on), so that the comparator of another additional sub-decoder compares the input codeword 1702 with the multiple codewords assigned to the multiple nodes corresponding to the next layer of the Tunstall tree received from the node memory of another additional sub-decoder based on the received addresses of the multiple nodes, and generates a decoded symbol of the input codeword 1702 of the next layer of the Tunstall tree based on the comparison. In this regard, the other additional sub-decoder is also used to output the decoded symbol of the input codeword 1702 of the next layer of the Tunstall tree to the symbol memory 1706. Therefore, in the same or similar manner as the first sub-decoder 1704, each additional sub-decoder may also include a selector that is communicatively coupled to the comparator and the node memory of the additional sub-decoder and is used to determine whether a determined node of a layer of the Tunstall tree associated with the additional sub-decoder has an associated grandchild node, and is used to obtain the above-mentioned addresses of multiple nodes corresponding to the next layer of the Tunstall tree relative to the layer of the Tunstall tree.
[0092] In each second exemplary embodiment, as described above, in the case where the Tunstall tree depth is greater than the number of sub-decoders in the LO decoder 1700, the last sub-decoder in the N-1 pipeline stages (e.g., Fig.17 The additional sub-decoder 1704d) in the illustrated example may iteratively decode multiple layers of the Tunstall tree to produce corresponding decoded symbols for each of the multiple layers of the Tunstall tree.
[0093] In each second example embodiment, according to the first example configuration or architecture, each of the first subdecoder 1704 and the one or more additional subdecoders 1704a, 1704b, 1704c, 1704d is used to determine (e.g., by a selector of the subdecoder) whether the determination node of the layer of the Tunstall tree associated with the subdecoder has an associated child node based on determining that the determination node of the layer of the Tunstall tree associated with the subdecoder does not have an associated grandchild node, wherein the child node is one layer below the determination node according to the Tunstall tree. In this regard, the LO decoder 1700 also includes a subtractor 1712, which is used to generate a decoded symbol 1705e relative to the input codeword 1702 of the next layer of the layer Tunstall tree relative to the Tunstall tree based on the input codeword 1702 and the codeword assigned to the determination node of the layer of the Tunstall tree associated with the subdecoder.
[0094] As described above, according to the above second example configuration or architecture, the LO decoder can be configured to have only one sub-decoder 1704, instead of Fig.17 1704a, 1704b, 1704c, 1704d). In this regard, the subdecoder can be used to iteratively decode multiple layers of the Tunstall tree to generate corresponding decoded symbols relative to each of the multiple layers of the Tunstall tree. As an example, assuming that the Tunstall tree has 8 layers, then one subdecoder 1704 can be used to iteratively decode 7 layers (i.e., layers 1 to 7) of the Tunstall tree when necessary, and the last layer is decoded with a subtractor when necessary. In this regard, a state machine can be provided (e.g., executed by the controller 1708) for controlling the subdecoder 1704 to iteratively decode each of the multiple layers of the Tunstall tree.
[0095] In various second exemplary embodiments, according to the second exemplary configuration or architecture, the node memory 1802 can be used to store multiple codewords of the Tunstall code respectively assigned to multiple nodes corresponding to the layers of the Tunstall tree for each of the multiple layers of the Tunstall tree, including the first layer of the Tunstall tree mentioned above, and for multiple nodes of the Tunstall tree corresponding to the layers of the Tunstall tree. In this regard, the comparator 1806 is used to determine that the determination node of the layer of the Tunstall tree has an associated grandchild node based on the sub-decoder 1704 (e.g., the selector 1807 of the sub-decoder 1704): based on the addresses of the multiple nodes of the Tunstall tree corresponding to the next layer of the Tunstall tree, compare the input codeword with the multiple codewords assigned to the multiple nodes of the Tunstall tree corresponding to the next layer of the Tunstall tree relative to the layer of the Tunstall tree received from the node memory 1802; based on the comparison, determine one of the multiple nodes corresponding to the next layer of the Tunstall tree as the determination node of the next layer of the Tunstall tree; and based on the determination node of the next layer of the Tunstall tree, generate a decoded symbol of the input codeword relative to the next layer of the Tunstall tree. In this regard, the sub-decoder 1704 is also used to output the decoded symbol 1705 of the input codeword 1702 relative to the next layer of the Tunstall tree to the symbol memory 1706.
[0096] In each second example embodiment, according to the second example configuration or architecture, the subdecoder 1704 is further configured to determine whether the determination node of the layer of the Tunstall tree has an associated child node (e.g., by the selector 1807 of the subdecoder 1704), based on determining that the determination node of the layer of the Tunstall tree does not have an associated grandchild node, wherein the child node is one layer below the determination node according to the Tunstall tree. In this regard, the LO decoder also includes a subtractor 1712, which is configured to generate a decoded symbol of the input codeword 1702 of the next layer of the Tunstall tree relative to the layer of the Tunstall tree based on the determination by the subdecoder 1704 that the determination node of the layer of the Tunstall tree has an associated child node and, based on the input codeword 1702 and the codeword assigned to the determination node of the layer of the Tunstall tree.
[0097] Therefore, in each second exemplary embodiment, each sub-decoder may be used to decode one symbol per clock cycle. As described above, each sub-decoder may include a comparator 1806, a selector 1807, and a node memory 1802. As described above, the comparator 1806 is used to compare the input codeword 1702 with the codewords assigned to a plurality of nodes corresponding to the layers of the Tunstall tree (i.e., all the stored codewords corresponding to the layers of the Tunstall tree), and output a decoded symbol relative to such a layer of the Tunstall tree. For example, Fig.13 As described in Algorithm 3 shown, comparator 1806 can be used to generate "1" or "0" when the stored codeword is less than or greater than / equal to the input codeword 1702, respectively. The outputs of all comparisons of the layers of the Tunstall tree are then one-hot encoded to generate corresponding decoded symbols. As described above, selector 1807 can be used to select the corresponding address of the next-level sub-decoder based on the decoded symbol of the current-level sub-decoder.
[0098] As described above, the node storage 1802 can store information of multiple nodes of the Tunstall tree, including, for each of the multiple nodes, a codeword assigned to the node, whether the node has child nodes, and whether the node has grandchild nodes. An example data format of such information associated with a node is as follows: Fig.16 Subtractor 1712 may be the last module in the pipeline of the LO decoder and is used to, when necessary, for example, by performing Fig.15 The operation of line 12 of Algorithm 4 shown in is to decode the leaf node. Specifically, the subtractor 1712 can be used to subtract the codeword of the determined node from the input codeword 1702 and output the last decoded symbol 1705e of the input codeword 1702.
[0099] The symbol memory 1706 may include a plurality of memory groups (or a plurality of register files), each memory group (or each register file) being used to store decoded symbols (decoding weights) (up to N-1 decoded symbols 1705, 1705a, 1705b, 1705c, 1705d) of corresponding codewords from subdecoders (up to N-1 subdecoders 1704, 1704a, 1704b, 1704c, 1704d) during decoding. Once the last decoded symbol 1705e of the input codeword 1702 is obtained from the subtractor 1712, or the plurality of memory groups are full, the controller 1708 may control the symbol memory 1706 to output (or read out) the decoded symbols of the input codeword 1702 stored in the corresponding memory group of the symbol memory 1706. Therefore, the controller 1708 may be used to control the symbol memory 1706 to output the decoded symbols of the corresponding codewords stored in the memory group for each of the plurality of memory groups. In this regard, the controller 1708 may be used to control the writing and reading of multiple memory groups of the symbol memory 1706. For example, in one cycle, the controller 1708 may write N-1 symbols decoded by N-1 sub-decoders into corresponding N-1 memory groups, and output N-1 decoded symbols of a codeword from another memory group and the last decoded symbol of the codeword from the subtractor 1712. If the number of symbols encoded into one codeword is greater than the number of sub-decoders in the decoder, the controller 1708 may be used to turn off the clocks of the sub-decoders except the last sub-decoder 1705d, while waiting for the last sub-decoder 1705d to iteratively decode additional symbols as described above.
[0100] In each second exemplary embodiment, the width of the decoded symbol may be 5 bits, the number of decoded symbols of a codeword may range from 2 to 6, and the width of the codeword may be 10 bits. Fig.16 The example data format of the node information associated with the node shown, in each second example embodiment, the example data format of the node information can be 15 bits, where bits [9:0] represent the codeword assigned to the node. The most significant bit indicates whether the node has a grandchild node (that is, whether the node's child node has a child node), for example, "1" indicates yes or true, and "0" indicates no or false. If the node has a grandchild node, bits [10:13] represent the starting address of the child node in the node memory (for example, the start / beginning of all addresses of the child node). Otherwise (for example, if the most significant bit indicates "0"), bit 10 indicates whether the node has a child node, for example, "1" indicates yes or true, and "0" indicates no or false. Those skilled in the art will understand that the present invention is not limited to Fig.16 The example sizes of the example data formats shown in are shown in FIG. , and data formats of various sizes may be employed as desired or appropriate, for example, based on the size of the codebook.
[0101] In each second exemplary embodiment, if one decoded symbol (or uncompressed symbol) per cycle is desired or required, the LO decoder may further include a multiple write port first-in-first-out (FIFO) buffer for converting parallel decoded symbols into sequential decoded symbols.
[0102] In each first exemplary embodiment, for the MO decoder 900, regarding the type of on-chip memory 906, for example, SRAM or register group can be selected as the on-chip memory. In this regard, SRAM is smaller but slower than the register group. Therefore, those skilled in the art can understand that the type of on-chip memory 906 can be selected as desired or appropriately according to various factors.
[0103] In each first exemplary embodiment, for the MO decoder 900, regarding the number of on-chip memory entries, from Figure 8 As can be seen from Algorithm 1 shown in , the more memory entries there are, the fewer decoding cycles are required. This is because the codebook pairs stored on the chip (e.g., Fig.10 906) and read fewer from off-chip, which is slower but consumes more energy. However, when the dictionary is too large to store on-chip or the area is limited, only a predetermined number (M) of the highest frequency codebook pairs are stored on-chip, where M corresponds to the number of memory entries of the on-chip memory 906. In this regard, as described above with reference to Fig. 11B As described, the pairs in the codebook are sorted by frequency, and the values are remapped according to the order. Therefore, pairs with low frequency can be read from off-chip. Therefore, those skilled in the art can understand that the number of on-chip memory entries can be determined as desired or appropriate according to various factors.
[0104] In each second exemplary embodiment, for the LO decoder, regarding the number of node memory groups in one sub-decoder, from Fig.13As can be seen from Algorithm 3 in , for example, the number of node memory groups in a sub-decoder reflects the maximum value of the weight for which the sub-decoder works well. If the number of node memory groups exceeds the maximum value of the weight, the LO decoder requires additional cycles to narrow the range of symbols and decode. For example, the maximum value of the rank output from Algorithm 3 may be equal to the number of node memory groups in a sub-decoder. If the maximum value of the weight is not greater than the number of node memory groups, the output from Algorithm 3 is the corresponding decoded symbol (e.g., the uncompressed weight). Otherwise, for example, the weight is from 0 to 63, and there are only 8 node memory groups. Therefore, log(8)(64)=2 runs are required to obtain the final result. The range (0,63) can be divided into 8 sub-ranges (0,7), (8,15), ..., (56,63). For example, in the first run, the result is one of the sub-ranges indexed by the rank (e.g., if the rank = 5, the result is (40,47)). In the second run, the result is one of the values of the sub-range indexed by the rank (e.g., for the sub-range (40,47), if the rank = 5, the result is 45).
[0105] In each second exemplary embodiment, for the LO decoder, regarding the number of sub-decoders, for example, each sub-decoder can execute in parallel Fig.15 The operation of Algorithm 4 shown in . Therefore, the number of sub-decoders limits the maximum number of symbols that can be decoded per cycle. Therefore, more sub-decoders can be provided to reduce the decoding time, however, this will lead to an increase in the use of hardware resources. Therefore, it can be understood by those skilled in the art that the number of sub-decoders can be determined as desired or appropriately according to various factors. Performance Evaluation
[0106] In various example embodiments, the MO and LO decoders are synthesized by Vivado and run on the FPGA.
[0107] Fig.19 A table (Table II) showing resource usage and corresponding clock cycles for decoding ResNet-50 and MobileNet-v2 CNNs with MO decoders of different memory entries and memory types is depicted. For example, it can be seen that when decoding ResNet-50 and MobileNet-v2, the 64-entry MO decoder reduces the memory capacity by 16 times compared to the 1024-entry MO decoder, while consuming 2.36 times and 2.49 times more cycles.
[0108] Fig. 20A table showing resource usage and corresponding cycles of LO decoders decoding ResNet-50 and MobileNet-V2 CNNs with different numbers of sub-decoders and node memory groups is depicted (Table III). For example, it can be seen that reducing the number of node memory groups from 27 to 10 reduces resource usage by about 20% while increasing decoding cycles by 3%. In addition, adding 7 sub-decoders can increase the speed by about 3 times. System-level impact on deep learning accelerators System Diagram
[0109] In various example embodiments, the MO or LO decoder may be integrated into a system-on-chip (SoC) platform (eg, a Pulpissimo SoC platform). Fig.21 A schematic diagram of a deep learning accelerator system 2100 according to various example embodiments of the present invention is depicted. The system 2100 includes a deep learning accelerator 2102 according to various example embodiments of the present invention, a memory 2104, and a decoder 2106, such as a MO or LO decoder as described above according to various example embodiments. Fig.21 As shown, the deep learning accelerator 2102 is used to directly receive input activations from the memory 2104. The decoder 2106 is used to receive compressed weights (corresponding to the input codewords as described above) and decode the compressed weights into uncompressed weights (corresponding to the decoded symbols as described above) of the deep learning accelerator 2102. Therefore, when performing inference, the input activations can be sent directly from the memory 2104 to the deep learning accelerator 2102, and the compressed weights can be extracted from the memory 2104 and decoded by the decoder 2106. The uncompressed weights decoded by the decoder 2016 can then be sent to the deep learning accelerator 2102. Evaluation results
[0110] It is assumed that the uncompressed weights require slow but large memory (e.g., eDRAM, which consumes 15pJ / byte). In addition, in the compressed case, the codebook and codewords are stored in on-chip memory (e.g., MRAM, which consumes 2pJ / byte), and decoding is done by reading the on-chip LUT (register file, which consumes 0.2pJ / byte). Each read access is also assumed to be a 32-bit word. In addition, it is assumed that the decoder clock and the accelerator clock frequency are the same.
[0111] Regarding main memory entry requirements, Fig. 22A table summarizing the memory capacity requirements for storing the weights of ResNet-50 and MobileNet-v2 CNNs with different encoding methods is depicted (Table IV). Specifically, 32-bit floating point, quantized 8-bit and 4-bit, two Tunstall encodings and Huffman encodings as described above according to various example embodiments are considered. It can be seen that the capacity required for Tunstall and Huffman encodings is reduced by 18.0 times and 2.3 times on average, respectively, compared to full precision and 4-bit weights. According to various example embodiments of the present invention, Huffman encoding occupies about 4% less capacity than both Tunstall MO and LO encodings.
[0112] Regarding the estimated memory access power consumption, Fig. 22 Table IV in also shows the estimated power consumption required to access all weights. From Table IV, it can be seen that for ResNet-50, the LO decoder consumes 2.7% of the energy to access all weights compared to full precision and 21.9% of the energy to access all weights compared to the state-of-the-art quantized 4-bit weights. For example, the MO decoder consumes only 0.9% and 7.6% compared to full precision and quantized 4-bit weights, respectively. It can also be seen that the power consumption of the MO decoder is similar to that of the Huffman decoder.
[0113] Regarding the weight reading clock cycle, Fig.23 A table showing memory access cycles in different approaches is depicted (Table V). For example, the LO decoder reads weights 9.9 times faster on ResNet-50 and 13.9 times faster on MobileNet-v2 compared to off-chip DRAM.
[0114] Regarding resource utilization overhead, after the decoder 2106 was integrated into Pulpissimo, they were synthesized by Vivado2020.1 without optimization. The MO 1024RAM decoder and LO 8SD 27-group decoder only bring 2.16% and 8.14% resource overhead, which is affordable. The MO 64RAM decoder only brings 0.48% overhead for such a system. Comparing Tunstall and Huffman Coding
[0115] As an example, the performance of a 7-level 24 memory bank LO Tunstall decoder and a 1024 entry MO Tunstall decoder will be compared with a 256 entry Huffman decoder. The Huffman decoder decodes one weight per clock cycle. Fig.24 A table (Table VI) showing the performance of the above-mentioned LO Tunstall decoder and MO Tunstall decoder for ResNet-50 is depicted. Codeword storage
[0116] Fig.25A and Fig.25B Two example Tunstall and Huffman codewords stored in two 32-bit main memory cells are shown respectively. Fig.25A There are 6 10-bit Tunstall codewords in , which can be decoded into 17 uncompressed symbols, and Fig.25B The number of bits of the Huffman codewords in varies from 1 to 7, which is very irregular. These 64-bit Huffman codewords represent 18 uncompressed symbols. In FIG. 25 , the two most significant bits represent the number of valid codewords in the 32-bit memory cell. Decode clock cycle
[0117] Regarding software decoding time, Fig.26 A table showing Tunstall and Huffman software decoding times (in microseconds) is depicted (Table VII). It can be seen that although the Huffman decoder outperforms the Tunstall decoder in some small simple layers, for the same CNN, Tunstall encoding is decoded about 1.5 times faster than Huffman encoding.
[0118] Regarding the hardware decoding clock cycle, such as Fig.24 As shown in Table VI, in ResNet-50, the 1024-entry MOTunstall decoder and the 7-level LO Tunstall decoder are 6.23 times and 3.29 times faster than the Huffman decoder.
[0119] Therefore, the decoders according to the embodiments and example embodiments of the present invention can be advantageously used in many commercial applications. For example, two hardware accelerated decoders are designed and implemented according to the example embodiments of the present invention, namely, a memory-oriented (MO) Tunstall decoder and a logic-oriented (LO) Tunstall decoder. For example, as described above, the decoder can be used for real-time decoding of compressed neural network parameters. For example, this is important for applying artificial intelligence in ultra-low power platforms. For example, as described above, the example embodiments adopt Tunstall encoding to further compress weights after state-of-the-art quantization. For example, compared with the full-precision 32-bit network, the MO decoder reduces memory usage by 19.58 times and 21.67 times at the inference level of ResNet-50 and MobileNet-v2, respectively, while the LO decoder reduces it by 19.76 times and 22.08 times, respectively. In addition, it is found that the MO and LO decoders are about 6 times and 3 times faster than Huffman encoding. Therefore, for example, this shows that the Tunstall decoder according to various example embodiments of the present invention is suitable for integration into a deep learning accelerator to significantly reduce memory usage while bringing little delay and resource overhead. However, it will be appreciated by those skilled in the art that decoders according to various embodiments or example embodiments of the present invention, such as on-the-fly decoding, are not limited to neural networks and artificial intelligence, and may be applied to various applications as desired or appropriate, such as providing an additional layer of security to further protect private keys and sensitive data.
[0120] Although the embodiments of the present invention have been specifically shown and described with reference to specific embodiments, it should be understood by those skilled in the art that various changes may be made in form and detail without departing from the spirit and scope of the present invention as defined in the appended claims. Therefore, the scope of the present invention is specified by the appended claims, and all changes within the meaning and scope of the equivalents of the claims are encompassed.
Claims
1. A decoder for decoding a code word of a Tunstall code, the decoder comprising: a sub-decoder, configured to receive an input codeword of the Tunstall code input to the decoder, and output a decoded symbol of the input codeword; a symbol memory for receiving and storing the decoded symbols of the input codewords from the sub-decoder; as well as a controller, configured to control the symbol memory to output one or more decoded symbols stored in the symbol memory, Wherein, the sub-decoder comprises: a node memory for storing, for a plurality of nodes of the Tunstall tree of the Tunstall code corresponding to a first layer of the Tunstall tree, a plurality of code words of the Tunstall code respectively assigned to the plurality of nodes; and A comparator is configured to compare the input codeword with the plurality of codewords received from the node memory and assigned to the plurality of nodes corresponding to the first layer of the Tunstall tree, and based on the comparison, generate the decoded symbol of the input codeword relative to the first layer of the Tunstall tree.
2. The decoder according to claim 1, wherein: Generating the decoded symbols for the input codewords of the first layer of the Tunstall tree comprises: Based on the comparison, determining one of the plurality of nodes corresponding to the first layer of the Tunstall tree as a determined node of the first layer of the Tunstall tree; and Based on the determined node, the decoded symbol of the input codeword relative to the first layer of the Tunstall tree is generated.
3. The decoder according to claim 2, wherein: The sub-decoder is also used for: Determining whether the determined node of the first level of the Tunstall tree has an associated grandchild node, the grandchild node being two levels below the determined node according to the Tunstall tree; as well as Based on determining that the determined node of the first layer of the Tunstall tree has the associated grandchild node, addresses of a plurality of nodes of the Tunstall tree corresponding to a next layer of the Tunstall tree relative to the first layer of the Tunstall tree are obtained.
4. The decoder according to claim 3, wherein: The sub-decoder also includes a selector, which is communicatively coupled to the comparator and the node memory, and the selector is used to determine whether the determined node of the first layer of the Tunstall tree has an associated grandchild node, and to obtain the addresses of the multiple nodes of the Tunstall tree corresponding to the next layer of the Tunstall tree.
5. A decoder according to claim 3 or 4, further comprising one or more additional sub-decoders, wherein the sub-decoder being a first sub-decoder associated with the first layer of the Tunstall tree, The one or more additional sub-decoders are respectively associated with one or more additional layers of the Tunstall tree, Each of the one or more additional sub-decoders is configured to receive the input codeword and output a decoded symbol of the input codeword relative to the layer of the Tunstall tree to which the additional sub-decoder is associated, and comprises: a node memory for storing, for a plurality of nodes of the Tunstall tree of the Tunstall code corresponding to the layer of the Tunstall tree associated with the additional sub-decoder, a plurality of code words of the Tunstall code respectively assigned to the plurality of nodes; as well as A comparator for comparing the input codeword with the plurality of codewords received from the node memory and assigned to the plurality of nodes corresponding to the layer of the Tunstall tree associated with the additional sub-decoder, and generating a decoded symbol of the input codeword relative to the layer of the Tunstall tree associated with the additional sub-decoder based on the comparison.
6. The decoder according to claim 5, wherein: Generating the decoded symbols for the input codewords of the layer of the Tunstall tree associated with the additional sub-decoder comprises: Based on the comparison, determining one of the plurality of nodes corresponding to the layer of the Tunstall tree as a determined node of the layer of the Tunstall tree; and Based on the determined node, the decoded symbol for the input codeword of the layer of the Tunstall tree associated with the additional sub-decoder is generated.
7. The decoder according to claim 6, wherein The first sub-decoder is used to output the addresses of the multiple nodes corresponding to the next layer of the Tunstall tree relative to the first layer of the Tunstall tree to an additional sub-decoder among the one or more additional sub-decoders associated with the next layer of the Tunstall tree based on determining that the determined node of the first layer of the Tunstall tree associated with the first sub-decoder has the associated grandchild node, so that the comparator of the additional sub-decoder compares the input codeword with the multiple codewords assigned to the multiple nodes corresponding to the next layer of the Tunstall tree received from the node memory of the additional sub-decoder based on the received addresses of the multiple nodes, and generates the decoded symbol of the input codeword relative to the next layer of the Tunstall tree associated with the additional sub-decoder based on the comparison, and The additional sub-decoder is further configured to output the decoded symbol of the input codeword of the next layer of the Tunstall tree associated with the additional sub-decoder to the symbol memory.
8. A decoder according to claim 6 or 7, wherein Each of the one or more additional sub-decoders is configured to output addresses of the plurality of nodes corresponding to a next layer of the Tunstall tree relative to the layer of the Tunstall tree to another additional sub-decoder associated with the next layer of the Tunstall tree based on determining that the determined node of the layer of the Tunstall tree associated with the additional sub-decoder has the associated grandchild node, so that the comparator of the another additional sub-decoder compares the input codeword with the plurality of codewords assigned to the plurality of nodes corresponding to the next layer of the Tunstall tree received from the node memory of the another additional sub-decoder based on the received addresses of the plurality of nodes, and generates the decoded symbol of the input codeword relative to the next layer of the Tunstall tree based on the comparison, and The another additional sub-decoder is further configured to output the decoded symbol of the input codeword relative to the next layer of the Tunstall tree to the symbol memory.
9. The decoder according to claim 6, wherein The first sub-decoder and each of the one or more additional sub-decoders are configured to determine whether the determined node of the layer of the Tunstall tree associated with the sub-decoder has an associated child node based on determining that the determined node of the layer of the Tunstall tree associated with the sub-decoder does not have an associated grandchild node, wherein the child node is one layer below the determined node according to the Tunstall tree, and The decoder also includes a subtractor for determining, based on the subdecoder, that the determined node of the layer of the Tunstall tree associated with the subdecoder has the associated child node, and generating a decoding symbol relative to the input codeword of the next layer of the Tunstall tree relative to the layer of the Tunstall tree based on the input codeword and the codeword assigned to the determined node of the layer of the Tunstall tree associated with the subdecoder.
10. The decoder according to claim 3 or 4, wherein: The node memory is used to store multiple codewords of the Tunstall code respectively assigned to the multiple nodes corresponding to the layer of the Tunstall tree for each of the multiple layers of the Tunstall tree, including the first layer of the Tunstall tree, and for the multiple nodes of the Tunstall tree corresponding to the layer of the Tunstall tree.
11. The decoder according to claim 10, wherein The comparator is used to determine, based on the sub-decoder, that the determined node of the layer of the Tunstall tree has the associated grandchild node: comparing the input codeword with the plurality of codewords received from the node memory and assigned to the plurality of nodes of the Tunstall tree corresponding to the next layer of the Tunstall tree relative to the layer of the Tunstall tree, based on addresses of the plurality of nodes of the Tunstall tree corresponding to the next layer of the Tunstall tree; Based on the comparison, determining one of the plurality of nodes corresponding to the next layer of the Tunstall tree as a determined node of the next layer of the Tunstall tree; as well as generating a decoded symbol for the input codeword of the next layer of the Tunstall tree based on the determined node of the next layer of the Tunstall tree, and The sub-decoder is further configured to output the decoded symbol of the input codeword of the next layer of the Tunstall tree to the symbol memory.
12. The decoder of claim 10, wherein The sub-decoder is further configured to determine whether the determined node of the layer of the Tunstall tree has an associated child node based on determining that the determined node of the layer of the Tunstall tree does not have the associated grandchild node, wherein according to the Tunstall tree, the child node is one layer below the determined node, and The decoder also includes a subtractor for determining, based on the subdecoder, that the determined node of the layer of the Tunstall tree has the associated child node, and generating a decoded symbol relative to the input codeword of the next layer of the Tunstall tree relative to the layer of the Tunstall tree based on the input codeword and the codeword assigned to the determined node of the layer of the Tunstall tree.
13. A decoder according to any one of claims 1 to 12, wherein The symbol memory includes a plurality of memory groups, each memory group is used to store one or more decoded symbols corresponding to the codeword, and The controller is configured to control the symbol memory to output the one or more decoded symbols of the corresponding codeword stored in the memory group for each of the plurality of memory groups.
14. A decoder according to any one of claims 1 to 13, wherein The input codewords correspond to compressed quantized weights of the neural network, and The one or more decoded symbols of the input codeword stored in the symbol memory and output from the decoder correspond to uncompressed quantized weights of the neural network.
15. The decoder according to any one of claims 1 to 14, wherein: The decoder is an integrated circuit.
16. A method of operating a decoder for decoding a codeword of a Tunstall code according to any one of claims 1 to 15, the method comprising: The subdecoder of the decoder receives an input codeword of the Tunstall code input to the decoder; The sub-decoder outputs a decoded symbol of the input codeword; The symbol memory of the decoder receives and stores the decoded symbols of the input codewords from the sub-decoder; as well as The controller of the decoder controls the symbol memory to output one or more decoded symbols stored in the symbol memory, The method further includes the comparator of the sub-decoder comparing the input codeword with the multiple codewords received from the node memory of the sub-decoder and assigned to the multiple nodes corresponding to the first layer of the Tunstall tree, and the comparator generating the decoding symbol of the input codeword relative to the first layer of the Tunstall tree based on the comparison.
17. A decoder for decoding a codeword of a Tunstall code, the decoder comprising: a symbol memory, comprising a plurality of memory entries, each memory entry storing one or more decoded symbols of a codeword of the Tunstall code corresponding to the memory entry; as well as A controller is configured to receive an input codeword of the Tunstall code input to the decoder, and control the symbol memory to output the one or more decoded symbols stored in one memory entry of the plurality of memory entries corresponding to the input codeword.
18. The decoder according to claim 17, wherein: Controlling the symbol memory to output the one or more decoded symbols stored in the one of the plurality of memory entries corresponding to the input codeword serves as an address of the one of the plurality of memory entries based on the input codeword.
19. A decoder according to claim 17 or 18, wherein The symbol memory is an on-chip memory, The controller is communicatively coupled to the symbol memory and the off-chip memory, wherein, The plurality of memory entries of the symbol memory respectively store the one or more decoded symbols of the first plurality of codewords of the Tunstall code, and the off-chip memory stores one or more decoded symbols of the codeword for each of the second plurality of codewords of the Tunstall code, and The controller is used to determine whether the one or more decoded symbols of the input codeword are located in the symbol memory or the off-chip memory, and control the determined one of the symbol memory and the off-chip memory to output the one or more decoded symbols of the input codeword.
20. The decoder of claim 19, wherein: The plurality of memory entries of the symbol memory are respectively updated with one or more decoded symbols of a plurality of remapped codewords located in a plurality of codebook entries of an updated codebook, replacing a plurality of codewords having a highest frequency count in a previous plurality of codebook entries.
21. A decoder according to any one of claims 16 to 20, wherein: Each of the plurality of memory entries of the symbol memory further stores a symbol number parameter indicating a number of decoded symbols of a corresponding codeword stored in the memory entry.
22. A decoder according to any one of claims 16 to 21, wherein The input codewords correspond to compressed quantized weights of the neural network, and The one or more decoded symbols stored in each of the plurality of memory entries of the symbol memory and the one or more decoded symbols of the input codeword output from the decoder correspond to uncompressed quantized weights of the neural network.
23. A decoder according to any one of claims 16 to 22, wherein: The decoder is an integrated circuit.
24. A method of operating a decoder for decoding a codeword of a Tunstall code according to any one of claims 16 to 23, the method comprising: The controller of the decoder receives an input codeword of the Tunstall code; as well as The controller of the decoder controls the symbol memory of the decoder to output one or more decoded symbols stored in one memory entry of the plurality of memory entries corresponding to the input codeword.
