Inline decompression

Abstract

Techniques and apparatus for decompressing stack-compressed data are described. Stack compression refers to the compression of data in one or more dimensions. Stack compression can be effective for very sparse uncompressed data blocks, i.e., data blocks containing many zeros. In stack compression, an uncompressed data block is compressed into a compressed data block by removing one or more zero words from the uncompressed data block. Map metadata is generated during compression that maps the zero words in the uncompressed data block. By using the map metadata, the compressed data block can be decompressed to restore the uncompressed data block.

Description

[Technical Field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This patent application claims the benefit of U.S. Non-Provisional Application No. 16 / 870,873, entitled "INLINE DECOMPRESSION," filed May 8, 2020, which is assigned to the assignee of the present application and is expressly incorporated herein by reference in its entirety.

[0002] Various aspects described herein relate generally to compression and decompression, and more particularly to inline decompression of vertically and / or horizontally compressed data. [Background technology]

[0003] Neural networks (NNs) tend to process data with many zeros. This can be particularly true in NNs that use a rectified linear unit (ReLU) activation function, which outputs zero when the input value is less than or equal to zero and the input value when the input value is positive. Also, if the network is pruned, many zeros can be present. For example, in a pruned recurrent neural network (RNN), nearly 90% of the weights can be pruned. Weights in a convolutional neural network (CNN) can similarly be sparse. Summary of the Invention [Means for solving the problem]

[0004] This Summary identifies features of some exemplary embodiments and is not an exclusive or comprehensive description of the disclosed subject matter. Whether a feature or aspect is included in or omitted from this Summary is not intended as an indication of the relative importance of such features. Additional features and aspects will be described or become apparent to those skilled in the art upon reading the following detailed description and viewing the drawings that form a part hereof.

[0005] An exemplary apparatus is disclosed. The apparatus may include a processor and a memory. The processor and memory may be configured to retrieve a compressed data block and map metadata. The compressed data block may include one or more words. The map metadata may be configured to map the words of the compressed data block to a generated decompressed data block. The processor and memory may also be configured to decompress the compressed data block to generate a decompressed data block according to the map metadata.

[0006] An exemplary method is disclosed. The method may include retrieving a compressed data block and map metadata. The compressed data block may include one or more words. The map metadata may be configured to map the words of the compressed data block to generated decompressed data blocks. The method may also include decompressing the compressed data block to generate decompressed data blocks according to the map metadata.

[0007] Another exemplary apparatus is disclosed. The apparatus may include means for retrieving a compressed data block and map metadata. The compressed data block may include one or more words. The map metadata may be configured to map the words of the compressed data block to generated decompressed data blocks. The apparatus may also include means for decompressing the compressed data block to generate decompressed data blocks according to the map metadata.

[0008] A non-transitory computer-readable medium storing computer-executable instructions for an apparatus is described. The executable instructions may include one or more instructions that cause the apparatus to retrieve compressed data blocks and map metadata. The compressed data blocks may include one or more words. The map metadata may be configured to map words of the compressed data blocks to generated decompressed data blocks. The executable instructions may also include one or more instructions that cause the apparatus to decompress the compressed data blocks to generate decompressed data blocks according to the map metadata.

[0009] Other objects and advantages associated with the embodiments disclosed herein will become apparent to those skilled in the art based on the accompanying drawings and detailed description.

[0010] The accompanying drawings are presented to help explain examples of one or more aspects of the disclosed subject matter and are provided by way of illustration only, not limitation. [Brief explanation of the drawings]

[0011] [Figure 1] FIG. 2 illustrates an example of stack compression that may be performed by a compression engine to generate compressed data blocks from uncompressed data blocks, according to one or more aspects. [Figure 2A] 2 illustrates stages in a decompression operation for decompressing the compressed data block of FIG. 1 to generate a decompressed data block that may be performed by a decompression engine, according to one or more embodiments. [Figure 2B] 2 illustrates stages in a decompression operation for decompressing the compressed data block of FIG. 1 to generate a decompressed data block that may be performed by a decompression engine, according to one or more embodiments. [Figure 2C] 2 illustrates stages in a decompression operation for decompressing the compressed data block of FIG. 1 to generate a decompressed data block that may be performed by a decompression engine, according to one or more embodiments. [Figure 2D] 2 illustrates stages in a decompression operation for decompressing the compressed data block of FIG. 1 to generate a decompressed data block that may be performed by a decompression engine, according to one or more embodiments. [Figure 3] FIG. 10 illustrates another example of stack compression that may be performed by a compression engine to generate a compressed data block from an uncompressed data block, according to one or more aspects. [Figure 4A]4 illustrates stages in a decompression operation for decompressing the compressed data block of FIG. 3 to generate a decompressed data block that may be performed by a decompression engine, according to one or more aspects. [Figure 4B] 4 illustrates stages in a decompression operation for decompressing the compressed data block of FIG. 3 to generate a decompressed data block that may be performed by a decompression engine, according to one or more aspects. [Figure 4C] 4 illustrates stages in a decompression operation for decompressing the compressed data block of FIG. 3 to generate a decompressed data block that may be performed by a decompression engine, according to one or more aspects. [Figure 4D] 4 illustrates stages in a decompression operation for decompressing the compressed data block of FIG. 3 to generate a decompressed data block that may be performed by a decompression engine, according to one or more aspects. [Figure 5] 1 is a block diagram of several sample aspects of components that may be employed within an apparatus configured to decompress compressed data blocks, in accordance with one or more aspects. [Figure 6] FIG. 1 illustrates an exemplary method performed by a decompression engine to decompress a compressed data block, according to one or more aspects. [Figure 7] FIG. 1 illustrates an exemplary method performed by a decompression engine to decompress a compressed data block, according to one or more aspects. [Figure 8] FIG. 1 illustrates an exemplary method performed by a decompression engine to decompress a compressed data block, according to one or more aspects. [Figure 9] 1A-1D are simplified block diagrams of several sample aspects of apparatuses configured to decompress compressed data blocks, according to one or more aspects. [Figure 10] 1 illustrates simulated results of compression achieved by a proposed compression / decompression technique, according to one or more aspects. DETAILED DESCRIPTION OF THE INVENTION

[0012] Aspects of the subject matter are provided in the following description and related drawings directed to particular examples of the disclosed subject matter. Alternative forms may be devised without departing from the scope of the disclosed subject matter. Additionally, well-known elements will not be described in detail or will be omitted so as not to obscure the relevant details.

[0013] The word "exemplary" is used herein to mean "serving as an example, instance, or illustration." Any aspect described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other aspects. Likewise, the term "aspect" does not require that all aspects include the discussed feature, advantage or mode of operation.

[0014] The terms used herein are merely descriptive of particular embodiments and should not be construed as limiting any embodiments disclosed herein. As used herein, the singular forms "a," "an," and "the" are intended to include the plural forms unless the context clearly dictates otherwise. Those skilled in the art will further understand that the terms "comprises," "comprising," "includes," and / or "including," when used herein, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0015] Further, various aspects may be described in terms of sequences of actions to be performed by, for example, elements of a computing device. Those skilled in the art will recognize that the various actions described herein may be performed by specific circuitry (e.g., an application-specific integrated circuit (ASIC)), by program instructions being executed by one or more processors, or by a combination of both. Furthermore, these sequences of actions described herein may be considered to be embodied entirely in any form of non-transitory computer-readable medium storing a corresponding set of computer instructions that, when executed, cause an associated processor to perform the functions described herein. Accordingly, the various aspects described herein may be embodied in several different forms, all of which are contemplated to be within the scope of the claimed subject matter. Additionally, for each aspect described herein, the corresponding form of any such aspect may be described herein, for example, as "logic configured to" perform the described actions and / or as other structural components configured to perform.

[0016] It was shown above that neural networks can have many zeros. As an example, the sparsity of an RNN weight matrix / tensor can be close to 90% sparsity. If the weights can be compressed, this can be advantageous for large RNNs. One advantage is that the amount of weights that can fit into on-chip memory or SRAM can be increased. Another advantage is that the bandwidth of on-chip static random access memory (SRAM) can be reduced. Yet another advantage can be that the bandwidth of dynamic RAM (e.g., double data rate (DDR) memory) can be reduced if it becomes necessary to leak the weights into memory (e.g., RAM, DRAM, DDR, SDR, ODR, etc.).

[0017] In one or more aspects, it is proposed to provide a technique for efficiently compressing and decompressing data with many zeros, for example, in a neural network environment. Note that weights in a neural network inference accelerator rarely change. Therefore, the weights can be compressed offline. The compressed weights and corresponding metadata, which can map zero and non-zero words in the original uncompressed data, can be loaded into a memory of the device and then into another memory of the device (e.g., tightly coupled memory (TCM)). The device can include a decompression engine that decompresses the compressed weights. The decompression engine can be purely hardware or a combination of hardware and software.

[0018] Although neural networks are discussed, this is not the only environment in which the proposed technique is applicable. The proposed technique can be applied to systems that introduce sparsity into weight tensors, such as machine learning (ML) networks that can be pruned. More generally, it is envisioned that the proposed technique can be applied to any environment in which data with a significant amount of zeros is processed, for example, when processing sparse matrices. In such instances, significant compression can be achieved by reducing the number of zeros stored in the compressed data relative to the original data.

[0019] As described above, the weights may be compressed by a compression engine. More generally, the original data may be compressed by a compression engine, and the proposed decompression technique may be performed by the decompression engine. The proposed decompression engine may be purely software, e.g., implemented as executable instructions stored in memory and executed by one or more general-purpose processors. However, for performance reasons, it may be preferable for the decompression engine to be implemented purely in hardware, or at least partially hardware-based (e.g., application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), etc.). Similarly, the compression engine may be implemented purely in software, purely in hardware, or a combination of software and hardware.

[0020] The compression engine and the decompression engine may be physically separate or may be at least partially integrated with each other within a single device. However, in the following description, the compression engine and the decompression engine are treated as being logically separate and independent components. In other words, the compression engine and the decompression engine may operate independently of each other. For example, the compression engine may operate to compress original data into compressed data and store the compressed data in a storage device regardless of when the compressed data is decompressed. The decompression engine may later obtain the compressed data (e.g., by accessing the storage device and / or by receiving it from a component having access to the storage device) and decompress the compressed data back to the original data. Of course, it is also contemplated that the compression engine and the decompression engine may communicate with each other (e.g., directly and / or through an intermediate component) so that the compression and decompression occur within some defined time frame.

[0021] In one or more aspects, the compression engine may perform stack compression to compress the original data into compressed data. Stack compression may be viewed as a compression scheme in which original data organized in multiple dimensions is compressed in at least one of those dimensions. For example, the original data may be organized in two dimensions (e.g., rows and columns), and the compressed data may be the original data compressed in the row dimension and / or the column dimension.

[0022] Before proceeding further, it should be noted that the data (e.g., the original data and / or the compressed data) need not be physically organized in multiple dimensions. For example, the original data could be stored in memory locations having consecutive addresses. In fact, at the extreme, the original data could be stored in one or more non-consecutive locations, provided that the order of the original data can be maintained (e.g., via pointers, tables, etc.). Of course, if physical memory is organized into multiple dimensions (e.g., pages, banks, rows, columns, words, etc.), it may also be advantageous to organize the (original and / or compressed) data to fit the memory dimensions or, in some cases, to make efficient use of the physical memory.

[0023] Figure 1 shows an example of stack compression that may be performed by a compression engine to generate compressed data from original data. Compression may be performed offline, i.e., regardless of when or even whether the compressed data is decompressed. This means that compression may be performed in software if compression speed is not an issue and / or if the software's compression performance is sufficient. Alternatively and / or additionally, compression may be performed in conjunction with a corresponding decompression (as described below with reference to Figures 2A-2D).

[0024] In one or more aspects, the original data, which may be of any size, may be divided into chunks, which are sometimes referred to as uncompressed data blocks. Each chunk, i.e., each uncompressed data block, may be stack-compressed into a compressed data block by a compression engine. A decompression engine may decompress each compressed data block to recover the corresponding uncompressed data block.

[0025] In FIG. 1 , line 110 represents an example of an uncompressed data block 122 (an exemplary chunk) of original uncompressed data that may be stack compressed. Line 110 may include multiple words (16 in this example) of original data. The original data in line 110 may be read and stored in first memory 120 in units of banks. That is, uncompressed data block 122 may be stored in first memory 120 and stored in N banks (or rows) and C columns of first memory 120. Again, note that "bank" and "column" are conceptual. The banks and / or columns may or may not have a correspondence with physical memory.

[0026] In one aspect, the read granularity of a read operation may be one word or one bank. In this example, one bank includes four words, i.e., four words of line 110 may be read at a time (for a read granularity of one bank) for storage in first memory 120. For simplicity, uncompressed data block 122 may be considered to be stored in N (N≧1) uncompressed banks, with each uncompressed bank n (n=0...N−1) including a sequence of C (C≧1) words. In FIG. 1 , there are four (N=4) uncompressed banks 125-n (n=0, 1, 2, 3, collectively uncompressed banks 125), each having four (C=4) words. Note that the words may be of any fixed length (e.g., 16 bits, 32 bits, etc.).

[0027] 1, a word can be a zero word or a non-zero word. A zero word designated as "0" can represent a word whose value is zero. Conversely, a non-zero word designated as something other than "0" (e.g., designated as "A," "B," "C," etc.) can represent a word whose value is not zero.

[0028] The uncompressed data blocks 122 in the uncompressed bank 125 may be stack-compressed in one dimension, for example, the row dimension, also called vertical compression, resulting in compressed data blocks 132 stored in the second memory 130 in M ​​(M≧1) compressed banks. Each compressed bank m, (m=0...M−1) may also include a column of C words. Note that the first and second memories 120, 130 may be the same or different.

[0029] In this row compression, uncompressed data block 122 may be compressed into compressed data block 132 by removing one or more zero words from the uncompressed data block. In one aspect, leading and intermediate zero words within each column of uncompressed bank 125 may be removed, i.e., compressed out, thereby pushing non-zero words down to compressed bank 135-m (m=0, 1, collectively compressed bank 135). For example, a non-zero value "F" occupying location (n=3, c=0) in uncompressed bank 125 (i.e., before compression) is seen to occupy location (m=0, c=0) in compressed bank 135, indicating that the zero words at locations (n=0, c=0), (n=1, c=0), and (n=2, c=0) in uncompressed bank 125 have been removed. As another example, the non-zero value "9" occupying location (n=3, c=3) in the uncompressed bank 125 before compression is seen to occupy location (m=1, c=3) in the compressed bank 135, indicating that the zero words in locations (n=1, c=3) and (n=2, c=3) have been removed.

[0030] Through vertical compression, the number of banks can be reduced, i.e., M < N. In FIG. 1, four uncompressed banks 125 in the first memory 120 are reduced to two compressed banks 135 in the second memory 130. The worst case can occur when there is one column c that has all non-zero words in the uncompressed bank 125, in which case M = N. Thus, generally, M ≤ N.

[0031] Some features / results of stack compression are described next. In one feature, the non-zero words of the original data of the uncompressed data block 122 can be maintained in the compressed data block 132 without modification, even if their locations may change. For example, the non-zero word "E" at the location (n = 2, c = 1) in the uncompressed bank 125 can be stored as the same non-zero word "E" at the location (m = 1, c = 1) in the compressed bank 135.

[0032] This means that the number of non-zero words in the compressed data block 132 remains equal to the number of non-zero words in the uncompressed data block 122. On the other hand, the number of zero words in the compressed data block 132 can be less than or equal to the number of zero words in the uncompressed data block 122. Basically, when M < N, the number of zero words in the compressed data block 132 becomes smaller.

[0033] In another feature, for the compressed data block 132 in the M compressed banks 135, there can be at least one column in which the words in the columns for all M compressed banks 135 are non-zero words. In FIG. 1, for example, it can be seen that the columns c = 1, 2, and 3 of the M compressed banks 135 are completely occupied by non-zero words. This is a natural consequence of the stack compression shown in that further row compression can be performed if all columns have at least one zero word.

[0034] In a further feature, it is also possible that for compressed data blocks 132 within the M compressed banks 135, there may be a column with both zero and non-zero words in a column. When such a column exists, each zero word in that column occupies a bank above the bank occupied by each non-zero word. More formally, for each column c, c=0...C-1, when both zero and non-zero words exist in that column c for the M compressed banks 135, each m compressed bank having a non-zero word in that column c is NZ is the number of compressed banks m that have zero words in that column. Z That is, for each column c of the M compressed banks 135, NZ <m Z For example, in FIG. 1, in column c=0, the non-zero word F is stored in the compressed bank 135-0(m NZ =0) and the zero word occupies bank 135-1(m Z One consequence is that the lowest bank (m=m NZ =0) is occupied by a non-zero word.

[0035] As part of the compression operation, the compression engine may generate map metadata 142. As described in more detail below, the map metadata 142 may be used by the decompression engine to decompress the compressed data block 132. Broadly, the map metadata 142 may map all non-zero words of the original uncompressed data. For example, with respect to N uncompressed banks 125, the map metadata 142 may map the locations of zero and non-zero words in the uncompressed data block 122. For example, each bit of the map metadata 142 may correspond to a word location in the N uncompressed banks 125, i.e., to a word in the uncompressed data block 122. Also, a one-bit value in the map metadata 142 may indicate whether the corresponding word in the uncompressed data block 122 is a zero word or a non-zero word. Thus, the size of the map metadata 142 may be a fraction (e.g., 1 / 8, 1 / 16, etc.) inversely proportional to the size of a word in the uncompressed data block 122. The map metadata 142 may be stored in the metadata storage device 140.

[0036] Although not shown, it should be noted that if all of the words in the original data line 110 are zero words, the map metadata 142 may correspondingly be filled with all zero bits. When this occurs, the uncompressed data block 122 may contain only zero words; that is, N uncompressed banks 125 may be filled with zero words. Also, the compressed data block 132 may be empty; that is, the number of compressed banks is M=0.

[0037] 1 , the mapped metadata 142 is shown as an N-row by C-column matrix, where each element (n, c), n = 0...N-1, c = 0...C-1, of the mapped metadata 142 corresponds to a word in bank n, column c of the uncompressed original data. As can be seen, each element (n, c) of the mapped metadata 142 is zero when the corresponding word in the uncompressed data block 122 is a zero word, and is non-zero when the corresponding word in the original data is a non-zero word. As indicated above, the size of each element (n, c) of the mapped metadata 142 may be smaller than the size of a word in the uncompressed data block 122. For example, each element (n, c) of the mapped metadata 142 may be a single bit indicating whether the corresponding word in the uncompressed data block 122 is a zero word or a non-zero word.

[0038] Before proceeding further, note the following: Up to this point, the exemplary compression described involved distinguishing between zero words and non-zero words and compressing and removing the zero words. However, this is only one example. It is fully contemplated to generalize the concept to distinguish between “removable” and “non-removable” words, where removable words may be words that have a removable value (which may be any value, such as zero), and non-removable words may be words that have a value other than a removable value. In this example, map metadata 142 may map to removable and non-removable words in uncompressed data block 122, and each bit of map metadata 142 may indicate whether the corresponding word in uncompressed data block 122 is a removable word or a non-removable word.

[0039] The concept can also be extended to allow for multiple removable values. For example, the original data could be such that three values ​​appear significantly more frequently than others. In this example, each element (n, c) could be two bits long, with one combination of the two bits (e.g.,

[11] ) representing a non-removable word and the other three (e.g., [00, 01, 10]) representing the first, second, and third values ​​of the removable word.

[0040] However, for ease of reference and explanation, the terms zero and non-zero will be used in the remainder of this disclosure, keeping in mind that the concepts described can be generalized straightforwardly.

[0041] The compression engine may also generate num-compressed-bank metadata 144. The num-compressed-bank metadata 144 may represent a value equal to the number M of compressed banks. The decompression engine may decompress the compressed data block 132 based on the num-compressed-bank metadata 144 in addition to the map metadata 142. The num-compressed-bank metadata 144 may also be stored in the metadata storage device 140.

[0042] Note that num-compressed-bank metadata 144 is optional, in that the compression engine is not required to provide it because it can be determined during decompression based on map metadata 142. For example, recall that compressed data block 132 is the result of the compression engine stack-compressing uncompressed data block 122 within N uncompressed banks 125 in one dimension (the row dimension in this example). A similar dimensional compression can be performed on map metadata 142 by the decompression engine to determine the number M of compressed banks. However, when num-compressed-bank metadata 144 is provided as part of compression, this means that fewer operations need to be performed during decompression.

[0043] 2A-2D illustrate various stages of a decompression operation that may be performed by a decompression engine on the compressed data of FIG. 1 to generate a decompressed data block that should be identical to the original uncompressed data block 122. In FIGS. 2A-2D, second memory 130 and metadata storage 140 are not shown to reduce complexity. The decompression operation is the focus of these figures. Therefore, when memory is mentioned in the context of a decompression operation, it is assumed that second memory 130 is referenced unless otherwise indicated.

[0044] In FIG. 2A, there may be C number of queues 250 corresponding to C columns of compression bank 135. Each queue 250-c may have a depth of S number of slots. The number of slots S may be any number, S≧1. However, in most examples, S=N / 2 may be sufficient. Each queue 250-c may have a separate counter, which is not shown to reduce complexity. Also, a map register 260 and a line register 270 may be provided to aid in decompression. For example, the line register 270 may be populated with decompressed data blocks as decompression occurs.

[0045] Additionally, a row index register 265 may be provided to hold a count of row index n, and a compressed bank index register 255 may be provided to hold a count of compressed bank index m. Note that some or all of the registers are not necessarily required. For example, rows of decompressed banks may be flushed out of the decompression engine. In this example, the decompressed data block may be flushed four words at a time, i.e., one bank at a time.

[0046] Recall from above that in some examples, the uncompressed data block 122 may contain only zero words. In this example, the operations performed by the decompression engine can be significantly simplified. For example, the decompression engine may first determine whether there is compressed data to be decompressed. This determination can be made by examining the num-compressed-bank metadata 144, which can be provided by the compression engine or derived from the map metadata 142. If the num-compressed-bank metadata 144 indicates that M = 0 (no compressed data), the decompression engine may simply add zero words to the entire line register 270 or to the entire decompressed data block.

[0047] On the other hand, if the num-compressed-bank metadata 144 indicates that M > 0, the decompression engine may decompress the compressed data block 132. FIGS. 2A-2D show exemplary stages when there is data to be decompressed, i.e., when M > 0. FIGS. 2A, 2B, 2C, and 2D each show extracting the values of bank 0, bank 1, bank 2, and bank 3 of the uncompressed data block 122 from the compressed data block 132.

[0048] The decompression process of FIG. 2A performed by the decompression engine to generate bank 0 of the decompressed data block (restore bank 0 of the uncompressed data block 122) may be as follows. · Initialize the row index n, o In FIG. 2A, the row index register 265 is set to 0 (n = 0). · Since the row index is n < N, read row n (i.e., row 0) from the map metadata 142 into the map register 260. o In FIG. 2A, the map register 260 is loaded with the row 0 entry [0, 0, 1, 1] of the map metadata 142. · Initialize the compressed bank index m. o In FIG. 2A, the compression bank index register 255 is set to 0 (m = 0). · Since the compression bank index is m < M, the words in compression bank m of the compressed data block 132 (i.e., bank 0 of the compression bank 135) are loaded into the queue 250, whereby the words in each column c of the compression bank m occupy the lowest unoccupied slot of the corresponding queue 250 - c. o In FIG. 2A, before loading the compressed data from the compression bank 135 - 0 into the queue 250, slot 0 of the queue 250 is all unoccupied. Therefore, the slots 0 of the queues 250 - c, c = 0, 1, 2, 3, are loaded with [F, C, A, B] from the compression bank 135 - 0. · Based on the entry of row n of the map metadata 142 (stored in the map register 260), a word is popped from the head, which is slot 0, of the queue 250, and the popped word is loaded into the corresponding location of the line register 270. o In FIG. 2A, the word locations 0 to 3 of the line register 270 correspond to row 0 of the map metadata 142, and also correspond to bank 0 of the uncompressed data block 122 and bank 0 of the decompressed data block. o The map register 260 is "AND - operated" with the queue 250, as shown below. · A "0" in column c of the map register 260 indicates that the corresponding queue 250 - c is not popped. Instead, a zero word is loaded into the corresponding location of the line register 270 (of the decompressed data block). · A "1" in column c of the map register 260 indicates that the corresponding queue 250 - c is popped and the popped word is loaded into the corresponding location of the line register 270 (of the decompressed data block). o The map register is loaded with the entry [0, 0, 1, 1] of row 0 of the map metadata 142, indicating that the data values "A", "B" in the heads (slots 0) of the queues 250 - 2, 250 - 3 are popped and loaded into the word locations 2 and 3 of the line register 270 corresponding to columns 2 and 3 of bank 0 of the decompressed data block. The word locations 0 and 1 of the line register 270 corresponding to columns 0 and 1 of bank 0 of the decompressed data block are filled with zeros. As a result, the values [0, 0, A, B] of bank 0 of the uncompressed banks 125 - 0 are restored.

[0049] In FIG. 2A (as well as FIGS. 2B - 2D), a multiplexer is included to show an example of how the map register 260 can be used to pop the queue 250 within the restoration process. As seen in one aspect, the entries within the map register 260 can be used to control the multiplexer. However, it should be noted that the actual implementation can be achieved in various ways.

[0050] To generate bank 1 of the decompressed data block (to restore bank 1 of the uncompressed data block 122), the decompression process of FIG. 2B executed by the decompression engine can be as follows. · Increment the row index n. o In FIG. 2B, the row index register 265 is incremented (n = 1). · Since the row index is n < N, read row n (i.e., row 1) from the map metadata 142 into the map register 260. o In FIG. 2B, the map register 260 is loaded with the row 1 entry [0, 1, 1, 0] of the map metadata 142. · Since the compressed bank index is m < M, increment the compressed bank index m. o In FIG. 2B, the compressed bank index register 255 is incremented (m = 1). · Since the incremented compressed bank index is m < M, load the words of compressed bank m of the compressed data block 132 (i.e., bank 1 of the compressed bank 135) into the queue 250, whereby the words of each column c of compressed bank m occupy the lowest unoccupied slot of the corresponding queue 250 - c. 2B, the lowest unoccupied slot for queues 250-0 and 250-1 is slot 1 for both because their heads (slot 0) have not been previously popped. Therefore, slot 1 for queues 250-0 and 250-1 is loaded with [0,E] from columns 0,1 of compression bank 135-1. For queues 250-2 and 250-3, the lowest unoccupied slot is slot 0 for both because their heads were previously popped. Therefore, slot 0 for queues 250-2 and 250-3 is loaded with [D,9] from columns 2 and 3 of compression bank 135-1. Pop a word from the head (slot 0) of the queue 250 based on the row n entry of the map metadata 142 (stored in the map register 260) and load the popped word into the corresponding location in the line register 270. o In FIG. 2B, word locations 4-7 of line register 270 correspond to row 1 of map metadata 142, which in turn corresponds to bank 1 of uncompressed data block 122 and to bank 1 of the decompressed data block. o Map register 260 is loaded with row 1 entry [0, 1, 1, 0] of map metadata 142, which indicates that data values ​​“C” and “D” in the heads (slot 0) of queues 250-1 and 250-2 are to be popped and loaded into word locations 5 and 6 of line register 270, which correspond to columns 1 and 2 of bank 1 of the decompressed data block. o Word locations 4 and 7 of line register 270 corresponding to columns 0 and 3 of bank 1 of the decompressed data block are filled with zeros. As a result, the value [0, C, D, 0] of bank 1 of uncompressed bank 125-1 is restored.

[0051] The decompression process of FIG. 2C performed by the decompression engine to generate bank 2 of decompressed data blocks (reconstructing bank 2 of uncompressed data blocks 122) may be as follows. Increment the row index n. o In FIG. 2C, the row index register 265 is incremented (n = 2). · Since the row index is n < N, row n (i.e., row 2) is read from the map metadata 142 into the map register 260. o In FIG. 2C, the map register 260 is loaded with the row 2 entry [0, 1, 0, 0] of the map metadata 142. · Since the compressed bank index is m < M, the compressed bank index m is incremented. o In FIG. 2B, the compressed bank index register 255 is incremented (m = 2). · Since the incremented compressed bank index m is not less than M, indicating that all compressed banks 135 have been read, no word is loaded from the compressed data block 132 (compressed bank 135). o In FIG. 2C, slots 0 of queues 250-0, 250-1, 250-2, 250-3 have the values [F, E, empty, 9], which is the state of the queues after popping the queues in FIG. 2B. · Based on the row n entry of the map metadata 142 (stored in the map register 260), a word is popped from the head (slot 0) of the queue 250 and loaded into the corresponding location of the line register 270. o In FIG. 2C, word locations 8 to 11 of the line register 270 correspond to row 2 of the map metadata 142, and also correspond to bank 2 of the uncompressed data block 122 and bank 2 of the decompressed data block. o The map register 260 is loaded with the row 2 entry [0, 1, 0, 0] of the map metadata 142, indicating that the data value "E" in the head of queue 250-1 is popped and loaded into word location 9 of the line register 270 corresponding to column 1 of bank 2 of the decompressed data block. o Word locations 8, 10, and 11 of the line register 270 corresponding to columns 0, 2, and 3 of bank 2 of the decompressed data block are filled with zeros. As a result, the value [0, E, 0, 0] of bank 2 of the uncompressed bank 125-2 is restored.

[0052] The decompression process of FIG. 2D executed by the decompression engine to generate bank 3 of the decompressed data block (restore bank 3 of the uncompressed data block 122) may be as follows. · Increment the row index n. o In FIG. 2D, the row index register 265 is incremented (n = 3). · Since the row index is n < N, read row n (i.e., row 3) from the map metadata 142 into the map register 260. o In FIG. 2D, the map register 260 is loaded with the row 3 entry [1, 0, 0, 1] of the map metadata 142. · Since the incremented compressed bank index m is not less than M, indicating that all compressed banks 135 have been read, no word is loaded from the compressed data block 132 (compressed bank 135). o In FIG. 2D, slot 0 of queues 250-0, 250-1, 250-2, 250-3 has the value [F, empty, empty, 9], which is the state of the queues after popping the queues in FIG. 2C. · Pop a word from the head (slot 0) of the queue 25 on the basis of the row n entry of the map metadata 142 (stored in the map register 260), and load the popped word into the corresponding location of the line register 270. o In FIG. 2D, word locations 12 to 15 of the line register 270 correspond to row 3 of the map metadata 142, and also correspond to bank 3 of the uncompressed data block 122 and bank 3 of the decompressed data block. o Map register 260 is loaded with row 3 entry [1, 0, 0, 1] of map metadata 142, which indicates that data values ​​“F” and “9” in the heads of queues 250-0 and 250-3 are to be popped and loaded into word locations 12 and 15 of line register 270, which correspond to columns 0 and 3 of bank 3 of the decompressed data block. o Word locations 13 and 14 of line register 270 corresponding to columns 1 and 2 of bank 3 of the decompressed data block are filled with zeros. As a result, the value [F, 0, 0, 9] of bank 3 of uncompressed bank 125-3 is restored. Moreover, the entire original data in line 110 is restored without any loss.

[0053] Note that in one or more aspects, the first bank (e.g., bank 0) may vary depending on the address to distribute bank reads. Also, the actual number of words per bank (e.g., actual C) and the number of banks (e.g., actual M) may vary. For example, if a 2K page size is assumed, memories 120, 130, and thus the uncompressed and compressed banks, may be organized into groups of 128x16b words (e.g., 8 banks with 16 words per bank) or into groups of 64x16b words (e.g., 16 banks with 4 words per bank). Of course, these are merely examples, and other configurations are possible. Also, compression is limited by the worst column. Better effective compression can be achieved with more banks. However, delays may increase as the number of banks increases.

[0054] FIG. 3 illustrates another example of stack compression that may be performed by a compression engine to generate compressed data blocks from original data. The compression of FIG. 3 may be performed offline. Alternatively and / or additionally, the compression of FIG. 3 may be performed in conjunction with a corresponding decompression (described with reference to FIGS. 4A-4D below). The stack compression of FIG. 3 differs from FIG. 1 in that additional compression in another dimension (e.g., in the column / horizontal dimension) may be performed. For example, after performing vertical compression, the compression engine may perform horizontal compression on the vertically compressed data.

[0055] In FIG. 3, the same original data as in FIG. 1 is used for illustration purposes. To reduce complexity, memories 120, 130 and metadata storage 140 are not shown in FIG. 3. In FIG. 3, after compression in a first dimension (e.g., vertical compression) to generate intermediate compressed data block 133, the state of compressed bank 135 is the same as compressed bank 135 in FIG. 1. That is, intermediate compressed data block 133 is the same as compressed data block 132 in FIG. 1. However, after compression in a second dimension (e.g., horizontal compression) to generate final compressed data block 332 (or simply compressed data block 332), the result is that values ​​can be shifted left to remove zero "holes" from intermediate compressed data block 133. This can result in saving reading zero words from memory and therefore saving power. For clarity, the compressed bank after horizontal compression is designated "335."

[0056] The stack compression of FIG. 3 (also called 2D compression) may share the characteristics of the stack compression of FIG. 1 (also called 1D compression) (e.g., a number of non-zero words equal to the original data, unmodified non-zero words, a number of zero words equal to or less than the original data, etc.).

[0057] There may be several different characteristics. For example, each compressed bank 335 is preceded by a non-zero word. That is, for each compressed bank m of compressed banks 335, each column c of that compressed bank m has a non-zero word. NZ is the value of each column c of that compressed bank m that has a zero word. Z It can be a lower column, i.e., c NZ <c Z For example, in compressed bank 335-1, it can be seen that non-zero words occupy columns 0, 1, 2, and then a zero word occupies column 3.

[0058] Although not shown, the difference between 2D compression and 1D compression may be negligible if all of the words in the original data line 110 are zero words. For example, the uncompressed data block 122 may contain only zero words, the map metadata 142 may be filled with only zero bits, and the compressed data block 332 may be empty, i.e., the number of compression banks is M=0.

[0059] Therefore, the difference in decompression can also be near zero. That is, the decompression engine may determine whether there is compressed data to decompress by, for example, examining num-compressed-bank metadata 144, which may be provided by the compression engine or derived by the decompression engine from map metadata 142. If num-compressed-bank metadata 144 indicates that M=0, the decompression engine may simply add zeros to the entire line register 270, i.e., add zeros to the entire decompressed data block.

[0060] 4A-4D illustrate various stages of a decompression operation that may be performed by a decompression engine on the 2D compressed data 332 of FIG. 3 to generate a decompressed data block when the compressed data block 332 is not empty (i.e., when M>0). In FIGS. 4A-4D, it may be assumed that memories 120 and / or 130 may be subjected to a "read under mask" (described below). Also in these figures, in addition to loading map metadata 142 and num-compressed-bank metadata 144, metadata storage 140 (not shown) may store column metadata 346 that maps zero and non-zero locations of compressed data block 332 after compression in the row dimension but before compression in the column dimension; i.e., column metadata 346 may map zero and non-zero locations (more generally, removable and non-removable locations) of intermediate compressed data block 133.

[0061] Similar to num-compressed-bank metadata 144, the compression engine may provide column metadata 346. However, again, this is optional from the perspective of the compression engine in that column metadata 346 can be determined by the decompression engine from map metadata 142. For example, note that 2D compressed data block 332 is the result of the compression engine stack-compressing uncompressed data block 122 within N uncompressed banks 125 in two dimensions. A similar two-dimensional compression can be performed on map metadata 142 by the decompression engine to determine column metadata 346. Nevertheless, when column metadata 346 is provided as part of the compression, fewer operations need to be performed during decompression.

[0062] 4A-4D, it may be assumed that queue 250, map register 260, line register 270, row index register 265, and compressed bank index register 255 are used in a similar manner. A shifter 475 may also be provided, for example, to provide column decompression capability.

[0063] The decompression process of FIG. 4A executed by the decompression engine to generate bank 0 of the decompressed data blocks (restore bank 0 of the uncompressed data blocks 122) may be as follows. · Initialize the row index n. o In FIG. 4A, the row index register 265 is set to 0 (n = 0). · Since the row index is n < N, read row n (i.e., row 0) from the map metadata 142 into the map register 260. o In FIG. 4A, the map register 260 is loaded with the row 0 entry [0, 0, 1, 1] of the map metadata 142. · Initialize the compressed bank index m. o In FIG. 4A, the compressed bank index register 255 is set to 0 (m = 0). · Since the compressed bank index is m < M, read row m (i.e., row 0) from the column metadata 346. o In FIG. 4A, the row 0 entry [1, 1, 1, 1] of the column metadata 346 is emphasized (in bold, italic). · Since the compressed bank index is m < M, perform a masked read to read the word of compressed bank m (i.e., bank 0 of compressed bank 335) of the compressed data block 332 and load the word into the queue 250. o The masked read is a read operation in which only k words from the left end of compressed bank m (bank 0) are read, and k is the number of 1s in row m (row 0) of the column metadata 346. In this example, since k = 4, all four words [F, C, A, B] of compressed bank 335-0 are read into the column shifter 475. o In FIG. 4A, the data [F, C, A, B] read from compressed bank 335-0 is not shifted in the shifter 475 because there are no zeros in the row 0 entry [1, 1, 1, 1] of the column metadata 346. o Slot 0 of the queue 250 is loaded with [F, C, A, B] from the shifter 475. Pop a word from the head (slot 0) of the queue 250 based on the row n entry of the map metadata 142 (stored in the map register 260) and load the popped word into the corresponding location in the line register 270. o In FIG. 4A, word locations 0-3 of line register 270 correspond to row 0 of map metadata 142, which in turn corresponds to bank 0 of uncompressed data block 122 as well as bank 0 of the decompressed data block. o The map register is loaded with the row 0 entry [0, 0, 1, 1] of the map metadata 142, which indicates that the data values ​​“A” and “B” in the heads (slot 0) of the queues 250-2 and 250-3 are to be popped and loaded into word locations 2 and 3 of the line register 270, which correspond to columns 2 and 3 of bank 0 of the decompressed data block. o Word locations 0 and 1 of line register 270 corresponding to columns 0 and 1 of bank 0 of the decompressed data block are filled with zeros. As a result, the value [0, 0, A, B] of bank 0 of uncompressed bank 125-0 is restored.

[0064] 2A-2D, in Figures 4A-4D, AND gates are included to show an example of how map register 260 is used to pop queue 250 in the restore process. The actual implementation may be achieved in a variety of ways. For example, in one aspect, entries in map register 260 may be used to control a multiplexer (not shown).

[0065] The decompression process of FIG. 4B performed by the decompression engine to generate bank 1 of decompressed data blocks (reconstructing bank 1 of uncompressed data blocks 122) may be as follows. Increment the row index n. o In FIG. 4B, the row index register 265 is incremented (n=1). · Since the row index is n < N, row n (i.e., row 1) is read from the map metadata 142 into the map register 260. o In FIG. 4B, the map register 260 is loaded with the row 1 entry [0, 1, 1, 0] of the map metadata 142. · Since the compressed bank index is m < M, increment the compressed bank index m. o In FIG. 4B, the compressed bank index register 255 is incremented (m = 1). · Since the incremented compressed bank index is m < M, read row m (i.e., row 1) from the column metadata 346. o In FIG. 4B, the row 1 entry [0, 1, 1, 1] of the column metadata 346 is emphasized (in bold, italic). · Since the incremented compressed bank index is m < M, perform a masked read to read the word of compressed bank m (i.e., bank 1 of compressed bank 335) of the compressed data block 332, shift the word from compressed bank m according to row m read from the column metadata 346, and load the shifted word corresponding to column c into the lowest non-occupied slot of the corresponding queue 250-c. o In this example, k = 3. Thus, the leftmost 3 words [E, D, 9] of compressed bank 335-1 are read into the shifter 475 when the masked read is performed. o In FIG. 4B, the read data [E, D, 9] is column-shifted within the shifter 475 according to the row 1 entry [0, 1, 1, 1] of the column metadata 346 (shifted by 1 column as [0, E, D, 9] due to the leading entry 0 in row 1 of the column metadata 346). For queues 250-0 and 250-1, their heads (slot 0) have not been previously popped, so the lowest unoccupied slot is slot 1 for both queues, and for queues 250-2 and 250-3, their heads have been previously popped, so the lowest unoccupied slot is slot 0 for both queues. Slot 1 for queues 250-0 and 250-1 is loaded with [0,E], and slot 0 for queues 250-2 and 250-3 is loaded with [D,9]. Pop a word from the head (slot 0) of the queue 250 based on the row n entry of the map metadata 142 (stored in the map register 260) and load the popped word into the corresponding location in the line register 270. o In FIG. 4B, word locations 4-7 of line register 270 correspond to row 1 of map metadata 142, which in turn corresponds to bank 1 of uncompressed data block 122 as well as bank 1 of the decompressed data block. o Map register 260 is loaded with row 1 entry [0, 1, 1, 0] of map metadata 142, which indicates that data values ​​“C” and “D” in the heads (slot 0) of queues 250-1 and 250-2 are to be popped and loaded into word locations 5 and 6 of line register 270, which correspond to columns 1 and 2 of bank 1 of the decompressed data block. o Word locations 4 and 7 of line register 270 corresponding to columns 0 and 3 of bank 1 of the decompressed data block are filled with zeros. As a result, the value [0, C, D, 0] of bank 1 of uncompressed bank 125-1 is restored.

[0066] The decompression process of FIG. 4C performed by the decompression engine to generate bank 1 of decompressed data blocks (reconstructing bank 1 of uncompressed data blocks 122) may be as follows. Increment the row index n. o In FIG. 4C, the row index register 265 is incremented (n=2). · Since the row index is n < N, row n (i.e., row 2) is read from the map metadata 142 into the map register 260. o In FIG. 4C, the map register 260 is loaded with the row 2 entry [0, 1, 0, 0] of the map metadata 142. · Since the compressed bank index is m < M, the compressed bank index m is incremented. o In FIG. 4B, the compressed bank index register 255 is incremented (m = 2). · Since the compressed bank index m is not less than M, indicating that all rows of the column metadata 346 have been read, no reading is performed from the column metadata 346. o In FIG. 4C, the rows of the column metadata 346 are not emphasized. · Since the incremented compressed bank index m is not less than M, indicating that all compressed banks 335 have been read, no word loading is performed from the compressed data block 332 (compressed bank 335). o In FIG. 4C, slot 0 of queues 250-0, 250-1, 250-2, 250-3 has the values [F, E, empty, 9], which is the state of the queues after popping the queues in FIG. 4B. · Based on the row n entry of the map metadata 142 (stored in the map register 260), a word is popped from the head (slot 0) of the queue 250 and the popped word is loaded into the corresponding location of the line register 270. o In FIG. 4C, word locations 8 to 11 of the line register 270 correspond to row 2 of the map metadata 142, and also correspond to bank 2 of the uncompressed data block 122 and bank 2 of the decompressed data block. o The map register 260 is loaded with the row 2 entry [0, 1, 0, 0] of the map metadata 142, indicating that the data value "E" in the head of queue 250-1 is popped and loaded into word location 9 of the line register 270 corresponding to column 1 of bank 2 of the decompressed data block. The word locations 8, 10, and 11 of the line register 270 corresponding to columns 0, 2, and 3 of bank 2 of the decompression data block are filled with zeros. As a result, the value [0, E, 0, 0] of bank 2 of the uncompressed bank 125-2 is restored.

[0067] The decompression process of FIG. 4D executed by the decompression engine to generate bank 3 of the decompression data block (restore bank 3 of the uncompressed data block 122) may be as follows. · Increment the row index n. o In FIG. 4D, the row index register 265 is incremented (n = 3). · Since the row index is n < N, read row n (i.e., row 3) from the map metadata 142 into the map register 260. o In FIG. 4D, the map register 260 is loaded with the row 3 entry [1, 0, 0, 1] of the map metadata 142. · Since the incremented compressed bank index m is not less than M, indicating that all compressed banks 335 have been read, no word is loaded from the compressed data block 332 (compressed bank 335). o In FIG. 4D, slot 0 of queues 250-0, 250-1, 250-2, 250-3 has the values [F, empty, empty, 9], which is the state of the queue after popping the queue in FIG. 4C. · Pop a word from the head (slot 0) of the queue 250 based on the row n entry of the map metadata 142 (stored in the map register 260), and load the popped word into the corresponding location of the line register 270. o In FIG. 4D, the word locations 12 to 15 of the line register 270 correspond to row 3 of the map metadata 142, and also correspond to bank 3 of the uncompressed data block 122 and bank 3 of the decompression data block. o Map register 260 is loaded with row 3 entry [1, 0, 0, 1] of map metadata 142, which indicates that data values ​​“F” and “9” in the heads of queues 250-0, 250-3 are to be popped and loaded into word locations 12 and 15 of line register 270, which correspond to columns 0 and 3 of bank 3 of the decompressed data block. o Word locations 13 and 14 of line register 270 corresponding to columns 1 and 2 of bank 3 of the decompressed data block are filled with zeros. As a result, the value [F, 0, 0, 9] of bank 3 of uncompressed bank 125-3 is restored. Moreover, the entire original data in line 110 is restored without any loss.

[0068] 5 illustrates an exemplary apparatus 500 configured to perform decompression on stack-compressed data. For example, the apparatus 500 may generate a decompressed data block by decompressing the compressed data block 132, 332, and the uncompressed data block 122 is restored within the decompressed data block. The apparatus 500 may include a memory 510 including multiple banks. Although not shown, the memory 510 may be configured to store the uncompressed data block 122 (e.g., N uncompressed banks 125) and / or the compressed data block 132, 332 (e.g., M compressed banks 135 and / or M compressed banks 335). The memory 510 may be composed of one or more volatile and / or nonvolatile memory components. The memory 500 may include SRAM and / or DRAM (e.g., DDR).

[0069] The apparatus 500 may also include a metadata storage device 520. Although not shown, the metadata storage device 520 may be configured to store the map metadata 142, the num-compressed-bank metadata 144, and / or the column metadata 346. The metadata storage device 520 may include buffers, registers, and / or memory. The memory of the metadata storage device 520 may be the same as or different from the memory used to store the uncompressed banks 125, the compressed banks 135, and / or the compressed banks 335. In one or more aspects, the metadata for the compressed data blocks (e.g., the map metadata 142, the num-compressed-bank metadata 144, and / or the column metadata 346) may be stored along with the compressed data blocks 132, 332.

[0070] Apparatus 500 may further include working storage 530. Although not shown, working storage 530 may include shifter 475, queue 250, map register 260, line register 270, and / or line 210. Working storage 530 may include buffers, registers, and / or memory.

[0071] The apparatus 500 may further include a decompression engine 540 configured to decompress the stack compressed data. The apparatus 500 may further include a processing system 550 configured to control the overall operation of the apparatus 500. The processing system 550 may include one or more processors (not shown). In one aspect, the processing system 550 may contribute to the decompression process, i.e., may be part of the decompression engine 540.

[0072] In one embodiment, the decompression engine 540 may be incorporated within a neural network (NN) system. That is, the device 500 may be an NN system, or at least a part of an NN system. In another embodiment, the decompression engine 540 may be configured, at least in part, in hardware. Furthermore, the decompression engine 540 may fully restore the original data.

[0073] FIG. 6 shows an exemplary method 600 executed by a decompression engine (e.g., any combination of the decompression engine 540 of apparatus 500 and / or processing system 550). In block 610, the decompression engine may initialize a row index n and a compressed bank index m. For example, the row index register 265 and the compressed bank index register 255 may be set to 0.

[0074] In block 620, the decompression engine may determine whether the decompression operation has completed. For example, the decompression engine may determine whether n < N. That is, the decompression engine may determine whether the value n in the row index register 265 is less than the number N of non-compressed banks. If not (branch from block 620, no), the decompression may be considered complete. On the other hand, if n < N (branch from block 620, yes), in block 630, the decompression engine may read row n from the map metadata 142.

[0075] In block 640, the decompression engine may determine whether all banks of the compressed data blocks 132, 332 (e.g., compressed banks 135, 335) have been read. For example, the decompression engine may determine whether m < M. That is, the decompression engine may determine whether the value m in the compressed bank index register 255 is less than the value M of the num-compressed-bank metadata 144. If not (no branch from block 640, no), the decompression engine may proceed to block 670 (further described below).

[0076] If so (branch from block 640, yes), in block 650, the decompression engine may load the queue 250 with words of compression bank m of compressed data block 132, 332 (e.g., load data from bank m of compression bank 135, 335). FIG. 7 shows an exemplary process performed by the decompression engine to implement block 650. The process shown in FIG. 7 may be applied to decompress a 1D compressed data block, such as compressed data block 132 stored in compression bank 135 of FIG. 1. In block 710, the decompression engine may read the words of compression bank m of compression bank 135 and load each word corresponding to each column c of compression bank m to occupy the lowest unoccupied slot of the corresponding queue 250-c (see, for example, FIGS. 2A-2D).

[0077] Regarding the slots of queue 250, note the following. In FIGS. 2A-2D and 4A-4D, each queue 250-c includes S slots, each of which is addressed as slot s, s=0...S-1. The head of each queue 250-c is slot 0, which is the lowest-addressed slot of queue 250-c. Thus, the lowest occupied slot, including head slot 0, is the unoccupied slot closest to the head. However, it is recognized that in other queue implementations, the head may be the highest-addressed slot S-1. In such implementations, compressed data may be read from compression bank m and loaded to occupy the highest unoccupied slot of the corresponding queue. But again, the highest unoccupied slot, including head slot S-1, is closest to the head. Thus, in one or more aspects, the "lowest unoccupied slot" may be interpreted more generally as the unoccupied slot "closest to the head of the queue."

[0078] 8 illustrates another exemplary process performed by the decompression engine to implement block 650. The process illustrated in FIG. 8 may be applied to decompress a 2D compressed data block, such as compressed data block 332 stored in compression bank 335 of FIG. 3. At block 805, the decompression engine may read row m of column metadata 346; e.g., the decompression engine may read the row of column metadata 346 based on the row value in compression bank index register 255. At block 810, the decompression engine may read words of compressed bank m of compressed data block 332 (e.g., read bank m of compression bank 335) based on the read row m of column metadata 346. For example, a read under a mask may be performed to read k words from the left end of compression bank 335-m, where k is the number of ones (or, more generally, the number of entries indicating non-zero words) in the read row m of column metadata 346. In block 820, the decompression engine may column-shift the words read from compression bank m of compression banks 335 according to row m read from column metadata 346. In block 830, the decompression engine may load each shifted word of compressed data block 332 corresponding to each column c of compression bank m to occupy the lowest unoccupied slot of corresponding queue 250-c (see, e.g., FIGS. 4A-4D).

[0079] 6, in block 660, the decompression engine may increment the compressed bank index m in the compressed bank index register 255. Then, in block 670, the decompression engine may pop from the head (slot 0) of each queue 250 based on the row n read from the map metadata 142 and output the popped word as decompressed bank data (see, e.g., FIGS. 2A-2D, 4A-4D). For example, the popped word may be placed into the corresponding column c of the line register 270 at location n and / or may be flushed as output. In block 680, the decompression engine may increment the row index n in the row index register 265. The decompression engine may then proceed to block 620.

[0080] Although not specifically described, capacity compression can be easily added by including the starting address of the line in the metadata. For example, a 1 KB line can be easily compressed. The metadata can be addressed using a given line index. Also, metadata (e.g., map metadata 142) can be read before compressing the data. This can provide the memory address (e.g., SRAM address) of the compressed data and can also provide the decompression metadata. As shown, the number of banks M can be calculated from the metadata.

[0081] In another aspect, the same decompression engine (e.g., the same decompressor hardware) may be used to decompress data whose words are of different lengths. For example, the same decompressor hardware may be used for INT8 words and FP16 (or INT16) words. In this example, the base decompression hardware may be designed for the minimum word length (e.g., INT8). Then, in the FP16 scheme, the lower byte (e.g., weight lower byte) and the upper byte of the data may be stored in separate blocks, and each block may be decompressed using a parallel decompression engine. In this example, the same metadata (e.g., the same map metadata 142) may be used for each block. On the other hand, if the engine is decompressing a base word (e.g., an INT8 word), parallel engines may be used to decompress different lines.

[0082] 9 illustrates an exemplary apparatus 900 configured to perform decompression on stack-compressed data. Apparatus 900 is represented as a series of interrelated functional modules 910-980 connected by a common bus. The apparatus 900 may include a module 910 for initializing location row and / or compressed bank indexes to perform block 610 of method 600, a module 920 for reading map metadata to perform block 630 of method 600, a module 930 for reading column metadata to perform block 805 of method 600, a module 940 for reading compressed banks to perform block 810 of method 600, a module 950 for column shifting to perform block 820 of method 600, a module 960 for loading compressed data into a queue to perform blocks 710 and 830 of method 600, a module 970 for incrementing location row and / or compressed bank indexes to perform blocks 660 and 680 of method 600, and a module 980 for popping data from a queue to perform block 670 of method 600. Each of modules 910-980 may be implemented in hardware or as a combination of hardware and software. For example, modules 910-980 may be implemented as any combination of components of device 500 of FIG.

[0083] The functionality of the modules in FIG. 9 may be implemented in various ways consistent with the teachings herein. In some designs, the functionality of these modules may be implemented as one or more electrical components. In some designs, the functionality of these blocks may be implemented as a processing system including one or more processor components. In some designs, the functionality of these modules may be implemented, for example, using at least a portion of one or more integrated circuits (e.g., ASICs). As described herein, an integrated circuit may include a processor, software, other related components, or some combination thereof. Thus, the functionality of different modules may be implemented, for example, as different subsets of an integrated circuit, as different subsets of a set of software modules, or a combination thereof. It will also be appreciated that a given subset (e.g., of an integrated circuit and / or a set of software modules) may provide at least a portion of the functionality for more than one module.

[0084] Additionally, the components and functions represented by FIG. 9, as well as other components and functions described herein, may be implemented using any suitable means. Such means may also be implemented, at least in part, using corresponding structure as taught herein. For example, components described above in conjunction with "modules for" components in FIG. 9 may also correspond to "means for" the similarly designated function. Thus, in some aspects, one or more of such means may be implemented using one or more of a processor component, an integrated circuit, or other suitable structure as taught herein.

[0085] The proposed decompression engine can provide both footprint (capacity) and bandwidth compression. Compression can be in both SRAM and DDR memory types. The decompression engine can operate at high speeds, e.g., SRAM speeds. Furthermore, decompression can restore the original data without loss.

[0086] FIG. 10 shows simulation results of the effective compression achieved by the proposed compression / decompression for a given number of banks and sparsity level. In FIG. 10, the numbers represent the relative size of the compressed data block to the uncompressed block. Therefore, smaller numbers represent greater compression. As can be seen, effective compression generally correlates with sparsity. This makes sense because compression is achieved by removing zero words from the uncompressed data block. Note that in one aspect, the vertical ratio in FIG. 10 may represent power, capacity, and bandwidth savings. The horizontal ratio may represent power savings when the same number of banks is used for storage and reading.

[0087] Those skilled in the art will appreciate that information and signals may be represented using any of a variety of different technologies and techniques. For example, the data, instructions, commands, information, signals, bits, symbols, and chips that may be referenced throughout the above description may be represented by voltages, currents, electromagnetic waves, magnetic fields or particles, optical fields or particles, or any combination thereof.

[0088] Furthermore, those skilled in the art will appreciate that the various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the aspects disclosed herein may be implemented as electronic hardware, computer software, or a combination of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the particular application and design constraints imposed on the overall system. Those skilled in the art may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present disclosure.

[0089] The various example logic blocks, modules, and circuits described in connection with the aspects disclosed herein may be implemented or performed using a general-purpose processor, a DSP, an ASIC, an FPGA or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A general-purpose processor may be a microprocessor, but alternatively, the processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of computing devices, e.g., a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration.

[0090] The methods, sequences, and / or algorithms described in connection with the aspects disclosed herein may be embodied directly in hardware, in software modules executed by a processor, or in a combination of the two. The software modules may reside in random access memory (RAM), flash memory, read-only memory (ROM), erasable programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor such that the processor can read information from, and write information to, the storage medium. Alternatively, the storage medium may be integral to the processor. The processor and the storage medium may reside in an ASIC. The ASIC may reside in a user terminal (e.g., UE). Alternatively, the processor and the storage medium may reside as discrete components in the user terminal.

[0091] In one or more exemplary aspects, the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored on or transmitted over as one or more instructions or code on a computer-readable medium. Computer-readable media includes both computer storage media and communication media, including any medium that facilitates transfer of a computer program from one place to another. Storage media may be any available medium that can be accessed by a computer. By way of example, and not limitation, such computer-readable media may comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer. Also, any connection is properly termed a computer-readable medium. For example, if software is transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included within the definition of medium. Disk and disc, as used herein, include compact discs (CDs), laser discs, optical discs, digital versatile discs (DVDs), floppy disks, and Blu-ray discs, where disks typically reproduce data magnetically and discs reproduce data optically using lasers. Combinations of the above should also be included within the scope of computer-readable media.

[0092] While the above disclosure illustrates exemplary aspects of the present disclosure, it should be noted that various changes and modifications can be made herein without departing from the scope of the present disclosure as defined by the appended claims. The functions, steps, and / or actions of the method claims in accordance with the aspects of the present disclosure described herein need not be performed in any particular order. Furthermore, although elements of the present disclosure may be described or claimed in the singular, the plural is contemplated unless limitation to the singular is explicitly stated. [Explanation of symbols]

[0093] 110 Line 120 First Memory 122 uncompressed data blocks 125 uncompressed banks 125-0 Uncompressed Bank 125-1 Uncompressed Bank 125-2 Uncompressed Bank 125-3 Uncompressed Bank 130 Second Memory 132 compressed data blocks 133 Intermediate Compressed Data Blocks 135 Compression Bank 135-0 Compression Bank 135-1 Compression Bank 140 Metadata Storage 142 Map Metadata 144 num-compressed-bank metadata 250 queues 250-0 queue 250-1 queue 250-2 queue 250-3 Queue 250-c queue 255 Compressed Bank Index Registers 260 Map Register 265 Row Index Register 270 Line Register 332 compressed data blocks 335 Compression Bank 335-0 Compression Bank 335-1 Compression Bank 346 Column Metadata 475 Shifter 500 devices 510 memory 520 Metadata Storage 530 Working Memory 540 Decompression Engine 550 Processing System 900 equipment 910 Module for initializing location rows and / or compressed bank indexes 920 Module for reading map metadata 930 Module for reading column metadata Module for reading 940 compressed banks Module for shifting 950 columns 960 Module for loading compressed data into a queue 970 Module for incrementing location row and / or compressed bank indexes 980 Module for popping data from a queue

Claims

1. An apparatus comprising a processor and a memory, wherein the processor and the memory are configured to read a compressed data block and map metadata, and uncompress the compressed data block to generate a decompressed data block according to the map metadata wherein the compressed data block includes one or more words, and the map metadata is configured to generate the decompressed data block by mapping the words of the compressed data block, the compressed data block is compressed from an uncompressed data block including a plurality of words, each word of the uncompressed data block being removable or non-removable, the removable words having removable values and the non-removable words having values other than the removable values, the uncompressed data block is compressed into the compressed data block by removing one or more removable words from the uncompressed data block, the map metadata and the uncompressed data block are arranged in the same N rows × C columns dimension, and the compressed data block is arranged in an M rows × C columns dimension, where M < N, the map metadata maps the removable and non-removable words of the uncompressed data block, and the uncompressed data block is restored within the decompressed data block. An apparatus.

2. The apparatus according to claim 1, wherein the plurality of words of the uncompressed data block correspond to weights of a neural network.

3. The compressed data block is arranged in row and column dimensions, including N (N≧1) compressed banks in the row dimension and D (D≧1) columns in the column dimension, the uncompressed data block is arranged in row and column dimensions, including N (N≧1) uncompressed banks in the row dimension and C columns in the column dimension, and the uncompressed data block is stack-compressed into the compressed data block, at least in the column dimension, such that D < C. The apparatus according to claim 1.

4. The compressed data block is arranged in row and column dimensions, including M (M≧1) compressed banks in the row dimension and C (C≧1) columns in the column dimension, The non-compressed data block includes N (N≧1) non-compressed banks in the row dimension and also includes C columns in the column dimension, and is arranged in the dimensions of the rows and columns. The non-compressed data block is stack-compressed into the compressed data block at least in the row dimension such that M < N. The apparatus according to claim 1, wherein at least one column of the compressed data block is a column in which all words of the at least one column are non-removable words.

5. The map metadata includes a matrix of N rows × C columns of elements, whereby each element (n, c) of the map metadata, where n = 0...N-1, c = 0...C-1, corresponds to a word in bank n, column c of the non-compressed data block. The apparatus according to claim 4, wherein each element of the map metadata includes a bit configured to indicate whether the corresponding word of the non-compressed data block is removable or non-removable.

6. Each element of the map metadata also corresponds to a word in bank n, column c of the decompressed data block. When decompressing the compressed data block to generate the decompressed data block, the processor and the memory fill the corresponding word of the decompressed data block with the removable word for each element of the map metadata indicating that the corresponding word of the non-compressed data block is removable, and load one of the non-removable words from the compressed data block into the corresponding word of the decompressed data block for each element of the map metadata indicating that the corresponding word of the non-compressed data block is non-removable. The apparatus according to claim 5, which is configured to perform the above.

7. The processor and the memory are configured to decompress the compressed data block to restore the non-compressed data block based on num-compressed-bank metadata in addition to the map metadata. The apparatus according to claim 5, wherein the num-compressed-bank metadata is configured to indicate the number M of the compressed banks of the compressed data block.

8. The processor and the memory Reading the num-compressed-bank metadata together with the compressed data block and the map metadata, and / or Determining the num-compressed-bank metadata based on the map metadata The apparatus according to claim 7, configured to perform the above.

9. The apparatus further includes at least C queues corresponding to the C columns, Each queue has a depth of the number S of slots (S≧1), When decompressing the compressed data block to generate the decompressed data block, the processor and the memory Initialize the row index n, Initialize the compressed bank index m, When the row index is n<N, execute a row decompression operation And is configured to perform the above, and the row decompression operation Read the row n of the map metadata based on the row index, When the compressed bank index is m<M, Load the word of the compressed bank m of the compressed data block into the queue, and Increment the compressed bank index m, Pop the word from the head of each queue c as decompressed bank data based on the row n read from the map metadata, Increment the row index n And include The decompression engine is configured to repeat the row decompression operation for each row index n. The apparatus according to claim 7.

10. When loading the word of the compressed bank m of the compressed data block into the queue, the processor and the memory are configured to load each word corresponding to each column c of the compressed bank m to occupy the lowest unoccupied slot of the corresponding queue c. The apparatus according to claim 9.

11. The uncompressed data block is stack-compressed in both the row and column dimensions, so that in each compressed bank m of the compressed data block, the non-removable words occupy the leftmost column c=0 of that compressed bank m. the processor and the memory are configured to load the words of the compressed bank m of the compressed data block into the queue based on column metadata configured to map removable and non-removable words of the compressed data block after compression in the row dimension and before compression in the column dimension; The processor and the memory reading the column metadata along with the compressed data blocks and the map metadata; and / or determining the column metadata based on the map metadata; configured to: the column metadata comprises an M row by C column matrix, whereby each element (m, c), m=0...M-1, c=0...C-1 of the column metadata corresponds to a word in bank m, column c of the compressed data block after the compression in the row dimension and before the compression in the column dimension; each element of the column metadata includes a bit configured to indicate whether a corresponding word in the compressed data block is removable or non-removable; In loading the words of the compressed bank m of the compressed data block into the queue, the processor and the memory: reading row m of the column metadata based on the compressed bank index; reading the word of the compressed bank m of the compressed data block based on the row m read from the column metadata; column-shifting the word read from the compressed bank m of the compressed data block according to the row m read from the column metadata; loading each shifted word of said compressed data block corresponding to each column c of said compression bank m to occupy a lowest unoccupied slot of a corresponding queue c; configured to: the decompression engine is configured, when reading the compressed bank m of the compressed data block, to perform a masked read based on the row m read from the column metadata; 10. The apparatus of claim 9, wherein a masked read is a read operation in which only k words from the left end of the compressed bank m are read, where k is a number of bits that indicate that a corresponding word of the uncompressed data block in the row m of the column metadata cannot be removed.

12. The apparatus of claim 1 , wherein uncompressed data is recovered without loss within the decompressed data block.

13. 1. A method of a decompression engine, comprising: reading the compressed data block and map metadata; decompressing the compressed data block to generate a decompressed data block in accordance with the map metadata; Including, wherein the compressed data block comprises one or more words; the map metadata is configured to map the words of the compressed data block to generate the decompressed data block; the compressed data block is compressed from an uncompressed data block comprising a plurality of words, each word of the uncompressed data block being removable or non-removable, a removable word having a removable value and a non-removable word having a value other than the removable value; the uncompressed data block is compressed into the compressed data block by removing one or more removable words from the uncompressed data block; The map metadata and the uncompressed data blocks are arranged in the same dimension of N rows by C columns, and the compressed data blocks are arranged in the dimension of M rows by C columns. <Nであり、 the map metadata maps the removable words and the non-removable words of the uncompressed data block; The uncompressed data blocks are reconstructed within the decompressed data blocks.

14. 14. A non-transitory computer-readable medium having stored thereon computer-executable instructions for an apparatus, the computer-executable instructions including one or more instructions that cause the apparatus to perform the method of claim 13.

Images