Inline decompression

Abstract

Techniques and apparatuses for decompressing data that has been stack compressed are described. Stack compression refers to compression of data in one or more dimensions. Stack compression can be efficient for very sparse uncompressed data blocks (i.e., data blocks containing many zeros). In stack compression, an uncompressed data block is compressed into a compressed data block by removing one or more zero words from the uncompressed data block. Mapping metadata of the zero words of the uncompressed data block is generated during compression. Using the mapping metadata, the compressed data block can be decompressed to recover the uncompressed data block.

Description

[0001] This patent application claims the benefit of U.S. nonprovisional application No. 16 / 870,873, filed May 8, 2020, entitled “INLINE DECOMPRESSION,” which has been assigned to the assignee of this application and is expressly incorporated herein by reference in its entirety. Technical Field

[0002] The various aspects described in this article generally involve compression and decompression, and in particular, inline decompression of data that has been vertically and / or horizontally compressed. Background Technology

[0003] In neural networks (NNs), data with many zeros is often processed. This is especially true in NNs with Corrected Linear Unit (ReLU) activation, which outputs zero when the input is zero and the input value when it is positive. If the network is pruned, there can also be numerous zeros. For example, in a pruned recurrent neural network (RNN), nearly 90% of the weights can be pruned. The weights in a convolutional neural network (CNN) can also be sparse. Summary of the Invention

[0004] This overview identifies some exemplary features and is not an exclusive or exhaustive description of the disclosed subject matter. Whether features or aspects are included in or omitted from this overview is not intended to indicate their relative importance. Additional features and aspects are described, and these will become apparent to those skilled in the art upon reading the following detailed description and examining the accompanying drawings that form a part of it.

[0005] An exemplary apparatus is disclosed. The apparatus may include a processor and a memory. The processor and the memory are configured to retrieve compressed data blocks and mapping metadata. The compressed data block may include one or more words. The mapping metadata is configured to map the words of the compressed data block to a generated decompressed data block. The processor and the memory are also configured to decompress the compressed data block according to the mapping metadata to generate the decompressed data block.

[0006] An exemplary method is disclosed. The method may include retrieving a compressed data block and mapping metadata. The compressed data block may include one or more words. The mapping metadata may be configured to map the words of the compressed data block to a generated decompressed data block. The method may also include decompressing the compressed data block according to the mapping metadata to generate the decompressed data block.

[0007] Another exemplary apparatus is disclosed. This apparatus may include means for retrieving a compressed data block and mapping metadata. The compressed data block may include one or more words. The mapping metadata may be configured to map the words of the compressed data block to a generated decompressed data block. The apparatus may also include means for decompressing the compressed data block according to the mapping metadata to generate the decompressed data block.

[0008] A non-transient computer-readable medium storing computer-executable instructions for a device is disclosed. These executable instructions may include one or more instructions that cause the device to retrieve a compressed data block and mapping metadata. The compressed data block may include one or more words. The mapping metadata may be configured to map the words of the compressed data block to a resulting decompressed data block. The executable instructions may also include one or more instructions that cause the device to decompress the compressed data block according to the mapping metadata to generate the decompressed data block.

[0009] Other objectives and advantages associated with the aspects disclosed herein will be apparent to those skilled in the art based on the accompanying drawings and detailed description. Attached Figure Description

[0010] The accompanying drawings are provided to help describe examples of one or more aspects of the disclosed subject matter, and these drawings are provided merely to illustrate the examples and not to limit the scope thereof:

[0011] Figure 1 The text describes an example of stack compression, which can be performed by a compression engine to generate compressed data blocks from uncompressed data blocks, based on one or more aspects.

[0012] Figures 2A-2D The explanation describes the functions that can be performed by the decompression engine based on one or more aspects for compressing... Figure 1 The various stages of the decompression operation to decompress compressed data blocks into decompressed data blocks;

[0013] Figure 3 This illustrates another example of stack compression, which can be performed by a compression engine to generate compressed data blocks from uncompressed data blocks, based on one or more aspects;

[0014] Figures 4A-4D The explanation describes the functions that can be performed by the decompression engine based on one or more aspects for compressing... Figure 3 The various stages of the decompression operation to decompress compressed data blocks into decompressed data blocks;

[0015] Figure 5 A block diagram illustrating several exemplary aspects of components that may be used in a device configured to decompress compressed data blocks, based on one or more aspects;

[0016] Figure 6-8 An exemplary method for decompressing compressed data blocks by a decompression engine according to one or more aspects is explained;

[0017] Figure 9 Simplified block diagrams illustrating several exemplary aspects of a device configured to decompress compressed data blocks according to one or more aspects are explained; and

[0018] Figure 10 The simulation results of compression achieved by the proposed compression / decompression technology are explained based on one or more aspects. Detailed Implementation

[0019] Various aspects of this subject matter are provided in the following description and related figures of specific examples of the disclosed subject matter. Alternative solutions can be devised without departing from the scope of the disclosed subject matter. Furthermore, well-known elements will not be described in detail or will be omitted to avoid obscuring the relevant details.

[0020] The term “exemplary” is used herein to mean “serving as an example, instance, or illustration.” Any aspect described herein as “exemplary” need not be construed as superior to or better than other aspects. Similarly, the term “aspect” does not require all aspects to include the features, advantages, or modes of operation discussed.

[0021] The terminology used herein describes specific aspects only and should not be construed as limiting any aspect disclosed herein. As used herein, the singular forms “a,” “some,” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. Those skilled in the art will further understand that the terms “comprising,” “having,” “including,” and / or “containing,” as used herein, specify the presence of the stated features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0022] Furthermore, the aspects can be described as sequences of actions performed by elements, such as computing devices. Those skilled in the art will recognize that the various actions described herein can be performed by special-purpose circuitry (e.g., application-specific integrated circuits (ASICs)), by program instructions being executed by one or more processors, or by a combination of both. Additionally, the sequences of actions described herein can be considered to be fully implemented within any form of non-transitory computer-readable medium storing a corresponding set of computer instructions that, upon execution, will cause the associated processor to perform the functionality described herein. Thus, the aspects described herein can be implemented in several different forms, all of which are contemplated to fall within the scope of the claimed subject matter. Furthermore, for each aspect described herein, any corresponding form of such aspect can be described herein as, for example, "logic configured to perform the described actions" and / or other structural components configured to perform the described actions.

[0023] The preceding text pointed out that neural networks can have many zeros. As an explanation, the sparsity of the RNN weight matrix / tensor can approach 90%. This can be advantageous for large RNNs if the weights can be compressed. One advantage is that the number of weights that can be accommodated in on-chip memory or SRAM can be increased. Another advantage is that the bandwidth of on-chip static random access memory (SRAM) can be reduced. Yet another advantage is that the bandwidth of dynamic RAM (e.g., double data rate (DDR) memory) can be reduced when it is necessary to overflow weights into memory (e.g., RAM, DRAM, DDR, SDR, QDR, etc.).

[0024] In one or more aspects, techniques are proposed for efficiently compressing and decompressing data with many zeros, such as in neural network environments. Note that the weights in a neural network inference accelerator tend not to change. Therefore, these weights can be compressed offline. The compressed weights and their corresponding metadata (which maps to the zero and non-zero words of the original uncompressed data) can be loaded into the device's memory and subsequently into another memory of the device (e.g., tightly coupled memory (TCM)). The device may include a decompression engine that decompresses the compressed weights. The decompression engine can be purely hardware or a combination of hardware and software.

[0025] While neural networks have been discussed, this is not the only context in which the proposed technique is applicable. The proposed technique can be applied to systems that introduce sparsity into weight tensors, such as pruned machine learning (ML) networks. Even more generally, it is envisioned that the proposed technique can be applied to any environment where data with a considerably large number of zeros is being processed, such as when dealing with sparse matrices. In such instances, significant compression can be achieved by reducing the number of zeros stored in the compressed data relative to the original data.

[0026] As mentioned above, the weights can be compressed by the compression engine. More generally, the raw data can be compressed by the compression engine, while the proposed decompression(s) can be executed by the decompression engine. The proposed decompression engine can be pure software, for example, implemented as executable instructions stored in memory and executed by one or more general-purpose processors. However, for performance reasons, it may be preferred that the decompression engine be implemented entirely in hardware or at least partially based on hardware (e.g., application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), etc.). Similarly, the compression engine can also be implemented entirely in software, entirely in hardware, or a combination of software and hardware.

[0027] The compression and decompression engines can be physically separate, or they can be at least partially integrated into each other within a single device. However, in the following discussion, the compression and decompression engines will be logically considered as separate and independent components. In other words, the compression and decompression engines can operate independently of each other. For example, the compression engine can operate to compress the raw data into compressed data and store the compressed data in storage, without considering when the compressed data will be decompressed. The decompression engine can later obtain the compressed data (e.g., by accessing the storage and / or by receiving it from a component that has access to the storage) and decompress the compressed data back into the original data. Of course, it is also envisioned that the compression and decompression engines can communicate with each other (e.g., directly and / or through an intermediary component) so that compression and decompression occur within a defined time frame.

[0028] In one or more aspects, a compression engine can perform stack compression to compress raw data into compressed data. Stack compression can be viewed as a compression scheme in which raw data organized in multiple dimensions is compressed in at least one of those dimensions. For example, the raw data may be organized in two dimensions (e.g., organized into rows and columns), and the compressed data may be the raw data compressed in the row and / or column dimensions.

[0029] Before proceeding further, it should be noted that data (e.g., raw and / or decompressed) does not need to be physically organized in multiple dimensions. For example, raw data may be stored in memory locations with coherent addresses. In fact, in extreme cases, raw data may be stored in one or more non-coherent locations if the order of the raw data can be maintained (e.g., via pointers, tables, etc.). Of course, if physical memory is organized in multiple dimensions (e.g., pages, groups, rows, columns, words, etc.), it can also be advantageous to organize data (raw and / or decompressed) to match these memory dimensions or otherwise utilize physical memory efficiently.

[0030] Figure 1 An example of stack compression, which can be performed by a compression engine to generate compressed data from raw data, is explained. This compression can be performed offline, i.e., without considering when or even if the compressed data will be decompressed. This means that compression can be performed in software where compression speed is not an issue and / or software compression performance is sufficient. As an alternative and / or supplement to this, compression can be combined with (see...) Figure 2A – Decompress the corresponding 2D description to execute.

[0031] In one or more aspects, the raw data (which can be of any size) can be divided into groups, also known as uncompressed data blocks. Each group (i.e., each uncompressed data block) can be compressed into a compressed data block by a compression engine stack. The decompression engine can decompress each compressed data block to recover the corresponding uncompressed data block.

[0032] exist Figure 1 In the diagram, line 110 represents an example of an uncompressed data block 122 (example block) that can be stack-compressed from raw, uncompressed data. Line 110 may include multiple words of raw data (16 words in this example). The raw data in line 110 can be read in groups and stored in the first memory 120. That is, the uncompressed data block 122 can be stored in the first memory 120. N Groups (or rows) and C The columns are stored in the first memory 120. Again, it should be noted that "group" and "column" are conceptual. Groups and / or columns may or may not have a correspondence with physical memory.

[0033] On one hand, the granularity of a read operation can be a word or a group. In this example, a group comprises four words; that is, four words of line 110 can be read at once (for group read granularity) for storage in the first memory 120. For clarity, the uncompressed data block 122 can be considered as stored in... N In the uncompressed groups, N ≥ 1 Each uncompressed group n (n = 0 ... N – 1 )include C Listing characters C ≥ 1 .exist Figure 1 In the middle, there are four ( N = 4 ) uncompressed groups 125-n ( n = 0、1、2、 3, collectively referred to as uncompressed group 125), each uncompressed group 125-n has four ( C = 4 (Number of words). Note that a word can be of any fixed length (e.g., 16 bits, 32 bits, etc.).

[0034] exist Figure 1 In this context, a word can be a zero word or a non-zero word. A zero word (displayed as "0") can represent a word with a value of zero. Conversely, a non-zero word (displayed as having content other than "0" (e.g., displayed as "A", "B", "C", etc.)) can represent a word with a value that is not zero.

[0035] Uncompressed data blocks 122 in uncompressed group 125 can be stacked and compressed in one dimension (e.g., in the row dimension) (also known as vertical compression), resulting in compressed data blocks 132... M One compressed group ( M ≥ 1 ) are stored in the second memory 130. Each compressed group m ( m = 0 ... M – 1 It may also include C Columns. Note that the first and second memories 120 and 130 may be the same or different.

[0036] In this row compression, uncompressed data block 122 can be compressed into compressed data block 132 by removing one or more zero words from the uncompressed data block. On one hand, leading and intermediate zero words in each column of the uncompressed group 125 can be removed (i.e., compressed) so that non-zero words appear in the compressed group 135-m ( m = 0、1 These are collectively referred to as compressed group 135 and pushed down. For example, it can be seen that the position occupied by uncompressed group 125 (i.e., before compression) is... n = 3, c = 0 The non-zero value "F" occupies position 135 in the compressed group. m = 0, c = 0 This indicates the position of the uncompressed group 125. n = 0, c = 0 ), ( n = 1, c = 0 )and( n = 2, c = 0 The zero word is removed. As another example, it can be seen that the position occupying the uncompressed group 125 before compression ( n = 3, c = 3 The non-zero value "9" occupies position 135 in the compressed group. m = 1, c = 3 This indicates the location ( n = 1, c = 3 )and( n = 2, c = 3 The zero in ) was removed.

[0037] By using vertical compression, the number of groups can be reduced, that is, M <N .exist Figure 1 In this process, the four uncompressed groups 125 in the first memory 120 are reduced to two compressed groups 135 in the second memory 130. When there is a column within the uncompressed group 125... c The worst-case scenario occurs when all words contain non-zero characters; in this case, M = N Therefore, generally speaking, M ≤ N .

[0038] Some characteristics / results of stack compression will now be described. One characteristic is that non-zero words of the original data in uncompressed data block 122 can be preserved in compressed data block 132 without modification, even if their position may change. For example, the position of uncompressed group 125 ( n = 2, c = 1 The non-zero word "E" in ) can be stored at position 135 of the compressed group ( m = 1, c = 1 The same non-zero word "E" in ).

[0039] This implies that the number of non-zero words in compressed data block 132 is still equal to the number of non-zero words in uncompressed data block 122. On the other hand, the number of zero words in compressed data block 132 can be less than or equal to the number of zero words in uncompressed data block 122. Basically, when M <N At that time, the number of zero words in compressed data block 132 will be less.

[0040] In another characteristic, regarding M Compressed data blocks 132 in compressed group 135 may exist such that they are applicable to all M Each compressed group of 135 words consists of at least one column of non-zero words. For example, in Figure 1 From this, we can see that M One compressed group 135 c = 1, 2 and 3 All rows are completely occupied by non-zero words. This is a natural consequence of the explained stack compaction, since further row compaction would occur if all columns had at least one zero word.

[0041] In further features, also regarding M Within a compressed data block 132 of compressed group 135, it is possible for a column to contain both zero words and non-zero words. When such a column exists, then within that column, each zero word occupies a group above the group occupied by each non-zero word. More formally, for each column... c , c = 0 ... C – 1 When the column c Targeting M When compressed group 135 contains both zero and non-zero words, in this column... c Each compressed group containing a non-zero wordm NZ It can be compared to the column. c Each compressed group containing a zero m Z The lower group, that is, for M Each column c of the compressed group 135, m NZ < m Z For example, in Figure 1 From this, it can be seen that in the list c = 0 From this, it can be seen that the non-zero word F occupies the compressed group 135-0 ( m NZ = 0 And zero occupies group 135-1 ( m Z = 1 One result is M The lowest group of each column of the compressed group 135 ( m = m NZ = 0 () is occupied by non-zero words.

[0042] As part of the compression operation, the compression engine can generate mapping metadata 142. As will be described in further detail below, mapping metadata 142 can be used by the decompression engine to decompress the compressed data block 132. Broadly speaking, mapping metadata 142 can map all non-zero words of the original uncompressed data. For example, for N uncompressed groups 125, mapping metadata 142 can map the positions of zero and non-zero words in the uncompressed data block 122. For example, each bit of mapping metadata 142 can correspond to the word position of the N uncompressed groups 125, that is, it can correspond to a word in the uncompressed data block 122. Moreover, the bit values ​​within mapping metadata 142 can indicate whether the corresponding word in the uncompressed data block 122 is a zero word or a non-zero word. Therefore, the size of mapping metadata 142 can be a fraction inversely proportional to the word size of the uncompressed data block 122 (e.g., 1 / 8, 1 / 16, etc.). Mapping metadata 142 can be stored in metadata storage 140.

[0043] Although not shown, note that if all words in the original data line 110 are zero words, the mapped metadata 142 can be correspondingly filled with all zero bits. When this occurs, the uncompressed data block 122 may contain only zero words. That is, N uncompressed groups 125 can be filled with zero words. Furthermore, the compressed data block 132 can be empty, i.e., the number of compressed groups... M = 0 .

[0044] exist Figure 1 In the example, mapping metadata 142 is interpreted as N line x C A matrix of columns, where each element of the mapping metadata 142 ( n , c )( n = 0 ... N – 1, c = 0 ... C – 1 ) corresponds to the uncompressed group of the original data n List c The text in the image. As shown, each element of the mapping metadata 142 ( n , c The value is zero when the corresponding word in the uncompressed data block 122 is zero, and non-zero when the corresponding word in the original data is non-zero. As indicated above, each element of the mapping metadata 142 ( n , c The size of the data block 122 can be smaller than the word size of the uncompressed data block 122. For example, each element of the mapping metadata 142 ( n , c () can be a single bit indicating whether the corresponding word of uncompressed data block 122 is a zero word or a non-zero word.

[0045] Before proceeding further, the following points should be noted. So far, the described interpretive compression has involved distinguishing between zero words and non-zero words, and compressing zero words. However, this is merely an example. It is conceivable to generalize this concept to distinguish between "removable" and "non-removable" words, where a removable word can be a word with a removable value (which can be any value, such as zero), while a non-removable word can be a word with a value other than a removable value. In this example, mapping metadata 142 can map to removable and non-removable words in the uncompressed data block 122, and each bit of the mapping metadata 142 can indicate whether the corresponding word in the uncompressed data block 122 is a removable or non-removable word.

[0046] Furthermore, this concept can be extended to allow for multiple removable values. For example, the original data might allow three values ​​to appear significantly more often than others. In this instance, each element ( n , c The word can be two bits long, where one combination of the two bits (e.g.,

[11] ) represents a non-removable word, while the other three combinations (e.g., [00, 01, 10]) represent the first, second, and third values ​​of a removable word.

[0047] However, for ease of reference and discussion, the rest of this disclosure will use zero and non-zero words. Please remember that the concepts discussed can be summarized in a straightforward manner.

[0048] The compression engine can also generate compressed group number metadata 144. Compressed group number metadata 144 can represent the compressed group number...M Equal values. In addition to metadata size 142, the decompression engine can also decompress compressed data blocks 132 based on compressed group number metadata 144. Compressed group number metadata 144 can also be stored in metadata storage 140.

[0049] It should be noted that the compressed group number metadata 144 is optional, as the compression engine does not need to provide it. This is because the compressed group number metadata 144 can be determined during decompression based on the mapping metadata 142. For example, recall that compressed data block 132 is the result of the compression engine stacking and compressing uncompressed data blocks 122 from N uncompressed groups 125 in one dimension (row dimension in this instance). A similar dimensional compression can be performed by the decompression engine on the mapping metadata 142 to determine the compressed group number. M However, when compressed group data 144 is provided as part of the compression, this means fewer operations will be required during decompression.

[0050] Figure 2A – 2D narration can be handled by the decompression engine Figure 1 The various stages of the decompression operation performed on the compressed data to generate decompressed data blocks (which should be identical to the original uncompressed data block 122). Figure 2A – In 2D, the second memory 130 and metadata storage 140 will not be described to reduce clutter. Decompression operations are the focus of these figures. Therefore, when memory is mentioned in the context of decompression operations, it can be assumed that the second memory 130 is being referenced, unless otherwise stated.

[0051] exist Figure 2A In, it may exist in the compressed group 135. C Corresponding columns C There are 250 queues. Each queue, 250-c, can have a depth of... S Number of time slots. S It can be any number S ≥ 1 However, in most instances, S = N / 2 That might be sufficient. Each queue 250-c can have a separate counter (which is not interpreted to reduce noise). Furthermore, a mapping register 260 and a line register 270 can be provided to assist decompression. For example, line register 270 can be filled with decompressed data blocks during decompression.

[0052] Additionally, row index register 265 can be provided to maintain row indexes. n The count, and can provide the compressed group index register 255 to maintain the compressed group index. mThe count. It should be noted that some or all of these registers are not required. For example, decompressed groups can be streamed from the decompression engine. In this example, decompressed data blocks can be streamed four words at a time (i.e., one group at a time).

[0053] Recall from the above, in some instances, the uncompressed data block 122 may consist of only zero words. In this instance, the operations performed by the decompression engine can be greatly simplified. For example, the decompression engine can first determine whether compressed data to be decompressed exists. This determination can be made by examining the compressed group number metadata 144, which may be provided by the compression engine or derived from the mapping metadata 142. If the compressed group number metadata 144 indicates... M = 0 (Without compressed data), the decompression engine can simply fill line register 270 with all-zero words—filling the decompressed data block.

[0054] On the other hand, if the compressed group data 144 indicates M>0 Then the decompression engine can decompress the compressed data block 132. Figure 2A – The 2D explanation describes what happens when there is data to be decompressed (i.e., when...). M>0 Example phase (at time). Figure 2A , 2B 2C and 2D respectively explain the extraction of group 0, group 1, group 2 and group 3 values ​​from the uncompressed data block 122 from the compressed data block 132.

[0055] Figure 2A The decompression process, executed by the decompression engine to generate group 0 of decompressed data blocks (group 0 of uncompressed data block 122), can be as follows:

[0056] Initialize row index n :

[0057] exist Figure 2A In the middle, row index register 265 is set to 0 ( n = 0 ).

[0058] Due to row index n < N Therefore, will be carried out n (That is, line 0) is read from mapping metadata 142 into mapping register 260:

[0059] exist Figure 2A In the mapping register 260, line 0 entry [0,0,1,1] of the mapping metadata 142 is loaded.

[0060] Initialize compressed group index m :

[0061] exist Figure 2AIn the compressed group index register 255, it is set to 0 ( m = 0 ).

[0062] Due to the compressed group index m < M Therefore, the compressed data block 132 is compressed into a compressed group. m (That is, the word of group 0 in compressed group 135) is loaded into queue 250 so that the compressed group m Each column c The word occupies 250 in the corresponding queue. c The minimum unused time slot:

[0063] exist Figure 2A In this context, before compressed data is loaded from compressed group 135-0 to queue 250, time slot 0 of these queues 250 is unoccupied. Therefore, time slot 0 of queue 250-c (c = 0, 1, 2, 3) is loaded with [F, C, A, B] from compressed group 135-0.

[0064] Based on the lines of mapping metadata 142 (stored in mapping register 260) n The entry pops the word from the head of queue 250 (which is slot 0) and loads the popped word into the corresponding position in line register 270:

[0065] exist Figure 2A In the middle, word positions 0-3 of line register 270 correspond to line 0 of the mapped metadata 142, and also to group 0 of the uncompressed data block 122 and group 0 of the decompressed data block.

[0066] Mapping register 260 is ANDed with queue 250 to indicate the following: the column in mapping register 260 c The "0" in the text indicates the corresponding queue 250- c It should not be popped from the stack. Instead, the zero word should be loaded into the corresponding location in line register 270 (of the decompressed data block).

[0067] Columns of Mapped Register 260 c The "1" indicates that the corresponding queue 250-c should be popped from the stack, and the popped word should be loaded into the corresponding position in the line register 270 (of the decompressed data block).

[0068] The mapping register is loaded with line 0 entry [0,0,1,1] of mapping metadata 142 to indicate that the data values ​​“A” and “B” in the head (slot 0) of queues 250-2 and 250-3 are to be popped from the stack and loaded into word positions 2 and 3 of line register 270 corresponding to columns 2 and 3 of group 0 of the decompressed data block.

[0069] The word positions 0 and 1 of line register 270 corresponding to columns 0 and 1 of group 0 of the decompressed data block are filled with zeros.

[0070] As a result, the group 0 value [0,0,A,B] of the uncompressed group 125-0 was restored.

[0071] exist Figure 2A In China (and in Figure 2B – In 2D, an example is included, which includes a multiplexer to indicate how the mapping register 260 can be used to pop queue 250 from the stack during recovery. As can be seen, in one aspect, entries in the mapping register 260 can be used to control the multiplexer. However, it should be noted that the actual implementation can be done in a variety of ways.

[0072] Figure 2B The decompression process, executed by the decompression engine to generate group 1 of decompressed data blocks (group 1 of uncompressed data block 122), can be as follows:

[0073] Incrementing row index n :

[0074] exist Figure 2B In the middle, row index register 265 is incremented ( n = 1 ).

[0075] Due to row index n < N Therefore, will be carried out n (That is, line 1) reads from mapping metadata 142 into mapping register 260:

[0076] exist Figure 2B In the mapping register 260, line 1 entry [0,1,1,0] of the mapping metadata 142 is loaded.

[0077] Due to the compressed group index m < M Therefore, the index is incremented via the compressed group. m :

[0078] exist Figure 2B In the middle, the compressed group index register 255 is incremented ( m = 1 ).

[0079] Due to the incremental compressed group index m < M Therefore, the compressed data block 132 is compressed into a compressed group. m (That is, the words of group 1 in compressed group 135) are loaded into queue 250 so that the compressed group m Each column c The word occupies 250 in the corresponding queue. c The minimum unused time slot:

[0080] exist Figure 2BIn the context of queues 250-0 and 250-1, the lowest unoccupied time slot for both is time slot 1, because their heads (time slot 0) were not previously popped from the stack. Thus, time slot 1 of queues 250-0 and 250-1 is loaded with [0, E] from columns 0 and 1 of compressed group 135-1.

[0081] For queues 250-2 and 250-3, the lowest unoccupied time slot for both is time slot 0, because their heads were previously popped from the stack. Thus, time slot 0 of queues 250-2 and 250-3 is loaded with [D,9] from columns 2 and 3 of compressed group 135-1.

[0082] Based on the lines of mapping metadata 142 (stored in mapping register 260) n The entry pops the word from the head (slot 0) of queue 250 from the stack and loads the popped word into the corresponding position in line register 270:

[0083] exist Figure 2B In the middle, word positions 4-7 of line register 270 correspond to line 1 of the mapped metadata 142, and also to group 1 of the uncompressed data block 122 and group 1 of the decompressed data block.

[0084] Mapping register 260 is loaded with line 1 entry [0,1,1,0] of mapping metadata 142 to indicate that the data values ​​“C” and “D” in the head (slot 0) of queues 250-1 and 250-2 are to be popped from the stack and loaded into word positions 5 and 6 of line register 270 corresponding to columns 1 and 2 of group 1 of the decompressed data block.

[0085] Word positions 4 and 7 of line register 270, corresponding to columns 0 and 3 of group 1 of the decompressed data block, are filled with zeros.

[0086] As a result, the group 1 value [0,C,D,0] of the uncompressed group 125-1 was restored.

[0087] Figure 2C The decompression process, executed by the decompression engine to generate group 2 of decompressed data blocks (group 2 of uncompressed data block 122), can be as follows:

[0088] Incrementing row index n :

[0089] exist Figure 2C In the middle, row index register 265 is incremented ( n = 2 ).

[0090] Due to row index n < N Therefore, will be carried out n (That is, line 2) read from mapping metadata 142 into mapping register 260: in Figure 2CIn the mapping register 260, the mapping metadata 142 is loaded with line 2 entry [0,1,0,0].

[0091] Due to the compressed group index m < M Therefore, the index is incremented via the compressed group. m :

[0092] exist Figure 2B In the middle, the compressed group index register 255 is incremented ( m = 2 ).

[0093] Due to the incremental compressed group index m is not less than M This indicates that all compressed groups 135 have been read, so no loading of words from compressed data block 132 (compressed group 135) will occur.

[0094] exist Figure 2C In the queues 250-0, 250-1, 250-2, and 250-3, slot 0 has the value [F, E, empty, 9]. This is because... Figure 2B The state of these queues after they are popped from the stack.

[0095] Based on the lines of mapping metadata 142 (stored in mapping register 260) n The entry pops the word from the head (slot 0) of queue 250 from the stack and loads the popped word into the corresponding position in line register 270:

[0096] exist Figure 2C In the middle, word positions 8-11 of line register 270 correspond to line 2 of the mapped metadata 142, and also to group 2 of the uncompressed data block 122 and group 2 of the decompressed data block.

[0097] Mapping register 260 is loaded with line 2 entry [0,1,1,0] of mapping metadata 142 to indicate that the data value “E” in the head of queue 250-1 is to be popped from the stack and loaded into word position 9 of line register 270 corresponding to column 1 of group 2 of the decompressed data block.

[0098] Word positions 8, 10, and 11 of line register 270 corresponding to columns 0, 2, and 3 of group 2 of the decompressed data block are filled with zeros.

[0099] As a result, the group 2 value [0,E,0,0] of the uncompressed group 125-2 was restored.

[0100] Figure 2D The decompression process, executed by the decompression engine to generate group 3 of decompressed data blocks (group 3 of uncompressed data block 122), can be as follows:

[0101] Incrementing row index n :

[0102] exist Figure 2D In the middle, row index register 265 is incremented ( n = 3 ).

[0103] Due to row index n < N Therefore, will be carried out n (That is, line 3) read from mapping metadata 142 into mapping register 260:

[0104] exist Figure 2D In the mapping register 260, the mapping metadata 142 is loaded with line 3 entry [1,0,0,1].

[0105] Due to the incremental compressed group index m is not less than M This indicates that all compressed groups 135 have been read, so no loading of words from compressed data block 132 (compressed group 135) will occur.

[0106] exist Figure 2D In the queue, slot 0 of 250-0, 250-1, 250-2, and 250-3 has the value [F, empty, empty, 9]. This is... Figure 2C The state of these queues after they are popped from the stack.

[0107] Based on the lines of mapping metadata 142 (stored in mapping register 260) n The entry pops the word from the head (slot 0) of queue 250 from the stack and loads the popped word into the corresponding position in line register 270:

[0108] exist Figure 2D In the middle, word positions 12-15 of line register 270 correspond to line 3 of the mapped metadata 142, and also to group 3 of the uncompressed data block 122 and group 3 of the decompressed data block.

[0109] Mapping register 260 is loaded with line 3 entry [1,0,0,1] of mapping metadata 142 to indicate that the data values ​​“F” and “9” in the headers of queues 250-0 and 250-3 are to be popped from the stack and loaded into word positions 12 and 15 of line register 270 corresponding to columns 0 and 3 of group 3 of the decompressed data block.

[0110] Word positions 13 and 14 of line register 270, corresponding to columns 1 and 2 of group 3 of the decompressed data block, are filled with zeros.

[0111] As a result, the group 3 value [F,0,0,9] of the uncompressed group 125-3 was recovered. In addition, all the original data in line 110 was recovered losslessly.

[0112] Note that, in one or more aspects, the first group (e.g., group 0) may vary depending on the address to which the group is to be read. Furthermore, the actual number of words in each group (e.g., actual...) C ) and the number of groups (e.g., actual M The number of pages can vary. For example, assuming a 2K page size, memory groups 120 and 130 (and therefore uncompressed and compressed groups) could be organized into groups of 128 x 16b words (e.g., organized into 8 groups of 16 words each) or groups of 64 x 16b words (e.g., organized into 16 groups of 4 words each). Of course, these are just examples, and other organization is possible. Compression is also limited by the worst-case column. More groups can achieve more efficient compression. However, latency increases with the number of groups.

[0113] Figure 3 Another example of stack compression, which can be performed by a compression engine to generate compressed data blocks from raw data, is explained. Figure 3 Compression can be performed offline. As a replacement and / or supplement to this, Figure 3 The compression can be used in conjunction with (see below) Figure 4A – Decompress the corresponding 4D description to execute. Figure 3 Stack compression and Figure 1 The difference lies in the ability to perform additional compression on another dimension (e.g., the column / horizontal dimension). For example, after performing vertical compression, the compression engine can perform horizontal compression on the vertically compressed data.

[0114] exist Figure 3 middle, Figure 1 The same original data will be used for the description. To reduce noise, Figure 3 Memory 120, 130 and metadata storage 140 are not shown. Figure 3 In the process, after compression in the first dimension (e.g., vertical compression) (which generates intermediate compressed data block 133), the state of the compressed group 135 is... Figure 1 The compressed group 135 is the same. That is, the intermediary compressed data block 133 is the same as... Figure 1 The compressed data block 132 is identical. However, after compression in the second dimension (e.g., horizontal compression) (to generate the final compressed data block 332 (or simply compressed data block 332)), the results are that these values ​​can be shifted left to remove zero "holes" from the intermediate compressed data block 133. This results in the omission of reading zero words from memory, thereby saving power. For illustrative purposes, the compressed group after horizontal compression is designated as "335".

[0115] Figure 3 Stack compression (also known as 2D compression) can be shared. Figure 1The stack compression (also known as 1D compression) features (e.g., the same number of non-zero words as the original data, unmodified non-zero words, the same number or fewer zero words as the original data, etc.).

[0116] Some different characteristics may also exist. For example, each compressed group 335 may begin with a non-zero word. That is, each compressed group in compressed group 335... m In the middle, the compressed group m Each column with a non-zero word c NZ It can be more than the compressed group m Each column with the word zero c Z The lower columns. For example, it can be seen that in the compressed group 335-1, non-zero words occupy columns 0, 1, and 2, and subsequently, zero words occupy column 3.

[0117] Although not shown, if all words in the original data line 110 are zero words, the difference between 2D compression and 1D compression may be very small. For example, uncompressed data block 122 may contain only zero words, mapped metadata 142 may be filled only with zero bits, and compressed data block 332 may be empty (i.e., compressed group number). M = 0 ).

[0118] Thus, there may be almost no difference in decompression as well. That is, the decompression engine can determine, for example, by examining the compressed group number metadata 144 (which may be provided by the compression engine or derived by the decompression engine from the mapping metadata 142). If the compressed group number metadata 144 indicates... M = 0 Then the decompression engine can simply fill line register 270 with all zeros (i.e., fill the decompressed data block).

[0119] Figures 4A-4D The explanation can be handled by the decompression engine. Figure 3 The 2D compressed data block 332 is executed when the compressed data block 332 is not empty (i.e., when M > 0 The decompression operation (at time) generates decompressed data blocks through various stages. Figures 4A-4DIn this context, it can be assumed that memories 120 and / or 130 can be “read under mask” (explained below). Also in these figures, in addition to load-mapped metadata 142 and compressed group metadata 144, metadata storage 140 (not shown) may also store column metadata 346, which can map the zero and non-zero positions of compressed data blocks 332 after compression in the row dimension and before compression in the column dimension. That is, the column metadata 346 can map the zero and non-zero positions (more generally, removable and non-removable positions) of intermediate compressed data blocks 133.

[0120] Similar to the compressed group metadata 144, the compression engine can provide column metadata 346. However, this is again optional from the compression engine's perspective, as column metadata 346 can be determined by the decompression engine based on mapping metadata 142. For example, note that 2D compressed data block 332 is the result of the compression engine performing stacked compression on N uncompressed data blocks 122 in N uncompressed groups 125 in two dimensions. A similar two-dimensional compression can be performed by the decompression engine on the mapping metadata 142 to determine column metadata 346. Nevertheless, when column metadata 346 is provided as part of the compression, fewer operations will be required during decompression.

[0121] exist Figures 4A-4D In this configuration, it can be assumed that queue 250, mapping register 260, line register 270, row index register 265, and compressed group index register 255 may also be used. Furthermore, shifter 475 may be provided, for example, to provide column decompression capability.

[0122] Figure 4A The decompression process, executed by the decompression engine to generate group 0 of decompressed data blocks (group 0 of uncompressed data block 122), can be as follows:

[0123] Initialize row index n :

[0124] exist Figure 4A In the middle, row index register 265 is set to 0 ( n = 0 ).

[0125] Due to row index n < N Therefore, will be carried out n (That is, line 0) is read from mapping metadata 142 into mapping register 260:

[0126] exist Figure 4A In the mapping register 260, line 0 entry [0,0,1,1] of the mapping metadata 142 is loaded.

[0127] Initialize compressed group index m:

[0128] exist Figure 4A In the compressed group index register 255, it is set to 0 ( m = 0 ).

[0129] Due to the compressed group index m < M Therefore, rows are read from column data 346. m (That is, line 0):

[0130] exist Figure 4A In the text, the row 0 entry [1,1,1,1] of column metadata 346 is highlighted ( Bold, Italic ).

[0131] Due to the compressed group index m < M Perform a masked read to read the compressed data block 332. m (That is, the words of group 0 of group 335 are compressed) and these words are loaded into queue 250.

[0132] Reading under the mask means only the compressed group is read. m (Group 0) The leftmost k The read operation in which each character is read, k It is row 346 of the data column. m The number of 1s in (row 0). In this example, because k = 4 Therefore, all four words [F,C,A,B] of the compressed group 335-0 are read into the column shifter 475.

[0133] exist Figure 4A In the data [F,C,A,B] read from the compressed group 335-0, it is not shifted in the shifter 475 because there is no zero in the row 0 entry [1,1,1,1] of the column data 346.

[0134] Slot 0 of queue 250 is loaded with [F,C,A,B] from shifter 475.

[0135] Based on the lines of mapping metadata 142 (stored in mapping register 260) n The entry pops the word from the head (slot 0) of queue 250 from the stack and loads the popped word into the corresponding position in line register 270:

[0136] exist Figure 4A In the middle, word positions 0-3 of line register 270 correspond to line 0 of the mapped metadata 142, and also to group 0 of the uncompressed data block 122 and group 0 of the decompressed data block.

[0137] The mapping register is loaded with line 0 entry [0,0,1,1] of mapping metadata 142 to indicate that the data values ​​“A” and “B” in the head (slot 0) of queues 250-2 and 250-3 are to be popped from the stack and loaded into word positions 2 and 3 of line register 270 corresponding to columns 2 and 3 of group 0 of the decompressed data block.

[0138] The word positions 0 and 1 of line register 270 corresponding to columns 0 and 1 of group 0 of the decompressed data block are filled with zeros.

[0139] As a result, the group 0 value [0,0,A,B] of the uncompressed group 125-0 was restored.

[0140] Similar to Figure 2A-2D ,exist Figures 4A-4D The example includes an AND gate to indicate how the mapping register 260 can be used to pop queue 250 from the stack during recovery. The actual implementation can be done in a variety of ways. For example, in one aspect, entries in the mapping register 260 can be used to control a multiplexer (not shown).

[0141] Figure 4B The decompression process, executed by the decompression engine to generate group 1 of decompressed data blocks (group 1 of uncompressed data block 122), can be as follows:

[0142] Incrementing row index n :

[0143] exist Figure 4B In the middle, row index register 265 is incremented ( n = 1 ).

[0144] Due to row index n < N Therefore, will be carried out n (That is, line 1) reads from mapping metadata 142 into mapping register 260:

[0145] exist Figure 4B In the mapping register 260, line 1 entry [0,1,1,0] of the mapping metadata 142 is loaded.

[0146] Due to the compressed group index m < M Therefore, the index is incremented via the compressed group. m :

[0147] exist Figure 4B In the middle, the compressed group index register 255 is incremented ( m = 1 ).

[0148] Due to the incremental compressed group index m < M Therefore, rows are read from column data 346. m (i.e., line 1):

[0149] exist Figure 4B In the text, the entry [0,1,1,1] in row 1 of column metadata 346 is highlighted. Bold, Italic ).

[0150] Due to the incremental compressed group index m < M Therefore, a masked read is performed to read the compressed data block 332. m (That is, the words of group 1 in compressed group 335) are based on the rows read from column data 346. m For from the compressed group m The word is shifted column by column and loaded with the column. c The corresponding shifted word occupies the corresponding queue 250- c The lowest unused time slot.

[0151] In this example, k = 3 Therefore, the three leftmost words [E,D,9] of the compression group 335-1 are read into the shifter 475 when the masked reading is executed.

[0152] exist Figure 4B In the process, the read data [E,D,9] is column-shifted in shifter 475 according to the row 1 entry [0,1,1,1] of the column data 346 (it is shifted by one column to [0,E,D,9] because the first entry 0 in row 1 of the column data 346 is shifted).

[0153] For queues 250-0 and 250-1, the lowest unoccupied time slot is time slot 1 (because their head (time slot 0) was not previously popped from the stack), while for queues 250-2 and 250-3, the lowest unoccupied time slot is time slot 0 (because their head (time slot 0) was previously popped from the stack). Time slot 1 of queues 250-0 and 250-1 is loaded with [0, E], while time slot 0 of queues 250-2 and 250-3 is loaded with [D, 9].

[0154] Based on the lines of mapping metadata 142 (stored in mapping register 260) n The entry pops the word from the head (slot 0) of queue 250 and loads the popped word into the corresponding position in line register 270: In Figure 4B In the middle, word positions 4-7 of line register 270 correspond to line 1 of the mapped metadata 142, and also to group 1 of the uncompressed data block 122 and group 1 of the decompressed data block.

[0155] Mapping register 260 is loaded with line 1 entry [0,1,1,0] of mapping metadata 142 to indicate that the data values ​​“C” and “D” in the head (slot 0) of queues 250-1 and 250-2 are to be popped from the stack and loaded into word positions 5 and 6 of line register 270 corresponding to columns 1 and 2 of group 1 of the decompressed data block.

[0156] Word positions 4 and 7 of line register 270, corresponding to columns 0 and 3 of group 1 of the decompressed data block, are filled with zeros.

[0157] As a result, the group 1 value [0,C,D,0] of the uncompressed group 125-1 was restored.

[0158] Figure 4C The decompression process, executed by the decompression engine to generate group 1 of decompressed data blocks (group 1 of uncompressed data block 122), can be as follows:

[0159] Incrementing row index n :

[0160] exist Figure 4C In the middle, row index register 265 is incremented ( n = 2 ).

[0161] Due to row index n < N Therefore, will be carried out n (That is, line 2) read from mapping metadata 142 into mapping register 260:

[0162] exist Figure 4C In the mapping register 260, the mapping metadata 142 is loaded with line 2 entry [0,1,0,0].

[0163] Due to the compressed group index m < M Therefore, the index is incremented via the compressed group. m :

[0164] exist Figure 4B In the middle, the compressed group index register 255 is incremented ( m = 2 ).

[0165] Due to the compressed group index m is not less than M This indicates that all rows of column data 346 have been read, therefore no reading of column data 346 will occur.

[0166] exist Figure 4C In the text, the rows containing column metadata 346 are not highlighted.

[0167] Due to the incremental compressed group index m is not less than M This indicates that all compressed groups 335 have been read, so no loading of words from compressed data block 332 (compressed group 335) will occur.

[0168] exist Figure 4C In the queues 250-0, 250-1, 250-2, and 250-3, slot 0 has the value [F, E, empty, 9]. This is because... Figure 4B The state of these queues after they are popped from the stack.

[0169] Based on the lines of mapping metadata 142 (stored in mapping register 260) n The entry pops the word from the head (slot 0) of queue 250 from the stack and loads the popped word into the corresponding position in line register 270:

[0170] exist Figure 4C In the middle, word positions 8-11 of line register 270 correspond to line 2 of the mapped metadata 142, and also to group 2 of the uncompressed data block 122 and group 2 of the decompressed data block.

[0171] Mapping register 260 is loaded with line 2 entry [0,1,0,0] of mapping metadata 142 to indicate that the data value “E” in the head of queue 250-1 is to be popped from the stack and loaded into word position 9 of line register 270 corresponding to column 1 of group 2 of the decompressed data block.

[0172] Word positions 8, 10, and 11 of line register 270 corresponding to columns 0, 2, and 3 of group 2 of the decompressed data block are filled with zeros.

[0173] As a result, the group 2 value [0,E,0,0] of the uncompressed group 125-2 was restored.

[0174] Figure 4D The decompression process, executed by the decompression engine to generate group 3 of decompressed data blocks (group 3 of uncompressed data block 122), can be as follows:

[0175] Incrementing row index n :

[0176] exist Figure 4D In the middle, row index register 265 is incremented ( n = 3 ).

[0177] Due to row index n < N Therefore, will be carried out n (That is, line 3) read from mapping metadata 142 into mapping register 260:

[0178] exist Figure 4D In the mapping register 260, the mapping metadata 142 is loaded with line 3 entry [1,0,0,1].

[0179] Due to the incremental compressed group index m is not less than MThis indicates that all compressed groups 335 have been read, so no loading of words from compressed data block 332 (compressed group 335) will occur.

[0180] exist Figure 4D In the queue, slot 0 of 250-0, 250-1, 250-2, and 250-3 has the value [F, empty, empty, 9]. This is... Figure 4C The state of these queues after they are popped from the stack.

[0181] Based on the lines of mapping metadata 142 (stored in mapping register 260) n The entry pops the word from the head (slot 0) of queue 250 from the stack and loads the popped word into the corresponding position in line register 270:

[0182] exist Figure 4D In the middle, word positions 12-15 of line register 270 correspond to line 3 of the mapped metadata 142, and also to group 3 of the uncompressed data block 122 and group 3 of the decompressed data block.

[0183] Mapping register 260 is loaded with line 3 entry [1,0,0,1] of mapping metadata 142 to indicate that the data values ​​“F” and “9” in the headers of queues 250-0 and 250-3 are to be popped from the stack and loaded into word positions 12 and 15 of line register 270 corresponding to columns 0 and 3 of group 3 of the decompressed data block.

[0184] Word positions 13 and 14 of line register 270, corresponding to columns 1 and 2 of group 3 of the decompressed data block, are filled with zeros.

[0185] As a result, the group 3 value [F,0,0,9] of the uncompressed group 125-3 was recovered. In addition, all the original data in line 110 was recovered losslessly.

[0186] Figure 5 An exemplary apparatus 500 configured to perform decompression on stack-compressed data is described. For example, apparatus 500 can generate a decompressed data block by decompressing compressed data blocks 132, 332, in which uncompressed data block 122 is recovered. Apparatus 500 may include a memory 510 comprising multiple groups. Although not described, memory 510 may be configured to store uncompressed data block 122 (e.g., ...). N Uncompressed groups 125) and / or compressed data blocks 132, 332 (e.g., un ...) M Each compressed group 135 and / or M(Compressed group 335). Memory 510 may include one or more volatile and / or non-volatile memory components. Memory 500 may include SRAM and / or DRAM (e.g., DDR).

[0187] The apparatus 500 may also include metadata storage 520. Although not described, metadata storage 520 may be configured to store mapping metadata 142, compressed group number metadata 144, and / or column metadata 346. Metadata storage 520 may include buffers, registers, and / or memory. The memory of metadata storage 520 may be the same as or different from the memory used to store uncompressed group 125, compressed group 135, and / or compressed group 335. In one or more aspects, metadata for compressed data blocks (e.g., mapping metadata 142, compressed group number metadata 144, and / or column metadata 346) may be stored together with compressed data blocks 132, 332.

[0188] The device 500 may further include working memory 530. Although not described, working memory 530 may include shifter 475, queue 250, mapping register 260, line register 270 and / or line 210. Working memory 530 may include buffers, registers and / or memory.

[0189] The apparatus 500 may further include a decompression engine 540 configured to decompress stack-compressed data. The apparatus 500 may further include a processing system 550 configured to control the overall operation of the apparatus 500. The processing system 550 may include one or more processors (not shown). In one aspect, the processing system 550 may contribute to the decompression process, i.e., it may be part of the decompression engine 540.

[0190] On one hand, the decompression engine 540 can be incorporated into a neural network (NN) system. That is, the device 500 can be an NN system, or at least a part of an NN system. On the other hand, the decompression engine 540 can be at least partially configured in hardware. Furthermore, the decompression engine 540 can completely recover the original data.

[0191] Figure 6 An exemplary method 600 performed by a decompression engine (e.g., any combination of decompression engine 540 of device 500 and / or processing system 550) is described. In block 610, the decompression engine may initialize the row index. n And compressed group index m For example, row index register 265 and compressed group index register 255 can be set to 0.

[0192] In box 620, the decompression engine can determine whether the decompression operation is complete. For example, the decompression engine can determine whether...n < N That is, the value in the decompression engine's feasible index register 265. n Is it less than the number of uncompressed groups? N If no (from the no branch of box 620), then decompression can be considered complete. On the other hand, if n < N (From the negative branch of box 620), then in box 630, the decompression engine can read rows from the mapping metadata 142. n .

[0193] In box 640, the decompression engine can determine whether all groups of compressed data blocks 132, 332 (e.g., compressed groups 135, 335) have been read. For example, the decompression engine can determine whether... m < M That is, the decompression engine can determine the value in the compressed group index register 255. m Is it less than the value of the compressed group data element 144? M If no (from the no branch of box 640), the decompression engine can proceed to box 670 (described further below).

[0194] If so (the branch is from box 640), then in box 650, the decompression engine can decompress the compressed data blocks 132 and 332. m The word (for example, from the compressed groups 135, 335) m The data is loaded into queue 250. Figure 7 The example process for implementing box 650, executed by the decompression engine, is explained. Figure 7 The process of interpreting data can be applied to 1D compressed data blocks (such as...) Figure 1 The compressed data blocks 132 stored in the compressed group 135 are decompressed. In block 710, the decompression engine can read the compressed data blocks 132 stored in the compressed group 135. m The word, and loaded with the compressed group. m Each column c Each corresponding character occupies 250 slots in its respective queue. c The lowest unoccupied time slot ( For example, see Figure 2A – 2D ).

[0195] Regarding time slot 250, please note the following. Figure 2A – 2D, 4A – In 4D, each queue of 250-c includes S There are 10 time slots, each of which is addressed as a time slot. s ( s = 0 … S – 1 The head of each queue 250-c is time slot 0, which is the lowest addressable time slot of queue 250-c. Therefore, including the head time slot... 0The lowest occupied slot is the unoccupied slot closest to the head. However, it should be recognized that in other queue implementations, the head can be the highest addressable slot. S – 1 In such implementations, from compressed groups m The read compressed data can be loaded to occupy the highest unoccupied slot in the corresponding queue. However, this also includes the header slot. S – 1 The highest unoccupied slot is closest to the head of the queue. Therefore, in one or more aspects, the "lowest unoccupied slot" can generally be interpreted as the unoccupied slot "closest to the head of the queue".

[0196] Figure 8 Another example process for implementing box 650, executed by the decompression engine, is explained. Figure 8 The process of interpreting data can be applied to 2D compressed data blocks (such as...) Figure 3 The compressed data blocks 332 stored in compressed group 335 are decompressed. In box 805, the decompression engine can read rows of column data 346. m For example, the decompression engine can read rows of column data 346 based on row values ​​in the compressed group index register 255. In box 810, the decompression engine can read rows of column data 346 based on the rows read. m To read the compressed group of compressed data block 332 m The word (e.g., reading the compressed group 335) m For example, a masked read can be performed to read compressed group 335-. m The far left k The character, among which k It is row 346 of the read column data. m The number of 1s (more generally, the number of entries with non-zero words). In box 820, the decompression engine can determine the number of rows read from column metadata 346. m To the compressed group from the compressed group 335 m The words read from the array are column-shifted. In box 830, the decompression engine can load the compressed data block 332 along with the compressed group. m Each column c Each corresponding shifted word occupies 250 bytes in its respective queue. c The lowest unoccupied time slot ( For example, see Figure 4A – 4D ).

[0197] Refer to the return Figure 6 In box 660, the decompression engine can increment the compressed group index in compressed group index register 255. m Then, in box 670, the decompression engine can use the lines read from the mapping metadata 142 as a basis. nPop the word from the head (slot 0) of each queue 250 and output the popped word as the decompressed group data. For example, see Figure 2A – 2D, 4A – 4D For example, a popped word can be placed at position... n The corresponding column is placed in line register 270. c And / or streamed as output. In box 680, the decompression engine can increment the row index in row index register 265. n Subsequently, the decompression engine can proceed to box 620.

[0198] Although not specifically explained, capacity compression can be easily added by including the starting address of the line in the metadata. For example, a 1KB line can be easily compressed. The metadata can be addressed using a given line index. Furthermore, the metadata (e.g., mapping metadata 142) can be read before the compressed data. This can provide the memory address of the compressed data (e.g., the SRAM address), and also the decompression metadata. As noted, the number of groups... M It can be calculated based on metadata.

[0199] On the other hand, the same decompression engine (e.g., the same decompression hardware) can be used to decompress data of different word lengths. For example, the same decompression hardware can be used for INT8 words and FP16 (or INT16) words. In this example, the basic decompression hardware can be designed for the smallest word length (e.g., INT8). Then, in the FP16 scheme, the low byte (e.g., the weighted low byte) and high byte of data can be stored in separate blocks, and a parallel decompression engine can be used to decompress each block. In this example, the same metadata (e.g., the same mapping metadata 142) can be used for each block. On the other hand, if the parallel engine is decompressing a basic word (e.g., an INT8 word), these engines can be used to decompress different lines.

[0200] Figure 9An example device 900 configured to decompress stacked data is described. Device 900 is represented as a series of interconnected functional modules 910–980 connected by a common bus. Apparatus 900 may include a module 910 of block 610 of method 600 for initializing position rows and / or compressed group indexes, a module 920 of block 630 of method 600 for reading mapping metadata, a module 930 of block 805 of method 600 for reading column metadata, a module 940 of block 810 of method 600 for reading compressed groups, a module 950 of block 820 of method 600 for column shifting, a module 960 of blocks 710 and 830 of method 600 for loading compressed data into a queue, a module 970 of blocks 660 and 680 of method 600 for incrementing position rows and / or compressed row indexes, and a module 980 of block 670 of method 600 for popping data from the queue. Each of modules 910–980 may be implemented in hardware or may be implemented as a combination of hardware and software. For example, modules 910-980 can be implemented as Figure 5 Any combination of the components of the device 500.

[0201] Figure 9 The functionality of the modules can be implemented in various ways consistent with the teachings herein. In some designs, the functionality of these modules can be implemented as one or more electrical components. In some designs, the functionality of these modules can be implemented as a processing system including one or more processor components. In some designs, the functionality of these modules can be implemented using, for example, at least a portion of one or more integrated circuits (e.g., ASICs). As discussed herein, integrated circuits may include processors, software, other related components, or some combination thereof. Therefore, the functionality of different modules can be implemented, for example, as different subsets of integrated circuits, different subsets of software modules, or combinations thereof. Moreover, it will be appreciated that a given subset (e.g., integrated circuits and / or software modules) can provide at least a portion of the functionality of more than one module.

[0202] in addition, Figure 9 The components and functions represented, as well as other components and functions described herein, can be implemented using any suitable means. Such means can also be implemented, at least in part, using the corresponding structures taught herein. For example, the above combination... Figure 9 The components described in the term "module for..." can also correspond to the functionality of "device for..." similarly named. Thus, in some aspects, one or more such devices can be implemented using one or more of the processor components, integrated circuits, or other suitable structures taught herein.

[0203] The proposed decompression engine offers both space (capacity) and bandwidth compression. Compression can be performed on both SRAM and DDR memory. The decompression engine can operate at a faster speed (e.g., SRAM speed). Furthermore, decompression can losslessly recover the original data.

[0204] Figure 10 Simulation results demonstrating efficient compression achieved using the proposed compression / decompression method at a given number of groups and sparsity level are presented. Figure 10 In this diagram, these numbers represent the relative sizes of compressed and uncompressed data blocks. Thus, smaller numbers indicate greater compression. As shown, effective compression is generally related to sparsity. This is logical, as compression is achieved by removing zeros from uncompressed data blocks. It's important to note that, on the one hand, Figure 10 The vertical ratio can represent savings in power, capacity, and bandwidth. The horizontal ratio can represent power savings when the same number of groups are used for storage and retrieval.

[0205] Those skilled in the art will appreciate that information and signals can be represented using any of a variety of different techniques and skills. For example, data, instructions, commands, information, signals, bits, symbols, and chips that may be referred to throughout the above description can be represented by voltage, current, electromagnetic waves, magnetic fields or magnetic particles, light fields or light particles, or any combination thereof.

[0206] Furthermore, those skilled in the art will appreciate that the various illustrative logic blocks, modules, circuits, and algorithmic steps described in connection with the aspects disclosed herein can be implemented as electronic hardware, computer software, or a combination of both. To clearly illustrate this interchangeability between hardware and software, the various illustrative components, blocks, modules, circuits, and steps are described above in a generalized manner in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the specific application and the design constraints imposed on the overall system. Those skilled in the art may implement the described functionality in different ways for each specific application, but such implementation decisions should not be construed as departing from the scope of this disclosure.

[0207] The various illustrative logic blocks, modules, and circuits described in conjunction with the aspects disclosed herein can be implemented or executed using a general-purpose processor, DSP, ASIC, FPGA, or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. The general-purpose processor may be a microprocessor, but in alternatives, it may be any conventional processor, controller, microcontroller, or state machine. The processor can also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors cooperating with a DSP core, or any other such configuration.

[0208] The methods, sequences, and / or algorithms described in conjunction with the aspects disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of both. The software module may reside in random access memory (RAM), flash memory, read-only memory (ROM), erasable programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), registers, hard disks, removable disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to a processor such that the processor can read and write information from / to the storage medium. In an alternative, the storage medium may be integrated into the processor. The processor and storage medium may reside in an ASIC. The ASIC may reside in a user terminal (e.g., a UE). In an alternative, the processor and storage medium may reside as discrete components in the user terminal.

[0209] In one or more exemplary aspects, the described functionality may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functionality may be stored or transmitted as one or more instructions or codes on or through a computer-readable medium. A computer-readable medium includes both computer storage media and communication media, including any medium that facilitates the transfer of a computer program from one location to another. A storage medium may be any available medium accessible to a computer. By way of example and not limitation, such computer-readable media may include RAM, ROM, EEPROM, CD-ROM or other optical disc storage, disk storage or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and is accessible to a computer. Similarly, any connection is also legitimately referred to as a computer-readable medium. For example, if the software is transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, then such coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of medium. As used in this article, disks and discs include compact discs (CDs), laser discs, optical discs, digital multi-purpose discs (DVDs), floppy disks, and Blu-ray discs. Disks typically reproduce data magnetically, while discs reproduce data optically using lasers. Combinations of these should also be included within the scope of computer-readable media.

[0210] Although the foregoing disclosure illustrates illustrative aspects of this disclosure, it should be noted that various changes and modifications may be made therein without departing from the scope of this disclosure as defined by the appended claims. The functions, steps, and / or actions in the method claims according to the aspects of this disclosure described herein need not be performed in any particular order. Furthermore, although elements of this disclosure may be described or claimed in the singular, pluralism is also contemplated unless explicitly stated to be limited to the singular.

Claims

1. An apparatus for decompression, comprising: At least one processor and at least one memory, wherein one or more of the at least one processor and the at least one memory are configured to: Read compressed data blocks and mapping metadata, wherein the compressed data blocks include one or more words, and the mapping metadata is configured to map the words of the compressed data blocks to the generated decompressed data blocks; as well as The compressed data block is decompressed according to the mapping metadata to generate the decompressed data block. The compressed data block is formed by compressing an uncompressed data block comprising multiple words, each word of which may or may not be removable. The uncompressed data block is compressed into the compressed data block by removing one or more removable words from it. The mapping metadata maps the removable and non-removable words of the uncompressed data block, and The uncompressed data blocks are recovered from the decompressed data blocks.

2. The apparatus as described in claim 1, The words in the uncompressed data block correspond to the weights of the neural network.

3. The apparatus of claim 1, wherein: The compressed data blocks are arranged in row and column dimensions, including those in the row dimension. N One compressed group, N ≥ 1 and the column dimensions D List, D ≥ 1 , The uncompressed data blocks are arranged according to the row dimension and the column dimension, including the data blocks in the row dimension. N One uncompressed group, N ≥ 1 And including the column dimension C Column, and The uncompressed data block has been stack-compressed into the compressed data block at least along the column dimension, so that... D < C .

4. The apparatus as claimed in claim 1, The compressed data blocks are arranged in row and column dimensions, including those in the row dimension. M One compressed group, M ≥ 1 and the column dimensions C List, C ≥ 1 , The uncompressed data blocks are arranged according to the row dimension and the column dimension, including the data blocks along the row dimension. N One uncompressed group, N ≥ 1 And also includes the column dimension C Column, and The uncompressed data block has been stack-compressed into the compressed data block at least along the row dimension, so that... M < N .

5. The apparatus of claim 4, wherein at least one column of the compressed data block satisfies the condition that all words in the at least one column are non-removable words.

6. The apparatus as claimed in claim 4, The mapping metadata includes N line x C A matrix of column elements, such that each element of the mapping metadata ( n , c ), n = 0 ... N – 1, c = 0 ... C – 1 Groups corresponding to the uncompressed data blocks n List c The characters in the middle, and Each element of the mapping metadata includes bits configured to indicate whether the corresponding word of the uncompressed data block is removable or non-removable.

7. The apparatus as claimed in claim 6, Each element of the mapping metadata also corresponds to a group of the decompressed data blocks. n List c The characters in the middle, and When decompressing the compressed data block to generate the decompressed data block, the processor and the memory are configured to: For each element of the mapping metadata indicating that the corresponding word of the uncompressed data block is not removable, the corresponding word of the decompressed data block is filled with the removable word; and For each element of the mapping metadata indicating that the corresponding word of the uncompressed data block is not removable, the corresponding word of the decompressed data block is loaded with one of the non-removable words from the compressed data block.

8. The apparatus of claim 6, wherein the processor and the memory are configured to: decompress the compressed data block to recover the uncompressed data block based on the compressed group number metadata and the mapping metadata, wherein the compressed group number metadata is configured to indicate the compressed group number of the compressed data block. M .

9. The apparatus of claim 8, wherein the processor and the memory are configured to: Read the compressed group metadata along with the compressed data blocks and the mapping metadata, and / or The compressed group number metadata is determined based on the mapping metadata.

10. The apparatus as claimed in claim 8, The device further includes components related to the... C At least the corresponding column C There are 1 queue, each with a depth of 1. S Number of time slots, S ≥ 1 , When decompressing the compressed data block to generate the decompressed data block, the processor and the memory are configured to: Initialize row index n ; Initialize compressed group index m ; When the row index n < N At that time, a line decompression operation is performed, which includes: The rows of the mapping metadata are read based on the row index. n ; When the compressed group index m < M hour, The compressed group of the compressed data block m The word is loaded into the queue; and Increment the compressed group index m ; Based on the rows read from the mapping metadata n The coming will come from each queue c The header word popped from the stack is the decompressed group data; and Increment the row index n ,and The decompression engine is configured to work for each row index. n Repeat the decompression operation described above.

11. The apparatus of claim 10, wherein the compressed group of the compressed data block is... m When a word is loaded into the queue, the processor and the memory are configured to load the compressed group. m Each column c Each corresponding character occupies its corresponding queue. c The lowest unused time slot.

12. The apparatus of claim 10, The uncompressed data blocks have been stacked and compressed in both the row and column dimensions to such that each compressed group of the compressed data blocks... m In the compressed group, the unremovable characters occupy space. m The leftmost column c = 0 ,and The processor and the memory are configured to compress the compressed data blocks into compressed groups based on column metadata. m The words are loaded into the queue, and the column metadata is configured to map the removable and non-removable words of the compressed data block after compression on the row dimension and before compression on the column dimension.

13. The apparatus of claim 12, wherein the processor and the memory are configured to: Read the column metadata along with the compressed data block and the mapping metadata, and / or The column metadata is determined based on the mapping metadata.

14. The apparatus as claimed in claim 12, The column metadata includes M line x C A matrix of columns is formed such that each element of the column metadata ( m, c ), m = 0 ... M – 1, c = 0 ... C – 1 The group corresponding to the compressed data blocks after compression in the row dimension and before compression in the column dimension. m List c The characters in the middle, Each element of the column metadata includes bits configured to indicate whether the corresponding word in the compressed data block is removable or non-removable. Among them, the compressed group of the compressed data block m When a word is loaded into the queue, the processor and the memory are configured to: Rows of column metadata read based on the compressed group index m ; Based on the rows read from the column metadata m To read the compressed group of the compressed data block m The character; Based on the rows read from the column metadata m For the compressed group from the compressed data block m The words read from the middle are shifted column by column; as well as Load the compressed data block and the compressed group m Each column c Each shifted word occupies its corresponding queue. c The lowest unused time slot.

15. The apparatus of claim 14, wherein the decompression engine is configured to: read the compressed group of the compressed data blocks m The time is based on the row read from the column metadata. m To perform a read under the mask.

16. The apparatus of claim 1, wherein the uncompressed data is losslessly recovered from the decompressed data block.

17. A method for a decompression engine, the method comprising: Read compressed data blocks and mapping metadata, wherein the compressed data blocks include one or more words, and the mapping metadata is configured to map the words of the compressed data blocks to the generated decompressed data blocks; as well as The compressed data block is decompressed according to the mapping metadata to generate the decompressed data block. The compressed data block is formed by compressing an uncompressed data block comprising multiple words, each word of which may or may not be removable. The uncompressed data block is compressed into the compressed data block by removing one or more removable words from it. The mapping metadata maps the removable and non-removable words of the uncompressed data block, and The uncompressed data blocks are recovered from the decompressed data blocks.

18. The method of claim 17, The words in the uncompressed data block correspond to the weights of the neural network.

19. The method of claim 17, wherein: The compressed data blocks are arranged in row and column dimensions, including those in the row dimension. N One compressed group, N ≥ 1 and the column dimensions D List, D ≥ 1 ; The uncompressed data blocks are arranged according to the row dimension and the column dimension, including the data blocks in the row dimension. N One uncompressed group, N ≥ 1 And including the column dimension C Column; and The uncompressed data block has been stack-compressed into the compressed data block at least along the column dimension, so that... D < C .

20. The method of claim 17, The compressed data blocks are arranged in row and column dimensions, including those in the row dimension. M One compressed group, M ≥ 1 and the column dimensions C List, C ≥ 1 , The uncompressed data blocks are arranged according to the row dimension and the column dimension, including the data blocks along the row dimension. N One uncompressed group, N ≥ 1 And also includes the column dimension C Column, and The uncompressed data block has been stack-compressed into the compressed data block at least along the row dimension, so that... M < N .

21. The method of claim 20, The mapping metadata includes N line x C A matrix of column elements, such that each element of the mapping metadata ( n , c ), n = 0 ... N – 1, c = 0 ... C – 1 Groups corresponding to the uncompressed data blocks n List c The characters in the middle, and Each element of the mapping metadata includes bits configured to indicate whether the corresponding word of the uncompressed data block is removable or non-removable.

22. The method as described in claim 21, Each element of the mapping metadata also corresponds to a group of the decompressed data blocks. n List c The characters in the middle, and The process of decompressing the compressed data block to generate the decompressed data block includes: For each element of the mapping metadata indicating that the corresponding word of the uncompressed data block is not removable, the corresponding word of the decompressed data block is filled with the removable word; and For each element of the mapping metadata indicating that the corresponding word of the uncompressed data block is not removable, the corresponding word of the decompressed data block is loaded with one of the non-removable words from the compressed data block.

23. The method of claim 21, wherein decompressing the compressed data block to generate a decompressed data block comprises: Initialize row index n ; Initialize compressed group index m ; When the row index n < N At that time, a line decompression operation is performed, which includes: The rows of the mapping metadata are read based on the row index. n ; When the compressed group index m < M hour, The compressed group of the compressed data block m The word is loaded into the context of the above. C Corresponding columns C In each of the queues, there is a depth. S Number of time slots, S ≥ 1 ;as well as Increment the compressed group index m ; Based on the rows read from the mapping metadata n The coming will come from each queue c The header word popped from the stack is the decompressed group data; and Increment the row index n , The method described includes indexing for each row. n Repeat the decompression operation described above.

24. The method of claim 23, wherein the compressed group of the compressed data block m Loading the words into the C queues includes: Loading with the compressed group m Each column c Each corresponding character occupies its corresponding queue. c The lowest unused time slot.

25. The method of claim 23, The uncompressed data blocks have been stacked and compressed in both the row and column dimensions to such that each compressed group of the compressed data blocks... m In the compressed group, the unremovable characters occupy space. m The leftmost column c = 0 , The method further includes reading column metadata, the column metadata including M line x C A matrix of columns is formed such that each element of the column metadata ( m, c ), m = 0 ... M – 1, c = 0 ... C – 1 The group corresponding to the compressed data blocks after compression in the row dimension and before compression in the column dimension. m List c The characters in the middle, Each element of the column metadata includes bits configured to indicate whether the corresponding word in the compressed data block is removable or non-removable, and The compressed data block is the compressed group. m The word is loaded into the C The queues include: Rows of column metadata read based on the compressed group index m ; Based on the rows read from the column metadata m To read the compressed group of the compressed data block m The character; Based on the rows read from the column metadata m For the compressed group from the compressed data block m The words read from the middle are shifted column by column; as well as Load the compressed data block and the compressed group m Each column c Each shifted word occupies its corresponding queue. c The lowest unused time slot.

26. The method of claim 17, wherein the uncompressed data is losslessly recovered from the decompressed data block.

27. An apparatus for decompression, comprising: A means for reading compressed data blocks and mapping metadata, the compressed data blocks comprising one or more words, and the mapping metadata being configured to map the words of the compressed data blocks to generated decompressed data blocks; as well as A means for decompressing the compressed data block according to the mapping metadata to generate the decompressed data block. The compressed data block is formed by compressing an uncompressed data block comprising multiple words, each word of which may or may not be removable. The uncompressed data block is compressed into the compressed data block by removing one or more removable words from it. The mapping metadata maps the removable and non-removable words of the uncompressed data block, and The uncompressed data blocks are recovered from the decompressed data blocks.

28. A non-transitory computer-readable medium storing computer-executable instructions for a device, the computer-executable instructions comprising: The device is configured to read one or more instructions of compressed data blocks and mapping metadata, the compressed data blocks comprising one or more words, and the mapping metadata being configured to map the words of the compressed data blocks to generated decompressed data blocks; as well as The device decompresses the compressed data block according to the mapped metadata to generate one or more instructions for the decompressed data block. The compressed data block is formed by compressing an uncompressed data block comprising multiple words, each word of which may or may not be removable. The uncompressed data block is compressed into the compressed data block by removing one or more removable words from it. The mapping metadata maps the removable and non-removable words of the uncompressed data block, and The uncompressed data blocks are recovered from the decompressed data blocks.

Images