Parallel decoding technology

Abstract

The present invention discloses parallel decoding technology. In various embodiments, a coding sequence including a variable length code (e.g., a compressed sequence for uncompressed data) is decoded in an iterative manner to generate a decoded symbol sequence. During each iteration, a group of threads decodes the code in the coding sequence in parallel to generate a symbol. Then, the thread group calculates an offset based on the size of the symbol. Subsequently, the thread group generates a continuous portion of the decoded sequence in parallel based on the symbol, the output address and the offset.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of priority to U.S. Provisional Patent Application "DATA PARALLEL DECODING OF ENTROPY CODED MESSAGES," Serial No. 63 / 088429, filed on October 6, 2020. The subject matter of this related application is incorporated herein by reference. Technical Field

[0003] Various embodiments relate generally to parallel processing systems and, more particularly, to parallel decoding techniques. Background Art

[0004] Lossless data encoding algorithms and corresponding data decoding algorithms are used to reduce the resources required to store and transmit data without causing any information loss. Typically, a lossless data encoding algorithm executed on a computing device maps a source sequence of text (literal) (e.g., bytes or alphabetic characters) represented as fixed-length "symbols" to an encoded "code" sequence of reduced size. Subsequently, the computing device and / or any number of other computing devices that obtain the encoded sequence execute a corresponding data decoding algorithm to map the encoded sequence to a decoded sequence of symbols. The decoded sequence of symbols represents a decoded sequence of text, which is a copy of the source sequence of text.

[0005] Lossless data encoding algorithms are often based on entropy coding techniques that are optimized for a combination of size reduction or "compression ratio" and decoding throughput. In some types of entropy coding, the length of a code is inversely proportional to the frequency of the corresponding symbol. Thus, relatively short codes are used to represent frequently used symbols, and relatively long codes are used to represent infrequently used symbols. In addition to entropy coding, some algorithms can map symbols representing duplicate or "copy" symbols to "copy" codes. Each copy symbol is a backreference that specifies the position and length of a text string in the source sequence.

[0006] One challenge associated with decoding a coded sequence as described above is that, due to the variable length of the codes, directly addressing and correctly decoding a specific code within the coded sequence in a non-serial manner is problematic. As a result, the data decoding algorithm cannot effectively parallelize the decoding of the resulting coded sequence, and thus the decoding throughput of a client device capable of parallel processing may be unnecessarily low.

[0007] For example, in one method of "data parallel" decoding, a subsequence-based encoding algorithm executed on a computing device divides a given sequence into symbol-aligned encoding subsequences and generates metadata specifying an input pointer for each encoding subsequence. The metadata is then stored and / or transmitted to other computing devices together with the encoding sequence consisting of the encoding subsequences. On the computing device and / or other computing devices, the subsequence-based decoding algorithm assigns the input pointer and the corresponding encoding subsequence to different threads. For each thread, the subsequence-based decoding algorithm also calculates an output pointer that describes the location where the thread stores the text represented by the decoded symbol in the decoding sequence. In parallel with other threads, each thread decodes the code in the assigned encoding subsequence in the order of the thread's input pointer and stores the text represented by the decoded symbol in the decoding sequence in the order of the thread's output pointer.

[0008] One disadvantage of performing data-parallel decoding on a subsequence basis is that transmitting and / or storing metadata reduces the compression ratio. Another disadvantage is that threads rarely access memory when reading from the encoded sequence and writing the decoded sequence. It is well known that sparse access to memory reduces processing efficiency and therefore reduces decoding throughput. Another disadvantage of performing data-parallel decoding on a subsequence basis is that a replicated code can be read by one thread before the original text string referenced by the replicated code is stored in the decoded sequence by another thread. To address this issue, replicated codes are often stored to delay processing, which consumes additional memory and further reduces decoding throughput.

[0009] As noted above, what is needed in the art are more efficient techniques for data-parallel decoding of coded sequences that include variable length codes. Summary of the invention

[0010] One embodiment of the present invention sets forth a method for decoding a coded sequence including a variable length code. The method includes determining a first set of codes based on the coded sequence; decoding the first set of codes to generate a first set of symbols; determining a first set of offsets based on the first set of symbols; and generating a first continuous portion of a decoded sequence based on the first set of symbols, the first set of offsets, and a first address.

[0011] At least one technical advantage of the disclosed technology relative to the prior art is that, through the disclosed technology, parallel decoding of a coding sequence including variable-length codes does not reduce the compression ratio. In this regard, because the arrangement of the codes in the coding stream is customized to facilitate data parallel processing, the threads in the thread group can collaborate to deterministically and iteratively access the code group for parallel decoding without the need for metadata that essentially reduces the compression ratio. In addition, unlike the prior art, because the threads read consecutive words from the coding sequence and write the text represented by consecutive symbols to the decoding sequence, the decoding throughput is not reduced due to sparse memory access. And, because the thread group generates the decoding sequence incrementally from beginning to end without any gaps, the duplicate code does not require delayed processing (in some prior art, this can have an adverse effect on memory usage and decoding throughput). Compared with the prior art methods, these technical advantages provide one or more technical improvements. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to be able to understand the above features of various embodiments in detail, the inventive concepts briefly summarized above may be described in more detail by reference to various embodiments (some of which are shown in the accompanying drawings). However, it should be noted that the drawings only illustrate typical embodiments of the inventive concepts and are therefore not to be considered as limiting the scope in any way, and there are other equally effective embodiments.

[0013] Figure 1 is a block diagram illustrating a system configured to implement one or more aspects of the present disclosure;

[0014] Figure 2 According to various embodiments, including Figure 1 A block diagram of a parallel processing unit in a parallel processing subsystem;

[0015] Figure 3 According to various embodiments, including Figure 2 A block diagram of a general processing cluster in a parallel processing unit of;

[0016] Figure 4 According to various embodiments Figure 3 A more detailed description of the decoding thread group;

[0017] Figure 5 According to various embodiments, Figure 4 A conceptual description of how the decode thread group generates a decode sequence; and

[0018] Figure 6 is a flow chart of method steps for decoding a coded sequence including a variable length code, according to various embodiments. DETAILED DESCRIPTION

[0019] In the following description, numerous specific details are set forth to provide a more thorough understanding of various embodiments. However, it will be apparent to one skilled in the art that the inventive concept may be practiced without one or more of these specific details.

[0020] As previously mentioned, one challenge associated with decoding coded sequences including variable length codes is that it is problematic to directly address and correctly decode specific codes within the coded sequence in a non-serial manner. In one conventional approach to data parallel decoding, a subsequence-based encoding algorithm divides a given sequence into symbol-aligned coded subsequences and generates metadata specifying an input pointer for each coded subsequence included in the coded sequence. The subsequence-based decoding algorithm uses the metadata to configure multiple threads to decode the coded subsequences in parallel.

[0021] A disadvantage of the above-described conventional subsequence-based data parallel decoding method is that transmitting and storing metadata reduces the compression ratio. Another disadvantage is that the decoding throughput may be reduced because threads rarely access memory when reading from the encoding sequence and writing to the decoding sequence. Another disadvantage of the conventional subsequence-based data parallel decoding method is that since the decoding sequence may include gaps, the subsequence-based decoding application may be forced to store duplicate code for delayed processing, thereby consuming additional memory and further reducing the decoding throughput.

[0022] To address the above issues, the entropy coding application 180 systematically rearranges the bits in the coded sequence relative to the corresponding symbols representing the source sequence to facilitate data parallel decoding. In this way, the entropy coding application 180 establishes a coding format that makes the coded sequence suitable for decoding using a group of threads or "thread groups" executed in parallel. As used herein, a "thread" can be a software thread or a hardware thread.

[0023] A software thread refers to an execution thread that executes on any type of processing unit. For purposes of explanation only, a group of software threads that decodes a coded sequence generated by the entropy coding application 180 is also referred to herein as a "decoding thread group." A "hardware thread" refers to a group of one or more fixed-function hardware units (e.g., replication engines) that can decode a coded sequence generated by the entropy coding application 180. For purposes of explanation only, a group of one or more fixed-function hardware units that decodes a coded sequence generated by the entropy coding application 180 is also collectively referred to herein as a "parallel decoding unit."

[0024] Typically, the entropy coding application 180 can be executed on any type of processing system, and the resulting coded sequence can be decoded by a decoding thread group or a parallel decoding unit included in the same processing system of different processing systems. For example, in some embodiments, the entropy coding application 180 and the parallel decoding application 190 are both executed on a multi-core processor. The entropy coding application 180 generates a coded sequence. In order to decode the coded sequence, the parallel decoding application 190 causes the decoding thread group to execute across multiple cores included in the multi-core processor. Some examples of multi-core processors include, but are not limited to, certain types of central processing units ("CPUs"), graphics processing units ("GPUs"), parallel processing units ("PPUs"), and accelerated processing units (*"APUs").

[0025] For the purpose of explanation only, the following Figure 1-6 The functions of the entropy encoding application 180, the parallel decoding application 190, the decoding thread group, and the parallel decoding unit are described in the context of some other embodiments implemented within the system 100. The system 100 includes, but is not limited to, a CPU 102 and a parallel processing subsystem 102 including any number of PPUs. As described in more detail below, in Figure 1-6 In the embodiment described in , entropy encoding application 180 is executed on CPU 102 to generate an encoded sequence. In some embodiments, to decode the encoded sequence, parallel decoding application 190 causes a decoding thread group to execute on a streaming multiprocessor ("SM") included in one of the PPUs. In other embodiments, parallel decoding application 190 may be omitted from system 100, and as Figure 2 As described, a parallel decoding unit included in the PPU decodes a coded sequence.

[0026] Note that the techniques described herein are illustrative rather than restrictive and may be modified without departing from the broader spirit and scope of the invention. Many modifications and variations of the functionality provided by the entropy encoding application 180, the parallel decoding application 190, the decoding thread groups, and the parallel decoding units will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments.

[0027] Example System Overview

[0028] Figure 1A block diagram of a system 100 configured to implement one or more aspects of the present disclosure is shown. As shown, system 100 includes, but is not limited to, a central processing unit ("CPU") 102 and a system memory 104, coupled to a parallel processing subsystem 112 via a memory bridge 105 and a communication path 113. Memory bridge 105 is further coupled to an input / output ("I / O") bridge 107 via a communication path 106, and I / O bridge 107 is in turn coupled to a switch 116.

[0029] In operation, I / O bridge 107 is configured to receive user input information from input device 108 (e.g., a keyboard or mouse) and forward the input information to CPU 102 for processing via communication path 106 and memory bridge 105. Switch 116 is configured to provide connections between I / O bridge 107 and other components of system 100, such as network adapter 118 and add-in cards 120 and 121.

[0030] As shown, I / O bridge 107 is coupled to system disk 114, which can be configured to store content and applications and data for use by CPU 102 and parallel processing subsystem 112. As a general matter, system disk 114 provides non-volatile storage for applications and data and can include a fixed or removable hard drive, flash memory device, CD-ROM, DVD-ROM, Blu-ray, HD DVD or other magnetic, optical or solid-state storage device. Finally, although not explicitly shown, other components, such as universal serial bus or other port connections, optical disk drives, DVD drives, film recording devices, etc., can also be connected to I / O bridge 107.

[0031] In various embodiments, memory bridge 105 may be a northbridge chip, and I / O bridge 107 may be a southbridge chip. In addition, communication paths 106 and 113, as well as other communication paths within system 100, may be implemented using any technically suitable protocol, including but not limited to accelerated graphics port, HyperTransport, or any other bus or point-to-point communication protocol known in the art.

[0032] In some embodiments, parallel processing subsystem 112 includes a graphics subsystem that provides pixels to display device 110, which may be any conventional cathode ray tube, liquid crystal display, light emitting diode display, etc. In such embodiments, parallel processing subsystem 112 includes circuits optimized for graphics and video processing, including, for example, video output circuits. As described below in Figure 2As described in more detail in , such circuitry may be incorporated into one or more parallel processing units ("PPUs") included within parallel processing subsystem 112. In other embodiments, parallel processing subsystem 112 includes circuitry optimized for general purpose and / or computational processing. Likewise, such circuitry may be incorporated into one or more PPUs included within parallel processing subsystem 112 that are configured to perform such general purpose and / or computational operations. In other embodiments, one or more PPUs included within parallel processing subsystem 112 may be configured to perform graphics processing, general purpose processing, and computational processing operations.

[0033] In some embodiments, the parallel processing subsystem 112 may be coupled to Figure 1 The parallel processing subsystem 112 may be integrated with one or more other elements of the CPU 102 to form a single system. For example, the parallel processing subsystem 112 may be integrated with the CPU 102 and other connected circuits on a single chip to form a system on a chip (“SoC”). In the same or other embodiments, any number of CPUs 102 and any number of parallel processing subsystems 112 may be distributed in any number of shared geographic locations and / or any number of different geographic locations and / or implemented in any combination in one or more cloud computing environments (i.e., packaged shared resources, software, data, etc.).

[0034] In some embodiments, system memory 104 may include, but is not limited to, any number of software applications, any number of device drivers (not shown), or any combination thereof. Each software application may reside in any number of memories and execute on any number of processors in any combination. As used herein, a "processor" may be any instruction execution system, device, or device capable of executing instructions. Some examples of processors include, but are not limited to, CPU 102, parallel processing subsystem 112, and PPU.

[0035] In some embodiments, at least one device driver is configured to manage processing operations of one or more PPUs within parallel processing subsystem 112. In the same or other embodiments, the device driver implements an application programming interface ("API") functionality that enables software applications to specify instructions for execution on one or more PPUs via API calls.

[0036] As shown, in some embodiments, system memory 104 includes, but is not limited to, an entropy encoding application 180 and / or a parallel decoding application 190. In some embodiments, entropy encoding application 180 is a software application that executes on CPU 102. In some other embodiments, entropy encoding application 180 is a software application that executes on any number of PPUs included in parallel processing subsystem 112. In the same or other embodiments, parallel decoding application 190 is a software application that executes instructions on any number of PPUs included in parallel processing subsystem 112 and / or any number of other parallel processing subsystems 112 (not shown). In some other embodiments, parallel decoding application 190 is a software application that executes on CPU 102.

[0037] In some embodiments, the entropy coding application 180 executes on an instance of the CPU 102 included in a server device, and any number of instances of the entropy coding application 180 execute on any number of instances of the parallel processing subsystem 112 distributed across any number of client devices located in different geographic locations.

[0038] In some embodiments, entropy encoding application 180 processes the source sequence (not in Figure 1 ) to generate a coded sequence (not shown in Figure 1 ), the encoded sequence reduces the resources required to store and transmit the source sequence without any information loss. As referred to herein, a "sequence" can be any sequence of discrete data portions of any number and / or type. Some examples of sequences include, but are not limited to, bit streams and byte streams.

[0039] For purposes of explanation only, discrete portions of data included in a source sequence are referred to herein as "text" and represented by "symbols." In some embodiments, each text is a byte or an alphabetic character. In the same or other embodiments, each symbol is a text symbol or a replicated symbol, wherein each text symbol represents a single text and each replicated symbol represents a copy of a resulting text string. More specifically, in some embodiments, a replicated symbol is a reverse reference specifying the location and length of an original text sequence that is identical to and continues with the associated text sequence in the source sequence.

[0040] As a part for generating coded sequence for given source sequence, entropy coding application 180 will be included in each symbol mapping in given coded sequence to variable length code based at least in part on entropy coding technology.As mentioned herein, each " code " can be any discrete part of data, and different codes can be associated with different code types.In addition, one group of symbols is mapped to the code of one group of variable length, and the code quantity that comprises is identical with the symbol quantity in this group of symbols.In certain embodiments, the length of code changes inversely proportional to the frequency of the corresponding symbol of the source sequence that represents association.

[0041] Implementing efficient data parallel decoding

[0042] As previously outlined herein, to address issues that may be associated with conventional methods of data parallel decoding, the entropy coding application 180 systematically rearranges the bits of the coded sequence relative to corresponding symbols representing the source sequence to facilitate data parallel decoding. In this manner, the entropy coding application 180 establishes a coding format that enables the coded sequence to be decoded using groups of threads that execute in parallel. Again, as used herein, a thread may be a software thread or a hardware thread.

[0043] In some embodiments, each thread in a thread group is associated with a different thread identifier ("ID") that uniquely identifies the thread in the thread group. In some embodiments, any number and / or type of techniques (e.g., prediction) may be used to disable one or more threads in a thread group for any period of time. In the same or other embodiments, threads in a thread group may synchronize, collaborate, communicate together, or any combination thereof in any technically feasible manner (e.g., via shared memory).

[0044] In some embodiments, the thread group can be configured to decode a given encoding sequence based on a single instruction, multiple data ("SIMD") model, where each thread processes a different data set based on a single instruction set. A thread group configured based on the SIMD model is also referred to herein as a "SIMD thread group". In some embodiments, the threads in the SIMD thread group execute in lockstep. In some other embodiments, the thread group can be configured to decode a given encoding sequence based on a single instruction, multiple thread ("SIMT") model, which allows different threads to more easily follow different execution paths relative to the SIMD model.

[0045] As shown, in some embodiments, entropy encoding application 180 encodes each of any number of source sequences based on thread group size 182 and word size 184. For purposes of explanation only, the functionality of entropy encoding application 180 is described herein in the context of encoding a single source sequence to generate a corresponding encoded sequence. However, thread group size 182 and word size 184 may be used to encode any number of source sequences and, therefore, may be used in conjunction with any number of encoded sequences.

[0046] The thread group size 182 specifies the total number of threads that will be used to decode the encoded sequence generated by the entropy encoding application 180. The thread group size 182 can be any integer greater than 1 and can be determined in any technically feasible manner. For example, in some embodiments, the thread group size 182 can be determined based on the architecture of the parallel processing subsystem 112.

[0047] In some embodiments, the entropy encoding application 180 formats the encoding sequence so that a thread group having a thread group size 182 can iteratively determine symbol groups and store corresponding word arrays in a continuous memory to incrementally generate a decoding sequence from beginning to end without gaps. For the purpose of explanation only, a "decoding iteration" refers to a sequence of steps during which threads in a thread group concurrently obtain different codes according to the encoding sequence, concurrently decode the codes to generate corresponding symbols, and concurrently store text represented by the symbols in a continuous memory.

[0048] In the same or other embodiments, entropy encoding application 180 may implicitly or explicitly indicate the number of threads to be used to process any portion of the encoded sequence during decoding, in any technically feasible manner. For example, in some embodiments, each block of the encoded sequence is associated with data parallel processing (e.g., compression blocks) or serial processing (e.g., header blocks). During decoding, all threads included in a thread group having thread group size 182 process blocks associated with data parallel processing, and a single thread in the thread group processes blocks associated with serial processing.

[0049] In some embodiments, word size 184 specifies the size of fixed-size words, also referred to herein as "words," into which entropy coding application 180 packs codes representing symbols in a source sequence. In the same or other embodiments, each word is associated with a thread, and each thread is associated with multiple words. Entropy coding application 180 then aggregates these words to produce a coded sequence. As a result, the coded sequence is a linear array of fixed-size words.

[0050] Word size 184 may specify an integer of any bit and may be determined in any technically feasible manner. In some embodiments, word size 184 may be determined based on the architecture of parallel processing subsystem 112. For example, in some embodiments, word size 184 is set to the number of bits (e.g., 32 bits, 64 bits, etc.) that parallel processing subsystem 112 can read from memory using a single read operation.

[0051] As shown, the thread group size 182 is symbolized as "T" and the word size 184 is symbolized as "B". For the purpose of explanation only, the numbering of the entropy encoding application 180 and the entropy decoding application 180 is zero-based. Accordingly, the thread IDs associated with the thread group having the thread group size 182 are from 0 to (T-1). The initial decoding iteration is the zeroth iteration.

[0052] In some embodiments, to encode the source sequence, the entropy coding application 180 performs any number and / or type of coding algorithms to map symbols representing the text included in the source sequence to a code sequence. Then, the entropy coding application 180 repeatedly assigns consecutive T code groups to T thread IDs. Each consecutive T code group corresponds to a different decoding iteration. In this way, in some embodiments, the entropy coding application 180 assigns the xth code relative to the source sequence to the thread corresponding to the thread ID of x%T so that it can be decoded during the floor(x / T) decoding iteration. Therefore, the xth symbol is determined by the thread corresponding to the thread ID of x%T.

[0053] For example, if the thread group size 182 is 32, the entropy coding application 180 assigns the 0th to 31st codes relative to the source sequence to threads corresponding to thread IDs 0-31, respectively, for decoding during the 0th decoding iteration. Then, the entropy coding application 180 assigns the 32nd to 63rd codes relative to the source sequence to threads corresponding to thread IDs 0-31, respectively, for decoding during the 1st decoding iteration, and so on.

[0054] In some embodiments, the entropy coding application 180 packs the codes into words of word size 184 and then generates a coded sequence based on the words. In operation, for each thread ID, the entropy coding application packs the bits of the assigned code into words of B bits each, which are arranged in order of increasing decoding iterations. It is worth noting that each word can start and / or end with a partial code. For example, if B is 16 bits, and the bit lengths of the first seven codes assigned to a given thread ID are 6, 5, 3, 4, 1, 8, and 5, respectively, relative to the source sequence, the entropy coding application 180 generates a word including the first three codes and the first half of the fourth code, and another word including the second half of the fourth code and the next three codes.

[0055] For the purpose of explanation only, the decoding iteration of a given word refers to the lowest decoding iteration associated with the first code included in the word. To generate the coded sequence, the entropy coding application arranges the words according to the primary criterion of increasing decoding iterations and the secondary criterion of increasing thread ID. Figure 5 An example of an encoding sequence generated by entropy encoding application 180 based on thread group size 182 and word size 184 is described in more detail.

[0056] In some embodiments, parallel decoding application 190 decodes any number of coded sequences formatted for data parallel decoding, as described above with respect to entropy encoding application 180. Parallel decoding application 190 may obtain (e.g., receive, read from memory, etc.) coded sequences from any number and / or type of devices internal or external to system 100 in any technically feasible manner. In some embodiments, parallel decoding application 190 may receive any number of coded sequences generated by any number of instances of entropy encoding application 180 executed on any number and / or type of processors. In the same or other embodiments, entropy encoding application 180 is omitted from system 100.

[0057] In some embodiments, the parallel decoding application 190 decodes each encoded sequence based on the thread group size 182 and the optional word size 184. The parallel decoding application 190 can determine the thread group size 182 and the word size 184 in any technically feasible manner. For example, in some embodiments, the thread group size 182 and the word size 184 are defined based on the architecture of the parallel processing subsystem 112.

[0058] For purposes of explanation only, the functionality of the parallel decode application 190 is described herein in the context of decoding a single encoded sequence to generate a corresponding decoded sequence. However, the thread group size 182 and the word size 184 may be common to any number of encoded sequences, and the parallel decode application 190 may therefore use the thread group size 182 and the word size 184 to decode any number of encoded sequences.

[0059] In some embodiments, parallel decoding application 190 configures threads in a thread group having thread group size 182 and executes on a processor (e.g., a multi-threaded processor included in parallel processing subsystem 112) to decode the encoded sequence in parallel. In the same or other embodiments, parallel decoding application 190 may dynamically adjust the number of threads that parallel decoding application 190 uses to decode different portions of the encoded sequence based on any number and / or type of criteria (e.g., disabling one or more threads in a thread group having thread group size 182). For example, in some embodiments, parallel decoding application 190 may configure all threads in a decoding thread group to process blocks associated with data parallel processing (e.g., compression blocks), and may configure a single thread in a decoding thread group to process blocks associated with serial processing (e.g., header blocks).

[0060] The parallel decoding application 190 may configure the thread groups to decode the encoded sequence in any technically feasible manner. In some embodiments, the parallel decoding application 190 configures T thread groups so that each thread group executes a different instance of the decoder kernel 192 in a SIMD manner, thereby generating a decoding thread group (in Figure 1 As shown, in some embodiments, decoder core 192 resides in a memory included in parallel processing subsystem 112. More specifically, in some embodiments, decoder core 192 resides in a parallel processing ("PP") memory that is coupled to one of the PPUs included in parallel processing subsystem 112. Figure 2 Describe the PPU and PP memory. Figure 4 and Figure 5 Describes the decode thread group in detail.

[0061] In some embodiments, any portion (including all) of the functionality of the entropy encoding application 180 described herein may be implemented in hardware. In the same or other embodiments, the entropy encoding application 180 is omitted from the system 100. In some embodiments, any portion (including all) of the functionality of the parallel decoding application 190 described herein may be implemented in fixed-function hardware capable of executing multiple hardware threads. Figure 2 In more detail, in some embodiments, a fixed-function parallel decoding unit (not in Figure 1 1 and 2 ) implement the functions of the parallel decoding application 190 and the decoding thread group. In the same or other embodiments, the parallel encoding application 190 is omitted from forming the system 100.

[0062] Note that the techniques described herein are illustrative rather than restrictive and may be varied without departing from the broader spirit and scope of the invention. Many modifications and variations of the functionality provided by the entropy encoding application 180, the parallel decoding application 190, and the decoder core 192 will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments.

[0063] It will be understood that the systems shown herein are illustrative and that variations and modifications are possible. The connection topology, including the number and arrangement of bridges, the number of CPUs 102, and the number of parallel processing subsystems 112, can be modified as desired. For example, in some embodiments, system memory 104 can be directly connected to CPU 102 rather than through memory bridge 105, and other devices can communicate with system memory 104 through memory bridge 105 and CPU 102. In other alternative topologies, parallel processing subsystem 112 can be connected to I / O bridge 107 or directly to CPU 102 rather than to memory bridge 105. In other embodiments, I / O bridge 107 and memory bridge 105 can be integrated into one chip rather than existing as one or more discrete devices. Finally, in some embodiments, Figure 1 One or more components shown in FIG. 1 may not exist. For example, the switch 116 may be eliminated, and the network adapter 118 and the add-in cards 120 and 121 may be directly connected to the I / O bridge 107.

[0064] Figure 2 According to various embodiments Figure 1 1 is a block diagram of a parallel processing unit ("PPU") 202 included in the parallel processing subsystem 112 of FIG. Figure 2 One PPU 202 is depicted, and as described above, the parallel processing subsystem 112 may include any number of PPUs 202. As shown, the PPU 202 is coupled to a local PP memory 204. The PPU 202 and the PP memory 204 may be implemented using one or more integrated circuit devices, such as programmable processors, application specific integrated circuits or memory devices, or in any other technically feasible manner.

[0065] In some embodiments, PPU 202 includes a graphics processing unit that can be configured to implement a graphics rendering pipeline to perform various operations related to generating pixel data based on graphics data provided by CPU 102 and / or system memory 104. When processing graphics data, PP memory 204 can be used as graphics memory, which stores one or more traditional frame buffers and, if necessary, one or more other rendering targets. Among other things, PP memory 204 can be used to store and update pixel data and deliver the final pixel data or display frame to display device 110 for display. In some embodiments, PPU 202 can also be configured for general processing and computing operations.

[0066] As previously combined Figure 1 As described, in some embodiments, system 100 includes a parallel decoding application 190 that configures T thread groups to each execute a different instance of a decoder kernel 192 in a SIMD manner, thereby generating a decoding thread group. As shown, in the same or other embodiments, decoder kernel 192 resides in PP memory 204.

[0067] In operation, CPU 102 is the main processor of system 100, controlling and coordinating the operation of other system components. In particular, CPU 102 issues commands that control the operation of PPU 202. In some embodiments, CPU 102 writes the command stream of PPU 202 into a data structure (in Figure 1 or Figure 2 The data structure may be located in system memory 104, PP memory 204, or another storage location accessible to both CPU 102 and PPU 202 (not shown in the figure). A pointer to the data structure is written to the push buffer to start processing the command stream in the data structure. PPU 202 reads the command stream from the push buffer and then executes the commands asynchronously with respect to the operation of CPU 102. In an embodiment where multiple push buffers are generated, an execution priority may be specified for each push buffer by an application through a device driver (not shown) to control the scheduling of different push buffers.

[0068] As also shown, the PPU 202 includes an I / O unit 205 that communicates with the rest of the system 100 via the communication path 113 and the memory bridge 105. The I / O unit 205 generates packets (or other signals) for transmission on the communication path 113, and also receives all incoming packets (or other signals) from the communication path 113, directing the incoming packets to the appropriate components of the PPU 202. For example, commands related to processing tasks may be directed to the host interface 206, while commands related to memory operations (e.g., reading or writing from the PP memory 204) may be directed to the crossbar unit 210. The host interface 206 reads each push buffer and transmits the command stream stored in the push buffer to the front end 212.

[0069] In some embodiments, I / O unit 205 transmits the command to copy engine 290. In the same or other embodiments, copy engine 290 implements any number and / or type of memory operations, including, but not limited to, any number and / or type of direct memory access operations. In some embodiments, copy engine 290 may copy data from one memory to another memory and / or move data within a memory. In the same or other embodiments, copy engine 290 performs any number and / or type of operations to enable any number of components of PPU 202 to access system memory 104 independently of CPU 102.

[0070] As depicted by the dashed box, in some embodiments, the copy engine 290 includes, but is not limited to, a parallel decoding unit 292. In some other embodiments, the parallel decoding unit 292 may be implemented in any component of the PPU 202 or as a separate unit in the PPU 202. The parallel decoding unit 292 may implement the functionality of any portion (including all) of the parallel decoding application 190 and / or decoding thread group described herein in any technically feasible manner. In some embodiments, the parallel decoding unit 292 includes, but is not limited to, one or more fixed-function hardware units that are collectively and / or individually capable of executing any number of hardware threads to efficiently decode the encoded sequence in parallel.

[0071] As above combined Figure 1 As described above, the connection of PPU 202 to the rest of system 100 may be different. In some embodiments, parallel processing subsystem 112 including at least one PPU 202 is implemented as an add-in card that can be inserted into an expansion slot of system 100. In other embodiments, PPU 202 may be integrated on a single chip with a bus bridge, such as memory bridge 105 or I / O bridge 107. Likewise, in other embodiments, some or all elements of PPU 202 may be included in a single integrated circuit or system chip with CPU 102.

[0072] In operation, the front end 212 transmits the processing tasks received from the host interface 206 to the work distribution unit (not shown) within the task / work unit 207. The work distribution unit receives a pointer to the processing task, which is encoded as task metadata ("TMD") and stored in a memory. The pointer to the TMD is included in a command stream, which is stored as a push buffer and received by the front end 212 from the host interface 206. The processing task that can be encoded as a TMD includes an index associated with the data to be processed, and state parameters and commands that define how to process the data. For example, the state parameters and commands can define the program to be executed on the data. The task / work unit 207 receives tasks from the front end 212 and ensures that the general processing cluster ("GPC") 208 is configured to a valid state before the processing tasks specified by each TMD are started. A priority can be specified for each TMD for arranging the execution of the processing task. The processing task can also be received from the processing cluster array 230. Optionally, a TMD may include a parameter that controls whether the TMD is added to the head or tail of a list of processing tasks (or a list of pointers to processing tasks), thereby providing another level of control over execution priority.

[0073] PPU 202 advantageously implements a highly parallel processing architecture based on a processing cluster array 230 that includes a set of C GPCs 208, where C ≥ 1. Each GPCs 208 is capable of concurrently executing a large number (e.g., hundreds or thousands) of threads, where each thread is an instance of a program (e.g., a kernel). In various applications, different GPCs 208 may be assigned to process different types of programs or perform different types of computations. The assignment of GPCs 208 may vary depending on the workload generated by each type of program or computation.

[0074] The memory interface 214 includes a set of D partition units 215, where D ≥ 1. Each partition unit 215 is coupled to one or more dynamic random access memories ("DRAMs") 220 residing within the PP memory 204. In some embodiments, the number of partition units 215 is equal to the number of DRAMs 220, and each partition unit 215 is coupled to a different one of the DRAMs 220. In other embodiments, the number of partition units 215 may be different than the number of DRAMs 220. It will be appreciated by those skilled in the art that the DRAMs 220 may be replaced with any other technically suitable memory device. In operation, various rendering targets, such as texture maps and frame buffers, may be stored across the DRAMs 220, allowing the partition units 215 to write portions of each rendering target in parallel to efficiently utilize the available bandwidth of the PP memory 204.

[0075] A given GPC 208 can process data to be written to any DRAM 220 within PP memory 204. Crossbar unit 210 is configured to route the output of each GPC 208 to the input of any partition unit 215 or any other GPC 208 for further processing. GPCs 208 communicate with memory interface 214 through crossbar unit 210 to read from or write to any number of DRAMs 220. In one embodiment, crossbar unit 210 has connections to I / O unit 205 in addition to connecting to PP memory 204 through memory interface 214, thereby enabling processing cores within different GPCs 208 to communicate with system memory 104 or other memory that is not local to PPU 202. In Figure 2 In the embodiment of FIG. 2 , the crossbar unit 210 is directly connected to the I / O unit 205. In various embodiments, the crossbar unit 210 can use virtual channels to separate the traffic flow between the GPCs 208 and the partition units 215.

[0076] Likewise, the GPCs 208 may be programmed to perform processing tasks associated with various applications, including, but not limited to, linear and nonlinear data transformations, filtering of video and / or audio data, modeling operations (e.g., applying laws of physics to determine the position, velocity, and other properties of an object), image rendering operations (e.g., tessellation shader, vertex shader, geometry shader, and / or pixel / fragment shader programs), general computational operations, and the like. In operation, the PPU 202 is configured to transfer data from the system memory 104 and / or the PP memory 204 to one or more on-chip memory units, process the data, and write the resulting data back to the system memory 104 and / or the PP memory 204. The resulting data may then be accessed by other system components, including the CPU 102, another PPU 202 within the parallel processing subsystem 112, or another parallel processing subsystem 112 within the system 100.

[0077] As described above, any number of PPUs 202 may be included in the parallel processing subsystem 112. For example, multiple PPUs 202 may be provided on a single add-in card, or multiple add-in cards may be connected to the communication path 113, or one or more PPUs 202 may be integrated into a bridge chip. The PPUs 202 in a multi-PPU system may be the same or different from each other. For example, different PPUs 202 may have different numbers of processing cores and / or different numbers of PP memories 204. In embodiments where there are multiple PPUs 202, these PPUs 202 may operate in parallel to process data at a higher throughput than a single PPU 202. Systems including one or more PPUs 202 may be implemented in a variety of configurations and form factors, including but not limited to desktops, laptops, handheld personal computers or other handheld devices, servers, workstations, game consoles, embedded systems, and the like.

[0078] Figure 3 According to various embodiments Figure 2 208. In operation, the GPC 208 may be configured to execute a large number of software threads in parallel to perform graphics, general processing, and / or computing operations. In some embodiments, each software thread executed on the GPC 208 is an instance of a specific program, executed on a specific set of input data. In some embodiments, SIMD instruction issuance technology is used to support the parallel execution of a large number of threads without providing multiple independent instruction units. In some other embodiments, SIMT technology is used to support the parallel execution of a large number of generally synchronized threads, using a common instruction unit configured to issue instructions to a group of processing engines within the GPC 208. Unlike the SIMD execution mechanism in which all processing engines generally execute the same instructions, SIMT execution allows different threads to more easily follow different execution paths through a given program. It will be understood by those of ordinary skill in the art that the SIMD processing mechanism is a functional subset of the SIMT processing mechanism.

[0079] The operation of GPC 208 is controlled by pipeline manager 305, which distributes processing tasks received from a work distribution unit (not shown) within task / work unit 207 to one or more SMs 310. Pipeline manager 305 can also be configured to control work distribution crossbar 330 by specifying the destination of processed data output by SMs 310.

[0080] In some embodiments, GPC 208 includes, but is not limited to, M sets of SMs 310, where M ≥ 1. In the same or other embodiments, each of SMs 310 includes, but is not limited to, a set of functional units (not shown), such as execution units and load-store units. Processing operations specific to any one functional unit may be pipelined, such that a new instruction may be issued for execution before a previous instruction has completed execution. Any combination of functional units may be provided within a given SM 310. In various embodiments, the functional units may be configured to support a variety of different operations, including integer and floating point arithmetic (such as addition and multiplication), comparison operations, Boolean operations (such as AND, OR, XOR), displacements, and calculations of various algebraic functions (such as planar interpolation and trigonometric, exponential and logarithmic functions, etc.). Advantageously, the same functional unit may be configured to perform different operations.

[0081] In operation, each of the SMs 310 is configured to process one or more thread groups. In the context of an SM 310, a "thread group" or "warp" refers to a group of threads that concurrently execute the same program on different input data, with each thread of the group being assigned to a different execution unit within the SM 310. A thread group may include fewer threads than the number of execution units within the SM 310, in which case some execution units may be idle during a cycle while the thread group is being processed. A thread group may also include more threads than the number of execution units within the SM 310, in which case processing may occur in consecutive clock cycles. Since each of the SMs 310 can support up to G thread groups simultaneously, it follows that up to G*M thread groups may be executing in a GPC 208 at any given time.

[0082] In addition, multiple related thread groups may be simultaneously active (in different stages of execution) within an SM 310. Such a collection of thread groups is referred to herein as a "cooperative thread array" ("CTA") or "thread array." The size of a particular CTA is equal to m*k, where k is the number of threads executing simultaneously in the thread group, which is typically an integer multiple of the number of execution units within an SM 310, and m is the number of simultaneously active thread groups within an SM 310.

[0083] In some embodiments, each thread in a given thread group is assigned a unique thread ID that can be accessed during the execution of the thread. The thread ID can be defined as a one-dimensional or more-dimensional numerical value that controls various aspects of the processing behavior of the thread. For example, the thread ID can be used to determine which part of the input data set the thread is to process and / or determine which part of the output data set the thread is to generate or write.

[0084] Despite Figure 3204 and 205. Although not shown, each of the SMs 310 contains a level 1 ("L1") cache or uses space in a corresponding L1 cache outside of the SM 310 to support, among other things, load and store operations performed by the execution units. Each SM 310 may also have access to a level 2 ("L2") cache (not shown) that is shared among all GPCs 208 in the PPU 202. In some embodiments, the L2 cache may be used to transfer data between threads. Finally, the SM 310 may also have access to off-chip "global" memory, which may include PP memory 204 and / or system memory 104. It should be understood that any memory external to the PPU 202 may be used as global memory. In addition, as shown in FIG. Figure 3 As shown, a level 1.5 ("L1.5") cache 335 may be included within GPC 208 and is configured to receive and hold data requested from memory by SM 310 via memory interface 214. Such data may include, but is not limited to, instructions, uniform data, and constant data. In embodiments having multiple SMs 310 within GPC 208, SMs 310 may advantageously share common instructions and data cached in L1.5 cache 335.

[0085] Each GPC 208 may have an associated memory management unit ("MMU") 320 that is configured to map virtual addresses to physical addresses. In various embodiments, the MMU 320 may reside within the GPC 208 or within the memory interface 214. The MMU 320 includes a set of page table entries ("PTEs") that map virtual addresses to physical addresses of tiles or memory pages, and optionally cache line indexes. The MMU 320 may include an address translation lookup buffer ("TLB") or buffer that may reside within the SMs 310, within one or more L1 buffers, or within the GPC 208.

[0086] In graphics and compute applications, GPC 208 may be configured such that each SM 310 is coupled to a texture unit 315 for performing texture mapping operations, such as determining texture sample locations, reading texture data, and filtering texture data.

[0087] In operation, each SM 310 transmits processed tasks to the work distribution crossbar 330 to provide the processed tasks to another GPC 208 for further processing, or to store the processed tasks in an L2 cache (not shown), PP memory 204, or system memory 104 via the crossbar unit 210. In addition, a pre-raster operation ("preROP") unit 325 is configured to receive data from the SM 310, direct the data to one or more raster operation ("ROP") units within the partition unit 215, perform optimizations for color blending, organize pixel color data, and perform address translation.

[0088] It will be appreciated that the core architecture described herein is illustrative and that variations and modifications are possible. Among other things, any number of processing units, such as SM 310, texture unit 315, or preROP unit 325, may be included within GPC 208. In addition, as described above in conjunction with Figure 2 As described above, the PPU 202 may include any number of GPCs 208 that are configured to be functionally similar to one another so that execution behavior does not depend on which GPC 208 receives a particular processing task. In addition, each GPC 208 operates independently of the other GPCs 208 in the PPU 202 to execute tasks of one or more application programs. In view of the above, one of ordinary skill in the art will appreciate that Figure 1-3 The architecture described in no way limits the scope of the present disclosure.

[0089] As shown, in some embodiments, one of the SMs 310 is configured to handle a decode thread group 390. The decode thread group 390 is a thread group having a thread group size 182 of T, wherein each of the T threads in the thread group concurrently executes a decoder kernel 192 on different data. In some embodiments, each thread of the decode thread group 390 is assigned to a different execution unit within the SM 310. The decode thread group 390 may be configured in any technically feasible manner. As described above in conjunction with Figure 1 As described above, in some embodiments, the parallel decoding application 190 configures the SM 310 to process the decoding thread group 390. For the purpose of explanation only, the reference to a thread used below refers to the thread of the decoding thread group 390 that is active at any given point in time. Figure 4 and Figure 5 The decoding thread group 390 is described in more detail.

[0090] Parallel decoding using shared input and output pointers

[0091] Figure 4 According to various embodiments Figure 34. The decoding thread group 390 decodes the encoded sequence 402 to generate a decoded sequence 498. As shown, in some embodiments, the decoding thread group 390 includes, but is not limited to, threads 410(0)-410(T-1), a shared input pointer 404, and a shared output pointer 406, where T is the thread group size 182. For purposes of explanation only, the thread IDs (not shown) of the threads 410(0)-410(T-1) are 0-(T-1), respectively.

[0092] In some embodiments, including Figure 4 In the embodiment depicted in FIG. 4 , the encoded sequence 402 is formatted for data parallel processing based on the thread group size 182 symbolized as T and the word size 184 symbolized as B. Furthermore, the encoded sequence 402 is optimized as a data parallel decoding process in which the threads 410 ( 0 )- 410 (T) each decode a single code from the encoded sequence 402 during each decoding iteration.

[0093] In some other embodiments, the encoding sequence 402 can be optimized for different types of data parallel processing and / or different types of data parallel decoding processes, and the techniques described herein are modified accordingly. For example, in some embodiments, the code in the encoding sequence 402 is not packed into fixed-length words, so the shared input pointer 404 is replaced with T different input pointers that are provided as metadata, where each input pointer is associated with a different thread.

[0094] Shared input pointer 404 and shared output pointer 406 are shared between threads 410(0)-410(T-1). In some embodiments, at any given point in time, shared input pointer 404 points to the next word to be read from encoded sequence 402. In the same or other embodiments, at any given point in time, shared output pointer 406 points to a location in memory allocated for decoded sequence 498 immediately following the last text written to decoded sequence 498 by decode thread group 390.

[0095] The format of the encoded sequence 402 and the communication capabilities of the threads 410 enable one of the threads 410 (referred to herein as the "leading thread") to deterministically advance the shared input pointer 404 and the shared output pointer 406 when the thread 410 decodes the encoded sequence 402. The leading thread may be determined in any technically feasible manner. In some embodiments, the leading thread is the thread 410(0) corresponding to the thread ID 0.

[0096] Importantly, the leading thread advances the shared output pointer 406 in a monotonically increasing manner from the beginning of the decode sequence 498 to the end of the decode sequence 498. In a corresponding manner, the decode thread group 390 generates the decode sequence 498 incrementally from the beginning to the end without gaps. In some embodiments, including Figure 4 In the embodiment depicted in , the leading thread also advances the shared input pointer 404 in a monotonically increasing manner from the beginning of the encoding sequence 402 to the end of the encoding sequence 402 .

[0097] Prior to the zeroth iteration, the parallel decoding application 190, the decoding thread group 390, and / or the leading thread initialize the shared input pointer 404 and the shared output pointer to point to the initial text word in the encoding sequence 402 and the beginning of the memory allocated to the decoding sequence 498, respectively. The parallel decoding application 190, the decoding thread group 390, and / or the leading thread may initialize the shared input pointer 404 and the shared output pointer 406 in any technically feasible manner.

[0098] As shown, in some embodiments, threads 410(0)-410(T-1) include, but are not limited to, input buffers 420(0)-420(T-1), refill flags 430(0)-430(T-1), input offsets 440(0)-440(T-1), decoders 462(0)-462(T-1), decoder sizes 464(0)-464(T-1), and output offsets 470(0)-470(T-1). For integers t from 0 to (T-1), input buffers 420(t), refill flags 430(t), input offsets 440(t), decoders 462(t), decoder sizes 464(t), and output offsets 470(t) are associated with thread 410(t) and are also collectively referred to herein as "specific thread variables" of thread 410(t). In some embodiments, each thread 410 writes to the thread-specific variables of thread 410 and reads from the thread-specific variables of any number of threads 410 (including thread 410).

[0099] In some embodiments, at any given point in time, the input buffer 420(t) stores any number of complete codes and / or any number of partial codes assigned to the thread 410(t) that have not yet been decoded. As needed, the thread 410(t) copies the words assigned to the thread 410(t) from the code sequence 402 to the input buffer 420(t). As needed, the thread 410(t) reads and deletes codes from the input buffer 420(t) in a first-in, first-out manner. In some embodiments, prior to the zeroth iteration, the parallel decoding application 190 and / or any number of threads 410 initialize the input buffer 420 to indicate that the input buffer 420 is empty.

[0100] The refill flag 430 enables the threads 410 to consume words from the encoded sequence 402 in a deterministic manner that ensures that each thread 410 reads the words assigned to the thread 410 by the input encoding application 180 during the encoding process. The input offsets 440(0)-440(T-1) are offsets relative to the shared input pointer 404. The sum of the shared input pointer 404 and the input offset 440(t) is the specific thread input pointer for the thread 410(t). As described in more detail below, in some embodiments, the input offset 440(0) is zero, and the threads 410(1)-440(T-1) determine the input offsets 440(1)-440(T-1), respectively, based on the refill flag 430.

[0101] Decoders 462(0)-462(T-1) respectively store one or more texts represented by symbols decoded by threads 410(0)-410(T-1) before threads 410(0)-410(T-1) simultaneously write decoders 462(0)-462(T-1) as a continuous text sequence into decode sequence 498. In some embodiments, each decoder 462 is a single text decoded based on a text encoding, or a text string decoded based on duplicate symbols.

[0102] Decode sizes 464(0)-464(T-1) specify the sizes of decoders 462(0)-462(T-1), respectively. Output offsets 470(0)-470(T-1) are offsets relative to shared output pointer 406. The sum of shared output pointer 406 and output offset 470(t) is the thread-specific output pointer for thread 410(t). As described in more detail below, in some embodiments, output offset 470(0) is zero, and threads 410(1)-440(T-1), respectively, determine output offsets 470(1)-470(T-1) based on decode size 464.

[0103] In some embodiments, to start a new decoding iteration, threads 410(0)-410(T-1) concurrently determine refill flags 430(0)-430(T-1) based on input buffers 420(0)-420(T-1), respectively. More specifically, thread 410(t) determines refill flag 430(t) based on input buffer 420(t). Thread 410(t) may determine refill flag 430(t) based on input buffer 420(t) in any technically feasible manner. In some embodiments, if input buffer 420(t) does not include at least one complete code, thread 410(t) sets refill flag 430(t) to 1 to indicate that the complete code of input buffer 420(t) is exhausted and needs to be refilled.

[0104] In some other embodiments, if the number of valid bits in the input buffer 420(t) is below a constant threshold, the thread 410(t) sets the refill flag 430(t) to 1. By setting the constant threshold to a value no less than the maximum possible code size (in valid bits), the thread 410(t) can conservatively determine the refill flag 430(t) without tracking the occupancy of the input buffer 420(t) based on the code included in the input buffer 420(t).

[0105] Subsequently, threads 410(1)-410(T-1) concurrently determine input offsets 440(1)-410(T-1) based on refill flags 430. For integers x from 1 to (T-1), thread 410(x) sets input offset 440(x) based on the sum of refill flags 430(0)-430(x-1). In some embodiments, input offset 440 is specified in words, and thread 410(x) sets input offset 440(x) to the sum of refill flags 430(0)-430(x-1). Thus, input offset 440(x) may have a value from 0 to x, where 0 indicates that none of input buffers 420(0)-420(x-1) need to be filled, and x indicates that all input buffers 420(0)-420(x-1) need to be refilled. The sum of the refill flags 430(0)-430(x-1) is also referred to herein as the (x-1)th element of the prefix sum of the refill flags 430. In some other embodiments, the input offset 440 is not specified in words, and the thread 410(x) converts the sum of the refill flags 430(0)-430(x-1) to a size unit associated with the input offset 440 to determine the input offset 440(x).

[0106] Subsequently, threads 410 with refill flag 430 set to one perform read step 450 in lockstep. Threads 410 with refill flag 430 set to zero skip read step 450. To perform read step 450(x), thread 410(x) reads a word from encoded sequence 402 according to the sum of shared input pointer 404 and input offset 440(x), and then appends the word to input buffer 420(x). Thus, threads 410 collectively read as few as zero and as many as T consecutive words from encoded sequence 402. Advantageously, memory accesses of encoded sequence 402 during decoding are therefore densely packed and aligned.

[0107] Threads 410(0)-410(T-1) then perform decoding steps 460(0)-460(T-1), respectively. In some embodiments, during decoding step 460(t), thread 410(t) reads and deletes a single code from input buffer 420(t) in a first-in, first-out manner. Figure 1As described above, because each word may begin and / or end with a partial word, a single code may include bits from multiple words. Thread 410(t) decodes the code to determine a single text symbol representing a text string stored as decoder 462(t) by thread 410(t) or a duplicate symbol representing a text string stored as decoder 462(t) by thread 410(t). Thread 410(t) then sets decoder size 464(t) equal to the size of decoder 462(t).

[0108] Thread 410(t) may decode the code in any technically feasible manner. For example, in some embodiments, thread 410(t) executes any number and / or type of decoding algorithms that reverse the mapping of symbols to codes used to generate encoded sequence 402. In some embodiments, the code may be a duplicate code. Advantageously, because decoding thread group 390 generates decoding sequence 498 incrementally from beginning to end without gaps, each original text string referenced backwards by the duplicate code is already stored in decoding sequence 498 or can be obtained from one of threads 410 with a lower thread ID. Therefore, decoding thread group 390 does not postpone processing duplicate code.

[0109] As shown, threads 410(1)-410(T-1) concurrently determine output offsets 470(1)-470(T-1) based on decode size 464, respectively. For an integer x from 1 to (T-1), thread 410(x) sets output offset 470(x) equal to the sum of decode sizes 464(0)-464(x-1). Subsequently, threads 410(0)-410(T-1) respectively perform write steps 480(0)-480(T-1) in lockstep. To perform write step 480(x), thread 410(x) copies decoder 462(x) to decode sequence 498, starting at a position specified by the sum of shared output pointer 406 and output offset 470(x). In this manner, threads 410 collectively write consecutive text groups corresponding to consecutive symbol groups to decode sequence 498. Advantageously, memory access to decode sequence 498 during decoding is therefore intensive. In some embodiments, each symbol is a byte and memory accesses of the encoded sequence 402 during decoding are also aligned.

[0110] The leading thread then executes leading thread step 490 to advance both the shared input pointer 404 and the shared output pointer 406. As shown, in some embodiments, thread 410(0) is the leading thread. In some embodiments, the shared input pointer 404 is specified in units of words, and the leading thread increases the shared input pointer 404 by the sum of the refill flags 430(0)-430(T-1). In some other embodiments, the shared input pointer 404 is not specified in units of words, and the leading thread converts the sum of the refill flags 430(0)-430(T-1) to a size unit associated with the shared input pointer 404 to increment the shared input pointer 404. The leading thread increases the shared output pointer 406 by decoding the sum of the sizes 464(0)-464(T-1).

[0111] Decode thread group 390 continues to perform decode iterations until decode thread group 390 has completed generating decode sequence 498. In some embodiments, the amount of code included in code sequence 402 is not a multiple of thread group size 182, and decode thread group 390 disables each thread 410 when thread 410 has decoded all of the code assigned to thread 410.

[0112] Note that the techniques described herein are illustrative rather than restrictive and may be varied without departing from the broader spirit and scope of the invention. Many modifications and variations of the functionality provided by the decode line group 390 will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments.

[0113] Figure 5 According to various embodiments Figure 4 498. For purposes of explanation only, Figure 5 Depicted is a beginning portion of an exemplary instance of an encoding sequence 402 associated with a thread group size 182 of three threads and a word size 184 of thirty-two bits. Figure 5 Also depicted is a portion of the decoding sequence 498 at a point in time after the decoding thread group 390 executes the decoding iterations 530 ( 0 )- 530 ( 3 ).

[0114] As shown, the encoded sequence 402 includes, but is not limited to, words 510(0)-510(5) and any number of other words 510 (depicted with ellipses). Each word 510 includes, but is not limited to, any integer or non-integer code 520. Although not shown, the source sequence is represented by symbols 598(0)-598(18), as well as any number of other symbols 598 (depicted with ellipses). Figure 5 In the described embodiments, as previously described in conjunction with Figure 1As depicted, entropy coding application 180 encodes symbols 598(0)-598(18) to generate codes 520(0)-520(18), respectively. Figure 5 In the depicted embodiment, for an integer x between 0 and 18, the entropy coding application 180 assigns a code 520(x) to the thread 410 (x%3) for decoding during a decoding iteration 530 (floor(x / 3)). Also, for each thread 410, the entropy coding application 180 packs the bits of the code 520 assigned to the thread 410 into a thirty-two-bit word 510 to generate the encoded sequence 402.

[0115] Thus, as shown, word 510(0) includes, but is not limited to, codes 520(0), 520(3), and 520(6). Word 510(1) includes, but is not limited to, codes 520(1) and 520(4). Word 510(2) includes, but is not limited to, codes 520(2) and 520(5), and the initial portion of code 520(8). Word 510(3) includes, but is not limited to, codes 520(7), 520(10), and 520(13). Word 510(4) includes, but is not limited to, the remainder of code 520(8) and codes 520(11) and 520(14). Word 510(5) includes, but is not limited to, codes 520(9), 520(12), 520(15), and 520(18).

[0116] Each code 520 is a text code or a copy code. Each text code is annotated with an integer, which specifies the index of the text 502 represented by the symbol 598 corresponding to the text code in the source sequence. Each copy code is annotated with an integer range, specifying the index of the text string 502 represented by the symbol 598 corresponding to the copy code in the source sequence. For the purpose of explanation only, the numbering of the source sequence is based on zero. Therefore, text 502 (0) corresponds to the first one of the text 502 included in the source sequence, and text 502 (1) corresponds to the next one of the text 502 included in the source sequence, and so on.

[0117] For example, code 520(0) is a text code annotated with zeros to indicate that code 520(0) corresponds to text 502(0) represented by symbol 598(0). Code 520(6) is a copy code annotated with 6-7 to indicate that code 520(6) corresponds to a copy of text string 502(6)-502(7), where the copy is represented by symbol 598(6). And code 520(18) is a text code annotated with 23 to indicate that code 520(18) corresponds to text 502(23) represented by symbol 598(18).

[0118] For purposes of explanation only, thread 410(0) and words 510(0) and 510(5) decoded by thread 410(0) are depicted by light shaded boxes, thread 410(1) and words 510(1) and 510(3) decoded by thread 410(1) are depicted by dark shaded boxes, and thread 410(2) and words 510(2) and 510(4) decoded by thread 410(2) are depicted by medium shaded boxes. In addition, circles annotated with numbers specifying decoding iterations 530 are superimposed on the arrows to represent words 510 read from encoded sequence 402 by decoding thread group 390, symbols 598 determined by decoding thread group 390 during decoding, and text 502 written into decode sequence 498 by decoding thread group 390 during each decoding iteration 530(0)-530(3).

[0119] As shown, during decoding iteration 530(0), threads 410(0)-410(2) read words 510(0)-510(2), respectively, from encoded sequence 402. Threads 410(0)-410(2) decode codes 520(0)-520(2), respectively, to determine symbols 598(0)-598(2), respectively. Threads 410(0)-410(2) then write text 502(0)-502(2), respectively represented by symbols 598(0)-598(2), to decode sequence 498.

[0120] During decoding iteration 530(1), none of threads 410(0)-410(2) reads from encoded sequence 402. Threads 410(0)-410(2) decode codes 520(3)-520(5), respectively, to determine symbols 598(3)-598(5), respectively. Threads 410(0)-410(2) then write text 502(3)-502(5), represented by symbols 598(3)-598(5), respectively, to decode sequence 498.

[0121] During decoding iteration 530(2), threads 410(1) and 410(2) read words 510(3) and 510(4), respectively, from encoding sequence 402 because word 510(1) stores two codes 520 and word 510(2) stores two half codes 520. In contrast, thread 410(0) does not read from encoding sequence 402 during decoding iteration 530(2) because word 510(0) stores three codes 520.

[0122] Thread 410(0) decodes code 520(6) as the duplicate code to determine symbol 598(6) representing the duplicate. More specifically, and for purposes of explanation only, symbol 598(6) represents a duplicate of text strings 502(2)-502(3). Accordingly, thread 410(0) writes text strings 502(6)-502(7) as duplicates of text strings 502(2)-502(3) to the decode sequence.

[0123] Threads 410(1) and 410(2) decode codes 520(7) and 520(8), respectively, to determine symbols 598(7)-598(8), respectively. Threads 410(1) and 410(2) then write text 502(8)-502(9), represented by symbols 598(7)-598(8), respectively, into decode sequence 498.

[0124] During decoding iteration 530(3), thread 410(0) reads word 510(5) from encoded sequence 402, but neither thread 410(1) nor thread 410(2) reads from encoded sequence 402. Threads 410(0)-410(2) decode code 520(9)-520(11), respectively, to generate symbols 598(9)-598(11), respectively. Threads 410(0)-410(2) then write text 502(10)-502(12), represented by symbols 598(9)-598(11), respectively, to decode sequence 498.

[0125] Advantageously, if Figure 5 As illustrated, memory accesses to the encoding sequence 402 and the decoding sequence 498 are closely spaced and aligned. Furthermore, because the decoding thread group 390 incrementally generates the decoding sequence 498 from beginning to end, the decoding thread group 390 does not need to delay processing duplicate code.

[0126] Figure 6 is a flow chart of method steps for encoding a sequence including a variable length code according to various embodiments. Figure 1-5 Although the method steps are described with reference to a system, those skilled in the art will understand that any system configured to perform the method steps in any order falls within the scope of the present invention.

[0127] As shown, the method 600 begins at step 602, where the parallel decoding application 190 configures the decoding thread group 390 to decode the encoded sequence 402. At step 604, the decoding thread group 390 sets the shared input pointer 404 to point to the beginning of the encoded sequence 402, sets the shared output pointer 406 to point to the beginning of the memory allocated for the decoded output, and each thread 410 initializes the input buffer 420 of the thread 410 to be empty. At step 606, each of the threads 410 determines the refill flag 430 of the thread 410 based on the input buffer 420 of the thread 410.

[0128] At step 608, each thread 410 for which the refill flag 430 is one sets the input offset 440 of the thread 410 to be equal to the sum of the refill flags 430 of the threads 410 with the lower thread ID. At step 610, each thread 410 for which the refill flag 430 is one reads the word 510 from the position in the encoding sequence 402 corresponding to the shared input pointer 404 of the thread 410 and the sum of the input offset 440, and appends the word 510 to the input buffer 420 of the thread 410.

[0129] At step 612, each of the threads 410 decodes the next code 520 in the input buffer 420 of the thread 410 to determine the decode 462 of the thread 410, removes the code 520 from the input buffer 420, and sets the decode size 464 of the thread 410 to be equal to the size of the decode 462 of the thread 410. At step 614, each thread 410 sets the output offset 470 of the thread 410 to be equal to the sum of the decode sizes 464 of the threads 410 with the lower thread ID. At step 616, each of the threads 410 writes the decode 462 of the thread 410 to the position in the decode sequence 498 corresponding to the sum of the shared output pointer 406 of the thread 410 and the output offset 470.

[0130] At step 618, the leading thread (e.g., thread 410(0)) advances the shared input pointer 404 by the sum of the refill flag 430 and advances the shared output pointer 406 by the sum of the decode size 464. At step 620, the decode thread group 390 deactivates any thread 410 that has completed decoding of the encoding 520 assigned to the thread 410. At step 622, the decode thread group 390 determines whether any threads 410 are activated. If, at step 622, the decode thread group 390 determines that no threads 410 are active, the method 600 terminates.

[0131] However, if at step 622, the decoding thread group 390 determines that at least one of the threads 410 is active, the method 600 returns to step 606, where the thread 410 determines the thread's refill flag 430. The decoding thread group 390 continues to loop through steps 606-622 until at step 622, the decoding thread group 390 determines that no thread 410 is active. The method 600 then terminates.

[0132] In summary, the disclosed technology can be used to efficiently decode a coded sequence by a thread group of software or hardware threads to produce a decoded sequence of incremental continuous symbol groups without gaps from beginning to end. In some embodiments, the entropy coding application generates a coded sequence based on a thread group size represented as T and a word size represented as B. The entropy coding application maps symbols representing the text included in the source sequence to a code sequence. The entropy coding application repeatedly assigns consecutive groups of T codes to T thread IDs from 0 to T-1. Therefore, the entropy coding application assigns the xth code with respect to the source sequence to the thread corresponding to the thread ID of x%T so as to decode during floor(x / T) decoding iterations. For each thread ID, the entropy coding application packs the bits of the assigned code into words with B bits each. In order to generate a coded sequence, the entropy coding application arranges words according to the primary criterion of increasing decoding iterations and the secondary criterion of increasing thread IDs.

[0133] In some embodiments, a parallel decoding application decodes an encoded sequence to generate a decoded output. The parallel decoding application configures T thread groups to each execute a different instance of a decoder kernel, thereby generating a decoding thread group. A leading thread included in the decoding thread group sets a shared input pointer to point to the beginning of the encoded sequence and sets a shared output pointer to point to the beginning of a memory allocated for the decoded output. Each thread initializes an input buffer of the thread to indicate that the input buffer is empty. The threads then perform decoding iterations until the decoding thread group completes decoding of the encoded sequence.

[0134] During each decoding iteration, each thread sets the thread's refill flag to zero if there is still at least one complete code in the thread's input buffer, otherwise the thread's refill flag is set to one. Each thread with a refill flag sets the thread's input offset equal to the sum of the refill flags of the threads with lower IDs. Each thread with a refill flag of one reads a word in the encoded sequence at a position corresponding to the sum of the shared input pointer and the thread's input offset, and the thread appends the word to the thread's input buffer. Importantly, during a decoding iteration, a thread with a refill flag of zero does not read from the encoded sequence.

[0135] Each thread extracts the first remaining code from the thread's input buffer and decodes the code to determine the thread's decoding. The thread can decode the code using any number and / or type of decoding algorithms. Each thread sets the thread's decoding size equal to the size of the decoding. Each thread sets the thread's output offset equal to the sum of the decoding sizes of the threads with lower thread IDs. Subsequently, each thread writes the thread's decoding to the decoded output, starting from a position corresponding to the sum of the thread's shared output pointer and output offset. To complete the decoding iteration, the leader increments the shared input pointer by the sum of the refill flags and increments the shared output pointer by the sum of the decoding sizes.

[0136] At least one technical advantage of the disclosed technology relative to the prior art is that, with the disclosed technology, a decoding thread group does not require metadata to efficiently decode a parallel encoding sequence including variable length codes. More specifically, the entropy coding application establishes a format by systematically rearranging the codes relative to corresponding symbols, so that the threads in the decoding thread group can deterministically access the code group for parallel decoding without any additional information. Moreover, unlike the prior art, since the threads read any number (including zero) of consecutive words from the encoding sequence and append a set of consecutive texts to the decoding sequence during each decoding iteration, the decoding throughput is not reduced due to sparse memory access. In addition, since the decoding thread group generates the decoded output incrementally from beginning to end without gaps, the text referenced by each duplicated symbol is available when the decoding thread group decodes the corresponding duplicated code. Therefore, relative to the prior art of storing duplicated codes to postpone processing, the decoding thread group can decode duplicated codes more efficiently. These technical advantages provide one or more technical improvements to the prior art methods.

[0137] Any and all combinations of any claim elements recited in any claim and / or any elements described in this application, in any manner, belong to the envisioned scope of embodiments and protection of the present invention.

[0138] The description of various embodiments is presented for the purpose of illustration, but is not intended to be exhaustive or limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments.

[0139] Aspects of embodiments of the present invention may be embodied as systems, methods or computer program products. Therefore, aspects of the present disclosure may take the form of complete hardware implementations, complete software implementations (including firmware, resident software, microcode, etc.) or implementations combining software and hardware aspects, which may be generally referred to herein as "modules", "systems" or "computers". In addition, any hardware and / or software technology, process, function, component, engine, module or system described in the present disclosure may be implemented as a circuit or a group of circuits. In addition, various aspects of the present disclosure may take the form of a computer program product embodied in one or more computer-readable media, on which a computer-readable program codec is embodied.

[0140] Any combination of one or more computer-readable media may be utilized. A computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples (non-exhaustive list) of computer-readable storage media would include the following: an electrical connection with one or more wires, a portable computer floppy disk, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory, a flash memory, an optical fiber, a portable optical disc read-only memory, an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this article, a computer-readable storage medium may be any tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device.

[0141] Various aspects of the present disclosure are described above with reference to the flowchart illustrations and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present disclosure. It is understood that each block of the flowchart illustration and / or block diagram and the combination of blocks in the flowchart illustration and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer or other programmable data processing device to produce a machine. These instructions, when executed by a processor of a computer or other programmable data processing device, can implement the functions / behaviors of one or more blocks specified in the flowchart and / or block diagram. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a specific application processor or a field programmable gate array.

[0142] The flow chart and block diagram in the figure illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each block in the flow chart or block diagram can represent a module, segment or part of a code, which includes one or more executable instructions for realizing a specified logical function. It should also be noted that in some alternative embodiments, the function indicated in the block may not appear in the order indicated in the figure. For example, two blocks displayed continuously may actually be basically executed simultaneously, or these blocks may sometimes be executed in reverse order, depending on the function involved. It will also be noted that each block in the block diagram and / or flow chart description, and the combination of the blocks in the block diagram and / or flow chart description, can be realized by a system based on special purpose hardware that performs a specified function or behavior, or a combination of special purpose hardware and computer instructions.

[0143] While the foregoing is directed to embodiments of the present disclosure, other and further embodiments of the disclosure may be devised without departing from the basic scope thereof, and the scope thereof is determined by the following claims.

Claims

1. A method for decoding a coded sequence comprising a variable length code, the method comprising: determining a first plurality of codes based on the coded sequence; decoding the first plurality of codes to generate a first plurality of symbols; determining a first plurality of offsets based on the first plurality of symbols; as well as generating a first continuous portion of a decoded sequence based on the first plurality of symbols, the first plurality of offsets, and a first address, Wherein, generating the first continuous part of the decoding sequence includes, for each offset included in the first plurality of offsets, writing different text represented by different symbols included in the first plurality of symbols to different positions in the decoding sequence, wherein the different positions are specified by the sum of the offset and the first address.

2. The method of claim 1 , wherein determining the first plurality of codes comprises: determining a plurality of refill flags based on the first plurality of input buffers; reading one or more consecutive fixed-length words from the encoding sequence based on the plurality of refill flags and a second address to generate a second plurality of input buffers; as well as A different code is extracted from each input buffer included in the second plurality of input buffers to generate the first plurality of codes.

3. The method of claim 1 , wherein decoding the first plurality of codes comprises: determining that a first code included in the first plurality of codes includes a duplicate code; and At least two texts are calculated based on at least one of: the decoded sequence or at least a second code included in the first plurality of codes and the first code. 4 . The method of claim 1 , wherein decoding the first plurality of codes comprises executing one or more decoding algorithms on the first plurality of codes.

5. The method of claim 1 , wherein determining the first plurality of offsets comprises: A first plurality of magnitudes is calculated based on the first plurality of symbols, and a prefix sum of the first plurality of magnitudes is calculated.

6. The method according to claim 1, further comprising: adding a sum of the first plurality of offsets to the first address to generate a second address; as well as A second contiguous portion of the decoded sequence is generated based on the encoded sequence and the second address.

7. The method of claim 1 , wherein determining the first plurality of codes comprises: determining a plurality of refill flags based on the first plurality of input buffers; calculating a second plurality of offsets based on a prefix sum of the plurality of refill flags; reading one or more consecutive fixed-length words from the encoded sequence based on the second plurality of offsets and second addresses to generate a second plurality of input buffers; as well as A different code is extracted from each input buffer included in the second plurality of input buffers to generate the first plurality of codes.

8. The method according to claim 7, further comprising: adding a sum of the second plurality of offsets to the second address to generate a third address; determining a second plurality of codes based on the encoding sequence, the third address, and the second plurality of input buffers; as well as A second continuous portion of the decoded sequence is generated based on the second plurality of codes.

9. The method of claim 1, wherein a group of threads executing in a multithreaded processor generates the first continuous portion of the decode sequence, and wherein a size of the group of threads matches a thread group size associated with the encode sequence.

10. One or more non-transitory computer-readable media comprising instructions that, when executed by one or more processors, cause the one or more processors to decode a coded sequence comprising a variable length code by performing the following steps: determining a first plurality of codes based on the coded sequence; decoding the first plurality of codes to generate a first plurality of symbols; determining a first plurality of offsets based on the first plurality of symbols; as well as generating a first continuous portion of a decoded sequence based on the first plurality of symbols, the first plurality of offsets, and a first address, Wherein, generating the first continuous part of the decoding sequence includes, for each offset included in the first plurality of offsets, writing different text represented by different symbols included in the first plurality of symbols to different positions in the decoding sequence, wherein the different positions are specified by the sum of the offset and the first address.

11. The one or more non-transitory computer-readable media of claim 10, wherein determining the first plurality of codes comprises: determining that a first input buffer included in the plurality of input buffers includes less than one code; in response, performing one or more read operations on the encoded sequence to determine a first fixed-length word; adding the first fixed-length word to the first input buffer; as well as A different code is extracted from each input buffer included in the plurality of input buffers to generate the first plurality of codes.

12. The one or more non-transitory computer-readable media of claim 10, wherein decoding the first plurality of codes comprises: determining that a first code included in the first plurality of codes includes a duplicate code; and At least two texts are read from the decoding sequence based on the replication code.

13. The one or more non-transitory computer-readable media of claim 10, wherein decoding the first plurality of codes comprises executing one or more decoding algorithms on the first plurality of codes.

14. The one or more non-transitory computer-readable media of claim 10, wherein: Determining the first plurality of offsets includes calculating a first plurality of magnitudes based on the first plurality of symbols, and calculating a prefix sum of the first plurality of magnitudes.

15. The one or more non-transitory computer-readable media of claim 10, further comprising: adding a sum of the first plurality of offsets to the first address to generate a second address; as well as A second contiguous portion of the decoded sequence is generated based on the encoded sequence and the second address.

16. The one or more non-transitory computer-readable media of claim 10, wherein determining the first plurality of codes comprises: determining a plurality of refill flags based on the first plurality of input buffers; calculating a second plurality of offsets based on a prefix sum of the plurality of refill flags; reading one or more consecutive fixed-length words from the encoded sequence based on the second plurality of offsets and second addresses to generate a second plurality of input buffers; as well as A different code is extracted from each input buffer included in the second plurality of input buffers to generate the first plurality of codes.

17. The one or more non-transitory computer-readable media of claim 16, further comprising: adding a sum of the second plurality of offsets to the second address to generate a third address; determining a second plurality of codes based on the encoding sequence, the third address, and the second plurality of input buffers; as well as A second continuous portion of the decoded sequence is generated based on the second plurality of codes.

18. The one or more non-transitory computer-readable media of claim 10, wherein a set of independent hardware processes associated with at least one fixed-function hardware unit determines the first plurality of codes based on the encoding sequence, and wherein, The size of the set of independent hardware processes matches the thread group size associated with the encoding sequence.

19. A system comprising: one or more memories storing instructions; and One or more processors coupled to the one or more memories, the processors performing the following steps when executing the instructions: determining a first plurality of codes based on the coded sequence; decoding the first plurality of codes to generate a first plurality of symbols; determining a first plurality of offsets based on the first plurality of symbols; as well as generating a first continuous portion of a decoded sequence based on the first plurality of symbols, the first plurality of offsets, and a first address, Wherein, generating the first continuous part of the decoding sequence includes, for each offset included in the first plurality of offsets, writing different text represented by different symbols included in the first plurality of symbols to different positions in the decoding sequence, wherein the different positions are specified by the sum of the offset and the first address.

Images