Method of encoding and decoding image content and system for transmitting image content

Abstract

The present disclosure relates to a method of encoding and decoding image content and a system of transmitting image content, in particular to a method for decoding image content from an encoded bitstream comprising a plurality of blocks, comprising: dividing a block comprising one or more components of the image content into N single samples and M sample groups corresponding to one of the components, wherein N and M are greater than or equal to 1; decoding each of the N single samples using a sign variable length code to generate one or more decoded single samples; decoding each of the M sample groups using a common prefix entropy code to generate one or more decoded sample groups, each of the M sample groups comprising a variable length prefix and one or more fixed length suffixes representing a plurality of samples; concatenating the decoded single samples and the decoded sample groups into a block of residuals; and reconstructing the image content based on previously reconstructed neighboring blocks and the block of residuals.

Description

[0001] Cross-references to related applications

[0002] This application claims priority and benefit to U.S. Provisional Patent Application No. 62 / 960,517, filed January 13, 2020, with the U.S. Patent and Trademark Office, and U.S. Term Patent Application No. 16 / 835,092, filed March 30, 2020, the entire disclosure of which is incorporated herein by reference. Technical Field

[0003] Embodiments of this disclosure relate to systems and methods for scalable throughput entropy encoders. Background Technology

[0004] Data compression involves encoding information using fewer bits compared to the original representation. Lossless data compression achieves encoding by removing statistical redundancy from the original representation of the information. Therefore, in lossless data compression, no information is lost, and the original representation of the information can be reconstructed by a decoder or decompressor. On the other hand, lossy data compression reduces the number of bits used by removing unnecessary or less important information. Therefore, in many cases, the original representation of the information cannot be perfectly reconstructed from data compressed using a lossy compression algorithm.

[0005] Entropy coding is a form of data compression in which fewer bits are used to encode the most likely symbols and more bits are used to encode the least likely symbols. In other words, in an entropy coding system, the number of bits used to represent a symbol can vary based on the probability that the symbol will appear in the original representation of the information. Examples of entropy coding include Huffman codes and Morse codes (for example, where a single bit is used to encode the most common letters in English, “E” and “T”, while four bits are used to represent less common letters such as “Q” and “Z”). Summary of the Invention

[0006] Embodiments of this disclosure relate to systems and methods for implementing scalable throughput entropy encoders.

[0007] According to one embodiment of the disclosure, a method for decoding image content from an encoded bitstream including a plurality of blocks includes: dividing, by a decoder circuit, a block including one or more components of the image content into N single samples and M sample groups corresponding to one of the one or more components, where N is greater than or equal to 1 and M is greater than or equal to 1; decoding, by the decoder circuit, each of the N single samples using a sign variable length code (SVLC) to generate one or more decoded single samples; decoding, by the decoder circuit, each of the M sample groups using a common prefix entropy code (CPEC) to generate one or more decoded sample groups, each of the M sample groups including a variable length prefix and one or more fixed length suffixes representing a plurality of samples; concatenating, by the decoder circuit, the decoded single samples and the decoded sample groups into a block of residuals; and reconstructing, by the decoder circuit, the image content based on a previously reconstructed neighboring block of the image content and the block of residuals.

[0008] N and M can be set according to a target decoder throughput by calculating an upper limit of M based on a number of samples of the block and a highest available throughput, calculating a number of variable length codes based on the number of samples of the block and the target decoder throughput, and calculating N based on the number of variable length codes and the upper limit of M.

[0009] Each of the M sample groups of the block can have a same number of fixed length suffixes. The block can be predictively encoded in a transform skip block prediction mode.

[0010] At least two of the M sample groups of the block can have different numbers of fixed length suffixes. The block can be predictively encoded in a transform mode or in a transform skip block prediction mode.

[0011] The block can include a plurality of components of the image content.

[0012] The encoded bitstream can further include a component skip flag indicating that all of the plurality of samples of a corresponding component of the block of the encoded bitstream are zero.

[0013] The encoded bitstream can further include a group skip flag indicating that all of the plurality of samples of a group of the M sample groups are zero.

[0014] According to one embodiment of the disclosure, a method for encoding image content includes: partitioning, by an encoder circuit, received image content into one or more blocks, each of the one or more blocks including a plurality of samples from one or more components of the image content; predictively encoding, by the encoder circuit, each block to generate a block of residuals; partitioning, by the encoder circuit, each of the block of residuals into N single samples and M groups of samples, where N is greater than or equal to 1 and M is greater than or equal to 1; encoding, by the encoder circuit, each of the N single samples using a sign variable length code (SVLC) to generate one or more SVLC-encoded samples; encoding, by the encoder circuit, each of the M groups of samples using a common prefix entropy code (CPEC) to generate one or more CPEC-encoded groups of samples, each of the M groups of samples including a variable length prefix and one or more fixed length suffixes; and combining, by the encoder circuit, the SVLC-encoded samples and the CPEC-encoded groups of samples to output an encoded bitstream.

[0015] N and M can be set according to a target decoder throughput by: calculating an upper limit of M based on a number of samples per block of the one or more blocks and a highest available throughput; calculating a number of variable length codes based on the number of samples per block and the target decoder throughput; and calculating N based on the number of variable length codes and the upper limit of M.

[0016] Partitioning each of the block of residuals can include dividing at least one of the block of residuals using uniform partitioning, where each of the M groups of samples of the at least one of the block of residuals can have a same number of fixed length suffixes. The at least one of the block of residuals can be predictively encoded in a transform skip block prediction mode.

[0017] Partitioning each of the block of residuals can include dividing at least one of the block of residuals using non-uniform partitioning, where at least two of the M groups of samples of the at least one of the block of residuals can have different numbers of fixed length suffixes. The at least one of the block of residuals can be predictively encoded in a transform mode or in a transform skip block prediction mode.

[0018] Each block can include a plurality of components of the image content.

[0019] The encoded bitstream can further include a component skip flag indicating that all of the plurality of samples of at least one corresponding component of the block are zero.

[0020] The encoded bitstream can further include a group skip flag indicating that all of the plurality of samples of a group of the M groups of samples are zero.

[0021] According to one embodiment of the present disclosure, a system for transmitting image content includes: an encoder circuit configured to: divide received image content including a plurality of components into one or more blocks, each of the one or more blocks including a plurality of samples from one of the plurality of components; predictively encode each block to generate a predictively encoded block; partition each of the predictively encoded blocks into N single samples and M sample groups, where N is greater than or equal to 1 and M is greater than or equal to 1; encode each of the N single samples using a sign variable length code (SVLC) to generate one or more SVLC encoded samples; encode each of the M sample groups using a common prefix entropy code (CPEC) to generate one or more CPEC encoded sample groups, each of the M sample groups including a variable length prefix and one or more fixed length suffixes; and combine the SVLC encoded samples and the CPEC encoded sample groups to output an encoded bitstream; and a decoder circuit configured to: receive the encoded bitstream from the encoder circuit; divide a block of the encoded bitstream into the N single samples and the M sample groups; decode each of the N single samples using the SVLC to generate one or more decoded single samples; decode each of the M sample groups using the CPEC to generate one or more decoded sample groups; reconstruct the predictively encoded block from the decoded single samples and the decoded sample groups; apply the prediction encoding to decode the predictively encoded block; and decode the image content from the decoded predictively encoded block.

[0022] N and M can be set according to a target decoder throughput by: calculating an upper limit of M based on a number of samples of a block and a highest available throughput; calculating a number of variable length codes based on the number of samples of the block and the target decoder throughput; and calculating N based on the number of variable length codes and the upper limit of M.

[0023] The encoder circuit can be further configured to: detect one or more factors of a communication environment in which at least one of the encoder circuit and the decoder circuit is operating; and dynamically update values of N and M based on the one or more factors.

[0024] The one or more factors can include one or more of: power, processor capability, a number of parallel working decoders in the decoder circuit, internal bandwidth, temperature conditions of the decoder circuit, and noise in a physical medium between the encoder circuit and the decoder circuit. BRIEF DESCRIPTION OF DRAWINGS

[0025] The accompanying drawings illustrate exemplary embodiments of the present disclosure and together with the description, serve to explain the principles of the present disclosure.

[0026] Figure 1is a block diagram of a system comprising an encoder and a decoder, wherein the encoder is configured to encode image content and the decoder is configured to decode the image content for display on a display device.

[0027] Figure 2 is a schematic illustration of a block of an image, wherein the block has a size of 8x2 (with 16 samples) and is divided into 4 groups of 4 samples each.

[0028] Figure 3 is a schematic depiction of a common prefix entropy code (CPEC) structure with groups of N=4 samples.

[0029] Figure 4 is a schematic depiction of a 3x1 block and its encoded form using CPEC in case of display stream compression (DSC).

[0030] Figure 5A is a schematic depiction of an 8x2 block divided into 4 uniformly sized groups (each group is a 2x2 portion of the block corresponding to 4 samples).

[0031] Figure 5B is a schematic depiction of an 8x2 block divided into 4 non-uniformly sized groups.

[0032] Figure 6 is a flowchart of a method for calculating the number of samples N encoded using a sign variable length code (SVLC) and the number of groups of samples M encoded using a CPEC for a given target decoder throughput according to one embodiment of the present disclosure.

[0033] Figure 7 is a flowchart of a method of encoding image content according to one embodiment of the present disclosure.

[0034] Figure 8 is a flowchart of a method of decoding image content according to one embodiment of the present disclosure.

[0035] Figure 9A is a schematic depiction of encoding a block using uniformly sized groups according to one embodiment of the present disclosure.

[0036] Figure 9B is a schematic depiction of encoding a block using non-uniformly sized groups according to one embodiment of the present disclosure.

[0037] Figure 9C is a schematic depiction of encoding a block according to a transform skip block prediction mode using non-uniformly sized groups according to one embodiment of the present disclosure. DETAILED DESCRIPTION

[0038] In the following detailed description, only certain exemplary embodiments of the disclosure are shown and described, by way of illustration. As those skilled in the art would realize, the disclosure can be implemented in many different forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided as examples so that this disclosure will be thorough and complete, and will fully convey the aspects and features of the disclosure to those skilled in the art. Therefore, processes, elements, and techniques that are not necessary for an understanding of the aspects and features of the disclosure can not be described. Throughout the drawings and written description, like reference numerals designate like elements and thus their descriptions will not be repeated.

[0039] Compression can be applied to data storage and data transmission in a wide range of contexts such as data archiving, as well as wired and wireless data communication over computer networks and over local connections. These local connections can include, for example, data transmission along data buses within computing devices (e.g., smartphones, tablet computers, laptop computers, and desktop computers) and / or connections to external devices, for example, through wired connections such as digital display interfaces (e.g., DisplayPort (DP) or Display Serial Interface (DSI)).

[0040] For convenience, embodiments of the disclosure will be presented herein in the context of digital display data, more specifically, displaying image content on a display panel of a display device. However, embodiments of the disclosure are not limited thereto, and the principles described herein can be applied to provide scalable throughput entropy encoders for other applications.

[0041] Figure 1 is a block diagram of a system including an encoder and a decoder, where the encoder is configured to encode image content and the decoder is configured to decode the image content for display on a display device. As shown in Figure 1 The image content 10 (e.g., a bitmap representation of a single image or a single frame or multiple frames of images) is fed to an application processor (AP) 100 (e.g., a central processing unit (CPU), a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), and / or a graphics processing unit (GPU) of a computing device) of a host 1, where the AP 100 can include an encoder 110, as shown in Figure 1In the embodiment shown in FIG. 1, the encoded (or compressed) bitstream 30 can be transmitted to a display driver integrated circuit (DDIC) 200 of a display device 2 (e.g., an external monitor, a television, or an integrated display panel of a smartphone, tablet computer, or laptop computer) through a physical medium 50 (e.g., through wiring such as a data bus or cable or other connector or through a wireless connection). The DDIC 200 includes a frame buffer 202 (or frame buffer, e.g., a random access memory such as dynamic random access memory) for receiving and storing the encoded bitstream 30 as it arrives through the physical medium 50, and a decoder 210 configured to decode the encoded (or compressed) bitstream 30 from the frame buffer 202 into a decompressed representation (or decoded representation) 18. In the case of lossless encoding, the decompressed representation 18 is identical (or substantially identical) to the original representation of the image content 10. In the case of lossy encoding, the decompressed representation 18 can be substantially similar (e.g., visually similar) to the original representation of the image content 10, such that the resulting data appears visually lossless. The DDIC 200 can then control the display panel 20 to display the decoded representation 18 of the image content 10, such as by supplying drive waveforms to the display panel 20 to control the luminance of individual pixels of the display panel 20 in accordance with the decoded representation 18 of the image content 10.

[0042] The DDIC 200 can include or be connected to components for demodulating signals received through the physical medium 50 (e.g., to generate a digital representation of the encoded bitstream 30 based on voltages applied to the physical medium 50). Likewise, the AP 100 can include or be connected to components for modulating signals to be applied to the physical medium 50 based on the encoded bitstream 30.

[0043] According to various embodiments of the present disclosure, the encoder 110 and the decoder 210 can be referred to as encoder circuitry and decoder circuitry, respectively, and can be implemented using various types of processing circuitry, as will be appreciated by those skilled in the art, and the encoder circuitry can be implemented using a different type of processing circuitry than the decoder circuitry. These types of processing circuitry can include, for example, general-purpose computer processing units (CPUs), graphics processing units (GPUs), digital signal processors (DSPs), field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), or combinations thereof (e.g., where different types of processing circuitry are used to implement different portions of an encoding process or pipeline or a decoding process or pipeline). Additionally, as will be appreciated by those skilled in the art, the various processing circuitry can be components of the same integrated circuit (e.g., as components of the same system on a chip or SoC), or can be components of different integrated circuits that can be connected through pins and wires on a printed circuit board.

[0044] Generally, image data or image content 10 is represented using a bitmap, where each location in the bitmap corresponds to a single pixel. For clarity, the term “pixel” will be used herein to refer to a picture element having multiple components (or channels). For example, in a red-green-blue (RGB) color space, these components include a red component (or red channel), a green component (or green channel), and a blue component (or blue channel). As another example, in a YCbCr color space, these components can include a luma (Y) component, a chroma blue (Cb) component, and a chroma red (Cr) component. As another example, in a YCoCg color space, these components can include a luma (Y) component, a chroma green (Cg) component, and a chroma orange (Co) component. Embodiments of the present disclosure are not limited in this regard, however. The value at each location in the bitmap represents a level (e.g., a gray level) of a component at that location. Thus, the image content 10 or image data can be considered to represent the color and luminance at each location in an image.

[0045] For discussion herein, each channel of the image content 10 will be processed independently. As is generally understood by those skilled in the art, the term “sample” refers to a digital value associated with one component of one pixel of the image content 10 (e.g., a “sample” can be a scalar value), the term “block” refers to a collection of samples (e.g., samples corresponding to a contiguous portion of the image content 10), and each block can be partitioned into one or more “groups” of samples. Figure 2 is an illustration of one block 12 of an image (image content 10) of birds, where the block 12 has a size of 8x2 (16 samples 14) and is partitioned into 4 groups 16A, 16B, 16C, and 16D, each having 4 samples.

[0046] A block can be encoded using spatial prediction within the image content 10 (e.g., a value of a sample can be encoded based on a prediction of the value of the sample from neighboring samples in the image, such as pixels above and to the left of the pixel in the image). Rather than using the original neighboring values, the reconstructed values are used for prediction. The difference between the predicted value of a sample and the actual value of the sample is referred to as a quantized residual, and these quantized residuals can be further used to represent the samples in an encoded representation of the image content 10.

[0047] For each block, one of a variety of different prediction coding modes can be used, where a particular prediction coding mode can be selected based on a test to determine which mode produces the least amount of distortion subject to a rate control constraint or, in other words, a rate-distortion cost, where rate relates to the amount of data needed to encode the block in that mode and distortion refers to the loss in quality after encoding (e.g., the size of the difference between the input block and the encoded block after being decoded). These prediction coding modes can include a "transform mode" that includes a transform (e.g., to the frequency domain) operation that can be more suitable for natural content and a "transform skip block prediction mode" that skips (or excludes) the transform and performs a block prediction operation and can be used for artificial or diagrammatic graphical content. For each block, there are a variety of modes (or options) available for encoding the block, and the encoder 110 can select the best mode (or option) for the block.

[0048] When operating in the transform mode, in some embodiments, the encoder 110 tests a set of intra-predictors for each block (denoted as X herein) to determine which intra-predictor produces the least rate-distortion cost. The selected intra-prediction mode is explicitly signaled in the bitstream, so that the decoder 210 only needs to parse the information and perform a single decoding operation. In cases where the block X is in the RGB color space, the data can be transformed into the YCoCg color space. In some cases, if the block X is in the YCbCr color space, no color space transform is applied, and the process continues in the YCbCr space.

[0049] In some embodiments, the encoder 110 computes an intra-predicted block P for a given set of intra-prediction modes in the transform space. The output of the intra-prediction is a residual block R (also referred to as residual R herein), which is the difference between the block X and the intra-predicted block P. In some embodiments, a discrete cosine transform (DCT) is then applied to the residual block R, resulting in a block of transform coefficients T. The transform coefficients T are then quantized to produce a block of quantized transform coefficients QT (also referred to as quantized transform coefficients QT herein). These quantized transform coefficients QT are the values that are transmitted in the bitstream, as embedded in an entropy coding group. Inverse quantization and inverse transform are applied, so that a distortion can be computed between the residual block R and a reconstructed residual block (also referred to as reconstructed residual herein). (The same inverse quantization and inverse transform operations can be performed by the decoder 210.) The rate-distortion cost information for each mode is computed from the rate and distortion mentioned above.

[0050] When operating in transform skip block prediction (BP) mode, in some embodiments, the current block is spatially predicted from a set of reconstructed neighboring samples (BP search range). Before prediction, the current block is partitioned into a set of sub-blocks (e.g., in the case of an 8x2 block, into 4 2x2 sub-blocks).

[0051] According to one embodiment, each sub-block is predicted from the BP search range using either a 2x2 partition or a pair of 2x1 partitions. In the former case, the 2x2 sub-block is represented by a single block prediction vector (BPV) that will generate a 2x2 prediction block P from the search range. In the case where a pair of 2x1 partitions is selected, the sub-block will be represented by two different BPVs. The first BPV will generate a 2x1 prediction block P for the upper two samples in the sub-block, while the second BPV will generate a 2x1 prediction block P for the lower two samples. The encoder 110 performs a search to find the BPVs that minimize the distortion of each 2x2 partition and 2x1 partition within the current block. The result is a set of prediction blocks P for both partition types and a set of BPVs. Next, the residual R is computed as R = X - P. Since there are two options for the partition type, two residual blocks R will be computed; one associated with the 2x2 partition and one associated with the 2x1 partition. The two residual blocks R can then be processed as follows (e.g., in parallel): first, forward quantization is performed on all residual samples, and the quantized residual QR is used to compute the entropy coding cost for each 2x2 sub-block; a second inverse quantization is performed to obtain the reconstructed residual from the reconstructed residual The distortion for each sub-block can be computed. Finally, for each 2x2 sub-block, the encoder 110 can select between the 2x2 partition and the 2x1 partition based on a rate / distortion trade-off. The syntax for the BP mode includes the set of BPVs and the entropy coded quantized residual QR for the 3 color components.

[0052] According to one embodiment, the decoder 210 receives from the bitstream the BP coded block including the quantized residual QR. Specifically, the decoder 210 applies an entropy decoder to decode the quantized residual QR while the BPV values and the partition structure are directly parsed. The BP search range is identical between the encoder 110 and the decoder 210 as it includes the causally available reconstructed samples. The partition structure and the BPV values are used to generate the prediction block P, while the quantized residual QR is inverse quantized to obtain the reconstructed residual Finally, the prediction block P and the reconstructed residual are added together to generate the reconstructed block, which is color space transformed if needed. For RGB source content, the BP is computed in the YCoCg color space. For YCbCr source content, the BP will be computed locally in the YCbCr color space.

[0053] When operating in transform mode, a transform such as a discrete cosine transform (DCT) is further applied to the residual, and the transformed coefficients are represented as values in a block. In other cases, such as operating in transform skip block prediction mode, the transform is omitted and the residual itself is represented as values in a block. In some cases, when lossless compression is used, the transform mode can be omitted. In some cases, when lossy compression is applied, the transform coefficients or the residual (in the case of transform mode or transform skip block prediction mode, respectively) are quantized.

[0054] Common prefix entropy coding (CPEC) is a technique for entropy coding a group of N samples (e.g., quantized residuals or quantized transform coefficients in the case of lossy compression), where a single prefix is used with N suffixes. In CPEC, the prefix is encoded using a variable length code (e.g., unary code) that represents the number of bits used to encode each of the N suffixes. Each of the N suffixes is encoded with the same number of bits (e.g., using a fixed length code). Figure 3 is a schematic depiction of a CPEC structure with a group of N = 4 samples. Thus, as shown in Figure 3 The CPEC structure 300 includes a prefix 302 and 4 suffixes 304A, 304B, 304C, and 304D, as shown in

[0055] The entropy decoder can take a full clock cycle to parse the prefix of a CPEC encoded group, as the prefix is of variable length. However, once the prefix representing the number of bits in each of the suffixes is decoded, the N suffixes can be shifted into another buffer to parse the individual samples without occupying additional time of the entropy decoder, and the entropy decoder can jump ahead in the frame buffer 202 (e.g., by the decoded number of bits times N) to decode the next group.

[0056] Additional information on entropy coding can be found in, for example, Jacobson, Natan, et al. “A new display stream compression standard under development in VESA.” Applications in Digital Image Processing XL, Vol. 10396, Society of Photo-Optical Instrumentation Engineers, 2017.

[0057] Entropy encoders and common prefix entropy codes (CPEC) are used in some standards such as Display Stream Compression (DSC). Figure 4 is a schematic depiction of a 3 x 1 block and its encoded form using common prefix entropy codes (CPEC) in the case of delta size unit variable length coding (DSU-VLC) used in Display Stream Compression (DSC). As shown inFigure 4 As shown in FIG. 5, a group of 3x1 (e.g., 3 adjacent samples in a row: s0, s1, and s2) can be entropy encoded using CPEC with one prefix (p0) 402 and 3 suffixes 404A, 404B, and 404C (suffix 0, suffix 1, and suffix 2, respectively). Thus, a 3-sample / clock throughput is achievable because parsing of the variable length prefix (p0) 402 can take a full clock cycle, but 3 suffixes can be decoded from the entropy decoder without additional time, e.g., because there is one variable length bin (VLB) for every 3 samples (s0, s1, and s2, as represented by suffix 0, suffix 1, and suffix 2, respectively) (in prefix 402).

[0058] As another example, in the case of VESA Display Compression-M (VDC-M), a block size of 8x2 is used, where each component of the block is partitioned into 4 groups. In some modes, the 8x2 block is partitioned into uniform groups. Figure 5A is a schematic depiction of an 8x2 block 510 divided into 4 uniformly sized groups (each group is a 2x2 portion of the block 510 corresponding to 4 samples), labeled as group 0, group 1, group 2, and group 3. The 16 samples of the block 510 are labeled S0 through S15. As shown in Figure 5A As shown in FIG. 6, each group is entropy encoded using CPEC with 4 suffixes and a corresponding variable length prefix to generate corresponding entropy encoded groups 520, 521, 522, and 523. Specifically, group 0 includes the signs representing samples S0, S1, S8, and S9, and is entropy encoded as entropy encoded group 520 using prefix P0; group 1 includes the signs representing samples S2, S3, S10, and S11, and is entropy encoded as entropy encoded group 521 using prefix P1; group 2 includes the signs representing samples S4, S5, S12, and S13, and is entropy encoded as entropy encoded group 522 using prefix P2; and group 3 includes the signs representing samples S6, S7, S14, and S15, and is entropy encoded as entropy encoded group 523 using prefix P3. Figure 5A The prefixes P0, P1, P2, and P3 are depicted using boxes of different widths to indicate that these variable length prefixes can have different lengths (e.g., encode the number of bits used to encode each of the suffixes in their respective groups). Likewise, the suffixes of each of the groups have lengths that can vary according to the values encoded by their corresponding prefixes.

[0059] In other modes, the 8x2 block is partitioned into non-uniform groups. Figure 5B is a schematic depiction of an 8x2 block 530 divided into 4 non-uniformly sized groups, labeled as group 0, group 1, group 2, and group 3. The 16 samples of the block 530 are labeled S0 through S15. As shown in Figure 5BIn the arrangement shown in FIG. 5B, group 0 has a size of 1 sample, group 1 has a size of 3 samples, group 2 has a size of 5 samples, and group 3 has a size of 7 samples. Specifically, group 0 includes the symbol representing sample SO and is entropy encoded as entropy encoded group 540 using prefix PO; group 1 includes the symbols representing samples S1, S2, and S8 and is entropy encoded as entropy encoded group 541 using prefix P1; group 2 includes the symbols representing samples S3, S4, S9, S10, and S11 and is entropy encoded as entropy encoded group 542 using prefix P2; and group 3 includes the symbols representing samples S5, S6, S7, S12, S13, S14, and S15 and is entropy encoded as entropy encoded group 543 using prefix P3. As with FIG. 5A, the prefix P0, P1, P2, and P3 are depicted using boxes of different widths to indicate that these variable length prefixes can have different lengths (e.g., encoding the number of bits used to encode each of the suffixes in their respective groups). Likewise, the suffixes of each of the groups have lengths that can vary according to the values encoded by their corresponding prefixes. Figure 5A As with FIG. 5A, Figure 5B As with FIG. 5A,

[0060] When both uniform and non-uniform groups are used in VDC-M, CPEC is applied on a per group basis. Thus, in VDC-M, an average throughput of 4 samples / clock is achievable, e.g., because in both cases 4 clock periods are used to decode the variable length prefixes that encode the 16 samples of a block together in 4 groups (e.g., 16 / 4 = 4).

[0061] Different applications can impose different design constraints, such as available bandwidth on the physical medium 50, image resolution of the image content 10, frame rate of the image content 10, decoder clock speed, and power requirements or constraints (e.g., more stringent constraints on power consumption in a handheld mobile device such as a smartphone compared to a larger mobile device such as a laptop computer and compared to a device plugged into a power grid such as a television). For example, in many cases, the DDIC 200 of the display device 2 is much slower compared to, for example, the AP 100 or other components implementing the encoder 110, or can be built on older technology. In designing the communication channel for various applications, a design tradeoff can be made between throughput (TP) and compression efficiency (CE), where higher throughput typically corresponds to lower compression efficiency, and higher compression efficiency typically corresponds to lower throughput. As mentioned above, the DSU-VLC encoding of the DSC uses one variable length code per 3 samples, enabling a throughput of 3 samples per clock, and the VDC-M encoding uses 4 variable length codes per block of 16 samples, enabling a throughput of 4 samples per clock. While these approaches used by the DSC and the VDC-M enable the system to achieve a particular throughput, they do not allow the encoding scheme to be adjusted to change the throughput to meet the design constraints of a particular application (e.g., compression efficiency requirements, decoding clock speed, power requirements).

[0062] Accordingly, aspects of embodiments of the present disclosure are directed to systems and methods for entropy encoding methods and / or protocols, where the throughput of the entropy encoding method can be adjusted by controlling a design parameter of the protocol. As mentioned above, the throughput of the decoder 210 (in terms of the number of samples per clock of the decoder 210) is typically limited based on the number of variable length codes that need to be resolved for each block of samples. Accordingly, some aspects of embodiments of the present disclosure are directed to controlling the throughput by setting the number of variable length codes used when encoding (or coding) a plurality of samples (e.g., the number of variable length codes used to encode a block of 16 samples).

[0063] In more detail, some aspects of embodiments of the present disclosure relate to a hybrid entropy coding scheme in which each component (e.g., in red, green, and blue components or in luma and chroma components) of a block is coded in such a way that one or more of the samples of the block are independently coded using a sign variable length code (SVLC) and the remaining samples of the block are partitioned into groups and coded using a CPEC that allocates a variable length binary word to each group. Examples of a sign variable length code (SVLC) include: unary code, Huffman code, Exponential-Golumb coding, Rice coding, and a concatenation of Exponential-Golumb and Rice codes, among others. In the following discussion, a block can be referred to as having N samples coded using a SVLC and M samples coded using a CPEC, where N and M are greater than or equal to zero. Some embodiments of the present disclosure relate to blocks in which both N and M are greater than or equal to 1.

[0064] Thus, in the hybrid coding method according to embodiments of the present disclosure, the total number of variable length binary words (VLBs) per component per block can be controlled by the parameters N and M, thereby allowing design control of the decoder throughput in units of samples per clock.

[0065] In more detail, the decoder throughput depends on the number of samples per component in a block and the total number of VLBs in a block:

[0066]

[0067]

[0068] where k≠0.

[0069] The total number of variable length binary words used per component in a block depends on the number of samples N coded using a SVLC and the number of groups of samples M coded using a CPEC:

[0070] #VLBs = N + M

[0071] Figure 6 is a flowchart of a method for calculating the number of samples N coded using a SVLC and the number of groups of samples M coded using a CPEC for a given target decoder throughput according to one embodiment of the present disclosure. In some embodiments, the method shown in Figure 6 is implemented using a suitable computing device such as a computer system including a processor and a memory, where the memory stores instructions that, when executed by the processor, cause the processor to perform the operations of the method associated with Figure 6 and output the calculated design parameters N and M.

[0072] In operation 610, the computer system according to one embodiment computes an upper limit on the number M of coded samples of the CPEC. This can be computed by setting the throughput to the estimated highest available throughput TP max

[0073]

[0074] For example, in some cases, each clock TP max = 4 samples represents a good tradeoff between performance and complexity (or compression efficiency):

[0075]

[0076] However, embodiments of the present disclosure are not limited to the case where the highest available throughput TP max is 4 samples per clock. For example, improvements in semiconductor technology and compression can result in cases where a TP max value higher than 4 samples per clock represents a good tradeoff between performance and complexity.

[0077] In operation 630, the computing system computes the number of variable length binary words (#VLBs) required to achieve the target decoder throughput TP target

[0078]

[0079] In operation 650, the computing system computes the number N of coded samples of the SVLC to achieve the target decoder throughput TP target based on the previously computed required number of VLBs (#VLBs) and the previously computed value of M:

[0080] #VLBs = N + M

[0081] or

[0082] N = #VLBs - M

[0083] Thus, the method described above with reference to Figure 6 provides a method for computing design parameters N and M based on a target decoder throughput.

[0084] For example, assume samples of block size 8 x 2 per component, and assume TP max = 4, the values of M and N can be computed as follows:

[0085] For a target decoder throughput of 2 samples per clock, per operation 610:

[0086] ​​

[0087] Every operation 630:

[0088]

[0089] And every operation 650:

[0090] N = VLBs - M

[0091] = 8 - 4

[0092] = 4

[0093] For a target decoder throughput of 3 samples per clock, every operation 610:

[0094]

[0095] Every operation 630:

[0096]

[0097] And every operation 650:

[0098] N = VLBs - M

[0099] = 5 - 4

[0100] = 1

[0101] For a target decoder throughput of 4 samples per clock, every operation 610:

[0102]

[0103] Every operation 630:

[0104]

[0105] And every operation 650

[0106] N = VLBs - M

[0107] = 4 - 4

[0108] = 0

[0109] For a target decoder throughput of 1 sample per clock, every operation 610:

[0110]

[0111] Every operation 630:

[0112]

[0113] And every operation 650:

[0114] N = VLBs - M

[0115] = 16 - 4

[0116] = 12

[0117] In some embodiments of the disclosure, the computing system computes parameters N and M according to a particular target compression efficiency, rather than computing parameters N and M for achieving a particular target decoder throughput TP target of the input. More specifically, compression efficiency can depend on: 1) compression ratio; 2) throughput; and 3) complexity of the codec (which can also depend on throughput). For a given complexity of the codec (e.g., keeping the complexity of the codec constant), a higher compression ratio will trade off throughput to achieve the same performance as a scheme with a lower compression ratio and high throughput. (For example, assume there are two codecs A and B that are roughly the same in complexity and performance, then codec A can operate at a compression ratio of 6: 1 with a throughput of 1 pixel / clock, while codec B can operate at a compression ratio of 4: 1 with a throughput of 4 pixels / clock.)

[0118] According to some embodiments, these design parameters are selected during the design phase of the system and are fixed for production of the finished product. However, embodiments of the disclosure are not so limited, and in some embodiments, parameters N and M that control the number of SVLC samples and CPEC groups are dynamically set in response to changing conditions (e.g., changing communication environment, where throughput can be reduced due to the need for additional error correction coding) during use of the system.

[0119] According to one embodiment of the disclosure, encoder 110 and / or decoder 210 dynamically determine a target decoder throughput or compression efficiency based on certain factors related to the communication environment in which at least one of encoder 110 and decoder 210 (e.g., one of encoder 110 and decoder 210 or both encoder 110 and decoder 210) is operating, and set parameters M and N of the protocol accordingly (e.g., according to the updated target decoder throughput, based on the factors). In some embodiments, the factors include power (e.g., whether the device is plugged into external power or plugged into battery power), processor capability (e.g., throttling due to thermal conditions and / or power consumption settings), number of decoder 210 working in parallel, internal bandwidth, thermal or temperature conditions of the decoder circuit, and noise or interference in the physical medium 50 between encoder 110 and decoder 210.

[0120] To enable the encoder 110 to configure or set the parameters M and N to determine the target decoder throughput or compression efficiency based on these factors, some embodiments of the present disclosure provide mechanisms for feedback to the encoder 110. In some embodiments, this is performed in real-time (e.g., during a live stream, where the target can change in the middle), or when the encoder 110 first starts encoding for a particular decoder 210 (and then the encoder 110 can take into account the specifications of the decoder 210 or other factors, and use those specifications to set the target). Examples of feedback types include the clock rate of the decoder 210, the current temperature of the decoder 210, the power status of the system that includes the decoder 210 (e.g., battery level or external power status), and the current status of the physical medium 50 through which the encoder 110 and decoder 210 communicate (e.g., wired or wireless connection, and the available bandwidth or throughput of the physical medium 50 due to interference). The feedback can be provided, for example, directly from the decoder-side system 200 (i.e., the DDIC 200) to the encoder-side system 100 (i.e., the AP 100), for example via the physical medium 50, or indirectly via a third system (e.g., a monitoring device configured to monitor the status of the decoder-side system 200 such as temperature and power status, and configured to provide information representing the monitored status to the encoder-side system 100).

[0121] According to some embodiments of the present disclosure, the decoder 210 also dynamically updates the parameters of the encoding scheme (values of M and N) according to the changes made to the parameters by the encoder 110, so it can implement the appropriate decoding scheme. In some embodiments, the encoder 110 explicitly indicates the changes to the decoder 210 (when the changes occur, or that the changes will occur at time T), and the explicit indication can be provided in-band (e.g., within the encoded bitstream 30 as encoded image content) or out-of-band (e.g., in a separate channel such as a separate communication stream in parallel with the encoded bitstream 30 on the same physical medium 50 or a different physical medium). In some embodiments, the decoder 210 independently considers the same factors as considered by the encoder 110, and performs the same analysis as the encoder 110 to predict when and how the encoder 110 updates the parameters of the encoding scheme (e.g., values of M and N).

[0122] Figure 7 is a flowchart of a method of encoding image content according to one embodiment of the present disclosure. According to some embodiments, reference is made to Figure 7The described operations are performed by the encoder 110 to generate the encoded bitstream 30 based on the supplied image content 10. In operation 710, each channel of the image content 10 is divided into a plurality of blocks of consecutive samples (e.g., samples from regions of adjacent pixels). In some embodiments, the image content 10 is converted between different color spaces, such as from RGB to YCoCg or YCbCr. In operation 730, the encoder 110 predictively encodes each block, where the predictive encoding can be, for example, transform mode encoding, transform skip block prediction mode encoding, etc. In more detail, the predictive encoding in operation 730 can include predicting values of each component (e.g., Y component, Cb component, and Cr component) of a block based on reconstructed sample values from neighboring blocks (e.g., previous rows and / or previous columns of the image content 10) using a predictor. The encoder 110 then computes residuals based on the difference between the predicted values and the actual values. In some embodiments, when using a lossy encoding scheme, the residuals are quantized to generate blocks of quantized residuals. As mentioned above, different blocks of the same image content can apply different types of predictive encoding (e.g., some blocks can use transform mode, while other blocks from the same image can use transform skip block prediction mode - in some embodiments, at the encoder 110, each predictive encoding mode is applied to each block to generate a plurality of encoded blocks, each encoded block for each of the different modes, and the encoder 110 outputs the encoded block with the lowest rate-distortion cost). At operation 750, the encoder 110 partitions each encoded block into N single samples and M sample groups (e.g., the encoder 110 takes N samples from the block and groups the remaining samples into M groups, where each of the M groups includes one or more samples, and where each of the samples of the block only appears once in the N single samples and M sample groups). For example, in some embodiments, the decoder 210 takes the first N samples of the block (e.g., samples S0to S(N-1)) as the N single samples, and partitions the remaining samples into the M sample groups.

[0123] In operation 770, the encoder 110 applies entropy encoding to the residuals. In operation 772, the encoder 110 encodes each of the N single samples using a symbol variable length code (SVLC) (such as unary encoding, Huffman encoding, exponential Golomb encoding, Rice encoding, and concatenation of exponential Golomb and Rice codes, etc.) to generate encoded single samples. Also, at operation 774, the encoder 110 encodes the M sample groups using a common prefix entropy code (CPEC) (or some other code where one or more samples are encoded using exactly one variable length prefix and one or more fixed length suffixes) to generate encoded sample groups.

[0124] At operation 790, the encoder 110 combines (e.g., concatenates) the encoded single samples with the encoded groups of samples to generate a data stream (e.g., the encoded bitstream 30).

[0125] Figure 8 is a flowchart of a method of decoding image content according to one embodiment of the present disclosure. According to some embodiments, with respect to Figure 8 The described operations are performed by the decoder 210 to generate the reconstructed image content 18 (i.e., the decompressed representation 18 or the decoded representation 18) based on the received encoded bitstream 30. The particular prediction coding technique (e.g., transform mode versus transform skip block prediction mode) can vary from block to block depending on the technique selected by the encoder 110 (e.g., to minimize rate-distortion cost). Thus, in some embodiments, the encoder 110 generates an explicit indication (e.g., a flag) of the prediction coding mode in the encoded bitstream 30, and in some embodiments, the decoder 210 automatically determines the prediction coding mode from the encoded bitstream 30. In operation 806, the decoder 210 determines the prediction coding mode for the current block of the encoded bitstream 30, such as by determining which of a plurality of different coding modes is indicated in the encoded bitstream 30, such as by a flag or other identifier. In operation 810, the decoder 210 partitions the received encoded bitstream 30 into N single samples and M groups of samples (e.g., the first N VLBs can be treated as N single samples, and the remaining data can be parsed as CPEC encoded groups). As noted above, to receive a single block, operation 810 can employ a number of clock cycles equal to the number of variable length binary words (VLBs) used to encode the block, as each VLB takes one clock cycle to parse, but if the VLBs act as prefixes to groups (e.g., groups of samples), then the fixed length suffixes can be shifted to another buffer for further processing without slowing down the decoder 210.

[0126] In operation 830, the decoder 210 entropy decodes the residuals. In operation 832, the decoder 210 decodes each of the N single samples using the SVLCs discussed above (e.g., unary codes or Huffman codes), and in operation 834, decodes each of the M groups of samples using CPEC, where the number of samples in each of the M groups of samples can be determined by the prediction coding mode of the block determined at operation 806 (e.g., whether the M groups of samples are encoded using uniform groups or non-uniform groups, as discussed below with respect to FIG. 9). Figure 9A and Figure 9B(Discussed in more detail). In some cases, N is zero and operation 832 is omitted. Then, in operation 850, decoder 210 concatenates samples from N single samples and M sample groups into a block of residuals (e.g., an encoded block), and then in operation 870 applies the corresponding predictive coding (e.g., the appropriate inverse operation performed by encoder 110) to reconstruct each component of the block (e.g., the Y component, Cb component, and Cr component). Typically, the reconstruction process is as described above regarding... Figure 7 The reverse of the described encoding process includes performing entropy decoding, dequantizing to generate reconstructed residuals, performing an inverse transform (if the transform mode is used in the block), calculating predicted samples based on neighboring samples (e.g., neighboring blocks of previously reconstructed data, such as blocks of previous rows and / or previous columns of image content 10), and then adding the residuals to the output of the predictor.

[0127] The following will discuss the target decoder throughput TP with 16 samples in an 8×2 block and 2 samples per clock. target Examples of encoding blocks of data according to embodiments of the present disclosure are described in more detail in the context of the following. However, embodiments of the present disclosure are not limited to the specific context presented below. For example, embodiments of the present disclosure can also be applied to blocks of different sizes and / or dimensions, such as 3×1, 4×8, and 8×8, and can also be selected as described above regarding Figure 6 The parameters N and M discussed are applied to different target decoder throughput TP, such as 3 samples per clock. target Values. Alternatively, samples can be encoded using two's complement or sign-amplitude representation.

[0128] An example CPEC output for an entropy-coded group containing four samples with sample values ​​{1, -2, -1, 0} is presented below. In this example, two's complement representation is used to generate the bits of the sample values ​​in the entropy-coded group. The number of bits required to faithfully reconstruct the group is 2. Specifically, in two's complement representation, n bits are needed to represent the range [-2]. n-1 ,2n -1 The data is in the range [-1]. Each sample value is represented by 2 bits in two's complement representation. Therefore, the prefix will indicate the value 2. For the group of sample values ​​{1, -2, -1, 0}, the bits output by the CPEC operation will be the prefix 110 (2-unary code) and 4 suffixes, where each sample is encoded using 2 bits such as "01101100". In this example, the output of the CPEC operation is only illustrative, and the actual output of the CPEC operation may vary depending on the actual implementation.

[0129] When symbol-amplitude representation is used, each sample requires n bits, where the absolute value (or amplitude) of all samples in the group is in [0, 2].n In a sign-magnitude representation, the sign bits are only informed for non-zero values. As an example, for an input set of sample values {1, -2, -1, 0}, the output of a CPEC operation in a sign-magnitude representation would be a prefix of 110 (unary code for 2), followed by a suffix that encodes the absolute values of each sample using 2 bits such as "01100100", and finally a sign bit "100" that uses 1 to represent a positive sign (for the first value 1) and 0 to represent a negative sign (for the second and third values -2 and -1). (Note that in this example, the sign value of 0 is not informed.) In some embodiments, 0 is used to represent a positive sign, and 1 is used to represent a negative sign.

[0130] The block can be partitioned into M groups in a uniform manner or a non-uniform manner. In a uniform partitioning, the number of samples (or fixed-length suffixes) in each of the M groups is the same. In a non-uniform partitioning, the number of samples varies from one group to another (e.g., at least two of the M groups have a different number of samples). The selection of uniform versus non-uniform groups can be made based on the prediction coding mode used to encode the block. For example, blocks that are encoded using a transform mode are typically better suited for non-uniform groups.

[0131] Figure 9A is a schematic depiction of encoding a block using uniform groups according to one embodiment of the disclosure. In the particular example shown in Figure 9A block 910 represents an 8x2 rectangle of samples taken from one component of an input image (e.g., from one component of image content 10). When block 910 is predictively encoded using a transform skip block prediction mode (e.g., where samples S0 through S15 of block 910 represent quantized residuals of base samples of the component of the image, rather than coefficients of a transform such as a DCT of a portion of the image), uniform groups can be applied.

[0132] In the example shown in Figure 9A the encoding scheme is designed for a throughput of 2 samples per clock. Thus, following the calculations described above with respect to Figure 6 the number of samples N that are encoded using sign variable length codes (SVLCs) is 4, and the number of sample groups M is also 4. Each of the M sample groups includes 1 prefix and 3 suffixes. This results in a total of 8 (N+M = 8) variable length bins (VLBs) in the bitstream per block of 16 samples (N+M*3 = 16), which provides the target decoder throughput of 2 samples per clock (16 samples / 8 clocks). The N samples that are encoded using SVLCs are in the first 4 VLBS, and the M sample groups are in the last 4 VLBS. Figure 9AThe sample was labeled as group 0920, and the M sample groups were labeled as group 1921, group 2922, group 3923, and group 4924. Table 1 below summarizes these findings. Figure 9A The five groups shown are their corresponding encoding methods and the total number of variable-length binary words (VLBs) in each group.

[0133] Table 1

[0134]

[0135]

[0136] exist Figure 9A In the arrangement shown in Table 1, the bit representations B0, B1, B8, and B9 of samples S0, S1, S8, and S9 in group 0920 are decoded sequentially because the boundaries between symbols are unknown or ambiguous due to each sample being encoded as a VLB. Figure 9A The bit representations B0, B1, B8, and B9 of samples S0, S1, S8, and S9 are depicted using boxes of different widths to indicate that these SVLC-encoded symbols have different lengths. According to... Figure 9A In the embodiment of the encoding scheme shown, group 0920 represents samples S0, S1, S8, and S9. However, embodiments of this disclosure are not limited thereto, and groups of SVLC-encoded samples can represent different samples of block 910.

[0137] Figure 9A Prefixes P1, P2, P3, and P4 are depicted using boxes of varying widths to indicate that these variable-length prefixes can have different lengths (e.g., the number of bits used to encode each of the suffixes in their respective groups). Similarly, the suffix for each group has a length that can vary depending on the value encoded by its corresponding prefix. Once their respective VLB prefixes P1, P2, P3, and P4 are resolved, the three suffixes in each of groups 1, 2, 3, and 4 can be resolved in parallel. Figure 9A In the embodiment of the encoding scheme shown, group 1921 represents samples S2, S3, and S10; group 2922 represents samples S4, S11, and S12; group 3923 represents samples S5, S6, and S13; and group 4924 represents samples S7, S14, and S15. However, embodiments of this disclosure are not limited thereto, and sample groups may represent different samples of block 910.

[0138] Figure 9B This is a schematic depiction of encoding using non-uniform grouped blocks according to an embodiment of the present disclosure. Figure 9BIn the particular example shown in FIG. 9, block 930 represents an 8x2 rectangle of samples taken from one component of an input image (e.g., from one component of image content 10). When block 930 is predictively encoded according to a transform mode (e.g., in which samples S0 through S15 of block 930 represent quantized coefficients of a transform such as a spatial frequency domain transform, e.g., a discrete cosine transform (DCT), of quantized residuals of the block), non-uniform groups can be applied.

[0139] In Figure 9B the example shown in FIG. 9, the encoding scheme is designed for a throughput of 2 samples per clock. Thus, following the calculations described above with respect to Figure 6 the number of samples N that are encoded using a sign variable length code (SVLC) is 4, and the number of groups M of samples is also 4. The 4 samples that are encoded using the SVLC are labeled as group 0 940 in Figure 9B FIG. 9, and the 4 groups of samples are labeled as group 1 941, group 2 942, group 3 943, and group 4 944. Each of the M groups of samples includes one prefix and a different number of suffixes. The number of suffixes can be set during design of the encoding scheme (e.g., while selecting parameters N and M based on a target decoder throughput and according to the size of the block). In Figure 9B the particular embodiment shown in FIG. 9, group 1 941 includes prefix P1 and only one suffix, and group 2 942 includes prefix P2 and only one suffix (e.g., each of group 1 941 and group 2 942 includes only one value). Group 3 943 includes prefix P3 and 4 suffixes, and group 4 includes prefix P4 and 6 suffixes. This results in a block of 16 samples (4 + 1 + 1 + 4 + 6 = 16) having a total of 8 (N + M = 8) variable length binaries (VLBs) in the bitstream, which provides a target decoder throughput of 2 samples per clock (16 samples / 8 clocks). The 5 groups shown in Figure 9B FIG. 9, as well as their corresponding encoding methods and total number of variable length binaries (VLBs) in each group, are summarized in Table 2 below.

[0140] Table 2

[0141] Group Index Encoding Method Number of Samples Total VLB Group 0 SVLC 4 4 Group 1 CPEC 1 1 Group 2 CPEC 1 1 Group 3 CPEC 4 1 Group 4 CPEC 6 1

[0142] In Figure 9B the arrangement shown in FIG. 9 and Table 2, the bit representations B0, B1, B2, and B8 of samples S0, S1, S2, and S8 in group 0 940 are sequentially decoded because the boundaries between the symbols are unknown or ambiguous due to each of the samples being encoded as a VLB. In Figure 9B the bit representations B0, B1, B2, and B8 of samples S0, S1, S2, and S8 are depicted using boxes of different widths in FIG. 9 to illustrate that these SVLC-encoded symbols have different lengths. According toFigure 9B As shown in the embodiment of the encoding scheme in FIG. 9, group 0 940 represents samples SO, SI, S2, and S8. However, embodiments of the present disclosure are not limited thereto, and the groups of SVLC-encoded samples can represent different samples of block 930.

[0143] As in FIG. 9, Figure 9A As in FIG. 9, Figure 9B The prefixes PI, P2, P3, and P4 are described using boxes of different widths to indicate that these variable-length prefixes can have different lengths (e.g., encoding the number of bits used to encode each of the suffixes in their respective groups). Likewise, the suffixes of each of the groups have lengths that can vary depending on the values encoded by their corresponding prefixes. Once their respective VLB prefixes PI, P2, P3, and P4 are parsed, the suffixes in each of groups 1, 2, 3, and 4 can be parsed in parallel. According to Figure 9B As shown in the embodiment of the encoding scheme in FIG. 9, group 1 941 represents sample S3, group 2 942 represents sample S4, group 3 943 represents samples S5, S9, S10, and S11, and group 4 944 represents samples S6, S7, S12, S13, S14, and S15. However, embodiments of the present disclosure are not limited thereto, and the groups of samples can have different sizes and can represent different samples of block 930.

[0144] In some embodiments of the present disclosure, the CPEC-encoded groups that include only one sample are instead encoded using SVLC. For example, referring to Figure 9B , groups 1 and 2 each include only one sample. Thus, instead of encoding each of these two samples with a corresponding variable-length prefix (PI and P2, respectively) and a corresponding suffix, these two samples can be directly encoded using SVLC (see, e.g., the encoding scheme described below in FIG. 10). Figure 9C). Some aspects of embodiments of the present disclosure relate to a method in which the encoder 110 selects between encoding a single sample using either an SVLC or a CPEC group having a single value. For example, the selection between an SVLC or a CPEC can depend on the sample distribution and the SVLC encoding method. As one example, assume that the sample value is -1. Using CPEC encoding and two's complement, the value -1 can be represented using a single bit having a value of 1. Thus, as described above, the prefix of a CPEC group specifying a suffix of length 1 is "10" and the suffix is a value of 1, for a total of 3 bits. On the other hand, when using an SVLC and, for example, Huffman coding, it can be found that -1 is a very unlikely sample value in a given environment and thus the particular codebook for that environment uses more than 3 bits to represent the sample value of -1. On the other hand, there can be cases where a sample value has a high probability and thus a short representation in the Huffman code, but also can require more bits to represent using two's complement and CPEC. In such a case, an SVLC can be a more efficient method for encoding that sample value. Thus, in some embodiments of the present disclosure, the encoder 110 dynamically selects whether to encode a single sample using an SVLC or a CPEC based on the efficiency of encoding the data using the two different techniques, and the encoder 110 can include a selected selection in the coded bitstream. In some embodiments, the selection of the SVLC or CPEC is determined based on a flag included in the coded bitstream 30 as part of determining the prediction encoding mode of the block in operation 806, and in some embodiments, the flag indicating whether a single sample is encoded using an SVLC or a CPEC is located in another portion of the coded bitstream 30, such as immediately preceding the M sample groups. Table 3 below summarizes the modifications to the encoding shown in Table 2 according to this embodiment. Figure 9B A modified version of the encoding shown in Table 2:

[0145] Table 3

[0146]

[0147]

[0148] Additionally, in some embodiments of the present disclosure, the partitioning of a block into multiple non-uniform groups is also applied when encoding the block using the transform skip block prediction mode. Figure 9C is a schematic depiction of encoding a block according to the transform skip block prediction mode using non-uniform groups according to one embodiment of the present disclosure. In the particular example shown in Figure 9C block 950 represents an 8x2 rectangle of samples taken from one component of an input image, e.g., from one component of the image content 10. When predictively encoding the block 950 according to the transform skip block prediction mode, the non-uniform groups can also be applied.

[0149] In the example shown in Figure 9C , the encoding scheme is designed for a throughput of 2 samples per clock. Thus, following the calculations described above with respect to Figure 6 , the number of samples N encoded using a sign variable length code (SVLC) is 4, and the number of groups of samples M is also 4. The 4 samples encoded using SVLC are labeled as group 0 960 in Figure 9C , and the 4 groups of samples are labeled as group 1 961, group 2 962, group 3 963, and group 4 964. Each of the M groups of samples includes one prefix and a different number of suffixes. The number of suffixes can be set during design of the encoding scheme (e.g., while selecting parameters N and M based on a target decoder throughput and according to the size of the block). In the particular embodiment shown in Figure 9C , group 1 961 and group 2 962 each include only one value. Thus, as described above and as shown in Figure 9C , in some embodiments, these samples are encoded using SVLC rather than CPEC. Group 3 963 includes prefix P3 and 5 suffixes, and group 4 includes prefix P4 and 5 suffixes. This results in a block of 16 samples (4 + 1 + 1 + 5 + 5 = 16) having a total of 8 (N + M = 8) variable length bins (VLBs) in the bitstream, which provides a target decoder throughput of 2 samples per clock (16 samples / 8 clocks). The 5 groups shown in Figure 9C , as well as their corresponding encoding methods and total number of variable length bins (VLBs) in each group, are summarized in Table 4 below.

[0150] Table 4

[0151]

[0152]

[0153] In the arrangement shown in Figure 9C and Table 4, the bit representations B0, B1, B8, and B9 of samples S0, S1, S8, and S9 in group 0 960 are sequentially decoded, as the boundaries between the symbols are unknown or ambiguous due to each of the samples being encoded as a VLB. In Figure 9C , the bit representations B0, B1, B8, and B9 of samples S0, S1, S8, and S9 are depicted using boxes of different widths to indicate that these SVLC-encoded symbols have different lengths. According to Figure 9C , group 0 960 represents samples S0, S1, S8, and S9. However, embodiments of the present disclosure are not limited thereto, and the groups of SVLC-encoded samples can represent different samples of block 950.

[0154] Figure 9C The prefixes P3 and P4 are depicted using boxes of different widths to indicate that these variable length prefixes can have different lengths (e.g., encoding the number of bits used to encode each of the suffixes in their respective groups). Likewise, the suffixes of each of groups 3 and 4 have lengths that can vary depending on the values encoded by their corresponding prefixes. Once their respective VLB prefixes P3 and P4 are parsed, the suffixes in each of groups 3 and 4 can be parsed in parallel. According to Figure 9C According to the embodiment of the encoding scheme shown in FIG. 9, group 1 961 represents sample S2, group 2 962 represents sample S10, group 3 963 represents samples S3, S11, S4, S12, and S5, and group 4 964 represents samples S7, S6, S13, S14, and S15. However, embodiments of the present application are not limited thereto, and the sample groups can have different sizes and can represent different samples of block 950. Because Figure 9C Figure 9C The group is chosen to be a consecutive group because the block is depicted as being encoded using a transform skip block prediction mode.

[0155] In some embodiments, a skip flag is used to signal when all samples within a group or within a component are zero. In some embodiments, a component skip flag is used when all samples in one component of a block are zero (e.g., the chroma orange component or the chroma green component in the YCoCg color space or the YCbCr color space, as the luminance component Y is rarely all zero).

[0156] In some embodiments, a group skip flag is used when all samples in one group are zero. In some embodiments, the samples that are encoded using SVLC are partitioned into one or more groups, and a group skip flag can be applied to all samples within a group of SVLC encoded samples. In some embodiments, a group skip flag is used with CPEC encoded groups. In some embodiments, a group skip flag is used only with CPEC encoded groups and not with other groups such as SVLC encoded groups. When a block is encoded using a transform mode, it is common to have groups that contain all zeros, as for example a block with low spatial frequency can result in many coefficients being zero.

[0157] Some aspects of embodiments of the present disclosure relate to adjusting the encoding arrangement to accommodate different sampling formats. For example, in a 4:2:2 chroma format, a chroma component block (e.g., a Cb component and a Cr component in a YCbCr format) contains half the number of samples of a luminance (Y) component block due to horizontal subsampling. As another example, in a 4:2:0 chroma format, a chroma component block contains one quarter the number of samples of a luminance (Y) component block due to both horizontal and vertical subsampling.

[0158] Accordingly, aspects of embodiments of the present disclosure are directed to maintaining the same decoding throughput for 4:2:2 chroma format and 4:2:0 chroma format. In some embodiments, the total number of groups M and the number of individual values N are calculated such that the total number of VLBs for the chroma components is no more than half (in the case of 4:2:2 chroma format) or one quarter (in the case of 4:2:0 chroma format) of the number of VLBs for the luma components. For example, with reference to the example embodiment described above having an 8x2 block size of a total of 16 samples, for 4:2:2 content, the number of luma samples is 16 and the number of chroma samples is 8, and thus the upper limit of M for CPEC encoding in the bitstream for 4:2:2 is 4 for luma blocks (16 / 4 = 4) and 2 for chroma blocks (8 / 4 = 2). In the case of 4:2:0 content, the number of luma samples is 16 and the number of chroma samples is 4, and thus the upper limit of M in the bitstream for 4:2:0 is 4 for luma blocks (16 / 4 = 4) and 1 for chroma blocks (4 / 4 = 1).

[0159] Accordingly, aspects of embodiments of the present disclosure are directed to systems and methods for implementing an encoding method in which the throughput of the encoding protocol, measured in units of symbols per decoder clock, is adjustable. Some aspects of embodiments of the present disclosure are directed to adjusting the throughput by modifying the number of variable length bins (VLBs) used to encode a given block in the encoded bitstream. In more detail, assuming that a decoder takes one full clock cycle to parse a given VLB, and thus the target decoder throughput can be achieved by dividing the number of samples per block by the target decoder throughput to arrive at a target number of VLBs per block. The target number of VLBs can then be controlled by encoding some of the samples using a symbol variable length code (SVLC) and encoding some of the samples using a code in which a single variable length code is shared by multiple samples that are encoded using a fixed length code (e.g., a common prefix entropy code or CPEC). Accordingly, aspects of embodiments of the present disclosure enable an adjustable protocol or class of protocols to meet the target decoder throughput of a particular application, allowing greater design flexibility when making tradeoffs between throughput and compression efficiency.

[0160] While the present disclosure has been described in connection with certain example embodiments, it will be understood that the present disclosure is not limited to the disclosed embodiments but is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims and their equivalents.

Claims

1. A method for decoding image content from an encoded bitstream comprising a plurality of blocks, the method comprising: partitioning, by decoder circuitry, a block comprising one or more components of the image content into N single samples and M sample groups by taking N single samples from the block and grouping the remaining samples of the block into M sample groups, wherein N is greater than or equal to 1 and M is greater than or equal to 1; decoding, by the decoder circuitry, each of the N single samples using a sign variable length code to generate one or more decoded single samples, each of the N single samples being encoded as a separate variable length binary word; decoding, by the decoder circuitry, each of the M sample groups using a common prefix entropy code to generate one or more decoded sample groups, each of the M sample groups comprising a variable length prefix and one or more fixed length suffixes representing a plurality of samples; concatenating, by the decoder circuitry, the decoded single samples and the decoded sample groups into a block of residuals; and reconstructing, by the decoder circuitry, the image content based on previously reconstructed neighboring blocks of the image content and the block of residuals.

2. The method of claim 1, wherein, N and M are set according to a target decoder throughput by: calculating an upper limit of M based on a number of samples of the block and a highest available throughput; calculating a number of variable length codes based on the number of samples of the block and the target decoder throughput; and calculating N based on the number of variable length codes and the upper limit of M. each of the M sample groups of the block has a same number of fixed length suffixes.

3. The method of claim 1, wherein, the block is predictively encoded in a transform skip block prediction mode.

4. The method of claim 3, wherein, at least two of the M sample groups of the block have different numbers of fixed length suffixes.

5. The method of claim 1, wherein, the block is predictively encoded in a transform mode or in a transform skip block prediction mode.

6. The method of claim 5, wherein, the block comprises a plurality of components of the image content.

7. The method of claim 1, wherein, the encoded bitstream further comprises a component skip flag indicating that all of the plurality of samples of a corresponding component of the block of the encoded bitstream are zero.

8. The method of claim 1, wherein, the encoded bitstream further comprises a group skip flag indicating that all of the plurality of samples of a group of the M sample groups are zero.

9. The method of claim 1, wherein, 10. A method for encoding image content, comprising: partitioning, by encoder circuitry, received image content into one or more blocks, each of the one or more blocks comprising a plurality of samples from one or more components of the image content; predictively encoding, by the encoder circuitry, each block to generate a block of residuals; partitioning, by the encoder circuitry, each of the blocks of residuals into N single samples and M sample groups by taking N single samples from the block of residuals and grouping the remaining samples of the block of residuals into M sample groups, wherein N is greater than or equal to 1 and M is greater than or equal to 1; ​ encoding, by the encoder circuit, each of the N single samples using a symbol variable length code to generate one or more symbol variable length code encoded samples, each of the N single symbol variable length code encoded samples being encoded as a separate variable length binary word; encoding, by the encoder circuit, each of the M sample groups using a common prefix entropy code to generate one or more common prefix entropy code encoded sample groups, each of the M sample groups including a variable length prefix and one or more fixed length suffixes; and combining, by the encoder circuit, the symbol variable length code encoded samples and the common prefix entropy code encoded sample groups to output an encoded bitstream.

11. The method of claim 10, wherein, N and M are set according to a target decoder throughput by: calculating an upper limit of M based on a number of samples per block of the one or more blocks and a highest available throughput; calculating a number of variable length codes based on the number of samples per block and the target decoder throughput; and calculating N based on the number of variable length codes and the upper limit of M. partitioning each of the blocks of residuals includes:

12. The method of claim 10, wherein, partitioning at least one of the blocks of residuals using uniform partitioning, wherein each of the M sample groups of the at least one block of residuals has a same number of fixed length suffixes. the at least one block of residuals is predictively encoded in a transform skip block prediction mode.

13. The method of claim 12, wherein, partitioning each of the blocks of residuals includes:

14. The method of claim 10, wherein, partitioning at least one of the blocks of residuals using non-uniform partitioning, wherein at least two of the M sample groups of the at least one block of residuals have different numbers of fixed length suffixes. the at least one block of residuals is predictively encoded in a transform mode or in a transform skip block prediction mode.

15. The method of claim 14, wherein, each block includes a plurality of components of the image content.

16. The method of claim 14, wherein, the encoded bitstream further includes a component skip flag indicating that all of the plurality of samples of at least one corresponding component of the block are zero.

17. The method of claim 10, wherein, the encoded bitstream further includes a group skip flag indicating that all of the plurality of samples of a group of the M sample groups are zero.

18. The method of claim 10, wherein, 19. A system for transmitting image content, comprising: an encoder circuit configured to: divide received image content including a plurality of components into one or more blocks, each of the one or more blocks including a plurality of samples from one of the plurality of components; predictively encode each block to generate predictively encoded blocks; partition each of the predictively encoded blocks into N single samples and M sample groups by taking N single samples from the predictively encoded block and grouping remaining samples of the predictively encoded block into M sample groups, wherein N is greater than or equal to one and M is greater than or equal to one; ​ encoding each of the N single samples using a symbol variable length code to generate one or more symbol variable length code encoded samples, each of the N single symbol variable length code encoded samples being encoded as a separate variable length binary word; encoding each of the M sample groups using a common prefix entropy code to generate one or more common prefix entropy code encoded sample groups, each of the M sample groups including a variable length prefix and one or more fixed length suffixes; and combining the symbol variable length code encoded samples and the common prefix entropy code encoded sample groups to output an encoded bitstream; and decoder circuitry configured to: receive the encoded bitstream from the encoder circuitry; divide a block of the encoded bitstream into the N single samples and the M sample groups; decode each of the N single samples using the symbol variable length code to generate one or more decoded single samples; decode each of the M sample groups using the common prefix entropy code to generate one or more decoded sample groups; reconstruct the predictively encoded block from the decoded single samples and the decoded sample groups; apply predictive coding to decode the predictively encoded block; and decode the image content from the decoded predictively encoded block.

20. The system of claim 19, wherein, N and M are set according to a target decoder throughput by: calculating an upper limit for M based on a number of samples of the block and a highest available throughput; calculating a number of variable length codes based on the number of samples of the block and the target decoder throughput; and calculating N based on the number of variable length codes and the upper limit for M.

21. The system of claim 20, wherein, the encoder circuitry is further configured to: detect one or more factors of a communication environment in which at least one of the encoder circuitry and the decoder circuitry is operating; and dynamically update values of N and M based on the one or more factors.

22. The system of claim 21, wherein, the one or more factors include one or more of the following: power, processor capability, a number of decoders in the decoder circuitry operating in parallel, internal bandwidth, temperature conditions of the decoder circuitry, and noise in a physical medium between the encoder circuitry and the decoder circuitry.

Images