Palette mode video coding using hierarchical palette table generation.

Abstract

The encoder [102] encodes an image portion [122] by recursively partitioning the portion into a partition hierarchy of levels. The top level has a single block representing the entire portion, and each lower level has four smaller blocks representing the corresponding larger block at the higher level. A palette table [124] is generated for each lowest-level block based on the pixels of the associated block. For each successively higher level, the encoder generates a palette table for each block at the current level by selecting palette colors based on the palette colors from the four palette tables of the four associated blocks at the next lowest level. A color index map [126] is then generated based on a final palette table selected from the palette tables generated for the partition hierarchy. A representation of the portion is then encoded using the final palette table and color index map to generate a corresponding segment of the encoded bitstream [108].

Description

[Background technology]

[0001] Certain video applications involve the transmission of "screen content" (i.e., computer-generated image content), such as remote desktop sharing, virtual desktop infrastructure for cloud computing, and cloud gaming applications. Such screen content is transmitted by itself or as hybrid content that represents a combination of screen content and camera-captured video content. However, unlike camera-captured video content, screen content, especially "desktop" screen content, is typically rich in text and other computer-generated graphics that present sharp edges with relatively little noise. Furthermore, screen content generally exhibits a smaller number of distinct colors within local regions of an image.

[0002] Some video coding standards have been extended to take advantage of the relatively small number of colors and relatively high contrast typically exhibited by screen content compared to camera-captured video to provide more efficient encoding of screen content. For example, the ITU-T High-Efficiency Video Coding (HEVC) standard provides a Screen Content Coding (SCC) extension that implements a palette mode with palette color selection followed by generation of a color index map based on the palette color selection. In this process, for each portion of an image containing screen content being encoded (i.e., for each coding unit), the encoder identifies a number of most significant representative colors (e.g., the top 8 colors) found in the portion and generates a palette table that indexes (i.e., maps an index to) each of the identified representative colors. The encoder then replaces each pixel in the portion of the image with the corresponding index value of the palette color from the palette table that most closely matches the color of the pixel to generate a corresponding color index map for the image portion being encoded. The palette table and color index map are then coded along with information about pixels from parts of the image that differ too much from any of the identified palette colors (these pixels are called "escape" pixels) to generate a corresponding segment of a coded bitstream, which is then transmitted to a destination device. The destination device then decodes the bitstream to obtain the palette table and color index map for the decoded block, and lossily or losslessly reconstructs the original block from these values ​​and any associated decoded residuals. The Alliance for Open Media Video 1 (AV1) coding standard provides a similar palette mode implementation.

[0003] A conventional process for generating a palette table for a given image portion to be encoded involves generating a single histogram of all of the pixel colors of the entire image portion and then selecting a representative color for the palette table from this histogram. While providing a compact bitstream, this palette table generation process is computationally intensive and therefore requires significant hardware resources in terms of silicon area, throughput and power consumption. To illustrate, the HEVC-SCC standard discussed above provides for a maximum coding unit size of 128x128 pixels each, which means that for each such coding unit of an image to be encoded, an encoder implementing a conventional palette table generation process would need to process 16,384 pixels to generate a corresponding histogram for each 128x128 coding unit, of which there may be hundreds or thousands in any given image depending on the resolution of the image. This process becomes even more complicated when the encoding system is configured to support multiple encoding partitions simultaneously, such as being able to support encoding of screen content video based on any or all of the typical 8x8, 16x16, 32x32, 64x64 and 128x128 coding unit partitions of the HEVC standard and other similar encoding standards.

[0004] The present disclosure may be better understood, and its numerous features and advantages made apparent to those skilled in the art by reference to the following drawings, in which: The use of the same reference numbers in different drawings indicates similar or identical items. [Brief description of the drawings]

[0005] [Figure 1] FIG. 2 is a block diagram of a source device having a palette mode encoder that utilizes hierarchical palette table generation for palette mode encoding of video including screen content, according to some embodiments. [Diagram 2]2 is a flow diagram illustrating a palette mode encoding process using hierarchical palette table generation implemented by the palette mode encoder of FIG. 1 in accordance with some embodiments. [Diagram 3] A diagram showing an example recursive hierarchical partitioning of a coding unit into sets of blocks of pixels at different levels of the hierarchy in accordance with some embodiments. [Figure 4-1] 4 is a block diagram illustrating an example embodiment of the method of FIG. 2 using the example recursive partitioning of the coding unit of FIG. 3, in accordance with some embodiments. [Figure 4-2] 4 is a block diagram illustrating an example embodiment of the method of FIG. 2 using the example recursive partitioning of the coding unit of FIG. 3, in accordance with some embodiments. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0006] Typical approaches to palette table generation for a coding unit (CU) in palette mode encoding based on histogram analysis or other clustering of all pixels represented by the CU are often computationally prohibitive, especially for encoders that support larger CU partitions or multiple simultaneous CU partitions. To reduce computational complexity while retaining sufficient color reproduction accuracy, the systems and methods described herein utilize a hierarchical palette table generation process to efficiently generate a palette table for each of one or more selected partition sizes for a selected CU encoded in palette mode. In this approach, a selected CU is recursively partitioned at different levels of a partitioning hierarchy. At the top level, a selected CU of K×K pixels is represented by a single block of K×K pixels. Each successive level below the top level has four smaller pixel blocks that represent the corresponding larger pixel block at the next higher level. At the lowest level of the partitioning hierarchy, the encoder generates a palette table for each block at the lowest level. Each of these palette tables contains a set of palette colors that represent the pixel colors present in the corresponding block, and in at least one embodiment is selected via histogram analysis of the pixel colors present in the block, via k-means, or via any of a variety of other clustering methods. Then, for each successively higher level in the partitioning hierarchy, the encoder generates a palette table for each block at the current level by selecting palette colors for the palette table based on palette colors from the four palette tables associated with the four blocks at the next lower level that correspond to the block at the current level. This may be performed via histogram collection or other clustering of the palette colors of the four palette tables. Then, at the top level, a single palette table is generated at this level from the four palette tables at the next highest level, which the encoder uses to generate the color index map for the selected CU.The encoder then encodes a representation of the CU using the final palette table and color index map to produce a corresponding segment of the encoded bitstream.

[0007] To illustrate using an exemplary 32x32 coding unit partition (i.e., K=32) and a minimum block size of 8x8 pixels, the CU is then designated as the top block at the top level, and for the next highest level, the top block becomes four second level blocks of 16x16 pixels each, and then each of these blocks is partitioned into four third level blocks of 8x8 pixels each. Starting at the lowest or bottom level (the third level in this example), the encoder uses histogram analysis, k-means analysis or other statistical analysis to generate an initial palette table for each of the blocks at this level, resulting in four palette tables for the bottom level. For ease of reference, an exemplary embodiment utilizing histogram collection (also identified as histogram analysis) is described herein, but it will be understood that reference to histogram collection / analysis shall include reference to any of the various clustering methods or other statistical methods such as k-means analysis, unless otherwise specified. At the second-lowest partitioning hierarchical level, the encoder identifies a set of candidate colors from the palette colors represented in the four palette tables of the four constituent blocks at the bottom level that together form a block at the second-lowest level. The encoder determines an intermediate palette table for the block at the second-lowest level through a statistical analysis of this set of candidate colors. This process is repeated for each of the three other blocks at the second-lowest level, resulting in four intermediate palette tables generated for the second-lowest level. Then, at the top level, the encoder identifies a set of candidate colors from the palette colors represented in the four palette tables of the four constituent blocks at the second-lowest level that together form a single block at the top level. The encoder determines a palette table for this block (representing the entire CU) through a statistical analysis of this set of candidate colors. The encoder then estimates, for each partitioning hierarchical level, the rate-distortion cost for encoding the CU using the corresponding palette table at that level.The palette table requiring the lowest rate-distortion cost is then designated as the “final” palette table for the CU, and the encoder then uses the final palette table to generate a color index map for the selected K×K coding unit, and encodes the color index map, the final palette table, and any quantized residuals for any escape pixels present to generate a corresponding portion of the encoded bitstream, which is then stored for later access or transmitted to a destination device, whereupon a decoder at the destination device decodes the bitstream to recover the encoded palette table, color index map, and quantized residuals for the coding unit, and uses the recovered palette table, color index map, and residuals to recover the original CU or a lossy representation thereof.

[0008] This hierarchical and recursive partitioning process for determining palette tables for blocks of pixels at each level from palette tables generated for constituent blocks at the next lower level provides efficient palette table generation through a reduction in the number of colors considered through the integration of palette color selection processes at lower levels into palette color selection processes at higher levels. For illustration, a conventional process for generating a palette table for a 64×64 CU would require an encoder to sort and count each of the 4,096 pixels in the CU to arrive at a palette table of, for example, eight representative colors. However, under the hierarchical palette color selection process described herein, only a small subset of pixels is sorted and counted for each relatively small block at the lowest level, and then only a small subset of the representative colors of each block at one level is considered for the corresponding block at the next highest level, resulting in fewer overall operations to arrive at a final palette table for a CU of a given size, and thus a more efficient process for generating a palette table for a CU, while also allowing the generation of separate palette tables at each level of the partitioning hierarchy that can be used for the individual partition sizes of the CU.

[0009] FIG. 1 illustrates a source device 100 having an encoding system 102 employing a palette mode encoding process utilizing hierarchically and recursively generated palette tables, according to some embodiments. The source device 100 may include any of a variety of devices configured to encode a video stream including screen content, including a server, a personal computer, a tablet computer, a mobile phone, or other computing-enabled mobile device supporting any of a variety of applications that generate screen content for transmission to or storage at a display device. Such services may include, for example, remote desktop services, wireless display services, screen sharing / collaboration services, cloud gaming services, and the like. The source device 100 includes a video source 104 and an encoding system 102, as well as various other components omitted from FIG. 1 for ease of illustration, including one or more wireless or wired network interfaces, input / output (I / O) devices and their controllers, one or more processors, memory and other storage components, and the like. The video source 104 operates to generate or otherwise provide a video stream comprised of a series of images 106 (also commonly referred to as "pictures" or "frames") for encoding by the encoding system 102 into a corresponding bitstream 108, which is provided for transmission via a network interface (not shown) to a destination device or otherwise provided for output, such as for storage on a local or networked storage device (not shown) so as to be available for subsequent access. The video source 104 may include, for example, a graphics processing unit (GPU) that operates to render some or all of the content contained in each image 106, a decoder that decodes previously encoded video data representing screen content or other image content, and the like.

[0010] In at least one embodiment, some or all of the images 106 included in the stream from the video source 104 are composed partially or completely of computer-generated screen content, which typically exhibits sharper contrast and fewer colors in local areas compared to camera-captured video images. The encoding system 102 is therefore configured to provide palette-based encoding (or "palette mode" encoding) of those images 106 identified as including screen content, either individually or as part of the corresponding stream. To this end, the encoding system 102 includes a hierarchical palette table generator 110 (hereinafter "palette table generator 110" for brevity), a color index map generator 112, a residual generator 114, a discrete cosign transform (DCT) quantizer 116, and an entropy encoder 118. Some or all of the components 110, 112, 114, 116, 118 may be implemented in source device 100 as one or more processors executing software instructions stored in system memory or another non-transitory computer-readable storage medium and configured to operate the one or more processors to perform the processes described herein. In other embodiments, some or all of the components 110, 112, 114, 116, 118 are implemented with hard-coded logic (e.g., application specific integrated circuits or ASICs), programmable logic (e.g., programmable logic arrays), or a combination thereof. Still further, in some embodiments, some or all of the components 110, 112, 114, 116, 118 are implemented with a combination of one or more processors executing software and hard-coded or programmable logic.

[0011] 2 illustrates a method 200 of operation of the encoding system 102 for palette mode encoding of images including screen content using a hierarchical and recursive palette table generation process according to some embodiments. For ease of explanation, reference is made to the exemplary hierarchical division of a CU illustrated by FIG. 3 and the exemplary embodiment of the method 200 illustrated by FIG. 4-1 and FIG. 4-2 in describing the method 200 below.

[0012] Method 200 begins at block 202, where encoding system 102 receives image 106 from video source 104 as part of a stream of images 106. For example, in some embodiments, each image 106 is buffered in a frame buffer (not shown) of source device 100 as it is generated or otherwise obtained by video source 104. In the following, it is assumed that the received image 106 includes sufficient screen content to benefit from palette mode encoding. For illustrative purposes, in some embodiments, palette mode is utilized on a per CU basis, and encoding system 102 calculates the rate-distortion cost of encoding the current CU using palette mode compared to the rate-distortion costs of other prediction modes, and utilizes palette mode encoding for the current CU if it represents the lowest rate-distortion cost of the encoding options. Otherwise, if palette mode is not indicated for image 106, image 106 is encoded using an appropriate conventional encoding technique, such as those promulgated by HEVC.

[0013] Further, in block 202, encoding system 102 logically partitions the J×K array of pixels represented by image 106 into an array 120 (FIG. 1) of two-dimensional coding units (CUs) 122, such as CUs 122-1 and 122-2 in FIG. 1. In certain embodiments, such as those found in the ITU-T Rec. H.264 standard, the CUs are all the same size (CUs are typically referred to instead as "macroblocks"). In other embodiments, such as those found in the ITU-T Rec. H.265 HEVC standard, an image may be partitioned into multiple coding tree units (CTUs), with each CTU further partitioned into multiple CUs of different sizes. For ease of explanation, CUs 122 of array 120 for image 106 are shown as being of the same size, although in other embodiments, array 120 may be composed of CUs 122 of different sizes, such as the CTU configurations discussed above.

[0014] In block 204, the encoding system 102 selects a CU 122 (e.g., CU 122-1 in FIG. 1) of the image 106 to be encoded. According to various standards, such as the HEVC standard, the image 106 may first be partitioned into slices, then each slice is partitioned into CTUs, then each CTU is partitioned into CUs, and then the selection of slices-CTUs-CUs may proceed in a defined order. Once the CU 122 is selected, in block 206, the palette table generator 110 utilizes a recursive and hierarchical palette table generation process, in which the CU 122 is recursively partitioned in a top-down approach into successive pixel blocks of progressively smaller size at each partition level of the partition hierarchy until a minimum block size is reached at the lowest partition level. Then, using a bottom-up approach, the palette table generator 110 performs histogram collection or other clustering on the pixels of each block at the lowest partition level to determine a set of candidate colors to include as palette colors in the corresponding palette table for each lowest level block. At the second-lowest partitioning level, the palette colors from the palette table of each of the lowest level blocks represented by the block at the second-lowest level serve as candidate colors for the block at the second-lowest level, and palette table generator 110 therefore performs histogramming or other clustering on these candidate colors to identify colors for inclusion in the palette table for the block at the second-lowest level. This process is performed for each block at the second-lowest level. The process of using the palette colors of each palette table generated for each constituent block at the next lower level as candidate colors for selection for the palette table of the corresponding block at the next higher level is repeated up to the top level of the partitioning hierarchy (representing the entire CU 122), at which point palette table generator 110 generates the palette table for the top level through histogramming or other clustering of the palette colors of the palette tables generated at the second-highest level of the partitioning hierarchy.That is, the palette table generator 110 generates intermediate palette tables for larger partitions of the CU 122 by recursively and hierarchically integrating the palette tables of the smaller partitions. This avoids the computationally intensive conventional process of exhaustively sorting and processing the entire pixels of the CU 122 as one block to generate a palette table. With the candidate palette tables generated for each partition level (e.g., 8×8 pixels, 16×16 pixels, 32×32 pixels, 64×64 pixels, 128×128 pixels, etc.), the palette table generator 110 or other components of the encoding system 102 then calculate the rate-distortion cost for encoding the CU 122 using the corresponding candidate palette table, which calculation is performed for each candidate palette table. The encoding system 102 then selects the candidate palette table that represents the lowest rate-distortion cost for encoding the CU 122 as the “final” palette table 124. An exemplary embodiment of the hierarchical palette table generation and selection process employed by the encoding system 102 in block 206 is described in more detail below with reference to the expanded process flow 207 of FIG. 2.

[0015] With the final palette table 124 for CU 122 determined using this hierarchical and recursive palette table generation and selection technique, in block 208, the color index map generator 112 performs a prediction based on a comparison of each of the original pixels of CU 122 with the palette colors represented in the palette table 124 to generate a color index map 126 that represents the pixels of CU 122 and is indexed to the final palette table 124. Any of a variety of standard or proprietary techniques can be used to generate the color index map 126 from the final palette table 124. By way of example, as understood in the art, a palette table such as the final palette table 124 includes a number of entries, each entry having an associated index value and a corresponding palette color selected from candidate colors identified in a histogram collection or other clustering of the color values ​​of the corresponding pixel block. Each palette color is composed of, for example, three color components (RGB or YCbCr). Thus, to generate color index map 126, for each pixel of CU 122, color index map generator searches final palette table 124 for a corresponding palette color with the minimum distortion and replaces the pixel color with the index number associated with the selected palette number in the corresponding position entry of color index map 126. For illustration purposes, assume that the pixel at (1,1) (i.e., row 1, column 1) of CU 122-1 has a color that is closest to the palette color (R150, B50, G75) found at index 1 of final palette table 124, for example. In this case, color index map generator 112 would place an index value of “1” in the (1,1) entry of color index map 126 to represent that the color of pixel (1,1) most closely resembles the palette color found at index 1 of final palette table 124.

[0016] Furthermore, in some lossless coding embodiments, the prediction performed by the color index map generator 112 identified one or more pixels that were not sufficiently similar to the palette colors and therefore required the addition of separate indices in the final palette table 124 and corresponding identifying entries in the color index map 126 to represent such pixels; such pixels are typically referred to as "escape pixels" and therefore their corresponding indices are referred to as "escape" indices. In lossy coding embodiments, the escape pixels are often coded as one of the palette colors instead or explicitly signaled after quantization, depending on which approach gives a better rate-distortion tradeoff.

[0017] Concurrent with the color index map generation process of block 208, in block 210, the residual generator 114 compares the original pixel colors with their selected palette colors to generate residuals representing the differences. These residuals are then quantized by the DCT quantizer 116 to generate quantized DCT coefficients. In block 212, the entropy encoder 118 encodes the palette table 124, the color index map 126, and the quantized DCT coefficients for the selected CU 122 into a corresponding segment of the bitstream 108.

[0018] The process of selecting the next CU 122 from the picture 106 to be coded, represented by blocks 204-212, is repeated for each CU 122 of the palette mode coded picture 106. After all such CUs have been coded, another iteration of the method 200 is repeated, beginning with block 202, receiving the next picture 106 in the video stream to be coded for transmission or storage.

[0019] For subsequent decoding of the encoded bitstream 108, any of a variety of standard or proprietary palette mode decoding processes may be employed that are consistent with how the bitstream 108 was encoded, such as the process provided by HEVC-SCC. Such a process typically involves, for each segment of the bitstream 108 that represents an encoded CU, a palette mode decoder decoding the segment to recover the palette table 124, the color index map 126, and the quantized DCT coefficients that represent the CU. The quantized DCT coefficients are dequantized to obtain a lossy or lossless representation of the residual, and the recovered palette table 124 and color index map are used together with the residual to recover the array of pixels represented by the original CU 122, or a lossy representation thereof.

[0020] As described above, the process of block 206 provides for the generation of the final palette table 124 using a hierarchical and recursive process that effectively merges intermediate palette tables generated for lower levels of the partitioning hierarchy to generate intermediate palette tables at higher levels of the partitioning hierarchy, and from these candidate palette tables, provides for the selection of the final palette table 124 that represents the lowest rate-distortion cost for use in encoding the CU 122. The expanded process flow 207 illustrates an exemplary embodiment of this process with reference to the exemplary hierarchical partitioning depicted in Figure 3 and exemplary implementations of this process as depicted in Figures 4-1 and 4-2.

[0021] The process of block 206 begins at block 220, where the CU 122 selected at block 204 is recursively partitioned into a partitioning hierarchy consisting of a lowest level (or bottom level), a highest level (or top level), and one or more intermediate levels between the lowest and highest levels if the partitioning hierarchy is more than two levels deep. The top level represents the entire pixel array of the CU 122 as a single top-level block. For each successively lower level of the partitioning hierarchy, the hierarchical palette table generator 110 partitions each block at the next higher level into an associated set of four blocks until a minimum block size of P×P pixels is reached at the lowest level, where P represents an integer greater than 1 and, in many embodiments, is the minimum CU size supported by the adopted coding standard. For example, the minimum block size would be set to 8, such that HEVC provides a minimum CU of 8×8 pixels in a CTU. Thus, the number of levels N in the partitioning hierarchy for a CU 122 of K×K pixels is set based on the relationship of the following equation: K=P*2 (N-1) Thus, if the minimum block size is 8x8 pixels (P=8) and CU 122 is 16x16 pixels (K=16), there will be only two partitioning levels (N=2), i.e., the top level and the bottom level. If the minimum block size is 8x8 pixels (P=8) and CU 122 is 128x128 pixels, there will be five partitioning levels (N=5), i.e., the top level, the bottom level, and three intermediate levels between them.

[0022] For illustrative purposes, an example of this recursive partitioning is shown in Figure 3. In the illustrated exemplary partitioning hierarchy 300, a 64x64 pixel CU is recursively partitioned until a minimum block size of 8x8 pixels is reached. Thus, the exemplary partitioning hierarchy 300 has four partitioning layers (64=8). *2^(4-1)→N=4). The entire 64×64 pixel array of a CU is considered to be a single block (BLK64) at the top level (Level 1) of the partitioning hierarchy. At a first intermediate level (Level 2), the 64×64 pixels of the CU are partitioned into a corresponding set of four 32×32 blocks, BLK[32,0] through BLK[32,3]. At a second intermediate level (Level 3), each of the 32×32 blocks at the first intermediate level is partitioned into a corresponding set of four 16×16 blocks, resulting in blocks BLK[16,0] through BLK[16,15] at the second intermediate level. As shown, block BLK[32,0] at the first intermediate level (Level 2) is partitioned into a corresponding set of blocks BLK[16,0], BLK[16,1], BLK[16,2], and BLK[16,3]. Similarly, block BLK[32,1] at the first intermediate level (Level 2) is partitioned into blocks BLK[16,4], BLK[16,5], BLK[16,6], BLK[16,7], etc. At the lowest level (Level 4) of the partitioning hierarchy 300, each 16×16 block at the second intermediate level is partitioned into a corresponding set of four 8×8 blocks, resulting in blocks BLK[8,0] through BLK[8,63]. As shown, BLK[16,0] at the second intermediate level is partitioned into the set of blocks BLK[8,0] through BLK[8,3]. BLK[16,1] is similarly partitioned into the set of blocks BLK[8,4] through BLK[8,7], and so on.

[0023] Returning to FIG. 2, once CU 122 has recursively partitioned into pixel blocks of decreasing size from the top to the bottom level of the partitioning hierarchy, such that each block at a non-bottom level is represented by four smaller blocks at the next lower level, in block 222 palette table generator 110 sets the current level being processed to the bottom level of the partitioning hierarchy and generates a separate palette table for each block at the lowest level of the partitioning hierarchy. In at least one embodiment, the palette generation process for each block at the lowest level includes performing histogram collection on the pixels in each block based on binning of pixel colors to generate a corresponding histogram of pixel colors present in the pixels of the block, and then selecting X subsets of the pixel colors present based on the corresponding bin size (i.e., the number of pixel colors that fall within each bin) as palette colors for the corresponding palette table. The value X is selectable or otherwise programmable, and is generally selected based on various parameters including minimum block size, coding efficiency, coding bit rate, etc. For example, palette tables are often limited to 16 palette colors or less, with 8 or 16 (i.e., X=8 or X=16) being frequent configurations. Thus, in the example partition hierarchy 300 of Figure 3, palette table generator 110 separately performs the sorting and other operations utilized for histogram analysis on each of the 64 8x8 pixel blocks at the lowest level (level 4) to generate 64 separate histograms, and generates 64 separate palette tables from these 64 separate histograms, each palette table containing, for example, up to 16 indexed palette colors.

[0024] Then, in block 224, the palette table generator 110 moves up one level (i.e., the current level being processed is set to the next higher level). In block 226, the palette table generator 110 selects a block of the current level being processed based on a specified selection order (e.g., horizontal scan or vertical scan). In block 228, the palette table generator 110 generates a palette table for the selected block in the current level from the palette colors of the palette table generated for the set of blocks in the previous (or lower) level that correspond to the selected block in the current level. In at least one embodiment, the palette table for the selected block in the current level is generated from the palette table of the corresponding set of blocks in the lower level by combining the palette colors of the blocks in the lower level into a set of candidate palette colors, then performing a histogram collection of this set of candidate palette colors to identify X histogram bins having the most frequently occurring palette colors, and generating a palette table for the selected block in the current level to include the X palette colors based on the identified X histogram bins. In the first iteration of block 228 (for the lowest intermediate level), this involves using the palette colors implemented in the palette table for the four lowest level blocks generated in block 222. In subsequent iterations (for higher intermediate or top levels), this involves using the palette colors implemented in the palette table for the four next lower level blocks generated in the previous iteration of block 228. Thus, it will be appreciated that rather than performing histogram collection for all of the pixels of the CU represented by the selected block at the current level, histogram collection is instead performed only on the palette colors of the palette tables of the lower level blocks, which in turn aggregate the palette colors of the palette tables of the even lower level blocks at higher levels of the partition hierarchy.Thus, significantly fewer operations are required to generate a histogram, and therefore a palette table, for a J×J block of pixels using this recursive approach as compared to conventional full-pixel histogram analysis approaches.

[0025] Once the palette tables for the selected blocks in the current level have been generated, in block 230 palette table generator 110 determines whether individual palette tables have been generated for each block in the current level. If not, another iteration of the block selection and block-by-block palette table generation process of blocks 226, 228, 230 is performed for the next block of the CU in the current partitioning level. Otherwise, if all blocks in the current partitioning level have had palette tables generated for them, in block 232 palette table generator 110 determines whether higher partitioning levels remain, i.e., whether the current level is the top level of the partitioning hierarchy. If not, palette table generator 110 returns to block 224 and sets the current level being processed to the next higher level in the partitioning hierarchy, and then sets another iteration of the block selection and block-by-block palette table generation process of blocks 226-332 for the current level being processed.

[0026] Otherwise, if the palette table generator 110 has climbed the partitioning hierarchy for the CU 122 being coded and is at the top level of the hierarchy, then in block 234, the coding system 102 designates the palette tables generated at each level of the partitioning hierarchy as candidate palette tables and then calculates the rate-distortion cost for coding the CU for each of the candidate palette tables. The candidate palette table requiring the lowest rate-distortion cost for coding the CU 122 is designated as the final palette table 124 for the CU 122, which is then used by the color index map generator 112 to generate the corresponding color index map 126 for the CU 122, as described above with reference to block 208.

[0027] 4-1 and 4-2, an exemplary embodiment of the method 200 of FIG. 2 is shown using an exemplary partition hierarchy 300 of a 64×64 CU as shown in FIG. 3. In this example, each palette table contains up to 16 palette colors (i.e., X=16). For a 64×64 pixel CU, the lowest partition level (level 4) generates a set of four 8×8 blocks for each 16×16 block of the next highest level (level 3). Each 8×8 block is statistically analyzed to generate a corresponding histogram of candidate pixel colors, and the histogram is used to identify up to 16 of the most frequently occurring pixel colors in the 8×8 block as palette colors indexed in the palette table of that block. For illustrative purposes, FIG. 4-1 shows histograms 401-0, 401-1, 401-2, 401-3 being collected for blocks BLK[8,0], BLK[8,1], BLK[8,2], BLK[8,3], respectively. Palette table generator 110 generates palette tables 402-0, 402-1, 402-2, 401-3 with up to 16 palette colors each selected based on histograms 401-0, 401-1, 401-2, 401-3, respectively. This same process is performed for each of the other 15 sets of four 8×8 blocks, resulting in a total of 64 palette tables being generated from the 64 8×8 blocks (BLK[8,0] through BLK[8,63]) at the lowest level (level 5) of partitioning hierarchy 300.

[0028] Then, at the lowest intermediate level (level 3) of the partition hierarchy 300, the palette table generator 110 generates a partition table for each of the 16 16×16 blocks (BLK[16,0] through BLK[16,15]) through a union of the four partition tables generated for the four 8×8 blocks at the bottom level corresponding to the blocks at the second level. For illustration, in FIG. 4-1, the indexed palette colors of each of the palette tables 402-0 through 402-3 are combined into a set 403 of up to 64 candidate colors (i.e., up to 16 palette colors per bottom level palette table×4 bottom level palette tables). The palette table generator 110 analyzes this set 403 to generate a histogram 404 (or other clustering) representing the frequency of occurrence of the candidate colors in the set 403, and then uses up to 16 of the most represented pixel colors in the histogram 404 to generate an intermediate palette table 405-0 for the 16×16 block BLK[16,0] at the second level (level 3) that represents the four 8×8 blocks BLK[8,0] through BLK[8,3] at the lowest level. This process is similarly performed for the other 15 16×16 blocks (BLK[16,1] through BLK[16,15]) at the lowest intermediate level. The result is four intermediate level palette blocks 405-0, 405-1, 405-2, 405-3.

[0029] Moving up to the highest intermediate level (level 2) in the partition hierarchy 300, the palette table generator 110 generates a partition table for each of the four 32x32 blocks (BLK[32,0] through BLK[32,3]) through a union of the four partition tables generated for the four 16x16 blocks at the lowest intermediate level (level 3) corresponding to the blocks at the third level. For illustration purposes, in FIG. 4-1, the indexed palette colors of each of the palette tables 405-0 through 405-3 are combined into a set 406 of up to 64 candidate colors. The palette table generator 110 analyzes this set 406 to generate a histogram 407 (or other clustering) representing the frequency of occurrence of the candidate colors in the set 406, and then uses up to 16 of the most represented pixel colors in the histogram 407 to generate an intermediate palette table 408-0 for the 32x32 block BLK[16,0] that represents the four 8x8 blocks BLK[8,0] through BLK[8,3] at the lowest level. This process is similarly performed for the other three 32x32 blocks (BLK[32,1] through BLK[32,3]) at the highest intermediate level. Referring to FIG. 4-2, the result is four intermediate level palette blocks 408-0, 408-1, 408-2, and 408-3.

[0030] At this point, palette table generator 110 has progressed to the highest or top level (level 1) of partition hierarchy 300 for CU 122. Thus, palette table generator 110 generates a top palette table 412 for the top partition hierarchy through a synthesis of the four partition tables 408-0 to 408-3 generated for the four 32×32 blocks at the highest intermediate level (level 2) by combining the palette colors of palette tables 408-0 to 408-3 into a set 409 of up to 64 candidate colors. Palette table generator 110 analyzes this set 409 to generate a histogram 410 (or other clustering) representing the frequency of occurrence of the candidate colors in set 409, and then uses up to 16 of the most represented pixel colors in histogram 410 to generate top palette table 412.

[0031] As the above-described example shows, the process of generating the top-level palette table 412 involves 64 instances of histogram collection for a set of 64 pixels, followed by three iterations of histogram collection for up to 64 candidate palette colors from lower level palette tables. It will be appreciated that this is computationally less complex than the conventional approach of a single histogram collection for the entire set of 4,096 pixels represented by the 64×64 pixel CU in the above example. Furthermore, this hierarchical approach facilitates the selective use of partitions of different sizes to more efficiently code a CU based on a rate-distortion analysis of the coding of the CU using each of the different palette tables at different partition levels. Thus, each of the palette tables 402-0, 402-1, 402-2, 402-3, 405-0, 408-0, 412 represents a candidate palette table that may be utilized as the final palette table 124 used to code the CU 122. Thus, for each of these candidate palette tables, the encoding system 102 can calculate a respective rate-distortion cost for encoding the CU 122 using the candidate palette table and select an intra-coding pipeline partition size based on which candidate palette table provides the lowest rate-distortion cost.

[0032] To illustrate, in the case of an 8×8 intra-coding pipeline for encoding a CU, the hierarchical partitioning unfolds to a single level, and at this level a palette table generated for an 8×8 block (e.g., palette table 402-0 generated for BLK[8,0]) may be used to generate a color index map 411 representing the CU. Similarly, in the case of a 16×16 intra-coding pipeline for encoding a CU, a palette table generated for the 16×16 partitioning level (e.g., palette table 405-0 for BLK[16,0]) may be used to generate a color index map 412 representing the CU. Similarly, in the case of a 32×32 intra-coding pipeline for encoding a CU, a palette table generated for the 32×32 partitioning level (e.g., palette table 408-0 for BLK[32,0]) may be used to generate a color index map 413 representing the CU, and so on. In the examples of Figures 4-1 and 4-2, the palette table 412 has the lowest rate-distortion cost, and therefore the palette table 412 is adopted as the final palette table 124 for encoding the CU in the 64x64 intra-coding pipeline to generate the color index map 126.

[0033] As disclosed herein, in some embodiments, a method for encoding an image including screen content at a source device includes, for a selected portion of the image, recursively partitioning the portion into a partitioning hierarchy consisting of multiple levels, where a top level of the partitioning hierarchy has a single block representing an entire set of pixels of the portion and each level below the top level has four smaller blocks of pixels representing a corresponding larger block of pixels at the next higher level; generating, at a bottom level of the partitioning hierarchy, a palette table for each block at the bottom level, where the palette table includes a set of palette colors representing pixel colors present in the corresponding block; for each successively higher level in the partitioning hierarchy, generating a palette table for each block at the current level by selecting palette colors for the palette table based on palette colors from four palette tables associated with four blocks at the next lower level that correspond to the block at the current level; selecting a final palette table from the palette tables generated for the multiple levels of the partitioning hierarchy; generating a color index map for the portion of the image based on the final palette table; and encoding a representation of the portion of the image using the final palette table and the color index map to generate a corresponding segment of an encoded bitstream for output by the source device.

[0034] In one aspect, generating a palette table for each block at a lowest level includes generating a first histogram representing the frequency of occurrence of binned pixel colors for pixels of the block, and selecting a specified maximum number of palette colors for indexing in the palette table based on the first histogram. In another aspect, generating a palette for a block at a higher level in the partitioning hierarchy includes grouping palette colors of each palette table of four lower level blocks associated with the block at the higher level into a set of candidate pixel colors, generating a second histogram representing the frequency of occurrence of the binned candidate pixel colors, and selecting a specified maximum number of palette colors for indexing in the palette table based on the second histogram. In yet another aspect, generating a palette table for a block at a higher level in the partitioning hierarchy includes grouping palette colors of each palette table of four lower level blocks associated with the block at the higher level into a set of candidate pixel colors, generating a clustering representing the frequency of occurrence of the binned candidate pixel colors, and selecting a specified maximum number of palette colors for indexing in the palette table based on the clustering.

[0035] In one embodiment, the blocks at the lowest level have a block size of P×P pixels, the portions of the image have a size of K×K pixels, and the number of levels N in the partitioning hierarchy is given by the formula K=P * In another aspect, selecting the final palette table includes determining, for each candidate palette table, a corresponding rate-distortion cost for encoding the portion of the image using the candidate palette table, and selecting the candidate palette table having the lowest corresponding rate-distortion cost as the final palette table. In yet another aspect, the portion of the image is a coding unit of a coding tree unit.

[0036] In some embodiments, a device for encoding an image including screen content includes a palette mode encoder that recursively partitions, for a selected portion of the image, the portion into a partitioning hierarchy consisting of a plurality of levels, where a top level of the partitioning hierarchy has a single block representing an entire set of pixels of the portion and each level below the top level has four smaller blocks of pixels representing a corresponding larger block of pixels at the next higher level, and at a lowest level of the partitioning hierarchy, generates a palette table for each block at the lowest level, where the palette table includes a set of palette colors representing pixel colors present in the corresponding block. for each successively higher level in the partitioning hierarchy, generating a palette table for each block at a current level by selecting palette colors for the palette table based on palette colors from four palette tables associated with four blocks at a next lower level that correspond to the block at the current level; selecting a final palette table from the palette tables generated for the multiple levels of the partitioning hierarchy; generating a color index map for the portion of the image based on the final palette table; and encoding a representation of the portion of the image using the final palette table and the color index map to generate a corresponding segment of the encoded bitstream.

[0037] In one aspect, the palette mode encoder is configured to generate a palette table for each block at a lowest level by generating a first histogram representing the frequency of occurrence of binned pixel colors for pixels of the block and selecting a specified maximum number of palette colors for indexing in the palette table based on the first histogram. In another aspect, the palette mode encoder is configured to generate a palette table for a block at a higher level in the partitioning hierarchy by grouping palette colors of each palette table of four lower level blocks associated with a block at the higher level into a set of candidate pixel colors, generating a second histogram representing the frequency of occurrence of the binned candidate pixel colors and selecting a specified maximum number of palette colors for indexing in the palette table based on the second histogram.

[0038] In one aspect, the palette mode encoder is configured to generate a palette table for a block at a higher level in the partitioning hierarchy by grouping palette colors of each palette table of four lower level blocks associated with the block at the higher level into a set of candidate pixel colors, generating a clustering representing the frequency of occurrence of the binned candidate pixel colors, and selecting a specified maximum number of palette colors for indexing in the palette table based on the clustering. In another aspect, the palette mode encoder selects a final palette table by determining, for each candidate palette table, a corresponding rate-distortion cost for encoding the portion of the image using the candidate palette table, and selecting the candidate palette table having the lowest corresponding rate-distortion cost as the final palette table. In yet another aspect, the palette mode encoder includes at least one processor and a memory for storing a set of instructions executable by the at least one processor.

[0039] In some embodiments, a method for encoding an image including screen content at a source device includes generating a palette table for each of a plurality of partitions of a partition hierarchy for a selected portion of the image by recursively using palette colors selected for a palette table for a set of smaller partitions of the portion of the image as candidate colors for a larger partition of the portion of the image, generating a color index map for the portion of the image based on a palette table selected from the generated palette tables for the portion of the image, and encoding a representation of the portion of the image using the selected palette table and color index map to generate a corresponding segment of an encoded bitstream for output by the source device. In another aspect, generating a palette table for a minimum partition of the portion of the image includes generating a first histogram representing a frequency of occurrence of binned pixel colors for pixels of the minimum partition, and selecting a specified maximum number of palette colors for indexing in the palette table based on the first histogram. In yet another aspect, generating a palette table for a partition other than the smallest partition includes grouping palette colors of each palette table of an associated next smaller partition into a set of candidate pixel colors, generating a second histogram representing a frequency of occurrence of the binned candidate pixel colors, and selecting a specified maximum number of palette colors for indexing in the palette table based on the second histogram.

[0040] In one aspect, the selected palette table is selected by determining, for each candidate palette table, a corresponding rate-distortion cost for encoding the portion of the image using the candidate palette table, and selecting the candidate palette table having the lowest corresponding rate-distortion cost as the selected palette table, hi another aspect, the portion of the image is a coding unit of a coding tree unit.

[0041] In some embodiments, a device for encoding an image including screen content includes a palette mode encoder configured to: generate a palette table for each of a plurality of partitions of a partition hierarchy for a selected portion of the image by recursively using palette colors selected for a palette table for a set of smaller partitions of the portion of the image as candidate colors for a larger partition of the portion of the image; generate a color index map for the portion of the image based on a palette table selected from the palette tables generated for the portion of the image; and encode a representation of the portion of the image using the selected palette table and color index map to generate a corresponding segment of an encoded bitstream for output by the device.

[0042] In one aspect, the palette mode encoder is configured to generate a palette table for a minimum partition of a portion of an image by generating a first histogram representing the frequency of occurrence of binned pixel colors for pixels of the minimum partition and selecting a specified maximum number of palette colors for indexing in the palette table based on the first histogram. In another aspect, the palette mode encoder is configured to generate a palette table for a partition of a portion other than the minimum partition by grouping palette colors of each associated next smaller partition into a set of candidate pixel colors, generating a second histogram representing the frequency of occurrence of the binned candidate pixel colors and selecting a specified maximum number of palette colors for indexing in the palette table based on the second histogram.

[0043] In one aspect, the palette mode encoder is configured to select the selected palette table by determining, for each candidate palette table, a corresponding rate-distortion cost for encoding the portion of the image using the candidate palette table, and selecting the candidate palette table having the lowest corresponding rate-distortion cost as the selected palette table. In another aspect, the palette mode encoder includes at least one processor and a memory for storing a set of instructions executable by the at least one processor.

[0044] In some embodiments, the systems and techniques described above are implemented using one or more integrated circuit (IC) devices (also referred to as integrated circuit packages or microchips), such as the encoding system 102 described with reference to FIGS. 1-4-2. Electronic design automation (EDA) and computer aided design (CAD) software tools can be used to design and manufacture these IC devices. These design tools are typically represented as one or more software programs. The one or more software programs include code executable by a computer system for operating the computer system to operate on code representing the circuits of one or more IC devices to perform at least a portion of a process for designing or adapting a manufacturing system for manufacturing the circuits. This code may include instructions, data, or a combination of instructions and data. The software instructions representing the design or manufacturing tools are typically stored in a computer readable storage medium accessible to the computing system. Similarly, code representing one or more stages of the design or manufacture of the IC devices is stored in and accessed from the same computer readable storage medium or a different computer readable storage medium.

[0045] A computer-readable storage medium includes any non-transitory storage medium or combination of non-transitory storage media that can be accessed by a computer system during use to provide instructions and / or data to the computer system. Such storage media may include, but are not limited to, optical media (e.g., compact discs (CDs), digital versatile discs (DVDs), Blu-ray discs), magnetic media (e.g., floppy disks, magnetic tape, magnetic hard drives), volatile memory (e.g., random access memory (RAM) or cache), non-volatile memory (e.g., read-only memory (ROM) or flash memory), or micro-electromechanical systems (MEMS) based storage media. The computer-readable storage medium (e.g., system RAM or ROM) may be internal to the computing system, the computer-readable storage medium (e.g., a magnetic hard drive) may be permanently attached to the computing system, the computer-readable storage medium (e.g., an optical disk or Universal Serial Bus (USB)-based flash memory) may be removably attached to the computing system, or the computer-readable storage medium (e.g., network-accessible storage (NAS)) may be coupled to the computer system via a wired or wireless network.

[0046] In some embodiments, certain aspects of the techniques described above are implemented by one or more processors of a processing system executing software. The software includes one or more sets of executable instructions stored or otherwise tangibly embodied in a non-transitory computer-readable storage medium. The software may include instructions and specific data that, when executed by the one or more processors, operate the one or more processors to perform one or more aspects of the techniques described above. The non-transitory computer-readable storage medium may include, for example, a magnetic or optical disk storage device, a solid-state storage device such as a flash memory, a cache, a random access memory (RAM), or other non-volatile memory device(s), etc. The executable instructions stored in the non-transitory computer-readable storage medium may be implemented as source code, assembly language code, object code, or other form of instructions that can be interpreted or otherwise executed by one or more processors.

[0047] In addition to the above, it should be noted that not all activities or elements described in the summary description are required, some of the specific activities or devices may not be required, one or more additional activities may be performed, and one or more additional elements may be included. Furthermore, the order in which the activities are listed is not necessarily the order in which they are performed. Also, the concepts have been described with reference to specific embodiments. However, those skilled in the art will appreciate that various changes and modifications can be made without departing from the scope of the invention as set forth in the claims. Thus, the specification and drawings should be regarded in an illustrative rather than a restrictive sense, and all such modifications are intended to be included within the scope of the invention.

[0048] Benefits, other advantages, and solutions to problems have been described above with regard to specific embodiments. However, the benefits, advantages, solutions to problems, and features from which any benefit, advantage, or solution may arise or be manifested are not to be construed as critical, essential, or essential features of any or all of the claims. Moreover, the specific embodiments described above are illustrative only, as the disclosed invention may be modified and practiced in different but similar manners apparent to those skilled in the art having the benefit of the teachings herein. No limitations are intended to the details of construction or design herein shown, other than as described in the appended claims. It is therefore apparent that the specific embodiments described above may be altered or modified, and all such variations are considered to be within the scope of the disclosed invention. Accordingly, the protection sought herein is set forth in the appended claims.

Claims

1. 1. A method for encoding an image including a screen content at a source device, comprising: for a selected portion of the image, recursively partitioning the selected portion of the image into a partitioning hierarchy consisting of a plurality of levels, the top level of the partitioning hierarchy having a single block representing an entire set of pixels of the selected portion of the image, and each level below the top level having four smaller blocks of pixels representing a corresponding larger block of pixels at the next higher level; generating, at a lowest level of the partition hierarchy, a palette table for each block at the lowest level, the palette table including a set of palette colors representing pixel colors present in a corresponding block; generating, for each successively higher level of the partition hierarchy, a palette table for each block at a current level by selecting a palette color for said palette table based on palette colors from four palette tables associated with four blocks at a next lower level that correspond to the block at the current level; selecting a final palette table from the palette tables generated for the plurality of levels of the partition hierarchy; generating a color index map for a selected portion of the image based on the final palette table; encoding a representation of a selected portion of the image using the final palette table and the color index map to generate a corresponding segment of an encoded bitstream for output by the source device. method.

2. generating a palette table for each block at the lowest level, generating a first histogram representing a frequency of occurrence of the binned pixel colors for the pixels of the block; selecting, based on the first histogram, up to a specified number of palette colors for indexing into the palette table.

2. The method of claim 1.

3. generating a palette of blocks for a higher level of the partitioning hierarchy, grouping palette colors of each of the palette tables of four lower level blocks associated with said higher level block into a set of candidate pixel colors; generating a second histogram representing the frequency of occurrence of the binned candidate pixel colors; selecting, based on the second histogram, up to the specified number of palette colors for indexing into the palette table. The method of claim 2.

4. generating a palette table for a block at a higher level of the partition hierarchy, grouping palette colors of each of the palette tables of four lower level blocks associated with said higher level block into a set of candidate pixel colors; generating a clustering representing a frequency of occurrence of the binned candidate pixel colors; selecting, based on the clustering, up to a specified number of palette colors for indexing into the palette table. The method according to any one of claims 1 to 3.

5. the lowest level block has a block size of P×P pixels; the selected portion of the image has a size of K×K pixels; The number of levels N of the partition hierarchy is expressed by the formula K=P*2^(N-1), The method according to any one of claims 1 to 4.

6. selecting the final pallet table comprises: determining, for each candidate palette table, a corresponding rate-distortion cost for encoding the selected portion of the image using the candidate palette table; selecting the candidate palette table with the lowest corresponding rate-distortion cost as the final palette table. The method according to any one of claims 1 to 5.

7. the selected portion of the image is a coding unit of a coding tree unit. The method according to any one of claims 1 to 6.

8. 1. A device for encoding an image including a screen content, comprising: Equipped with a palette mode encoder, The palette mode encoder includes: for a selected portion of the image, recursively partitioning the selected portion of the image into a partitioning hierarchy consisting of a plurality of levels, the top level of the partitioning hierarchy having a single block representing an entire set of pixels of the selected portion of the image, and each level below the top level having four smaller blocks of pixels representing a corresponding larger block of pixels at the next higher level; generating, at a lowest level of the partition hierarchy, a palette table for each block at the lowest level, the palette table including a set of palette colors representing pixel colors present in a corresponding block; generating, for each successively higher level of the partition hierarchy, a palette table for each block at a current level by selecting a palette color for said palette table based on palette colors from four palette tables associated with four blocks at a next lower level that correspond to the block at the current level; selecting a final palette table from the palette tables generated for the plurality of levels of the partition hierarchy; generating a color index map for a selected portion of the image based on the final palette table; encoding a representation of a selected portion of the image using the final palette table and the color index map to generate a corresponding segment of an encoded bitstream; 4. The method of claim 3, device.

9. The palette mode encoder includes: generating a first histogram representing a frequency of occurrence of the binned pixel colors for the pixels of the block; selecting, based on the first histogram, up to a specified number of palette colors for indexing into the palette table; and generating a palette table for each block at the lowest level by The device of claim 8.

10. The palette mode encoder includes: grouping palette colors of each of the palette tables of four lower level blocks associated with said higher level block into a set of candidate pixel colors; generating a second histogram representing the frequency of occurrence of the binned candidate pixel colors; selecting, based on the second histogram, up to the specified number of palette colors for indexing into the palette table; generating a palette table for a block at a higher level of the partition hierarchy by The device of claim 9.

11. The palette mode encoder includes: grouping palette colors of each of the palette tables of four lower level blocks associated with said higher level block into a set of candidate pixel colors; generating a clustering representing a frequency of occurrence of the binned candidate pixel colors; selecting, based on said clustering, up to a specified number of palette colors for indexing into said palette table; generating a palette table for a block at a higher level of the partition hierarchy by A device according to any one of claims 8 to 10.

12. The palette mode encoder includes: determining, for each candidate palette table, a corresponding rate-distortion cost for encoding the selected portion of the image using the candidate palette table; selecting the candidate palette table with the lowest corresponding rate-distortion cost as the final palette table; and selecting the final pallet table by A device according to any one of claims 8 to 11.

13. the palette mode encoder comprising at least one processor and a memory for storing a set of instructions executable by the at least one processor; A device according to any one of claims 8 to 12.

14. 1. A method for encoding an image including a screen content at a source device, comprising: generating a palette table for each of a plurality of partitions of a partition hierarchy for the selected portion of the image by recursively using palette colors selected for a set of smaller partitions of the selected portion of the image as candidate colors for larger partitions of the selected portion of the image, the partition hierarchy including a plurality of levels, a highest level of the plurality of levels having at least one block representing an entire set of pixels of the selected portion of the image, and each level below the highest level having two or more smaller blocks of pixels representing a corresponding larger block of pixels at the next higher level; generating a color index map for the selected portion of the image based on a palette table selected from the plurality of palette tables generated for the selected portion of the image; encoding a representation of a selected portion of the image using the selected palette table and the color index map to generate a corresponding segment of an encoded bitstream output by the source device. method.

15. generating a palette table for a minimum partition of the selected portion of the image, generating a first histogram representing a frequency of occurrence of the binned pixel colors for the smallest partition of pixels; selecting, based on the first histogram, up to a specified number of palettes for indexing into the palette table.

15. The method of claim 14.

16. Generating a palette table for a partition of the selected portion other than the smallest partition includes: grouping the palette colors of each palette table of the associated next smaller partition into a set of candidate pixel colors; generating a second histogram representing the frequency of occurrence of the binned candidate pixel colors; selecting, based on the second histogram, up to the specified number of palette colors for indexing into the palette table.

16. The method of claim 15.

17. The selected pallet table is determining, for each candidate palette table, a corresponding rate-distortion cost for encoding the selected portion of the image using the candidate palette table; selecting the candidate palette table with the lowest corresponding rate-distortion cost as the selected palette table; Selected by, The method of any one of claims 14 to 16.

18. the selected portion of the image is a coding unit of a coding tree unit. The method of any one of claims 14 to 17.

19. 1. A device for encoding an image including a screen content, comprising: Equipped with a palette mode encoder, The palette mode encoder includes: generating a palette table for each of a plurality of partitions of a partition hierarchy for the selected portion of the image by recursively using palette colors selected for a set of smaller partitions of the selected portion of the image as candidate colors for larger partitions of the selected portion of the image, the partition hierarchy including a plurality of levels, a highest level of the plurality of levels having at least one block representing an entire set of pixels of the selected portion of the image, and each level below the highest level having two or more smaller blocks of pixels representing a corresponding larger block of pixels at the next higher level; generating a color index map for the selected portion of the image based on a palette table selected from the plurality of palette tables generated for the selected portion of the image; encoding a representation of a selected portion of the image using the selected palette table and the color index map to generate a corresponding segment of an encoded bitstream output by the device; [0023] device.

20. The palette mode encoder includes: generating a first histogram representing the frequency of occurrence of the binned pixel colors for the smallest partition of pixels; selecting, based on the first histogram, up to a specified number of palette colors for indexing into the palette table; generating a palette table of the smallest partition of the selected portion of the image by 20. The device of claim 19.

21. The palette mode encoder includes: grouping the palette colors of each palette table of the associated next smaller partition into a set of candidate pixel colors; generating a second histogram representing the frequency of occurrence of the binned candidate pixel colors; selecting, based on the second histogram, up to the specified number of palettes for indexing into the palette table; and generating a palette table for a partition of the selected portion other than the smallest partition by 21. The device of claim 20.

22. The palette mode encoder includes: determining, for each candidate palette table, a corresponding rate-distortion cost for encoding the selected portion of the image using the candidate palette table; selecting the candidate palette table with the lowest corresponding rate-distortion cost as the selected palette table; and selecting the selected palette table by A device according to any one of claims 19 to 21.

23. the palette mode encoder comprising at least one processor and a memory for storing a set of instructions executable by the at least one processor; A device according to any one of claims 19 to 22.

Images