Using a Palette Predictor in Video Coding and Decoding

Abstract

In an example aspect, a method of video processing includes performing a conversion between a current block of a video and an encoded / decoded representation of the video using a palette mode. During the conversion, a predictor palette is used to predict a palette of representative sample values of the current block in the palette mode. Based on characteristics of the current block, updating of the predictor palette is prohibited after conversion of the current block according to a rule.

Description

[0001] Cross - reference to related applications

[0002] According to the applicable patent laws and / or the rules applicable to the Paris Convention, this application timely claims the priority and benefits of International Patent Application PCT / CN2019 / 105554 filed on September 12, 2019. For all legal purposes, the entire disclosure of the above - mentioned application is incorporated by reference into a part of the disclosure of this application. Technical field

[0003] This patent document relates to video coding and decoding technologies, devices, and systems. Background art

[0004] Currently, efforts are being made to improve the performance of current video codec technologies to provide better compression ratios or video coding and decoding schemes that allow for lower complexity or parallel implementation. Industry experts have recently proposed several new video coding and decoding tools, and tests are currently being conducted to determine their effectiveness. Summary of the invention

[0005] Devices, systems, and methods related to digital video coding and decoding, particularly related to the management of motion vectors, are described. The described methods can be applied to existing video coding and decoding standards (e.g., High Efficiency Video Coding (HEVC) or Versatile Video Coding) and future video coding and decoding standards or video codecs.

[0006] In one representative aspect, the disclosed technology can be used to provide a method for video processing. The method includes performing a conversion between a current block of a video and a coded - decoded representation of the video using a palette mode, wherein a palette of representative sample values is used to code - decode the current block. During the conversion, a predictor palette is used to predict the palette of representative sample values. Based on the characteristics of the current block, updating the predictor palette is prohibited after the conversion of the current block according to a rule.

[0007] In another representative aspect, the disclosed technology can be used to provide a method for video processing. The method includes performing a conversion between a current block of a video and a coded - decoded representation of the video using a palette mode, wherein a palette of representative sample values is used to code - decode the current block. During the conversion, a predictor palette is used to predict the palette of representative sample values. Whether to perform a change to the predictor palette is determined according to the color components of the current block.

[0008] In another representative aspect, the disclosed techniques can be used to provide a method for video processing. The method includes performing a conversion between a current block in a video unit of a video and an encoded / decoded representation of the video using a palette mode, where a palette of representative sample values is used to encode and decode the current block. During the conversion, multiple predictor palettes are used to predict the palette of representative sample values.

[0009] In another representative aspect, the disclosed techniques can be used to provide a method for video processing. The method includes performing a conversion between a current block in a video unit of a video and an encoded / decoded representation of the video using a palette mode, where a palette of representative sample values is used to encode and decode the current block. During the conversion, a predictor palette is used to predict the palette of representative sample values. Before the conversion of the first block in the video unit or after the conversion of the last video block in a previous video unit, the predictor palette is reset or reinitialized according to a rule.

[0010] In another representative aspect, the disclosed techniques can be used to provide a method for video processing. The method includes performing a conversion between a video unit of a video and an encoded / decoded representation of the video using a palette mode. The video unit includes multiple blocks. During the conversion, a shared predictor palette is used by all the multiple blocks to predict the palette of representative sample values for each of the multiple blocks in the predictor palette mode.

[0011] In another representative aspect, the disclosed techniques can be used to provide a method for video processing. The method includes performing a conversion between a current block of a video and an encoded / decoded representation of the video using a palette mode, where a palette of representative sample values is used to encode and decode the current block. During the conversion, a predictor palette is used to predict the palette of representative sample values, and a counter indicating the usage frequency of a corresponding entry is maintained for each entry of the predictor palette.

[0012] In another representative aspect, the disclosed techniques can be used to provide a method for video processing. The method includes performing a conversion between a current block of a video and an encoded / decoded representation of the video using a palette mode, where a palette of representative sample values is used to encode and decode the current block to predict the palette of representative sample values of the current block. The number of entries of the palette signaled in the encoded / decoded representation is in the range of [0, the maximum allowed size of the palette - the number of palette entries derived during the conversion].

[0013] In another representative aspect, the disclosed technology can be used to provide a method for video processing. The method includes performing a conversion between a current block in a video unit of a video and an encoded / decoded representation of the video using a palette mode, where a palette of representative sample values is used to encode and decode the current block. During the conversion, a predictor palette is used to predict the palette of representative sample values, and the size of the predictor palette is adaptively adjusted according to rules.

[0014] In another representative aspect, the disclosed technology can be used to provide a method for video processing. The method includes performing a conversion between a current block in a video unit of a video and an encoded / decoded representation of the video using a palette mode, where a palette of representative sample values is used to encode and decode the current block. During the conversion, a predictor palette is used to predict the palette of representative sample values, and the size of the palette of representative samples or the predictor palette is determined according to rules, where the rules allow the size to change between video units of the video.

[0015] In another representative aspect, the disclosed technology can be used to provide a method for video processing. The method includes performing a conversion between a current block in a video unit of a video and an encoded / decoded representation of the video using a palette mode, where a palette of representative sample values is used to encode and decode the current block. During the conversion, a predictor palette is used to predict the palette of representative sample values. The predictor palette is reinitialized when a condition is met, where the condition is met when the video unit is the first video unit in a video unit row and a syntax element indicating that wavefront parallel processing is enabled for the video unit is included in the encoded / decoded representation.

[0016] In another representative aspect, the disclosed technology can be used to provide a method for video processing. The method includes performing a conversion between a video block in a video unit and an encoded / decoded representation of the video block using a palette mode, where, during the conversion, a predictor palette is used to predict the current palette information of the video block. Further, the predictor palette is selectively reset before the conversion between the video block and the bitstream representation of the video block.

[0017] In another representative aspect, the disclosed technology can be used to provide another method for video processing. The method includes performing a conversion between a video block in a video unit and an encoded / decoded representation of the video block using a palette mode, where, during the conversion, a predictor palette is used to predict the current palette information of the video block. Further, when multiple encoded / decoded units of the video unit have a common shared area, the predictor palette is a shared predictor palette.

[0018] In another representative aspect, the disclosed technology can be used to provide another method for video processing. The method includes performing a conversion between a video block in a video unit and an encoded / decoded representation of the video block using a palette mode, wherein, during the conversion, a predictor palette is used to predict the current palette information of the video block, and further, the size of the predictor palette is adaptively changed according to one or more conditions.

[0019] In another representative aspect, the disclosed technology can be used to provide another method for video processing. The method includes performing a conversion between a video block in a video unit and an encoded / decoded representation of the video block using a palette mode, wherein, during the conversion, a predictor palette is used to predict the current palette information of the video block, and further, the predictor palette is updated based on the size or number of entries in the predictor palette.

[0020] In another representative aspect, the disclosed technology can be used to provide another method for video processing. The method includes performing a conversion between a video block in a video unit and an encoded / decoded representation of the video block using a palette mode, wherein, during the conversion, a predictor palette is used to predict the current palette information of the video block, and further, the entries of the predictor palette are reordered or modified.

[0021] In another representative aspect, the disclosed technology can be used to provide another method for video processing. The method includes performing a conversion between a video block in a video unit and an encoded / decoded representation of the video block using a palette mode, wherein, during the conversion, a predictor palette is used to predict the current palette information of the video block, and further, the use of the predictor palette is indicated by maintaining a counter that tracks the number of times the predictor palette is used.

[0022] In another example aspect, the method described above can be implemented by a video decoder device including a processor.

[0023] In another example aspect, the method described above can be implemented by a video encoder device including a processor.

[0024] Further, in a representative aspect, a device in a video system is disclosed, the device including a processor and a non-transitory memory having instructions thereon. When the instructions are executed by the processor, the processor is caused to implement any one or more of the disclosed methods.

[0025] Further, a computer program product stored on a non-transitory computer-readable medium is disclosed, the computer program product including program code for performing any one or more of the disclosed methods.

[0026] The above and other aspects and features of the disclosed technology are described in more detail in the accompanying drawings, the specification, and the claims. Description of the Drawings

[0027] Figure 1 An example of a block encoded and decoded in palette mode is shown.

[0028] Figure 2 An example of notifying palette entries using predictor palette signaling is shown.

[0029] Figure 3 Examples of horizontal traversal scan and vertical traversal scan are shown.

[0030] Figure 4 An example of the encoding and decoding of palette indices is shown.

[0031] Figure 5 An example of a picture with an 18×12 luminance CTU divided into 12 slices and 3 raster scan stripes is shown.

[0032] Figure 6 An example of a picture with an 18×12 luminance CTU divided into 24 slices and 9 rectangular stripes is shown.

[0033] Figure 7 An example of a picture divided into 4 slices, 11 bricks, and 4 rectangular stripes is shown.

[0034] Figure 8 An example of a picture with 28 sub-pictures is shown.

[0035] Figure 9 It is a block diagram of an example hardware platform for implementing the visual media decoding or visual media encoding techniques described in this document.

[0036] Figure 10 It is a block diagram of an example video processing system that can implement the disclosed technology.

[0037] Figure 11 A flowchart of an example method for video encoding and decoding is shown.

[0038] Figure 12 It is a flowchart representation of a method for video processing according to the present technology.

[0039] Figure 13 It is a flowchart representation of another method for video processing according to the present technology.

[0040] Figure 14 It is a flowchart representation of another method for video processing according to the present technology.

[0041] Figure 15It is a flowchart representation of another video processing method according to the present technology.

[0042] Figure 16 It is a flowchart representation of another video processing method according to the present technology.

[0043] Figure 17 It is a flowchart representation of another video processing method according to the present technology.

[0044] Figure 18 It is a flowchart representation of another video processing method according to the present technology.

[0045] Figure 19 It is a flowchart representation of another video processing method according to the present technology.

[0046] Figure 20 It is a flowchart representation of another video processing method according to the present technology.

[0047] Figure 21 It is a flowchart representation of yet another video processing method according to the present technology. Detailed implementation manners

[0048] 1. Video encoding and decoding of HEVC / H.265

[0049] Video encoding and decoding standards have mainly evolved through the development of well-known ITU-T and ISO / IEC standards. ITU-T developed H.261 and H.263, ISO / IEC developed MPEG-1 and MPEG-4 Visual, and the two organizations jointly developed H.262 / MPEG-2 video, H.264 / MPEG-4 Advanced Video Coding (AVC), and H.265 / HEVC standards. Since H.262, video encoding and decoding standards have been based on a hybrid video encoding and decoding structure that utilizes temporal prediction and transform coding. To explore future video encoding and decoding technologies beyond HEVC, VCEG and MPEG jointly established the Joint Video Exploration Team (JVET) in 2015. Since then, JVET has adopted many new methods and incorporated them into a reference software called the Joint Exploration Model (JEM). In April 2018, a JVET was established between VCEG (Q6 / 16) and ISO / IEC JTC1 SC29 / WG11 (MPEG) to work on the VVC standard with the goal of reducing the bit rate by 50% compared to HEVC.

[0050] 2. Palette mode

[0051] 2.1 Palette mode of HEVC Screen Content Coding Extension (HEVC-SCC)

[0052] 2.1.1. Concept of the palette mode

[0053] The basic idea behind the palette mode is that pixels in a CU are represented by a small set of representative color values. This set is called the palette. Also, out-of-palette samples can be indicated by signaling escape symbols for subsequent (possibly quantized) component values. Such pixels are called escape pixels. The palette mode is shown in Figure 1 As Figure 1 shown, for each pixel with three color components (luma and two chroma components), an index to the palette is established and the block can be reconstructed based on the values found in the palette.

[0054] 2.1.2. Encoding and decoding of palette entries

[0055] For palette-encoded / decoded blocks, the following key aspects are introduced:

[0056] 1) Construct the current palette based on the predictor palette and the new entries (if any) signaled for the current palette.

[0057] 2) Divide the current sample / pixel into two categories: one category (the first category) includes samples / pixels in the current palette, and the other category (the second category) includes samples / pixels outside the current palette.

[0058] a. For samples / pixels in the second category, quantization (at the encoder) is applied to the sample / pixel and the quantization value is signaled; and dequantization (at the decoder) is applied.

[0059] 2.1.2.1. Predictor palette

[0060] For the encoding and decoding of palette entries, a predictor palette is maintained, which is updated after decoding a palette-encoded block.

[0061] 2.1.2.1.1. Initialization of the predictor palette

[0062] Initialize the predictor palette at the start of each slice and each tile.

[0063] Signal the palette and the maximum size of the predictor palette in the SPS. In HEVC-SCC, the palette_predictor_initializer_present_flag is introduced in the PPS. When this flag is 1, the entries used to initialize the predictor palette are signaled in the bitstream.

[0064] Depending on the value of palette_predictor_initializer_present_flag, the size of the predictor palette is reset to 0 or initialized using the predictor palette initializer entry signaled in the PPS. In HEVC-SCC, a predictor palette initializer of size 0 is enabled to allow explicit disabling of the predictor palette initialization at the PPS level.

[0065] The corresponding syntax, semantics, and decoding processes are defined as follows. Newly added text is shown in bold and underlined italics. Any deleted text is marked with [[ ]].

[0066] 7.3.2.2.3 Sequence parameter set screen content coding extension syntax

[0067]

[0068] palette_mode_enabled_flag being equal to 1 specifies that the decoding process for the palette mode can be used for intra blocks. palette_mode_enabled_flag being equal to 0 specifies that the decoding process for the palette mode shall not be applied. When not present, the value of palette_mode_enabled_flag is inferred to be equal to 0.

[0069] palette_max_size specifies the maximum allowed size of the palette. When not present, the value of palette_max_size is inferred to be 0.

[0070] delta_palette_max_predictor_size specifies the difference between the maximum allowed size of the palette predictor and the maximum allowed size of the palette. When not present, the value of delta_palette_max_predictor_size is inferred to be 0. The derivation of the variable PaletteMaxPredictorSize is as follows:

[0071] PaletteMaxPredictorSize = palette_max_size + delta_palette_max_predictor_size (2 - 1)

[0072] The bitstream conformance requirement is that when palette_max_size is equal to 0, the value of delta_palette_max_predictor_size shall be equal to 0.

[0073] The sps_palette_predictor_initializer_present_flag being equal to 1 specifies that the sequence palette predictor is initialized using the sps_palette_predictor_initializers specified in this clause. The sps_palette_predictor_initializer_flag being equal to 0 specifies that the entries in the sequence palette predictor are initialized to 0. When not present, the value of sps_palette_predictor_initializer_flag is inferred to be equal to 0.

[0074] The requirement for bitstream conformance is that when palette_max_size is equal to 0, the value of sps_palette_predictor_initializer_present_flag shall be equal to 0.

[0075] sps_num_palette_predictor_initializer_minus1 plus 1 specifies the number of entries in the sequence palette predictor initializer.

[0076] The requirement for bitstream conformance is that the value of sps_num_palette_predictor_initializer_minus1 plus 1 shall be less than or equal to PaletteMaxPredictorSize.

[0077] sps_palette_predictor_initializers[comp][i] specifies the value of the comp-th component of the i-th palette entry used to initialize the array PredictorPaletteEntries in the SPS. For values of i in the range 0 to sps_num_palette_predictor_initializer_minus1 (inclusive), the value of sps_palette_predictor_initializers[0][i] shall be in the range 0 to (1 << BitDepth Y ) – 1 (inclusive), and the values of sps_palette_predictor_initializers[1][i] and sps_palette_predictor_initializers[2][i] shall be in the range 0 to (1 << BitDepth C ) – 1 (inclusive).

[0078] 7.3.2.3.3 Picture Parameter Set Screen Content Coding Extension Syntax

[0079]

[0080] The pps_palette_predictor_initializer_present_flag being equal to 1 specifies that the palette predictor initializer for a picture that references the PPS is derived from the palette predictor initializer specified by the PPS. The pps_palette_predictor_initializer_flag being equal to 0 specifies that the palette predictor initializer for a picture that references the PPS is inferred to be equal to the palette predictor initializer specified by the active SPS. When not present, the value of pps_palette_predictor_initializer_present_flag is inferred to be equal to 0. The requirement for bitstream conformance is that the value of pps_palette_predictor_initializer_present_flag shall be equal to 0 when palette_max_size is equal to 0 or palette_mode_enabled_flag is equal to 0.

[0081] pps_num_palette_predictor_initializer specifies the number of entries in the picture palette predictor initializer.

[0082] The requirement for bitstream conformance is that the value of pps_num_palette_predictor_initializer shall be less than or equal to PaletteMaxPredictorSize.

[0083] Initialize the palette predictor variables as follows:

[0084] – If the coding tree unit is the first coding tree unit in a slice, then the following applies:

[0085] – Invoke the initialization process for the palette predictor variables as specified in Clause 9.3.2.3.

[0086] – Otherwise, if entropy_coding_sync_enabled_flag is equal to 1 and CtbAddrInRs % PicWidthInCtbsY is equal to 0 or TileId[CtbAddrInTs] is not equal to TileId[CtbAddrRsToTs[CtbAddrInRs - 1]], then the following applies:

[0087] – Derive the spatial neighbor block T using the position (x0, y0) of the top - left luma sample of the current coding tree block as followsFigure 2 ) the position (xNbT, yNbT) of the top - left luminance sample is as follows:

[0088] (xNbT, yNbT) = (x0 + CtbSizeY, y0 - CtbSizeY) (9 - 3)

[0089] - Invoke the availability derivation process of the blocks in the z - scan order specified in Clause 6.4.1, where the position (xCurr, yCurr) set to be equal to (x0, y0) and the neighboring position (xNbY, yNbY) set to be equal to (xNbT, yNbT) are used as inputs, and the output is assigned to availableFlagT.

[0090] - The synchronization process calls of context variables, Rice parameter initialization status, and palette predictor variables are as follows:

[0091] - If availableFlagT is equal to 1, then invoke the synchronization process of context variables, Rice parameter initialization status, and palette predictor variables specified in Clause 9.3.2.5, with TableStateIdxWpp, TableMpsValWpp, TableStatCoeffWpp, PredictorPaletteSizeWpp, and TablePredictorPaletteEntriesWpp as inputs.

[0092] - Otherwise, the following applies:

[0093] - Invoke the initialization process of the palette predictor variables specified in Clause 9.3.2.3.

[0094] - Otherwise, if CtbAddrInRs is equal to slice_segment_address and dependent_slice_segment_flag is equal to 1, then invoke the synchronization process of context variables and Rice parameter initialization status specified in Clause 9.3.2.5, with TableStateIdxDs, TableMpsValDs, TableStatCoeffDs, PredictorPaletteSizeDs, and TablePredictorPaletteEntriesDs as inputs.

[0095] - Otherwise, the following applies:

[0096] - Invoke the initialization process of the palette predictor variables specified in Clause 9.3.2.3.

[0097] 9.3.2.3 Initialization Process of Palette Predictor Entries

[0098] The output of this process is the initialized palette predictor variables PredictorPaletteSize and PredictorPaletteEntries.

[0099] The variable numComps is derived as follows:

[0100] numComps = (ChromaArrayType == 0)? 1 : 3 (9 - 8)

[0101] – If pps_palette_predictor_initializer_present_flag is equal to 1, the following applies:

[0102] – PredictorPaletteSize is set to be equal to pps_num_palette_predictor_initializer.

[0103] – The array PredictorPaletteEntries is derived as follows:

[0104]

[0105] – Otherwise (pps_palette_predictor_initializer_present_flag is equal to 0), if sps_palette_predictor_initializer_present_flag is equal to 1, the following applies:

[0106] – PredictorPaletteSize is set to be equal to sps_num_palette_predictor_initializer_minus1 plus 1.

[0107] – The array PredictorPaletteEntries is derived as follows:

[0108]

[0109] – Otherwise (pps_palette_predictor_initializer_present_flag is equal to 0 and sps_palette_predictor_initializer_present_flag is equal to 0), PredictorPaletteSize is set to be equal to 0.

[0110] 2.1.2.1.2. Use of Predictor Palette

[0111] For each entry in the palette predictor, a reuse flag is signaled to indicate whether it is part of the current palette. This is shown in Figure 2 . The reuse flag is sent using run - length coding with zero. After that, the number of new palette entries is signaled using an exponential Golomb (EG) code of order 0 (e.g., EG - 0). Finally, the component values of the new palette entries are signaled.

[0112] 2.1.2.2. Update of Predictor Palette

[0113] The update of the predictor palette is performed using the following steps:

[0114] (1) Before decoding the current block, there is a predictor palette, denoted as PltPred0

[0115] (2) The current palette table is constructed by first inserting the items in PltPred0 and then inserting the new entries of the current palette.

[0116] (3) Construct PltPred1:

[0117] a. First, add the items in the current palette table (which may include items in PltPred0)

[0118] b. If not full, add the unreferenced items in PltPred0 in ascending order of entry index.

[0119] 2.1.3. Coding and Decoding of Palette Index

[0120] As Figure 3 shown, horizontal and vertical traversal scans are used to code and decode the palette index. The scan order is explicitly signaled in the bitstream using palette_transpose_flag. For the remainder of the sub - clause, it is assumed that the scan is horizontal.

[0121] Two palette sample modes are used to code and decode the palette index: "COPY_LEFT" and "COPY_ABOVE". In the "COPY_LEFT" mode, the palette index is assigned to the decoded index. In the "COPY_ABOVE" mode, the palette index of the samples in the above row is copied. For both the "COPY_LEFT" and "COPY_ABOVE" modes, a run value is signaled, which specifies the number of subsequent samples to be coded and decoded using the same mode.

[0122] In palette mode, the index value of an escape sample is the number of palette entries. Also, when the escape symbol is part of a run in "COPY_LEFT" or "COPY_ABOVE" mode, the escape component values are signaled for each escape symbol. The encoding and decoding of the palette index are shown in Figure 4 as follows.

[0123] This syntax order is completed as follows. First, the number of index values for the CU is signaled. Then the actual index values for the entire CU are signaled using truncated binary encoding. Both the number of indices and the index values are encoded and decoded in bypass mode. This groups the bypass binary bits related to the indices together. Then, the palette sample mode (if necessary) and the runs are signaled in an interleaved manner. Finally, the component escape values for the escape samples corresponding to the entire CU are grouped together and encoded and decoded in bypass mode. The binarization of the escape samples is an EG encoding with three orders, e.g., EG-3.

[0124] After signaling the index values, the additional syntax element last_run_type_flag is signaled. This syntax element, combined with the number of indices, eliminates the need to signal the run value corresponding to the last run in the block.

[0125] In HEVC-SCC, the palette mode also applies to 4:2:2, 4:2:0, and monochrome chroma formats. For all chroma formats, the signaling of the palette entries and the palette index is almost the same. If it is a non-monochrome format, each palette entry consists of 3 components. For the monochrome format, each palette entry consists of a single component. For subsampled chroma directions, the chroma samples are associated with the luma sample indices divisible by 2. After reconstructing the palette index of the CU, if a sample has only a single component associated with it, only the first component of the palette entry is used. The only difference in the signaling lies in the escape component values. For each escape sample, the number of escape component values signaled may be different, depending on the number of components associated with that sample.

[0126] In addition, there is an index adjustment process in the palette index encoding and decoding. When signaling the palette index, the left neighboring index or the above neighboring index should be different from the current index. Thus, by removing one possibility, the range of the current palette index can be reduced by 1. After that, the index is signaled using truncated binary (TB) binarization.

[0127] The text related to this part is shown below, where CurrPaletteIndex is the current palette index and adjustedRefPaletteIndex is the predicted index.

[0128] The variable PaletteIndexMap[xC][yC] specifies the palette index, which is the index of the array represented by CurrentPaletteEntries. The array indices xC, yC specify the position (xC, yC) of the sample relative to the luminance sample at the upper left corner of the picture. The value of PaletteIndexMap[xC][yC] should be in the range from 0 to MaxPaletteIndex (inclusive).

[0129] The derivation of the variable adjustedRefPaletteIndex is as follows:

[0130]

[0131]

[0132] When CopyAboveIndicesFlag[xC][yC] is equal to 0, the derivation of the variable CurrPaletteIndex is as follows:

[0133] if(CurrPaletteIndex >= adjustedRefPaletteIndex)

[0134] CurrPaletteIndex++

[0135] 2.1.3.1. Decoding process of the palette encoding / decoding block

[0136] 1) Read the prediction information to mark which entries in the predictor palette will be reused;

[0137] (palette_predictor_run)

[0138] 2) Read the new palette entries of the current block

[0139] a.num_signaled_palette_entries

[0140] b.new_palette_entries

[0141] 3) Construct CurrentPaletteEntries based on a) and b)

[0142] 4) Read the escape symbol presence flag: palette_escape_val_present_flag to derive MaxPaletteIndex

[0143] 5) Encode and decode how many samples that are not encoded in the copy mode / run mode

[0144] a. num_palette_indices_minus1

[0145] b. For each sample not encoded / decoded in copy mode / run mode, encode / decode palette_idx_idc in the current plt table

[0146] 2.2. Palette Mode in VVC

[0147] 2.2.1. Palette in the Dual-Tree

[0148] In VVC, the dual-tree codec structure is used to encode / decode intra-coded strips. Therefore, the luminance component and the two chrominance components may have different palettes and palette indices. In addition, the two chrominance components share the same palette and palette indices.

[0149] 2.2.2. Palette as a Separate Mode

[0150] In some embodiments, the prediction mode of the codec unit can be MODE_INTRA, MODE_INTER, MODE_IBC, and MODE_PLT. The binarization of the prediction mode is changed accordingly.

[0151] When IBC is disabled, on an I slice, the first binary bit is used to indicate whether the current prediction mode is MODE_PLT. On a P / B slice, the first binary bit is used to indicate whether the current prediction mode is MODE_INTRA. If not, an additional binary bit is used to indicate whether the current prediction mode is MODE_PLT or MODE_INTER.

[0152] When IBC is enabled, on an I slice, the first binary bit is used to indicate whether the current prediction mode is MODE_IBC. If not, the second binary bit is used to indicate whether the current prediction mode is MODE_PLT or MODE_INTRA. On a P / B slice, the first binary bit is used to indicate whether the current prediction mode is MODE_INTRA. If it is an intra mode, the second binary bit is used to indicate whether the current prediction mode is MODE_PLT or MODE_INTRA. If not, the second binary bit is used to indicate whether the current prediction mode is MODE_IBC or MODE_INTER.

[0153] Example syntax text is shown as follows.

[0154] Codec Unit Syntax

[0155]

[0156]

[0157] 2.3. Segmentation of Pictures, Sub - pictures, Strips, Slices, Tiles, and CTUs

[0158] Sub - picture: A rectangular region of one or more strips within a picture.

[0159] Strip: An integral number of tiles of a picture that are uniquely contained within a single NAL unit. A strip consists of a contiguous sequence of multiple complete slices or complete tiles of a slice.

[0160] Slice: A rectangular region of CTUs within a specific slice column and a specific slice row of a picture.

[0161] Tile: A rectangular region of CTU rows within a specific slice of a picture. A slice can be divided into multiple tiles, each tile consisting of one or more CTU rows within the slice. A slice that is not divided into multiple tiles is also called a tile. However, a tile that is a true subset of a slice is not called a slice.

[0162] Tile scan: A specific order sorting of CTU segmentation of a picture, where CTUs are sorted consecutively in the CTU raster scan of a tile, tiles within a slice are sorted consecutively in the raster scan of the tiles of the slice, and slices within a picture are sorted consecutively in the raster scan of the slices of the picture.

[0163] A picture is divided into one or more slice rows and one or more slice columns. A slice is a sequence of CTUs that covers a rectangular region of the picture.

[0164] A slice is divided into one or more tiles, each tile consisting of multiple CTU rows within the slice.

[0165] A slice that is not divided into multiple tiles is also called a tile. However, a tile that is a true subset of a slice is not called a slice.

[0166] A strip contains multiple slices of a picture or multiple tiles of a slice.

[0167] A sub - picture contains one or more strips that jointly cover a rectangular region of the picture.

[0168] Two strip modes are supported, namely the raster - scan strip mode and the rectangular strip mode. In the raster - scan strip mode, a strip contains a sequence of slices in the slice raster scan of the picture. In the rectangular strip mode, a strip contains multiple tiles of the picture that jointly form a rectangular region of the picture. The tiles within a rectangular strip are in the raster - scan order of the tiles of the strip.

[0169] Figure 5 An example of the raster - scan strip segmentation of a picture is shown, where the picture is divided into 12 slices and 3 raster - scan strips.

[0170] Figure 6Shows an example of rectangular strip segmentation of a picture, where the picture is divided into 24 slices (6 slice columns and 4 slice rows) and 9 rectangular strips.

[0171] Figure 7 Shows an example of a picture segmented into slices, tiles, and rectangular strips, where the picture is divided into 4 slices (2 slice columns and 2 slice rows), 11 tiles (the upper left slice contains 1 tile, the upper right slice contains 5 tiles, the lower left slice contains 2 tiles, and the lower right slice contains 3 tiles), and 4 rectangular strips.

[0172] Figure 8 Shows an example of sub - picture segmentation of a picture, where the picture is segmented into 28 sub - pictures of different dimensions.

[0173] When encoding and decoding a picture using three separate color planes (separate_colour_plane_flag equals 1), a strip contains only CTUs of one color component, which is identified by the corresponding value of colour_plane_id, and each color - component array of the picture consists of strips with the same colour_plane_id value. Encoded and decoded strips with different colour_plane_id values within a picture can be interleaved under the following constraint: for each value of colour_plane_id, the encoded and decoded strip NAL units with that colour_plane_id value shall be in the order of increasing CTU addresses in the tile scan order of the first CTU of each encoded and decoded strip NAL unit.

[0174] When separate_colour_plane_flag equals 0, each CTU of the picture is exactly contained in one strip. When separate_colour_plane_flag equals 1, each CTU of a color component is exactly contained in one strip (for example, the information of each CTU of the picture exists exactly in three strips, and these three strips have different colour_plane_id values).

[0175] 2.4. Wavefront with 1 - CTU delay

[0176] In VVC, a one - CTU - delay wavefront parallel processing (WPP) is adopted instead of the two - CTU - delay in the HEVC design. WPP processing enables multiple parallel processes with limited encoding and decoding losses, but the two - CTU - delay may hinder the parallel processing ability. Since the target resolution is getting larger and the number of CPUs is increasing, it is asserted that improving the parallel processing ability by using the proposed one - CTU - delay is beneficial for reducing encoding and decoding latency and will make full use of the CTU capacity.

[0177] 3. Example problems in the existing embodiments

[0178] DMVR and BIO do not involve the original signal during the refinement of motion vectors, which may lead to inaccurate motion information of the coding / decoding block. In addition, DMVR and BIO sometimes adopt fractional motion vectors after motion refinement, while screen video usually uses integer motion vectors, which makes the current motion information more inaccurate and the coding / decoding performance worse.

[0179] (1) The current palette is constructed based on the prediction of the previous coding / decoding palette. The current palette is only re-initialized before decoding a new CTU row or a new slice when the entropy_coding_sync_enabled_flag is equal to 1. However, in practical applications, parallel encoders are preferred, where different CTU rows can be pre-coded / decoded without referring to the information of other CTU rows.

[0180] (2) The way to process the predictor palette update process is fixed. That is, the entries inherited from the previous predictor palette and the new entries in the current palette are inserted in sequence. If the number of entries is still less than the size of the predictor palette, further entries not inherited from the previous predictor palette will be added. This design does not consider the importance of different entries in the current predictor palette and the previous predictor palette.

[0181] (3) The size of the predictor palette is fixed, and after decoding a block, it must be updated to fill all entries, which is sub-optimal because some of these entries may never be referenced.

[0182] (4) The size of the current palette is fixed, regardless of the color component. For example, fewer chroma samples can be used compared to luminance.

[0183] 4. Example techniques and embodiments

[0184] The detailed embodiments described below should be regarded as examples for explaining the general concept. These embodiments should not be interpreted in a narrow sense. In addition, these embodiments can be combined in any way.

[0185] In addition to DMVR and BIO mentioned below, the methods described below may also be applicable to other decoder motion information derivation techniques.

[0186] Re - hierarchical predictor palette

[0187] 1. It is proposed to reset or re-initialize the predictor palette (e.g., entries and / or the size of the predictor palette) before decoding the first block in a new video unit.

[0188] a. Alternatively, after decoding the last block in a video unit, the predictor palette (e.g., the entries and / or the size of the predictor palette) may be reset or re-initialized.

[0189] b. In one example, the video unit is a sub-region / CTU / CTB / multiple CTUs / multiple CUs / CTU row / slice / tile / sub-picture / view, etc. of a CTU (e.g., VPDU).

[0190] i. Alternatively, in addition, even if wavefront is disabled (e.g., entropy_coding_sync_enabled_flag equals 0), the above method is invoked.

[0191] c. In one example, the video unit is a chrominance CTU row.

[0192] i. Alternatively, in addition, the predictor palette may be reset or re-initialized before decoding the first chrominance CTB in a new chrominance CTU row.

[0193] ii. Alternatively, in addition, when applying the dual-tree and the current split tree is the chrominance codec tree, the above method is invoked.

[0194] d. In one example, the size of the predictor palette (e.g., PredictorPaletteSize in the specification) is reset to 0.

[0195] e. In one example, the size of the predictor palette (e.g., PredictorPaletteSize in the specification) is reset to the number of entries in the sequence palette predictor initializer (e.g., sps_num_palette_predictor_initializer_minus1 plus 1) or the maximum number of entries allowed in the predictor palette (e.g., PaletteMaxPredictorSize).

[0196] f. The initialization of the predictor palette (e.g., PredictorPaletteEntries) before encoding / decoding a new sequence / picture can be used to initialize the predictor palette before encoding / decoding a new video unit.

[0197] g. In one example, when entropy_coding_sync_enabled_flag equals 1, the predictor palette after encoding / decoding the upper CTB / CTU can be used to initialize the predictor palette before encoding / decoding the current CTB / CTU.

[0198] 2. It is proposed to prohibit updating the predictor palette after encoding / decoding a specific palette coding block.

[0199] a. In one example, whether to update the predictor palette can depend on the decoding information of the current block.

[0200] i. In one example, whether to update the predictor palette can depend on the block dimensions of the current block.

[0201] 1. In one example, if the width of the current block is not greater than a first threshold (denoted by T1) and the height of the current block is not greater than a second threshold (denoted by T2), the update process is disabled.

[0202] 2. In one example, if the product of the width of the current block and the height of the block is not greater than a first threshold (denoted by T1), the update process is disabled.

[0203] 3. In one example, if the width of the current block is not less than a first threshold (denoted by T1) and the height of the current block is not less than a second threshold (denoted by T2), the update process is disabled.

[0204] 4. In one example, if the product of the width of the current block and the height of the block is not less than a first threshold (denoted by T1), the update process is disabled.

[0205] 5. In the above examples, T1 / T2 can be predefined or signaled.

[0206] a) In one example, T1 / T2 can be set to 4, 16, or 1024.

[0207] b) In one example, T1 / T2 can depend on the color component.

[0208] 3. A shared predictor palette can be defined, where the same predictor palette can be used for all CUs / PUs under the shared region.

[0209] a. In one example, a shared region can be defined for an MxN region (e.g., 16×4 or 4×16 region) using TT partitioning.

[0210] b. In one example, a shared region can be defined for an MxN region (e.g., 8×4 or 4×8 region) using BT partitioning.

[0211] c. In one example, a shared region can be defined for an MxN region (e.g., 8×8 region) using QT partitioning.

[0212] d. Alternatively, in addition, a shared predictor palette can be constructed before encoding / decoding all blocks within the shared region.

[0213] e. In one example, an indication of a prediction entry in the predictor palette (e.g., palette_predictor_run) may be signaled together with the first palette coding block within the shared region.

[0214] i. Alternatively, in addition, for the remaining coding blocks within the shared region, signaling of the indication of the prediction entry in the predictor palette (e.g., palette_predictor_run) may be skipped.

[0215] f. Alternatively, in addition, after decoding / encoding the blocks within the shared region, the update of the predictor palette may always be skipped.

[0216] 4. A counter may be maintained for each entry of the predictor palette to indicate the frequency of its use.

[0217] a. In one example, for each new entry added to the predictor palette, the counter may be set to a constant K.

[0218] i. In one example, K may be set to 0.

[0219] b. In one example, when an entry is marked for reuse in the coding / decoding palette block, the corresponding counter may be incremented by a constant N.

[0220] i. In one example, N may be set to 1.

[0221] 5. It is proposed to adaptively change the size of the predictor palette instead of using a fixed-size predictor palette.

[0222] a. In one example, it may change between video units (blocks / CUs / CTUs / slices / bricks / sub-pictures) and another video unit.

[0223] b. In one example, the size of the predictor palette may be updated according to the size of the current palette.

[0224] i. In one example, the size of the predictor palette may be set to the size of the current palette after decoding / encoding the current block.

[0225] ii. In one example, the size of the predictor palette may be set to the size of the current palette minus or plus an integer value represented as K.

[0226] 1. In one example, K may be signaled / instantly derived.

[0227] c. In one example, the size of the predictor palette can depend on the block size. Let S be the predefined size of the predictor palette for the palette-coded block.

[0228] i. In one example, a palette-coded block with a size less than or equal to T can use a predictor palette with a size less than S.

[0229] 1. In one example, the first K entries (K <= S) in the palette predictor can be used.

[0230] 2. In one example, a subsampled version of the palette predictor can be used.

[0231] ii. In one example, a palette-coded block with a size greater than or equal to T can use a predictor palette with a size equal to S.

[0232] iii. In the above examples, K and / or T are integers and can be based on

[0233] 1. Video content (e.g., screen content or natural content)

[0234] 2. Messages signaled in DPS / SPS / VPS / PPS / APS / Picture Header / Strip Header / Slice Group Header / Largest Coding Unit (LCU) / Coding Unit (CU) / LCU Row / LCU Group / TU / PU Block / Video Coding Unit

[0235] 3. The position of CU / PU / TU / Block / Video Coding Unit

[0236] 4. Indication of color format (e.g., 4:2:0, 4:4:4, RGB, or YUV)

[0237] 5. Coding tree structure (e.g., dual-tree or single-tree)

[0238] 6. Strip / Slice Group type and / or Picture type

[0239] 7. Color component

[0240] 8. Temporal layer ID

[0241] 9. Profile / Level / Tier of the standard

[0242] d. In one example, after encoding / decoding a palette block, the predictor palette can be customized according to the entry counter.

[0243] i. In one example, entries with a counter less than the threshold T can be discarded.

[0244] ii. In one example, entries with the smallest counter values can be discarded until the size of the predictor palette is less than a threshold T.

[0245] e. Alternatively, in addition, after decoding / encoding the palette coding block, the predictor palette can be updated based only on the current palette.

[0246] i. Alternatively, in addition, after decoding / encoding the palette coding block, the predictor palette can be updated to the current palette.

[0247] 6. Entries of the current palette and / or the predictor palette before encoding / decoding the current block can be reordered / modified before being used to update the predictor palette.

[0248] a. In one example, reordering can be applied according to the decoding information / reconstruction of the current sample.

[0249] b. In one example, reordering can be applied according to the counter values of the entries.

[0250] c. Alternatively, in addition, the number of occurrences of samples / pixels (in the current palette and / or outside the current palette) can be counted.

[0251] i. Alternatively, in addition, samples / pixels with a larger counter (e.g., occurring more frequently) can be placed before another sample / pixel with a smaller counter.

[0252] 7. Information on escaped samples can be used to update the predictor palette.

[0253] a. Alternatively, in addition, an update of the predictor palette using the escaped information can be conditionally invoked.

[0254] i. In one example, when the predictor palette is not full after inserting the current palette, escaped sample / pixel information can be added to the predictor palette.

[0255] 8. Updating / initializing / resetting the predictor palette can depend on the color component.

[0256] a. In one example, the rule for determining whether to update the predictor palette can depend on the color component, such as luminance or chrominance.

[0257] 9. A set of multiple predictor palettes can be maintained and / or updated.

[0258] a. In one example, one predictor palette can have information for one or all color components.

[0259] b. In one example, a predictor palette may have information for two color components (e.g., Cb and Cr).

[0260] c. In one example, at least one global palette and at least one local palette may be maintained.

[0261] i. In one example, the predictor palette may be updated based on the global palette and the local palette.

[0262] d. In one example, a palette associated with the last K palette coding / decoding blocks (in encoding / decoding order) may be maintained.

[0263] e. In one example, palettes for the luminance component and the chrominance component may be predicted based on different predictor palettes (e.g., different indices of a set of multiple predictor palettes).

[0264] f. Alternatively, in addition, bullet 1 may be applied to a set of predictor palettes.

[0265] g. Alternatively, in addition, an index / multiple indices of a predictor palette in a set of predictor palettes may be signaled for a sub-region of a CU / PU / CTU / CTB / CTU or CTB.

[0266] Regarding palette / predictor palette size

[0267] 10. The size of the palette may vary between video units and another video unit.

[0268] a. In one example, it may vary between a video unit (block / CU / CTU / slice / tile / sub-picture) and another video unit.

[0269] b. In one example, it may depend on the decoding information of the current block and / or neighboring (adjacent or non-adjacent) blocks.

[0270] 11. The size of the palette and / or the predictor palette may depend on the block dimension and / or the quantization parameter.

[0271] 12. The size (or the number of entries therein) of the palette and / or the predictor palette may be different for different color components.

[0272] a. In one example, an indication of the size of the palette and / or the predictor palette for the luminance component and the chrominance component may be signaled explicitly or implicitly.

[0273] b. In one example, an indication of the size of the palette and / or the predictor palette for each color component may be signaled explicitly or implicitly.

[0274] c. In one example, whether signaling notifies the indication of multiple sizes may depend on the use of the dual tree and / or the slice / picture type.

[0275] Signaling of palette

[0276] 13. The compliant bitstream shall satisfy that the number of entries directly signaled for the current block (e.g., num_signaled_palette_entries) shall be in the range of [0, palette_max_size - NumPredictedPaletteEntries], and this range is a closed range including 0 and palette_max_size - NumPredictedPaletteEntries.

[0277] a. How to binarize num_signaled_palette_entries may depend on the allowed range.

[0278] i. Truncated binarization coding and decoding can be used instead of EG-0 th 。

[0279] b. How to binarize num_signaled_palette_entries may depend on the decoded information (e.g., block dimensions).

[0280] Regarding wavefront with 1 - CTU

[0281] 14. It is proposed to re-initialize the predictor palette (e.g., entries and / or sizes) when parsing the CTU syntax ends (e.g., in Clause 7.3.8.2 of VVC), entropy_coding_sync_enabled_flag is equal to 1, and the current CTB is the first in the new CTU row or the current CTB is not in the same tile as its previous CTB.

[0282] a. Alternatively, in addition, maintain PredictorPaletteSizeWpp and PredictorPaletteEntriesWpp to record the updated size and entries of the predictor palette after encoding / decoding the above CTU.

[0283] i. Alternatively, in addition, PredictorPaletteSizeWpp and PredictorPaletteEntriesWpp can be used for encoding / decoding the current block in the current CTU.

[0284] b. In one example, when parsing the CTU syntax in Clause 7.3.8.2 is completed, if entropy_coding_sync_enabled_flag is equal to 1 and CtbAddrInRs % PicWidthInCtbsY is equal to 0 or BrickId[CtbAddrInBs] is not equal to BrickId[CtbAddrRsToBs[CtbAddrInRs - 1]], the storage procedure of the context variables specified in Clause 9.3.2.3 is called, with TableStateIdx0Wpp, TableStateIdx1Wpp, TableMpsValWpp, PredictorPaletteSizeWpp, and PredictorPaletteEntriesWpp (when palette_mode_enabled_flag is equal to 1) as the output.

[0285] Overview

[0286] 15. Whether and / or how the above method is applied may be based on the following:

[0287] a. Video content (such as screen content or natural content)

[0288] b. Messages signaled in DPS / SPS / VPS / PPS / APS / picture header / strip header / slice group header / largest coding unit (LCU) / coding unit (CU) / LCU row / LCU group / TU / PU block / video coding unit

[0289] c. Positions of CU / PU / TU / block / video coding unit

[0290] d. Decoding information of the current block and / or its neighboring blocks

[0291] i. Block dimensions / block shapes of the current block and / or its neighboring blocks

[0292] e. Indication of color format (such as 4:2:0, 4:4:4, RGB, or YUV)

[0293] f. Coding tree structure (such as dual tree or single tree)

[0294] g. Strip / slice group type and / or picture type

[0295] h. Color components (for example, it can be applied only to the luminance component and / or chrominance component)

[0296] i. Temporal layer ID

[0297] j. Profile / level / hierarchy of the standard

[0298] 5. Additional Embodiments

[0299] In the following embodiments, newly added text is shown in bold and underlined italic text. Any deleted text is marked with [[ ]].

[0300] 5.1. Embodiment #1

[0301] 9.3.1 Overview

[0302] This procedure is called when parsing a syntax element using the descriptor ae(v) in Clauses 7.3.8.1 to 7.3.8.12.

[0303] The input to this procedure is a request for the value of the syntax element and the value of the previously parsed syntax element.

[0304] The output of this procedure is the value of the syntax element.

[0305] The initialization procedure specified in Clause 9.3.2 is called when starting to parse one or more of the following:

[0306] 1. The strip segment data syntax specified in Clause 7.3.8.1,

[0307] 2. The CTU syntax specified in Clause 7.3.8.2 and the CTU is the first CTU in the [[slice]], [[slice]]

[0308] 3. The CTU syntax specified in Clause 7.3.8.2, where [[entropy_coding_sync_enabled_flag]] is equal to 1 and the associated luma CTB is the first luma CTB in the CTU row of the [[slice]]. [[slice]]

[0309] The parsing process of the syntax element is as follows:

[0310] When [[cabac_bypass_alignment_enabled_flag]] is equal to 1, for a request for the value of the syntax element for the syntax element coeff_abs_level_remaining[] or coeff_sign_flag[] and escapeDataPresent is equal to 1, the calibration procedure before calibration bypass decoding specified in Clause 9.3.4.3.6 is called.

[0311] For each requested value of the syntax element, the derived binarization specified in Clause 9.3.3 is performed.

[0312] The binarization of the syntax element and the parsed binary bit sequence determine the decoding process, as described in Clause 9.3.4.

[0313] When processing a request for the value of a syntax element for the syntax element pcm_flag and the decoded value of pcm_flag is equal to 1, the decoding engine is initialized after decoding any pcm_alignment_zero_bit and all pcm_sample_luma and pcm_sample_chroma data specified in Clause 9.3.2.6. The storage procedure for context variables is applied as follows:

[0314] – When parsing the CTU syntax in Clause 7.3.8.2 is completed, entropy_coding_sync_enabled_flag is equal to 1, and CtbAddrInRs % PicWidthInCtbsY is equal to 1, or both CtbAddrInRs are greater than 1 and TileId[CtbAddrInTs] is not equal to TileId[CtbAddrRsToTs[CtbAddrInRs - 2]], call the storage procedure for context variables, Rice parameter initialization status, and palette predictor variables specified in Clause 9.3.2.4, with TableStateIdxWpp, TableMpsValWpp, TableStatCoeffWpp (when persistent_rice_adaptation_enabled_flag is equal to 1), PredictorPaletteSizeWpp, and PredictorPaletteEntriesWpp (when palette_mode_enabled_flag is equal to 1) as outputs.

[0315] – When parsing the overall slice segment data syntax in Clause 7.3.8.1 is completed, dependent_slice_segments_enabled_flag is equal to 1 and end_of_slice_segment_flag is equal to 1, call the storage procedure for context variables, Rice parameter initialization status, and palette predictor variables specified in Clause 9.3.2.4, with TableStateIdxDs, TableMpsValDs, TableStatCoeffDs (when persistent_rice_adaptation_enabled_flag is equal to 1), PredictorPaletteSizeDs, and PredictorPaletteEntriesDs (when palette_mode_enabled_flag is equal to 1) as outputs.

[0316] 5.2. Example #2

[0317] 9.3 CABAC parsing process of strip data

[0318] 9.3.1 Overview

[0319] The input to this process is a request for the value of a syntax element and the value of previously parsed syntax elements.

[0320] The output of this process is the value of the syntax element.

[0321] When starting to parse the CTU syntax specified in Clause 7.3.8.2 and one or more of the following conditions are true, the initialization process specified in Clause 9.3.2 is called

[0322] – The CTU is the first CTU in the tile.

[0323] – The value of entropy_coding_sync_enabled_flag is equal to 1, and the CTU is the first CTU in the CTU row of the tile.

[0324] The parsing process of the syntax element is as follows

[0325] For each requested value of the syntax element, the binarization is derived as specified in Subclause 9.3.3.

[0326] The binarization of the syntax element and the sequence of parsed binary bits determine the decoding flow, as described in Subclause 9.3.4.

[0327] The storage process of the context variables is applied as follows

[0328] – When ending the parsing of the CTU syntax in Clause 7.3.8.2, entropy_coding_sync_enabled_flag is equal to 1, and CtbAddrInRs % PicWidthInCtbsY is equal to 0 or BrickId[CtbAddrInBs] is not equal to BrickId[CtbAddrRsToBs[CtbAddrInRs - 1]], the storage process of the context variables specified in Clause 9.3.2.3 is called, where TableStateIdx0Wpp, TableStateIdx1Wpp, and TableMpsValWpp are used as the output.

[0329] 9.3.2 Initialization process

[0330] 9.3.2.1 Overview

[0331] – The output of this process is the initialized CABAC internal variables.

[0332] – The context variables of the arithmetic decoding engine are initialized as follows:

[0333] – If the CTU is the first CTU in the tile, the initialization process of the context variables is called as specified in Clause 9.3.2.2, and the variables PredictorPaletteSize[0 / 1 / 2] are initialized to 0.

[0334] – Otherwise, if entropy_coding_sync_enabled_flag is equal to 1, and CtbAddrInRs % PicWidthInCtbsY is equal to 0 or BrickId[CtbAddrInBs] is not equal to BrickId[CtbAddrRsToBs[CtbAddrInRs - 1]], the following applies:

[0335] – Use the position (x0, y0) of the top - left luma sample of the current CTB to derive the position (xNbT, yNbT) of the top - left luma sample of the spatial neighboring block T( Figure 9 - 2 ) as follows:

[0336] (xNbT,yNbT) = (x0,y0 - CtbSizeY) (9 - 3)

[0337] – Call the derivation process of neighboring block availability specified in Clause 6.4.4, where the position (xCurr, yCurr) is set to be equal to (x0, y0), the neighboring position (xNbY, yNbY) is set to be equal to (xNbT, yNbT), checkPredModeY is set to be equal to FALSE, cIdx is set to be equal to 0 as input, and the output is assigned to availableFlagT.

[0338] – The synchronization process of the context variables is called as follows:

[0339] – If availableFlagT is equal to 1, call the synchronization process of the context variables specified in Clause 9.3.2.4, where TableStateIdx0Wpp, TableStateIdx1Wpp, TableMpsValWpp are used as input, and the variable PredictorPaletteSize is initialized to 0.

[0340] – Otherwise, call the initialization process of the context variables as specified in Clause 9.3.2.2, and the variable PredictorPaletteSize is initialized to 0.

[0341] – Otherwise, the initialization process of the call context variables as specified in Clause 9.3.2.2 is followed, and the variable PredictorPaletteSize is initialized to 0.

[0342] – Initialize the decoding engine registers ivlCurrRange and ivlOffset with 16-bit register precision by calling the initialization process of the arithmetic decoding engine as specified in Subclause 9.3.2.5.

[0343] 9.3.2.3 Storage Process of Context Variables

[0344] The inputs to this process include:

[0345] – CABAC context variables indexed by ctxTable and ctxIdx.

[0346] The outputs to this process include:

[0347] – Variables tableStateSync0, tableStateSync1, and tableMPSSync that contain the values of variables pStateIdx0, pStateIdx1, and valMps used in the initialization process of the context variables, and these context variables are assigned to all syntax elements in Clauses 7.3.8.1 to 7.3.8.11, except end_of_brick_one_bit and end_of_subset_one_bit.

[0348]

[0349] For each context variable, the corresponding entries pStateIdx0, pStateIdx1, and valMps in tables tableStateSync0, tableStateSync1, and tableMPSSync are initialized to the corresponding pStateIdx0, pStateIdx1, and valMps.

[0350]

[0351] Alternatively, the following may apply:

[0352]

[0353] 5.3. Example #3

[0354]

[0355] Alternatively, in the above table can be set to another integer value, such as a fixed value or a predictor palette size.

[0356] 6. Example embodiments of the disclosed technology

[0357] Figure 9 is a block diagram of a video processing device 900. The device 900 can be used to implement one or more methods described herein. The device 900 can be embodied in a smartphone, a tablet computer, a computer, an Internet of Things (IoT) receiver, etc. The device 900 can include one or more processors 902, one or more memories 904, and video processing hardware 906. The processor 902 can be configured to implement one or more methods described in this document. The memory 904 can be used to store data and code for implementing the methods and techniques described herein. The video processing hardware 906 can be used to implement some of the techniques described in this document in hardware circuitry and can be partially or fully part of the processor 902 (e.g., a graphics processing unit core GPU or other signal processing circuitry).

[0358] In this document, the term "video processing" can refer to video encoding, video decoding, video compression, or video decompression. For example, a video compression algorithm can be applied during the conversion from a pixel representation of a video to a corresponding bitstream representation, and vice versa. For example, the bitstream representation of a current video block can correspond to bits that are co-located within the bitstream or scattered at different locations as defined by the syntax. For example, a macroblock can be encoded based on transformed and decoded error residual values and can also be decoded using bits in the header and other fields in the bitstream.

[0359] It will be understood that the disclosed methods and techniques will benefit video encoder and / or decoder embodiments incorporated into a video processing device, such as a smartphone, a laptop computer, a desktop computer, and similar devices, by allowing the use of the techniques disclosed in this document.

[0360] Figure 10 is a block diagram of an example video processing system 1000 that can implement the various techniques disclosed herein. Various embodiments can include some or all of the components of the system 1000. The system 1000 can include an input 1002 for receiving video content. The video content can be received in a raw or uncompressed format (e.g., 8 or 10-bit multi-component pixel values) or can be received in a compressed or encoded format. The input 1002 can represent a network interface, a peripheral bus interface, or a storage interface. Examples of network interfaces include wired interfaces such as Ethernet, Passive Optical Network (PON), etc., and wireless interfaces such as Wi-Fi or cellular interfaces.

[0361] System 1000 may include an encoding / decoding component 1004, which may implement various encoding / decoding or coding methods described in this document. The encoding / decoding component 1004 may reduce the average bit rate of a video from the input 1002 to the output of the encoding / decoding component 1004 to produce an encoded / decoded representation of the video. Thus, encoding / decoding techniques are sometimes referred to as video compression or video transcoding techniques. The output of the encoding / decoding component 1004 may be stored or transmitted via a connected communication, represented by component 1006. The stored or transmitted bitstream (or encoded / decoded) representation of the video received at the input 1002 may be used by component 1008 to generate pixel values or a displayable video that is sent to the display interface 1010. The process of generating a user-visible video from the bitstream representation is sometimes referred to as video decompression. Additionally, although certain video processing operations are referred to as "encoding / decoding" operations or tools, it will be understood that encoding / decoding tools or operations are used at the encoder, and the corresponding decoding tools or operations that reverse the encoding / decoding results will be performed by the decoder.

[0362] Examples of a peripheral bus interface or a display interface may include Universal Serial Bus (USB), High-Definition Multimedia Interface (HDMI), or Displayport, etc. Examples of a storage interface include Serial Advanced Technology Attachment (SATA), PCI, IDE interface, etc. The techniques described in this document may be embodied in various electronic devices, such as mobile phones, laptops, smartphones, or other devices capable of performing digital data processing and / or video display.

[0363] Figure 11 is a flowchart of an example method 1100 for video processing. At 1110, method 1100 includes performing a conversion between a video block in a video unit and an encoded / decoded representation of the video block using a palette mode, wherein, during the conversion, a predictor palette is used to predict the current palette information of the video block, and further wherein, the predictor palette is selectively reset before the conversion between the video block and the bitstream representation of the video block.

[0364] Some embodiments may be described using the following clause-based format.

[0365] 1. A method for video processing, comprising:

[0366] performing a conversion between a video block in a video unit and an encoded / decoded representation of the video block using a palette mode, wherein, during the conversion, a predictor palette is used to predict the current palette information of the video block, and further wherein, the predictor palette is selectively reset before the conversion between the video block and the bitstream representation of the video block.

[0367] 2. The method according to clause 1, wherein the video unit comprises one of the following: one or more coding tree units, one or more coding tree blocks, a sub-region of a coding tree unit or a coding tree block, or a view of a coding tree block row / slice / tile / sub-picture / coding tree unit.

[0368] 3. The method according to any one of clauses 1-2, wherein the delayed wavefront parallel processing is disabled during the conversion.

[0369] 4. The method according to clause 3, wherein the entropy_coding_sync_enabled_flag is set to be equal to 0.

[0370] 5. The method according to clause 1, wherein the video unit is a row of chrominance coding tree units.

[0371] 6. The method according to clause 5, wherein the predictor palette is reset before decoding the first chrominance coding tree block (CTB) in the new chrominance CTU row.

[0372] 7. The method according to clause 5, wherein when applying the dual coding tree and the current split of the dual coding tree is a chrominance coding tree unit, the predictor palette is reset.

[0373] 8. The method according to clause 1, wherein the size of the predictor palette is reset to zero.

[0374] 9. The method according to clause 1, wherein the size of the predictor palette is reset to the number of entries in the sequence palette predictor initializer or the maximum number of allowed entries.

[0375] 10. The method according to clause 9, wherein the sequence palette predictor initializer is used to initialize the palette predictor before applying it to the video unit.

[0376] 11. The method according to clause 1, wherein when the entropy_coding_sync_enabled_flag is set to 1, the palette predictor applied to the previous video block is re-initialized before applying it to the video unit.

[0377] 12. The method according to clause 1, wherein the update of the predictor palette is prohibited based on the coding information associated with the video unit.

[0378] 13. The method according to clause 12, wherein the coding information includes the dimension of the video unit.

[0379] 14. The method according to clause 13, wherein updating the predictor palette is prohibited based on the dimension of the video unit reaching one or more threshold conditions.

[0380] 15. The method according to clause 14, wherein the one or more threshold conditions are predefined.

[0381] 16. The method according to clause 14, wherein the one or more threshold conditions are signaled explicitly or implicitly in the codec representation of the video unit.

[0382] 17. A method for video processing, comprising:

[0383] Performing a conversion between a video block in a video unit and a codec representation of the video block using a palette mode, wherein during the conversion, a predictor palette is used to predict the current palette information of the video block, and further, wherein when multiple codec units of the video unit have a common shared area, the predictor palette is a shared predictor palette.

[0384] 18. The method according to clause 17, wherein the shared area is associated with any one of the following: TT partition, BT partition, QT partition.

[0385] 19. The method according to clause 17, wherein the shared predictor palette is constructed before being applied to multiple codec units.

[0386] 20. The method according to clause 17, wherein an indication of the use of the shared predictor palette is signaled explicitly or implicitly in the codec representation associated with the first palette codec unit of the shared area.

[0387] 21. The method according to clause 17, further comprising:

[0388] Skipping updating the shared predictor palette after the codec unit applied to multiple codec units.

[0389] 22. A method for video processing, comprising:

[0390] Performing a conversion between a video block in a video unit and a codec representation of the video block using a palette mode, wherein during the conversion, a predictor palette is used to predict the current palette information of the video block, and further, wherein the size of the predictor palette is adaptively changed according to one or more conditions.

[0391] 23. The method according to clause 22, wherein one or more conditions are associated with at least the following: the size of the previous palette information, the dimensions of the video unit, the content of the video unit, the color format of the video unit, the color components of the video unit, the codec tree structure of the video block, the relative position of the video block in the codec representation, the temporal layer ID of the video block, the slice / tile group type and / or picture type of the video block, or the profile / level / tier of the video block.

[0392] 24. A method for video processing, comprising:

[0393] Performing a conversion between a video block in a video unit and a codec representation of the video block using a palette mode, wherein during the conversion, a predictor palette is used to predict the current palette information of the video block, and further wherein the predictor palette is updated based on the size or number of entries in the predictor palette.

[0394] 25. The method according to clause 24, wherein the size of the predictor palette is updated from a previous video block to a current video block.

[0395] 26. The method according to clause 24, wherein the size of the predictor palette is signaled implicitly or explicitly in the codec representation.

[0396] 27. The method according to clause 24, wherein the size of the predictor palette depends on one or more of the following: the dimensions of the video block, the quantization parameter of the video block, or one or more color components of the video block.

[0397] 28. A method for video processing, comprising:

[0398] Performing a conversion between a video block in a video unit and a codec representation of the video block using a palette mode, wherein during the conversion, a predictor palette is used to predict the current palette information of the video block, and further wherein the entries of the predictor palette are reordered or modified.

[0399] 29. The method according to clause 28, wherein the entries of the predictor palette are reordered or modified when the entropy_coding_sync_enabled_flag is equal to 1.

[0400] 30. The method according to clause 28, wherein the entries of the predictor palette are reordered or modified when the end of the codec tree unit syntax is encountered.

[0401] 31. The method according to clause 28, wherein the entries of the predictor palette are reordered or modified when the current CTB is the first CTB in a new CTU row or when the current CTB and the previous CTB are not in the same tile.

[0402] 32. A method for video processing, comprising:

[0403] Performing a conversion between a video block in a video unit and an encoded / decoded representation of the video block using a palette mode, wherein during the conversion, a predictor palette is used to predict the current palette information of the video block, and further wherein the use of the predictor palette is indicated by maintaining a counter that tracks the number of times the predictor palette is used.

[0404] 33. The method according to any of the preceding clauses, wherein enabling or disabling the predictor palette is associated with at least one of the following: the size of the previous palette information, the dimensions of the video block, the content of the video block, the color format of the video block, the color components of the video block, the codec tree structure of the video block, the relative position of the video block in the encoded / decoded representation, the temporal layer ID of the video block, the slice / group type and / or picture type of the video block, or the profile / level / tier of the video block.

[0405] 34. The method according to any of the preceding clauses, wherein more than one predictor palette is used during the conversion.

[0406] 35. A video decoding apparatus, comprising a processor configured to implement the method recited in one or more of clauses 1 to 34.

[0407] 36. A video encoding apparatus, comprising a processor configured to implement the method recited in one or more of clauses 1 to 34.

[0408] 37. A computer program product having computer code stored thereon, which when executed by a processor causes the processor to implement the method recited in any one of clauses 1 to 34.

[0409] 38. A method, apparatus, or system described in this document.

[0410] Figure 12 is a flowchart representation of a method 1200 for video processing according to the present technology. At operation 1210, method 1200 includes performing a conversion between a current block of a video and an encoded / decoded representation of the video using a palette mode, in which a palette of representative sample values is used to encode and decode the current block. During the conversion, a predictor palette is used to predict the palette of representative sample values, and based on the characteristics of the current block, updating of the predictor palette is prohibited according to a rule after the conversion of the current block.

[0411] In some embodiments, the characteristics of the current block include coding and decoding information associated with the current block. In some embodiments, the characteristics of the current block include the dimensions of the current block. In some embodiments, the rule specifies that updating the predictor palette is prohibited when the width of the current block is less than or equal to a first threshold and the height of the current block is less than or equal to a second threshold. In some embodiments, the rule specifies that updating the predictor palette is prohibited when the height of the current block is less than or equal to the first threshold. In some embodiments, the rule specifies that updating the predictor palette is prohibited when the width of the current block is greater than or equal to the first threshold and the height of the current block is greater than or equal to the second threshold. In some embodiments, the rule specifies that updating the predictor palette is prohibited when the height of the current block is greater than or equal to the first threshold.

[0412] In some embodiments, the first threshold or the second threshold is predefined or signaled in the coded representation. In some embodiments, the first threshold is 4, 16, or 1024. In some embodiments, the second threshold is 4, 16, or 1024. In some embodiments, the first threshold or the second threshold is based on the color components of the current block.

[0413] Figure 13 is a flowchart representation of a method 1300 for video processing according to the present technology. At operation 1310, method 1300 includes performing a conversion between a current block of a video and a coded representation of the video using a palette mode, in which a palette of representative sample values is used to code and decode the current block. During the conversion, a predictor palette is used to predict the palette of representative sample values, and it is determined whether to change the predictor palette based on the color components of the current block.

[0414] In some embodiments, changing the predictor palette includes updating, initializing, or resetting the predictor palette. In some embodiments, the color components include luminance or chrominance components. In some embodiments, the predictor palette includes information corresponding to the color components of the current block. In some embodiments, the predictor palette includes information corresponding to all color components of the current block. In some embodiments, the predictor palette includes information corresponding to two chrominance components of the current block.

[0415] Figure 14 is a flowchart representation of a method 1400 for video processing according to the present technology. At operation 1410, method 1400 includes performing a conversion between a current block in a video unit of a video and a coded representation of the video using a palette mode, in which a palette of representative sample values is used to code and decode the current block. During the conversion, multiple predictor palettes are used to predict the palette of representative sample values.

[0416] In some embodiments, the predictor palette of the current block is updated based at least on a global palette and a local palette. In some embodiments, multiple predictor palettes are associated with K blocks in a video unit that has been encoded or decoded using a palette mode. In some embodiments, palettes for different color components are determined based on different predictor palettes of the multiple predictor palettes. In some embodiments, the multiple predictor palettes are reset or reinitialized before the transformation of the first block in a video unit or after the transformation of the last block in a previously transformed video unit. In some embodiments, the index of the predictor palette of the multiple predictor palettes is signaled in a coding unit, a prediction unit, a coding tree unit, a coding tree block, a sub-region of a coding tree unit, or a sub-region of a coding tree block in a coded representation.

[0417] Figure 15 is a flow chart representation of a method 1500 for video processing according to the present technology. At operation 1510, method 1500 includes performing a transformation between a current block in a video unit of a video and a coded representation of the video using a palette mode, in which a palette of representative sample values is used to encode or decode the current block. During the transformation, a predictor palette is used to predict the palette of representative sample values. According to a rule, the predictor palette is reset or reinitialized before the transformation of the first block in a video unit or after the transformation of the last video block in a previous video unit.

[0418] In some embodiments, a video unit includes a sub-region of a coding tree unit, a virtual pipeline data unit, one or more coding tree units, a coding tree block, one or more coding units, a coding tree unit row, a slice, a tile, a sub-picture, or a view of a video. In some embodiments, the rule that the predictor palette is reset or reinitialized applies to a video unit regardless of whether wavefront parallel processing of multiple video units is enabled. In some embodiments, a video unit includes a coding tree unit row corresponding to a chrominance component. In some embodiments, the first block includes a first coding tree block corresponding to a chrominance component in a coding tree unit row. In some embodiments, the rule that the predictor palette is reset or reinitialized applies to a video unit in the case where a dual-tree segmentation is applied and the current segmentation tree is a coding tree corresponding to a chrominance component. In some embodiments, the size of the predictor palette is reset or reinitialized to 0. In some embodiments, the size of the predictor palette is reset or reinitialized to the number of entries in a sequence palette predictor initializer or the maximum number of entries allowed in a predictor palette signaled in a coded representation.

[0419] In some embodiments, before converting a new video unit, the predictor palette is further reset or re-initialized. In some embodiments, in the case of enabling wavefront parallel processing of multiple video units, the predictor palette for converting the current coding tree block or the current coding tree unit is determined based on the converted coding tree blocks or coding tree units.

[0420] Figure 16 is a flowchart representation of a video processing method 1600 according to the present technology. At operation 1610, method 1600 includes performing a conversion between a video unit of a video and a coded representation of the video using a palette mode. The video unit includes a plurality of blocks. During the conversion, a shared predictor palette is used by all of the plurality of blocks to predict the palette of representative sample values for each of the plurality of blocks in the palette mode.

[0421] In some embodiments, a ternary tree segmentation is applied to the video unit, and wherein the shared predictor palette is used for a video unit having a dimension of 16×4 or 4×16. In some embodiments, a binary tree segmentation is applied to the video unit, and the shared predictor palette is used for a video unit having a dimension of 8×4 or 4×8. In some embodiments, a quadtree segmentation is applied to the video unit, and the shared predictor palette is used for a video unit having a dimension of 8×8. In some embodiments, the shared predictor palette is constructed before the conversion of all of the plurality of blocks within the video unit.

[0422] In some embodiments, for a first coded block of a plurality of blocks in a region, an indication of a predicted entry in the shared predictor palette is signaled in the coded representation. In some embodiments, for the remainder of the plurality of blocks in the region, the indication of the predicted entry in the shared predictor palette is omitted in the coded representation. In some embodiments, the update of the shared predictor palette is skipped after the conversion of one of the plurality of blocks in the region.

[0423] Figure 17 is a flowchart representation of a video processing method 1700 according to the present technology. At operation 1710, method 1700 includes performing a conversion between a current block of a video and a coded representation of the video using a palette mode, in which a palette of representative sample values is used to code the current block. During the conversion, the predictor palette is used to predict the palette of representative sample values, and a counter indicating the usage frequency of the corresponding entry is maintained for each entry of the predictor palette.

[0424] In some embodiments, for a new entry to be added to the predictor palette, the counter is set to K, where K is an integer. In some embodiments, K = 0. In some embodiments, whenever the corresponding entry is reused during the conversion of the current block, the counter is incremented by N, where N is a positive integer. In some embodiments, N = 1.

[0425] In some embodiments, before the predictor palette is used for transformation, the entries of the predictor palette are reordered according to a rule. In some embodiments, the rule stipulates that the entries of the predictor palette are reordered according to the codec information of the current sample. In some embodiments, the rule stipulates that the entries of the predictor palette are reordered according to the counter of each corresponding entry of the entries in the predictor palette.

[0426] In some embodiments, a second counter is used to indicate the occurrence frequency of a sample. In some embodiments, in the predictor palette, a first sample with a higher occurrence frequency is located before a second sample with a lower occurrence frequency. In some embodiments, the predictor palette is updated according to a rule using the escape samples in the current block. In some embodiments, the rule stipulates that the predictor palette is updated using the escape samples when a condition is met. In some embodiments, the condition is met when the predictor palette is not full after inserting the current block of the current block.

[0427] Figure 18 is a flowchart representation of a video processing method 1800 according to the present technology. At operation 1810, method 1800 includes performing a transformation between a current block of a video and a codec representation of the video using a palette mode, in which a palette of representative sample values is used to codec the current block to predict the palette of representative sample values of the current block. The number of palette entries signaled in the codec representation is in the range of [0, the maximum allowable size of the palette - the number of palette entries derived during transformation], which is a closed range including 0 and the maximum allowable size of the palette - the number of palette entries derived during transformation.

[0428] In some embodiments, the number of palette entries signaled in the codec representation is binarized based on this range. In some embodiments, a truncated binary codec process is used to binarize the number of entries signaled in the codec representation. In some embodiments, the number of entries signaled in the codec representation is binarized based on the characteristics of the current block. In some embodiments, the characteristics include the dimensions of the current block.

[0429] Figure 19 is a flowchart representation of a video processing method 1900 according to the present technology. At operation 1910, method 1900 includes performing a transformation between a current block in a video unit of a video and a codec representation of the video using a palette mode, in which a palette of representative sample values is used to codec the current block. During the transformation, a predictor palette is used to predict the palette of representative sample values, and wherein the size of the predictor palette is adaptively adjusted according to a rule.

[0430] In some embodiments, the size of the predictor palette in the dual-tree split is different from that in the single-tree split. In some embodiments, a video unit includes a block, a coding / decoding unit, a coding / decoding tree unit, a slice, a tile, or a sub-picture. In some embodiments, the rule specifies that the predictor palette has a first size for a video unit and a different second size for the transformation of a subsequent video unit. In some embodiments, the rule specifies that the size of the predictor palette is adjusted according to the size of the current palette for the transformation. In some embodiments, the size of the predictor palette is equal to the size of the current palette determined after the transformation of the current block. In some embodiments, the size of the predictor palette is equal to the size of the current palette determined after the transformation of the current block plus or minus an offset, where the offset is an integer. In some embodiments, the offset is signaled in the coded representation. In some embodiments, the offset is derived during the transformation.

[0431] In some embodiments, the rule specifies a predefined size S for the predictor palette of the current block, and the rule further specifies that the size of the predictor palette is adjusted according to the size of the current block. In some embodiments, when the size of the current block is less than or equal to T, the size of the predictor palette is adjusted to be less than the predefined size S, where T and S are integers. In some embodiments, the first K entries in the predictor palette are used for the transformation, where K is an integer and K ≤ S. In some embodiments, a subsampled predictor palette with a size less than the predefined size S is used for the transformation. In some embodiments, when the size of the current block is greater than or equal to T, the size of the predictor palette is adjusted to the predefined size S.

[0432] In some embodiments, K or T is determined based on characteristics of the video. In some embodiments, characteristics of the video include the content of the video. In some embodiments, characteristics of the video include information signaled in any of the following: a decoder parameter set, a slice parameter set, a video parameter set, a picture parameter set, an adaptive parameter set, a picture header, a slice header, a slice group header, a largest coding unit (LCU), a coding / decoding unit, an LCU row, an LCU group, a transform unit, a picture unit, or a video coding / decoding unit in the coded representation. In some embodiments, characteristics of the video include the position of a coding / decoding unit, a picture unit, a transform unit, a block, or a video coding / decoding unit within the video. In some embodiments, characteristics of the video include an indication of the color format of the video. In some embodiments, characteristics of the video include the coding / decoding tree structure applicable to the video. In some embodiments, characteristics of the video include the slice type, slice group type, or picture type of the video. In some embodiments, characteristics of the video include the color components of the video. In some embodiments, characteristics of the video include the temporal layer identifier of the video. In some embodiments, characteristics of the video include the profile, level, or tier of the video standard.

[0433] In some embodiments, the rule stipulates that the size of the predictor palette is adjusted according to one or more counters of each entry in the predictor palette. In some embodiments, during the transformation, entries with a counter less than a threshold T, where T is an integer, are discarded. In some embodiments, the entry with the smallest counter is discarded until the size of the predictor palette is less than a threshold T, where T is an integer.

[0434] In some embodiments, the rule stipulates that the predictor palette is updated based only on the current palette used for the transformation. In some embodiments, the predictor palette is updated to the current palette of the subsequent block after the transformation.

[0435] Figure 20 FIG. 2000 is a flowchart of a method 2000 for video processing according to the present technology. The method 2000 includes, at operation 2010, performing a transformation between a current block in a video unit of a video and an encoded / decoded representation of the video using a palette mode, in which a palette of representative sample values is used to encode and decode the current block. During the transformation, a predictor palette is used to predict the palette of representative sample values, and the size of the palette of representative samples or the predictor palette is determined according to a rule that allows the size to vary between video units of the video.

[0436] In some embodiments, a video unit includes a block, a coding / decoding unit, a coding / decoding tree unit, a block, a tile, or a sub-picture. In some embodiments, the size of the palette of representative samples or the predictor palette is further determined based on characteristics of the current block or neighboring blocks of the current block. In some embodiments, the characteristics include the dimensions of the current block or neighboring blocks. In some embodiments, the characteristics at least include the quantization parameter of the current block or neighboring blocks. In some embodiments, the characteristics include the color components of the current block or neighboring blocks.

[0437] In some embodiments, different sizes of the palette of representative samples or the predictor palette are used for different color components. In some embodiments, the size of the palette of representative samples or the predictor palette of the luminance component and the chrominance component is indicated in the encoded / decoded representation. In some embodiments, the size of the palette of representative samples or the predictor palette of each color component is indicated in the encoded / decoded representation. In some embodiments, the signaling of different sizes in the encoded / decoded representation is based on the use of a dual-tree segmentation, a slice type, or a picture type used for the transformation.

[0438] Figure 21It is a flowchart representation of a video processing method 2100 according to the present technology. At operation 2110, method 2100 includes performing a conversion between a current block in a video unit of a video and an encoded / decoded representation of the video using a palette mode, in which a palette of representative sample values is used to encode / decode the current block. During the conversion, a predictor palette is used to predict the palette of representative sample values. The predictor palette is reinitialized when a condition is met, where the condition is met when the video unit is the first video unit in a video unit row and an encoded / decoded representation includes a syntax element indicating that wavefront parallel processing is enabled for the video unit.

[0439] In some embodiments, a video unit includes a coding tree unit or a coding tree block. In some embodiments, the condition is met if the current block and a previous block are not in the same tile. In some embodiments, after the conversion of a video unit, at least one syntax element is maintained to record the size of the predictor palette and / or the number of entries in the predictor palette. In some embodiments, at least one syntax element is used for the conversion of the current block.

[0440] In some embodiments, a storage procedure for context variables of a video is invoked in case (1) the current block is in the first column of a picture or case (2) the current block and a previous block are not in the same tile. In some embodiments, the output of the storage procedure includes at least the size of the predictor palette or the number of predictor palette entries.

[0441] In some embodiments, the applicability of the one or more methods described above is based on characteristics of the video. In some embodiments, characteristics of the video include the content of the video. In some embodiments, characteristics of the video include information signaled in any of the following: decoder parameter sets, slice parameter sets, video parameter sets, picture parameter sets, adaptive parameter sets, picture headers, slice headers, picture group headers, largest coding units (LCUs), coding units, LCU rows, LCU groups, transform units, picture units, or video coding units in a coded representation. In some embodiments, characteristics of the video include the position of a coding unit, picture unit, transform unit, block, or video coding unit within the video. In some embodiments, characteristics of the video include characteristics of a current block or neighboring blocks of the current block. In some embodiments, characteristics of the current block or neighboring blocks of the current block include the dimensions of the current block or the dimensions of the neighboring blocks of the current block. In some embodiments, characteristics of the video include an indication of the color format of the video. In some embodiments, characteristics of the video include a coding tree structure applicable to the video. In some embodiments, characteristics of the video include the slice type, group type, or picture type of the video. In some embodiments, characteristics of the video include the color components of the video. In some embodiments, characteristics of the video include the temporal layer identifier of the video. In some embodiments, characteristics of the video include the profile, level, or tier of the video standard.

[0442] In some embodiments, the transformation includes encoding the video into a coded representation. In some embodiments, the transformation includes decoding the coded representation to generate pixel values of the video.

[0443] Some embodiments of the disclosed techniques include making a decision or determination to enable a video processing tool or mode. In one example, when a video processing tool or mode is enabled, the encoder will use or implement the tool or mode in the processing of video blocks, but may not necessarily modify the resulting bitstream based on the use of the tool or mode. That is, the transformation from video blocks to the bitstream representation of the video will use the video processing tool or mode when the video processing tool or mode is enabled based on the decision or determination. In another example, when a video processing tool or mode is enabled, the decoder will process the bitstream knowing that the bitstream has been modified based on the video processing tool or mode. That is, the transformation from the bitstream representation of the video to video blocks will be performed using the video processing tool or mode enabled based on the decision or determination.

[0444] Some embodiments of the disclosed technology include making a decision or determination to disable a video processing tool or mode. In one example, when a video processing tool or mode is disabled, the encoder will not use the tool or mode when converting video blocks into a bitstream representation of the video. In another example, when a video processing tool or mode is disabled, the decoder will process the bitstream knowing that the bitstream has not been modified using the video processing tool or mode enabled based on the decision or determination.

[0445] The disclosed and other solutions, examples, embodiments, modules, and functional operations described in this document can be implemented in digital electronic circuitry or in computer software, firmware, or hardware, including the structures disclosed in this document and their structural equivalents, or combinations of one or more of them. The disclosed embodiments and other embodiments can be implemented as one or more computer program products, e.g., one or more modules of computer program instructions encoded on a computer-readable medium for execution by, or to control the operation of, a data processing apparatus. The computer-readable medium can be a machine-readable storage device, a machine-readable storage substrate, a memory device, a composition of matter affecting a machine-readable propagated signal, or a combination of one or more of them. The term “data processing apparatus” encompasses all apparatus, devices, and machines for processing data, e.g., including programmable processors, computers, or multiple processors or computers. In addition to hardware, the apparatus can also include code that creates an execution environment for the computer programs being discussed, e.g., code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them. A propagated signal is an artificially generated signal, e.g., a machine-generated electrical, optical, or electromagnetic signal that is generated for encoding information for transmission to a suitable receiver apparatus.

[0446] A computer program (also called a program, software, software application, script, or code) can be written in any form of programming language, including a compiled or interpreted language, and can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. Programs can be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the relevant program, or in multiple coordinated files (e.g., files that store one or more modules, subroutines, or portions of code). A computer program can be deployed to execute on one computer or to execute on multiple computers located at one site or distributed across multiple sites and interconnected by a communication network.

[0447] The processes and logical flows described in this document can be executed by one or more programmable processors that execute one or more computer programs to perform functions by operating on input data and generating output. The processing and logical flows can also be executed by special-purpose logic circuitry, such as an FPGA (Field Programmable Gate Array) or ASIC (Application Specific Integrated Circuit), and the apparatus can also be implemented as special-purpose logic circuitry.

[0448] By way of example, processors suitable for the execution of a computer program include both general and special purpose microprocessors, and any one or more processors of any type of digital computer. Generally, a processor will receive instructions and data from a read only memory or a random access memory or both. The essential elements of a computer are a processor for executing instructions and one or more memory devices for storing instructions and data. Generally, a computer will also include, or be operatively coupled to receive data from or transfer data to, one or more mass storage devices for storing data, such as magnetic disks, magneto-optical disks, or optical disks. However, a computer need not have such devices. Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media and memory devices, including by way of example semiconductor memory devices, such as EPROM, EEPROM, and flash memory devices; magnetic disks, such as internal hard disks or removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks. The processor and the memory can be supplemented by, or incorporated in, special-purpose logic circuitry.

[0449] Although this patent document contains many details, these details should not be construed as limiting the scope of any subject matter or of what may be claimed, but rather as descriptions of features specific to particular embodiments of particular technologies. Certain features described in this patent document in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment can also be implemented separately in multiple embodiments or in any suitable sub-combination. Moreover, although the above features may be described as acting in certain combinations and even initially claimed as such, in some cases, one or more features from a claimed combination can be excluded from the combination, and the claimed combination can be directed to a sub-combination or a variant of a sub-combination.

[0450] Similarly, although operations are depicted in the drawings in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. In addition, the separation of various system components in the embodiments described in this patent document should not be understood as requiring such separation in all embodiments.

[0451] Only some embodiments and examples are described, and other embodiments, enhancements, and variations may be made based on what is described and illustrated in this patent document.

Claims

1. A video processing method, comprising: For the conversion between the current block of a video and the bitstream of the video, determining to apply a palette prediction mode to the current block, wherein, in the prediction mode, the reconstructed samples are represented by a set of representative color values, and the set of representative color values includes at least one of the following: 1) a palette predictor, 2) escape samples, or 3) palette information included in the bitstream; Constructing a current palette of the current block based on a palette prediction table, wherein the current palette is used to derive the reconstructed samples of the current block, and the palette prediction table includes three color components; Performing the conversion based on the current palette; and Determining whether to update the palette prediction table based on the characteristics of the current block, wherein the palette prediction table is updated based on the current palette; The characteristics of the current block include color components, width, and height; When the current block at least satisfies that the width of the current block is not greater than a first threshold or the height of the current block is not greater than a second threshold, disabling the update process of the palette prediction table, wherein the update process includes: (1) inserting an entry of the current block, and (2) after determining that the palette prediction table is not full, inserting non-referenced entries for previously decoded blocks from the palette prediction table; When the current block at least satisfies that the current block is a luma block and the size of the current block is greater than 16, allowing the palette prediction table to be updated, wherein the update includes a reset process; and When the current block is a luma block of which the tree type is a binary tree, constructing palettes of different sizes for the current block and the chroma block corresponding to the current block.

2. The method according to claim 1, wherein In the case where a local binary tree is applied to the current block and the current block is a chroma block, prohibiting the update of the palette prediction table.

3. The method according to claim 1, wherein, In the case where a single tree is applied to the current block, the palette prediction table includes three color components, wherein, in the case where a binary tree is applied to the current block and the current block is a chroma block, the palette prediction table includes two chroma color components, and wherein, in the case where a binary tree is applied to the current block and the current block is a luma block, the palette prediction table includes one color component.

4. The method according to claim 1, wherein The conversion includes encoding the current block into the bitstream.

5. The method according to claim 1, wherein The conversion includes decoding the current block from the bitstream.

6. A device for processing video data, the device comprising a processor and a non-transitory memory storing instructions thereon, wherein, The instructions, when run by the processor, cause the processor to: For the conversion between the current block of a video and the bitstream of the video, determining to apply a palette prediction mode to the current block, wherein, in the prediction mode, the reconstructed samples are represented by a set of representative color values, and the set of representative color values includes at least one of the following: 1) a palette predictor, 2) escape samples, or 3) palette information included in the bitstream; Constructing a current palette of the current block based on a palette prediction table, wherein the current palette is used to derive the reconstructed samples of the current block, and the palette prediction table includes three color components; Performing the conversion based on the current palette; and Determine whether to update the palette prediction table based on the characteristics of the current block, wherein the palette prediction table is updated based on the current palette; the characteristics of the current block include color components, width, and height; when the current block at least satisfies that the width of the current block is not greater than a first threshold or the height of the current block is not greater than a second threshold, disable the update process of the palette prediction table, wherein the update process includes: (1) inserting an entry of the current block, and (2) after determining that the palette prediction table is not full, inserting unreferenced entries for previously decoded blocks from the palette prediction table; when the current block at least satisfies that the current block is a luminance block and the size of the current block is greater than 16, allow updating the palette prediction table, wherein the update includes a reset process; and when the current block is a luminance block with a tree type of double tree, construct palettes of different sizes for the current block and the chrominance block corresponding to the current block.

7. The apparatus according to claim 6, wherein In the case where the local double tree is applied to the current block and the current block is a chrominance block, prohibit updating the palette prediction table.

8. A non-transitory computer-readable storage medium having instructions stored thereon, the instructions causing a processor to: For the conversion between the current block of the video and the bitstream of the video, it is determined to apply a palette prediction mode to the current block, wherein, In the prediction mode, the reconstructed samples are represented by a set of representative color values, and the set of representative color values includes at least one of the following: 1) a palette predictor, 2) escape samples, or 3) palette information included in the bitstream; Construct the current palette of the current block based on the palette prediction table, wherein the current palette is used to derive the reconstructed samples of the current block, and the palette prediction table includes three color components; Perform the conversion based on the current palette; and Determine whether to update the palette prediction table based on the characteristics of the current block, wherein the palette prediction table is updated based on the current palette; the characteristics of the current block include color components, width, and height; when the current block at least satisfies that the width of the current block is not greater than a first threshold or the height of the current block is not greater than a second threshold, disable the update process of the palette prediction table, wherein the update process includes: (1) inserting an entry of the current block, and (2) after determining that the palette prediction table is not full, inserting unreferenced entries for previously decoded blocks from the palette prediction table; when the current block at least satisfies that the current block is a luminance block and the size of the current block is greater than 16, allow updating the palette prediction table, wherein the update includes a reset process; and when the current block is a luminance block with a tree type of double tree, construct palettes of different sizes for the current block and the chrominance block corresponding to the current block.

9. A non-transitory computer-readable recording medium storing a bitstream of a video generated by a method executed by a video processing device, wherein, The method includes: For the conversion between the current block of a video and the bitstream of the video, determine to apply a palette prediction mode to the current block, where in the prediction mode, reconstructed samples are represented by a set of representative color values, and the set of representative color values includes at least one of the following: 1) a palette predictor, 2) escape samples, or 3) palette information included in the bitstream; Construct a current palette of the current block based on a palette prediction table, where the current palette is used to derive the reconstructed samples of the current block, and the palette prediction table includes three color components; Generate the bitstream based on the current palette; and Determine whether to update the palette prediction table based on the characteristics of the current block, where the palette prediction table is updated based on the current palette; the characteristics of the current block include color components, width, and height; When the current block at least satisfies that the width of the current block is not greater than a first threshold or the height of the current block is not greater than a second threshold, disable the update process of the palette prediction table, where the update process includes: (1) inserting an entry of the current block, and (2) after determining that the palette prediction table is not full, inserting un-referenced entries for previously decoded blocks from the palette prediction table; When the current block at least satisfies that the current block is a luma block and the size of the current block is greater than 16, allow updating the palette prediction table, where the update includes a reset process; and When the current block is a luma block with a tree type of double tree, construct palettes of different sizes for the current block and the chroma block corresponding to the current block.

10. A method for storing a bitstream of a video, including: For the conversion between the current block of a video and the bitstream of the video, determine to apply a palette prediction mode to the current block, where in the prediction mode, reconstructed samples are represented by a set of representative color values, and the set of representative color values includes at least one of the following: 1) a palette predictor, 2) escape samples, or 3) palette information included in the bitstream; Construct a current palette of the current block based on a palette prediction table, where the current palette is used to derive the reconstructed samples of the current block, and the palette prediction table includes three color components; Generate the bitstream based on the current palette; Store the bitstream in a non-transitory computer-readable recording medium; and Determine whether to update the palette prediction table based on the characteristics of the current block, where the palette prediction table is updated based on the current palette; the characteristics of the current block include color components, width, and height; When the current block at least satisfies that the width of the current block is not greater than a first threshold or the height of the current block is not greater than a second threshold, disable the update process of the palette prediction table, where the update process includes: (1) inserting an entry of the current block, and (2) after determining that the palette prediction table is not full, inserting un-referenced entries for previously decoded blocks from the palette prediction table; When the current block satisfies at least that the current block is a luminance block and the size of the current block is greater than 16, updating of the palette prediction table is allowed, where the updating includes a reset process; and When the current block is a luminance block with a tree type of double tree, palettes of different sizes are constructed for the current block and the chrominance block corresponding to the current block.

Images