Palette Mode and Intra Block Copy Prediction

Abstract

A video processing method is described. The method includes: for the conversion between a video unit of a video and the codec representation of the video, determining, according to a rule, a maximum number of entries of a palette of representative values to be used during the conversion; and performing the conversion using the palette, and wherein the rule specifies the maximum number of entries according to the characteristics of the video unit.

Description

[0001] Cross - Reference to Related Applications

[0002] This application is a national stage application in China of international patent application No. PCT / US2020 / 034913 filed on May 28, 2020, and this application claims the priority and benefits of international patent application No. PCT / CN2019 / 089506 filed on May 31, 2019. The entire disclosure of the foregoing application is incorporated herein by reference as part of the disclosure of this application. Technical Field

[0003] This application document relates to video and image encoding and decoding technologies. Background Art

[0004] Digital video occupies the largest bandwidth usage on the Internet and other digital communication networks. As the number of connected user devices capable of receiving and displaying video increases, the bandwidth demand for digital video use is expected to continue to grow. Summary of the Invention

[0005] The disclosed technology can be implemented by video or image decoder or encoder embodiments, where a palette coding / decoding mode is used for encoding or decoding video.

[0006] In one example aspect, a video processing method is disclosed. The method includes: for the conversion between a video unit of a video and the coded / decoded representation of the video, determining a maximum number of entries of a palette of representative values used during the conversion according to a rule; and performing the conversion using the palette, and wherein the rule specifies the maximum number of entries according to the characteristics of the video unit.

[0007] In another example aspect, another video processing method is disclosed. The method includes: for the conversion between a current video block of a video and the coded / decoded representation of the video, determining one or more coding / decoding parameters for de - blocking processing of the current video block based on the use of a palette mode coding / decoding tool for the current video block according to a rule; and performing the conversion using the one or more coding / decoding parameters, wherein the rule specifies that one or more coding / decoding parameters of a video block coded / decoded using a palette mode coding / decoding tool are derived differently from the parameters for other coding / decoding tools, and wherein the palette mode coding / decoding tool includes: during encoding, representing the current video block as the coded / decoded representation using a palette of representative sample values; or during decoding, reconstructing the current video block from the coded / decoded representation using a palette of representative sample values.

[0008] In another example aspect, another video processing method is disclosed. The method includes: for the conversion between a current video block of a video and the codec representation of the video, determining, based on a rule, one or more parameters for deblocking processing to be applied to the current video block based on the use of a Composite Palette Mode (CPM) codec tool for the current video block; and performing the conversion using the one or more parameters, and wherein the rule specifies that the codec representation includes an indication of the one or more parameters using syntax elements that are the same as the syntax elements of an adjacent video block coded using another coding mode, and wherein the CPM codec tool includes representing or reconstructing the current video block by combining the use of a palette of representative entries with one or more samples derived by an Intra Block Copy mode.

[0009] In another example aspect, another video processing method is disclosed. The method includes: for the conversion between a current video block of a video and the codec representation of the video, determining, based on a rule and based on the coding mode of an adjacent video block, parameters for deblocking processing to be applied to the current video block; and performing the conversion based on the determination, and wherein the rule specifies using the same parameters for the case of coding the adjacent video block using a Palette Mode Combined with Intra Prediction (PCIP) or another coding mode, and wherein the PCIP includes using intra prediction to derive a predicted block of the current video block and using palette information associated with the current video block to refine the predicted block.

[0010] In another example aspect, another video processing method is disclosed. The method includes: performing a conversion between a current video block of a video and the codec representation of the video, wherein the current video block is represented in the codec representation according to a coding mode that is a palette coding mode, or a palette mode combined with an intra prediction mode (PCIP), or a composite palette mode (CPM); wherein the codec representation includes a flag for the current video block that indicates, according to a rule, whether the current video block has non-zero coefficients; wherein the rule specifies that the flag has a value according to the coding mode or the presence of escape pixels in the codec representation of the current video block, wherein the palette coding mode includes representing or reconstructing the current video block using a palette of representative sample values, wherein the CPM includes representing or reconstructing the current video block by combining the use of a palette of representative entries with one or more samples derived by an Intra Block Copy mode, and wherein the PCIP includes using intra prediction to derive a predicted block of the current video block and using palette information to refine the predicted block.

[0011] In another example aspect, another video processing method is disclosed. The method includes: for the conversion between a video including a plurality of video blocks and an encoded / decoded representation of the video, determining the applicability of deblocking processing to each of the plurality of video blocks based on the conditions of each video block among the plurality of video blocks related to the encoding / decoding mode, where the encoding / decoding mode uses a palette of representative values to encode / decode the corresponding video block, and where the encoding / decoding mode includes a palette mode, a palette mode combined with an intra prediction mode (PCIP), or a composite palette mode (CPM); and performing the conversion based on the determination, and where the palette mode is applicable to the corresponding video block to encode / decode the pixels of the corresponding video block using a palette of component values, and where the PCIP is applicable to the corresponding video block to derive a prediction block of the corresponding video block using intra prediction and refine the prediction block using palette information, and where the CPM is applicable to the corresponding video block to represent or reconstruct samples within the corresponding video block by combining the use of a palette of representative entries with one or more samples derived by an intra block copy mode.

[0012] In another example aspect, another video processing method is disclosed. The method includes: performing a conversion between a current video block encoded / decoded using a composite palette mode (CPM) and an encoded / decoded representation of the current video block, where the encoded / decoded representation includes an indication of the CPM at the video region level, the indication of the CPM being separate from an indication of an intra mode, an inter mode, an intra block copy mode, or a palette mode applicable to the video region, and where the CPM allows for reconstructing samples in the current video block by combining the use of a palette of representative entries with one or more samples derived by an intra block copy mode.

[0013] In another example aspect, another video processing method is disclosed. The method includes: performing a conversion between a current video block of a color component of a video and an encoded / decoded representation of the current video block, where the current video block is encoded / decoded using an encoding / decoding mode including a palette mode combined with an intra prediction mode (PCIP), or a composite palette mode (CPM), where syntax elements related to the encoding / decoding mode are selectively included in the encoded / decoded representation based on characteristics of the color components of the video, or a segmentation structure or planar encoding for the current video block, and where the PCIP is applicable to the current video block to derive a prediction block of the corresponding video block using intra prediction and refine the prediction block using palette information, and where the CPM is applicable to the current video block to represent or reconstruct samples within the corresponding video block by combining the use of a palette of representative entries with one or more samples derived by an intra block copy mode.

[0014] In another example aspect, another video processing method is disclosed. The method includes: performing a conversion between a current chrominance block of a video and an encoded / decoded representation of the video, wherein the current chrominance block is encoded / decoded using an encoding / decoding mode, and wherein, based on encoding / decoding information of one or more selected regions of a luminance block corresponding to the current chrominance block, the encoded / decoded representation selectively includes an indication of the encoding / decoding mode.

[0015] In another example aspect, another video processing method is disclosed. The method includes: performing a conversion between a video picture including a plurality of video blocks and an encoded / decoded representation of the video picture, and wherein the video blocks are encoded / decoded using an encoding / decoding mode that encodes / decodes the video blocks using a palette of representative values, and signaling a syntax element of the encoding / decoding mode based on values of one or more prediction modes applied to the current video block.

[0016] In another example aspect, another video processing method is disclosed. The method includes: performing a first conversion between a first encoded / decoded representation of a current video block and the current video block encoded using a Composite Palette Mode (CPM) that allows reconstruction of samples in the current video block by selectively using a palette of representative entries, wherein the current video block has a motion vector associated with the current video block; and performing a second conversion between a next video block and a second encoded / decoded representation of the next video block, wherein during the second conversion, motion information associated with the current video block is used to predict motion information of the next video block.

[0017] In another example aspect, another video processing method is disclosed. The method includes: performing a first conversion between a first encoded / decoded representation of a current video block and the current video block encoded using a Palette Mode combined with Intra Prediction (PCIP) that allows using intra prediction to derive a predicted block of the current video block and allows refining the current predicted block using palette information, wherein the current video block has an intra prediction direction associated with the current video block; and performing a second conversion between a next video block and a second encoded / decoded representation of the next video block, wherein during the second conversion, the intra mode of the current video block is used as an intra mode predictor for the next video block.

[0018] In another example aspect, another video processing method is disclosed. The method includes: determining a prohibited use of a scaling matrix during a conversion between a video picture including one or more video blocks and an encoded / decoded representation of the video picture due to the use of an encoding / decoding mode related to a palette; and performing the conversion based on the determination of the prohibited use of the scaling matrix.

[0019] In another example aspect, another video processing method is disclosed. The method includes: determining an allowed use of a scaling matrix during a conversion between a video picture including one or more video blocks and an encoded / decoded representation of the video picture due to the use of an encoding / decoding mode related to a palette; and performing the conversion based on the determination of the allowed use of the scaling matrix.

[0020] In another example aspect, the above method may be implemented by a video decoder device including a processor.

[0021] In another example aspect, the above method may be implemented by a video encoder device including a processor.

[0022] In yet another example aspect, these methods may be implemented in the form of processor-executable instructions and stored on a computer-readable program medium.

[0023] This application document will further describe these and other aspects. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 An illustration of intra block copy is shown.

[0025] Figure 2 Examples of spatial domain adjacent candidates are shown.

[0026] Figure 3 Examples of blocks encoded / decoded in palette mode are shown.

[0027] Figure 4 Examples of using a palette predictor to signal palette entries are shown.

[0028] Figure 5 Examples of horizontal and vertical back-and-forth scans are shown.

[0029] Figure 6 Examples of encoding / decoding of palette indices are shown.

[0030] Figure 7 Examples of multi-type tree partitioning patterns are shown.

[0031] Figure 8 Examples of pixels involved in filter on / off decision and strong / weak filter selection are shown.

[0032] Figure 9 An example implementation method of the palette mode in combination with intra prediction is shown.

[0033] Figure 10 An example implementation of the composite palette mode is shown.

[0034] Figure 11 Examples of 67 modes (directions) for intra prediction are shown.

[0035] Figure 12 Examples of the left neighbor and the upper neighbor of the current video block are shown.

[0036] Figure 13 Examples of the corresponding luma regions of chroma blocks are shown.

[0037] Figure 14A and Figure 14B is a block diagram of an example of

[0038] a video processing device.

[0039] Figures 15A to 15E is a flowchart of an example of a video processing method according to some embodiments based on the disclosed technology.

[0040] Figures 16A to 16F is a flowchart of an example of a video processing method according to some embodiments based on the disclosed technology. Detailed implementation

[0041] This application document provides various techniques that can be used by a decoder of an image or video bitstream to improve the quality of decompressed or decoded digital video or images. For the sake of brevity, the term "video" is used herein to include both a sequence of pictures (traditionally called video) and a single image. In addition, a video encoder may also implement these techniques during the encoding process in order to reconstruct decoded frames for further encoding.

[0042] Section headings are used in this application document for ease of understanding and do not limit the embodiments and techniques to the corresponding sections. Thus, the embodiments of one section can be combined with the embodiments of other sections.

[0043] 1. Overview

[0044] This patent application document relates to video coding and decoding technology. Specifically, this application document relates to combined coding and decoding including triangular prediction modes. It can be applied to existing video coding and decoding standards, such as HEVC, or to standards under consideration (Versatile Video Coding). It may also be applicable to future video coding and decoding standards or video codecs.

[0045] 2. Preliminary discussion

[0046] Video coding standards have mainly evolved by developing well-known ITU-T and ISO / IEC standards. ITU-T developed H.261 and H.263, ISO / IEC developed MPEG-1 and MPEG-4 Visual, and the two organizations jointly developed H.262 / MPEG-2 Video, H.264 / MPEG-4 Advanced Video Coding (AVC), and H.265 / HEVC [1] standards. Since H.262, video coding standards have been based on a hybrid video coding structure, in which temporal prediction and transform coding are employed. To explore future video coding technologies beyond HEVC, VCEG and MPEG jointly established the Joint Video Exploration Team (JVET) in 2015. Since then, JVET has adopted many new methods and incorporated them into a reference software called the Joint Exploration Model (JEM) [3][4]. In April 2018, the Joint Video Experts Team (JVET) between VCEG (Q6 / 16) and ISO / IEC JTC1 SC29 / WG11 (MPEG) was established to work on the VVC standard aiming at a 50% bitrate reduction compared to HEVC.

[0047] The latest version of the VVC draft, namely Versatile Video Coding (Draft 4), can be found at the following URL:

[0048] phenix.itsudparis.eu / jvet / doc_end_user / current_document.php?id=5755

[0049] The latest reference software for VVC called VTM can be found at the following URL:

[0050] vcgit.hhi.fraunhofer.de / jvet / VVCSoftware_VTM / tags / VTM-4.0

[0051] 2.1 Intra Block Copy

[0052] Intra Block Copy (IBC) (also known as current picture reference) is adopted in the HEVC Screen Content Coding Extension (HEVC-SCC) and the current VVC Test Model (VTM-4.0). IBC extends the concept of motion compensation from inter coding to intra coding. As Figure 1As shown, when applying IBC, the current block is predicted from a reference block in the same picture. Before encoding or decoding the current block, the samples in the reference block must have been reconstructed. Although IBC is not efficient for most sequences captured by cameras, it shows significant coding and decoding gains for screen content. The reason is that there are many repetitive patterns in screen content pictures, such as icons and text characters. IBC can effectively eliminate the redundancy between these repetitive patterns. In HEVC-SCC, if the current picture is selected as its reference picture, the coding unit (CU) for inter coding can apply IBC. In this case, the MV is renamed as the block vector (BV), and the BV always has integer pixel precision. To be compatible with the main profile of HEVC, the current picture is marked as a "long-term" reference picture in the decoded picture buffer (DPB). It should be noted that, similarly, in the multi-view / 3D video coding standard, the inter-view reference pictures are also marked as "long-term" reference pictures.

[0053] After the BV finds its reference block, the prediction is generated by copying the reference block. The residual can be obtained by subtracting the reference pixels from the original signal. Then, transformation and quantization can be applied as in other coding modes.

[0054] Figure 1 An illustration of intra block copy is shown.

[0055] However, when the reference block is outside the picture, or overlaps with the current block, or is outside the reconstructed area, or is outside the valid area restricted by certain constraints, some or all of the pixel values are not defined. Basically, there are two solutions to this problem. One is to prohibit such cases, for example, in terms of bitstream consistency. The other is to apply padding to those undefined pixel values. The following subsections describe the solutions in detail.

[0056] 2.2 IBC Extension in HEVC Screen Content Coding

[0057] In the screen content coding extension of HEVC, when a block uses the current picture as a reference, it should be ensured that the entire reference block is within the available reconstructed area, as shown in the following specification text:

[0058] The derivation of the variables offsetX and offsetY is as follows:

[0059] offsetX = (ChromaArrayType == 0)? 0 : (mvCLX[0] & 0x7? 2 : 0) (2-1)

[0060] offsetY = (ChromaArrayType == 0)? 0 : (mvCLX[1] & 0x7? 2 : 0) (2-2)

[0061] For bitstream consistency, when the reference picture is the current picture, the luma motion vector mvLX shall comply with the following constraints:

[0062] – Invoke the derivation process of z-scan order block availability specified in Clause 6.4.1, where (xCurr, yCurr) set to be equal to (xCb, yCb), and the adjacent luma position (xNbY, yNbY) set to (xPb + (mvLX[0] >> 2) - offsetX, yPb + (mvLX[1] >> 2) - offsetY) are used as inputs, and its output shall be TRUE.

[0063] – Invoke the derivation process of z-scan order block availability specified in Clause 6.4.1, where (xCurr, yCurr) set to be equal to (xCb, yCb), and the adjacent luma position (xNbY, yNbY) set to (xPb + (mvLX[0] >> 2) + nPbW - 1 + offsetX, yPb + (mvLX[1] >> 2) + nPbH - 1 + offsetY) are used as inputs, and its output shall be equal to TRUE.

[0064] – One or all of the following conditions shall be true:

[0065] – The value of (mvLX[0] >> 2) + nPbW + xB1 + offsetX is less than or equal to 0.

[0066] – The value of (mvLX[1] >> 2) + nPbH + yB1 + offsetY is less than or equal to 0.

[0067] – The following condition shall be satisfied:

[0068] (xPb + (mvLX[0] >> 2) + nPbSw - 1 + offsetX) / CtbSizeY - xCurr / CtbSizeY <= yCurr / CtbSizeY - (yPb + (mvLX[1] >> 2) + nPbSh - 1 + offsetY) / CtbSizeY (2 - 3)

[0069] Therefore, there will be no situation where the reference block overlaps with the current block or the reference block is outside the picture. There is no need to fill the reference or prediction block.

[0070] 2.3 IBC in the VVC Test Model

[0071] In the current VVC test model (i.e., the VTM-4.0 design), the entire reference block should have the current coding tree unit (CTU) and not overlap with the current block. Therefore, there is no need to pad the reference or prediction block. The IBC flag is coded as the prediction mode of the current CU. Therefore, for each CU, there are a total of three prediction modes: MODE_INTRA, MODE_INTER, and MODE_IBC.

[0072] 2.3.1 IBC Merge Mode

[0073] In the IBC Merge mode, an index pointing to an entry in the IBC Merge candidate list is parsed from the bitstream. The construction of the IBC Merge list can be summarized in the following sequential steps:

[0074] · Step 1: Derive spatial candidates

[0075] · Step 2: Insert HMVP candidates

[0076] · Step 3: Insert pairwise average candidates

[0077] In deriving the spatial Merge candidates, as Figure 2 shown, among the candidates located at the positions depicted by A1, B1, B0, A0, and B2, up to four Merge candidates are selected at most. The derivation order is A1, B1, B0, A0, and B2. Position B2 is considered only when any of the PUs at positions A1, B1, B0, A0 is not available (e.g., because it belongs to another stripe or slice) or is not coded using the IBC mode. After adding the candidate at position A1, the insertion of the remaining candidates is subject to a redundancy check to ensure that candidates with the same motion information are excluded from the list in order to improve the coding efficiency.

[0078] After inserting the spatial candidates, if the IBC Merge list size is still less than the maximum IBC Merge list size, IBC candidates from the HMVP table can be inserted. A redundancy check is performed when inserting HMVP candidates

[0079] Finally, the pairwise average candidates are inserted into the IBC Merge list.

[0080] When the reference block identified by a Merge candidate is outside the picture, or overlaps with the current block, or is outside the reconstruction region, or is outside the valid region restricted by certain constraints, that Merge candidate is called an invalid Merge candidate.

[0081] Note that invalid Merge candidates can be inserted into the IBC Merge list.

[0082] VVC adopts JVET-N0843. In JVET-N0843, the BV predictors for the Merge mode and the AMVP mode in IBC will share a common predictor list, which consists of the following elements:

[0083] · 2 spatial neighboring positions (such as Figure 2 A1, B1 in

[0084] · 5 HMVP entries

[0085] · The default zero vector

[0086] For the Merge mode, the first 6 entries of this list will be used; for the AMVP mode, the first 2 entries of this list will be used. And the list meets the requirements of the shared Merge list area (sharing the same list within the SMR).

[0087] In addition to the above BV predictor candidate list, JVET-N0843 also proposes to simplify the pruning operation between the HMVP candidates and the existing Merge candidates (A1, B1). In the simplification, since it only compares the first HMVP candidate with the spatial Merge candidates, at most 2 pruning operations will be performed.

[0088] In the latest VVC and VTM5, on top of the current bitstream constrained by the previous VTM and VVC versions, explicit syntax constraints are used to disable the 128x128 IBC mode, which makes the presence of the IBC flag depend on the CU size < 128x128.

[0089] 2.3.2 IBC AMVP Mode

[0090] In the IBC AMVP mode, the AMVP index pointing to an entry in the IBC AMVP list is parsed from the bitstream. The construction of the IBC AMVP list can be summarized as the following sequential steps:

[0091] · Step 1: Derive spatial candidates

[0092] ο Check A0, A1 until an available candidate is found.

[0093] ο Check B0, B1, B2 until an available candidate is found.

[0094] · Step 2: Insert HMVP candidates

[0095] · Step 3: Insert zero candidates

[0096] After inserting the spatial candidates, if the IBC AMVP list size is still smaller than the maximum IBC AMVP list size, IBC candidates from the HMVP table can be inserted.

[0097] Finally, insert the zero candidate into the IBC AMVP list.

[0098] 2.3.3 Chrominance IBC mode

[0099] In the current VVC, motion compensation in the chrominance IBC mode is performed at the sub-block level. The chrominance block is divided into several sub-blocks. Each sub-block determines whether the corresponding luma block has a block vector and, if so, determines its validity. There are encoder constraints in the current VTM, where if all sub-blocks in the current chrominance CU have valid luma block vectors, the chrominance IBC mode is tested. For example, for a YUV 420 video, if the chrominance block is NxM, then the co-located luma region is 2Nx2M. The sub-block size of the chrominance block is 2x2. Performing chrominance mv derivation has several steps, and then block copy processing is performed.

[0100] 1) First, divide the chrominance block into (N>>1)*(M>>1) sub-blocks.

[0101] 2) Each sub-block with the top-left sample (coordinates at (x, y)) grabs the corresponding luma block covering the same top-left sample (coordinates at (2x, 2y)).

[0102] 3) The encoder checks the block vector (bv) of the grabbed luma block. If any of the following conditions are met, the bv is considered invalid.

[0103] a. There is no bv for the corresponding luma block.

[0104] b. The predicted block identified by the bv has not been reconstructed yet.

[0105] c. The predicted block identified by the bv overlaps partially or fully with the current block.

[0106] 4) Set the chrominance motion vector of the sub-block to the motion vector of the corresponding luma sub-block.

[0107] When all sub-blocks find valid bvs, the IBC mode is allowed at the encoder.

[0108] The decoding process of the IBC block is listed below. In the IBC mode, the part related to chrominance mv derivation is highlighted in bold and italic.

[0109] 8.6.1 General decoding process of the coded / decoded unit coded / decoded in IBC prediction

[0110] The input to this process is:

[0111] – Luma position (xCb, yCb), specifying the top-left sample of the current coded / decoded block relative to the top-left luma sample of the current picture;

[0112] – The variable cbWidth specifies the width of the current coded block in the luma samples,

[0113] – The variable cbHeight specifies the height of the current coded block in the luma samples,

[0114] – The variable treeType specifies whether to use a single tree or a dual tree, and if a dual tree is used, it specifies whether the current tree corresponds to the luma or chroma component.

[0115] The output of this process is the modified reconstructed picture before loop filtering.

[0116] Call the derivation process of the quantization parameter specified in Clause 8.7.1, where the luma position (xCb, yCb), the width cbWidth of the current coded block in the luma samples, the height cbHeight of the current coded block in the luma samples, and the variable treeType are used as inputs.

[0117] The decoding process of the coded unit coded in the IBC prediction mode includes the following sequential steps:

[0118] 1. The motion vector components of the current coded unit are derived as follows:

[0119] 1. If treeType is equal to SINGLE_TREE or DUAL_TREE_LUMA, the following applies:

[0120] – Call the derivation process of the motion vector components specified in Clause 8.6.2.1, where the luma coded block position (xCb, yCb), the luma coded block width cbWidth, and the luma coded block height cbHeight are used as inputs and the luma motion vector mvL[0][0] is used as the output.

[0121] – Call the derivation process of the chroma motion vector specified in Clause 8.6.2.9, where when treeType is equal to SINGLE_TREE, the luma motion vector mvL[0][0] is used as the input and the chroma motion vector mvC[0][0] is used as the output.

[0122] – The number of luma coded sub-blocks numSbX and numSbY in the horizontal and vertical directions are both set to be equal to 1.

[0123] 1. Otherwise, if treeType is equal to DUAL_TREE_CHROMA, the following applies:

[0124] – The derivation of the number of luma coded sub-blocks numSbX in the horizontal direction and numSbY in the vertical direction is as follows:

[0125] numSbX = (cbWidth >> 2) (8 - 886)

[0126] numSbY = (cbHeight >> 2) (8 - 887)

[0127]

[0128]

[0129] – Invoke the derivation process of the chrominance motion vector specified in Clause 8.6.2.9, where mvL[xSbIdx][ySbIdx] is used as the input and mvC[xSbIdx][ySbIdx] is used as the output.

[0130] – The requirement for bitstream consistency is that when the reference picture is the current picture, the chrominance motion vector mvC[xSbIdx][ySbIdx] shall comply with the following constraints::

[0131] – When invoking the derivation process of block availability specified in Clause 6.4.X [Ed.(BB): Check process for adjacent block availability tbd], where the current chrominance position (xCurr, yCurr) set to be equal to (xCb / SubWidthC, yCb / SubHeightC) and the adjacent chrominance position (xCb / SubWidthC + (mvC[xSbIdx][ySbIdx][0] >> 5), yCb / SubHeightC + (mvC[xSbIdx][ySbIdx][1] >> 5)) are used as the input, the output shall be equal to TRUE.

[0132] – When invoking the derivation process of block availability specified in Clause 6.4.X [Ed.(BB): Check process for adjacent block availability tbd[, where the current chrominance position (xCurr, yCurr) set to be equal to (xCb / SubWidthC, yCb / SubHeightC) and the adjacent chrominance position (xCb / SubWidthC + (mvC[xSbIdx][ySbIdx][0] >> 5) + cbWidth / SubWidthC - 1, yCb / SubHeightC + (mvC[xSbIdx][ySbIdx][1] >> 5) + cbHeight / SubHeightC - 1) are used as the input, the output shall be equal to TRUE.

[0133] – One or all of the following conditions shall be true:

[0134] –(mvC[xSbIdx][ySbIdx][0]>>5)+xSbIdx*2+2 is less than or equal to 0.

[0135] –(mvC[xSbIdx][ySbIdx][1]>>5)+ySbIdx*2+2 is less than or equal to 0.

[0136] 2. The derivation of the predicted samples of the current coding unit is as follows:

[0137] – If treeType is equal to SINGLE_TREE or DUAL_TREE_LUMA, the predicted samples of the current coding unit are derived as follows:

[0138] · Call the decoding process of the ibc block specified in Clause 8.6.3.1, where the luma coding block position (xCb, yCb), luma coding block width cbWidth and luma coding block height cbHeight, the number of luma coding sub-blocks in the horizontal direction numSbX and vertical direction numSbY, the luma motion vector mvL[xSbIdx][ySbIdx] (where xSbIdx = 0..numSbX-1, and ySbIdx = 0..numSbY–1) and the variable cIdx set to be equal to 1 are used as inputs, and the in-frame block copy (ibc) predicted samples (predSamples) of the (cbWidth)x(cbHeight) array predSamplesL of the predicted luma samples are used as outputs.

[0139] – Otherwise, if treeType is equal to SINGLE_TREE or DUAL_TREE_CHROMA, the predicted samples of the current coding unit are derived as follows:

[0140] · Call the decoding process of the ibc block specified in Clause 8.6.3.1, where the luma coding block position (xCb, yCb), luma coding block width cbWidth and luma coding block height cbHeight, the number of luma coding sub-blocks in the horizontal direction numSbX and vertical direction numSbY, the chroma motion vector mvC[xSbIdx][ySbIdx] (where xSbIdx = 0..numSbX-1, and ySbIdx = 0..numSbY–1) and the variable cIdx set to be equal to 1 are used as inputs, and the (cbWidth / 2)x(cbHeight / 2) array predSamples Cb of the ibc predicted samples (predSamples) of the predicted chroma samples of the chroma component Cb are used as outputs.

[0141] · Invoke the decoding process of the IBC block specified in Clause 8.6.3.1, where the luminance coding / decoding block position (xCb, yCb), luminance coding / decoding block width cbWidth, and luminance coding / decoding block height cbHeight, the number of luminance coding / decoding sub-blocks in the horizontal direction numSbX and vertical direction numSbY, the chrominance motion vector mvC[xSbIdx][ySbIdx] (where xSbIdx = 0..numSbX - 1 and ySbIdx = 0..numSbY – 1), and a variable cIdx set to be equal to 2 are used as inputs, and a (cbWidth / 2) x (cbHeight / 2) array predSamples of predicted chrominance samples of the chrominance component Cr Cr is used as the output for the IBC predicted samples (predSamples).

[0142] 3. Set the variables NumSbX[xCb][yCb] and NumSbY[xCb][yCb] to be equal to numSbX and numSbY respectively.

[0143] 4. The derivation of the residual samples of the current coding / decoding unit is as follows:

[0144] – Invoke the decoding process of the residual signal of the coding / decoding block coded / decoded in the inter prediction mode specified in Clause 8.5.8, where when treeType is equal to SINGLE_TREE or treeType is equal to DUAL_TREE_LUMA, the position (xTb0, yTb0) set to be equal to the luminance position (xCb, yCb), the width nTbW set to be equal to the luminance coding / decoding block width cbWidth, the height nTbH set to be equal to the luminance coding / decoding block height cbHeight, and a variable cIdxset set to 0 are used as inputs, and an array resSamples L is used as the output.

[0145] – Invoke the decoding process of the residual signal of the coding / decoding block coded / decoded in the inter prediction mode specified in Clause 8.5.8, where when treeType is equal to SINGLE_TREE or treeType is equal to DUAL_TREE_CHROMA, the position (xTb0, yTb0) set to be equal to the chrominance position (xCb / 2, yCb / 2), the width nTbW set to be equal to the chrominance coding / decoding block width cbWidth / 2, the height nTbH set to be equal to the chrominance coding / decoding block height cbHeight / 2, and a variable cIdxset set to 1 are used as inputs, and an array resSamples Cb is used as the output.

[0146] – Invoke the decoding process of the residual signal of the coded block coded in the inter prediction mode specified in Clause 8.5.8, where when treeType is equal to SINGLE_TREE or treeType is equal to DUAL_TREE_CHROMA, the position (xTb0, yTb0) equal to the chroma position (xCb / 2, yCb / 2), the width nTbW equal to cbWidth / 2 of the chroma coded block, the height nTbH equal to cbHeight / 2 of the chroma coded block, and the variable cIdxset equal to 2 are used as inputs, along with the array resSamples Cr as output.

[0147] 5. The derivation of the reconstructed samples of the current coded unit is as follows:

[0148] - Invoke the picture reconstruction process of the color component specified in Clause 8.7.5, where when treeType is equal to SINGLE_TREE or treeType is equal to DUAL_TREE_LUMA, the block position (xB, yB) equal to (xCb, yCb), the block width bWidth equal to cbWidth, the block height bHeight equal to cbHeight, the variable cIdx equal to 0, the (cbWidth)x(cbHeight) array predSamples equal to predSamples L and the (cbWidth)x(cbHeight) array resSamples equal to resSamples L are used as inputs and the output is the modified reconstructed picture before loop filtering.

[0149] – Invoke the picture reconstruction process of the color component specified in Clause 8.7.5, where when treeType is equal to SINGLE_TREE or treeType is equal to DUAL_TREE_CHROMA, the block position (xB, yB) equal to (xCb / 2, yCb / 2), the block width bWidth equal to cbWidth / 2, the block height bHeight equal to cbHeight / 2, the variable cIdx equal to 1, the (cbWidth / 2)x(cbHeight / 2) array predSamples equal to predSamples Cb and the (cbWidth / 2)x(cbHeight / 2) array resSamples equal to resSamples Cb are used as inputs and the output is the modified reconstructed picture before loop filtering.

[0150] – Invoke the picture reconstruction process for the color components specified in Clause 8.7.5, where when treeType is equal to SINGLE_TREE or treeType is equal to DUAL_TREE_CHROMA, set the block position (xB, yB) equal to (xCb / 2, yCb / 2), set the block width bWidth equal to cbWidth / 2, set the block height bHeight equal to cbHeight / 2, set the variable cIdx to 2, set the (cbWidth / 2) x (cbHeight / 2) array predSamples equal to predSamples Cr and the (cbWidth / 2) x (cbHeight / 2) array resSamples equal to resSamples Cr as inputs, and the output is the modified reconstructed picture before in-loop filtering.

[0151] 2.4 Adaptive Motion Vector Resolution (AMVR)

[0152] In HEVC, when use_integer_mv_flag in the slice header is equal to 0, the motion vector difference (MVD) (between the motion vector of the CU and the predicted motion vector) is signaled in quarter-luminance samples. In VVC, an adaptive motion vector resolution (AMVR) scheme at the CU level is introduced. AMVR allows the MVD of a CU to be coded and decoded with different precisions. The MVD of the current CU can be adaptively selected as follows, depending on the mode of the current CU (normal AMVP mode or affine AVMP mode):

[0153] – Normal AMVP mode: Quarter-luminance samples, integer-luminance samples, or four-luminance samples.

[0154] – Affine AMVP mode: Quarter-luminance samples, integer-luminance samples, or 1 / 16-luminance samples.

[0155] If the current CU has at least one non-zero MVD component, the MVD resolution indication at the CU level is signaled conditionally. If all MVD components (i.e., the horizontal and vertical MVDs of reference list L0 and reference list L1) are zero, a quarter-luminance sample MVD resolution is inferred.

[0156] For a CU with at least one non-zero MVD component, a first flag is signaled to indicate whether the precision of the quarter-luma sample MVD is used for the CU. If the first flag is 0, no further signaling is required and the precision of the quarter-luma sample MVD is used for the current CU. Otherwise, a second flag is signaled to indicate whether integer-luma samples or the precision of the four-luma sample MVD is used for normal AMVP CUs. The same second flag is used to indicate whether integer-luma samples or the precision of the 1 / 16-luma sample MVD is used for affine AMVP CUs. To ensure that the reconstructed MVs have the expected precision (quarter-luma sample, integer-luma sample, or four-luma sample), the motion vector predictor of the CU is rounded to the same precision as the MVD before being added to the MVD. The motion vector predictor is rounded towards zero (i.e., a negative motion vector predictor is rounded towards positive infinity and a positive motion vector predictor is rounded towards negative infinity).

[0157] The encoder uses RD checking to determine the motion vector resolution of the current CU. To avoid always performing three CU-level RD checks for each MVD resolution, in VTM4, the RD check for MVD precision other than quarter-luma sample is only conditionally invoked. For the normal AVMP mode, first the RD costs for quarter-luma sample MVD precision and integer-luma sample MV precision are calculated. Then, the RD cost of the integer-luma sample MVD precision is compared with the RD cost of the quarter-luma sample MVD precision to determine whether it is necessary to further check the RD cost of the four-luma sample MVD precision. When the RD cost of the quarter-luma sample MVD precision is much smaller than the RD cost of the integer-luma sample MVD precision, the RD check for the four-luma sample MVD precision is skipped. For the affine AMVP mode, if the affine inter mode is not selected after checking the rate-distortion costs of the affine merge / skip mode, merge / skip mode, normal AMVP mode with quarter-luma sample MVD precision, and affine AMVP mode with quarter-luma sample MVD precision, then the 1 / 16-luma sample MV precision and 1-pixel MV precision affine inter modes are not checked. Additionally, the affine parameters obtained in the quarter-luma sample MV precision affine inter mode are used as the starting search points for the 1 / 16-luma sample and quarter-luma sample MV precision affine inter modes.

[0158] 2.5 Palette Mode in HEVC Screen Content Coding Extension (HEVC-SCC)

[0159] The basic concept behind the palette mode is to represent the samples in a CU by a small set of color values. This set is called the palette. It can also indicate samples by (possibly quantized) component values signaled by escape symbols outside the palette. Figure 3 This concept is illustrated.

[0160] In the palette mode of HEVC-SCC, the prediction method is used for encoding and decoding the palette and the index picture.

[0161] 2.5.1 Encoding and decoding of palette entries

[0162] To encode and decode the palette entries, a palette predictor is maintained. The maximum size of the palette and the palette predictor are signaled in the SPS. In HEVC-SCC, palette_predictor_initializer_present_flag is introduced in the PPS. When this flag is 1, the entries used to initialize the palette predictor are signaled in the bitstream. The palette predictor is initialized at the start of each CTU row, each slice, and each strip. Using the initializer entries of the palette predictor signaled in the PPS, the palette predictor is reset to 0 or initialized, depending on the value of palette_predictor_initializer_present_flag. In HEVC-SCC, a palette predictor initializer of size 0 is enabled to allow explicit disabling of palette predictor initialization at the PPS level.

[0163] For each entry in the palette predictor, a multiplexing flag is signaled to indicate whether it is part of the current palette. This is shown in Figure 4 . Run-length encoding with zero is used to send the multiplexing flag. Thereafter, the number of new palette entries is signaled using an exponential Golomb code of order 0. Finally, the component values of the new palette entries are signaled.

[0164] 2.5.2 Encoding and decoding of palette indices

[0165] As shown in Figure 5 , horizontal and vertical raster scans are used to encode and decode the palette indices. The scan order is explicitly signaled in the bitstream using palette_transpose_flag. For the remainder of this subsection, it is assumed that the scan is horizontal.

[0166] The palette indices are encoded and decoded using two main palette sample modes: 'INDEX' and 'COPY_ABOVE'. As mentioned before, the escape symbol is also signaled in the 'INDEX' mode and is assigned an index equal to the maximum palette size. This mode is signaled using a flag other than the top row or when the previous mode was 'COPY_ABOVE'. In the 'COPY_ABOVE' mode, the palette index of the sample in the previous row is copied. In the 'INDEX' mode, the palette index is signaled explicitly. For both the 'INDEX' and 'COPY_ABOVE' modes, a run value is signaled, which specifies the number of subsequent samples that are also encoded and decoded using the same mode. When the escape symbol is part of a run in the 'INDEX' or 'COPY_ABOVE' mode, an escape component value is signaled for each escape symbol. Figure 6 Shows the encoding and decoding of palette indices.

[0167] This syntax order is completed as follows. First, the number of index values of the CU is signaled. After that, the actual index values of the entire CU are signaled using truncated binary encoding. Both the number of indices and the index values are encoded in bypass mode. This groups the bypass bins related to the indices together. Then the palette sample modes (if needed) and runs are signaled in an interleaved manner. Finally, the component escape values corresponding to the escape samples of the entire CU are grouped together and encoded in bypass mode.

[0168] After signaling the index values, the additional syntax element last_run_type_flag is signaled. This syntax element, together with the number of sequence numbers, eliminates the need to signal the run value corresponding to the last run in the block.

[0169] In HEVC-SCC, the palette mode is also enabled for 4:2:2, 4:2:0, and monochrome chroma formats. For all chroma formats, the signaling of palette entries and palette indices is almost the same. If it is a non-monochrome format, each palette entry consists of 3 components. For the monochrome format, each palette entry consists of a single component. For the chroma direction of subsampling, the chroma samples are associated with the luminance sample indices divisible by 2. After reconstructing the palette index for the CU, if a sample is associated with only a single component, only the first component of the palette entry is used. The only difference in signaling is the escape component value. For each escape sample, the number of escape component values signaled may vary depending on the number of components associated with that sample.

[0170] 2.6 Coefficient Encoding and Decoding in Transform Skip Mode

[0171] In JVET-M0464 and JVET-N0280, several modifications to the coefficient coding and decoding in the transform skip (TS) mode were proposed to adapt the residual coding and decoding to the statistical and signaling characteristics of transform skipping.

[0172] The proposed modifications are listed as follows.

[0173] No final significant scan position : Since the residual signal reflects the spatial residual after prediction and no energy compaction through transformation is performed on TS, there is no longer a higher probability of trailing zeros or insignificant levels appearing at the lower right corner of the transform block. Therefore, signaling of the last significant scan position is omitted in this case.

[0174] Sub-block CBF : The absence of signaling of the last significant scan position requires the following modification to the coded_sub_block_flag of the sub-block signaled by the coded_sub_block_flag of TS:

[0175] · Due to quantization, the above-mentioned insignificant sequences may still locally appear inside the transform block. Therefore, as described above, the last significant scan position is removed, and the coded_sub_block_flag is decoded for all sub-blocks except for the case where the coefficients of all CGs except the last one are zero. Therefore, it is not necessary to decode the coded_sub_block_flag for the last CG.

[0176] · The coded_sub_block_flag for the sub-block covering the DC frequency position (the upper-left sub-block) represents a special case. In VVC Draft 3, the coded_sub_block_flag for this sub-block is never signaled and is always inferred to be equal to 1. When the last significant scan position is in another sub-block, this means that there is at least one significant level outside the DC sub-block. Therefore, although the coded_sub_block_flag for this sub-block is inferred to be equal to 1, the DC sub-block can only contain zeros / non-significant levels. In the case of the absence of the last scan position information in the TS, the coded_sub_block_flag for each sub-block is signaled. This also includes the coded_sub_block_flag for the DC sub-block, except when all other coded_sub_block_flag syntax elements are already equal to 0. In this case, the coded_sub_block_flag for the DC is inferred to be equal to 1 (inferDcSbCbf = 1). Since there must be at least one significant level in this DC sub-block, the sig_coeff_flag syntax element at the first position (0,0) is not signaled and is derived to be equal to 1 (inferSbDcSigCoeffFlag = 1), and conversely, all other sig_coeff_flag syntax elements in the DC sub-block are equal to 0.

[0177] · Changed the context modeling of the coded_sub_block_flag. The context model index is counted as the sum of the coded_sub_block_flag on the left side and the coded_sub_block_flag on the upper side of the current sub-block, rather than the logical disjunction of the two.

[0178] Sig_coeff_flag context modeling : Modified the local template in the sig_coeff_flag context modeling to include only the left neighbor (NB0) and the upper neighbor (NB1) of the current scan position. The context model offset is the number of valid adjacent positions sig_coeff_flag[NB0] + sig_coeff_flag[NB1]. Therefore, the selection of different context sets depending on the diagonal d within the current transform block is removed. This results in three context models for encoding / decoding the sig_coeff_flag flag and a separate context model.

[0179] Abs_level_gt1_flag and par_level_flag context modeling : The abs_level_gt1_flag and par_level_flag use a single context model.

[0180] Abs_remainder encoding / decoding: Although the empirical distribution of the absolute levels of the transform skip residuals is generally still fitted to a Laplacian or geometric distribution, there is greater instability than for the absolute levels of the transform coefficients. In particular, the variance within a window of consecutive realizations is higher for the absolute levels of the residuals. This motivates the following modifications to the abs_remainder syntax binarization and context modeling:

[0181] Using a higher cutoff value in binarization, i.e. the transition point from encoding and decoding using sig_coeff_flag, abs_level_gt1_flag, par_level_flag and abs_level_gt3_flag to Rice code for abs_remainder, and a dedicated context model for each binary file position will result in higher compression efficiency. Increasing the cutoff value will result in more "greater than X" flags, such as introducing abs_level_gt5_flag, abs_level_gt7_flag, and so on, until the cutoff value is reached. The cutoff value itself is fixed to 5 (numGtFlags=5).

[0182] The template derived from the rice parameter is modified, that is, only the left and upper neighbors of the previous scan position are considered similar to the local template modeled by the sig_coeff_flag context.

[0183] Coeff_sign_flag context modeling : Due to instabilities within symbol sequences and the fact that prediction residuals are often biased, context models can be used to encode and decode symbols even if the global empirical distribution is almost uniformly distributed. A single dedicated context model is used for encoding and decoding of a symbol, and the symbol is parsed after sig_coeff_flag to keep all context encoding and decoding bins together.

[0184] 2.7 Quantized Residual Block Differential Pulse Coded Modulation (QR-BDPCM)

[0185] In JVET-M0413, Quantized Residual Block Differential Pulse Coded Modulation (QR-BDPCM) is proposed to efficiently encode and decode screen content.

[0186] The prediction directions used in QR-BDPCM can be vertical and horizontal prediction modes. Intra prediction for the entire block is performed by sample replication in a prediction direction (horizontal or vertical prediction) similar to intra prediction. The residual is quantized, and the increment between the quantized residual and the quantized value of its predicted value (horizontal or vertical) is encoded and decoded. This can be described as follows: For a block of size M (rows) × N (columns), using the unfiltered samples from the upper or left block boundary samples, after performing intra prediction horizontally (copying the left adjacent pixel values of the entire prediction block row by row) or vertically (copying the upper adjacent row to each row in the prediction block), let r i,j , 0 ≤ i ≤ M - 1, 0 ≤ j ≤ N - 1 be the prediction residual. Let Q(r i,j ), 0 ≤ i ≤ M - 1, 0 ≤ j ≤ N - 1 represent the quantized form of the residual r i,j , where the residual is the difference between the original block and the predicted block value. Then, block DPCM is applied to the quantized residual samples, resulting in a modified M × N array with elements When signaling vertical BDPCM, there is: When signaling vertical BDPCM, there is:

[0187]

[0188] For horizontal prediction, similar rules apply, and by

[0189]

[0190] the quantized residual samples are obtained.

[0191] The quantized residual samples are sent to the decoder.

[0192] On the decoder side, the above calculations are reversed to produce Q(r i,j ), 0 ≤ i ≤ M - 1, 0 ≤ j ≤ N - 1. For the vertical prediction case,

[0193]

[0194] For the horizontal case,

[0195]

[0196] the inverse Q -1 (Q(r i，j )) of the quantized residual is added to the intra-block prediction value to produce the reconstructed sample value.

[0197] The main advantage of this scheme is that the inverse of DPCM can be dynamically completed simply by performing the addition of the prediction factor when parsing the coefficients or after parsing.

[0198] The draft text of QR-BDPCM is changed as follows.

[0199] 7.3.6.5 Coding and Decoding Unit Syntax

[0200]

[0201]

[0202]

[0203] bdpcm_flag[x0][y0] equal to 1 specifies that there is bdpcm_dir_flag in the coding and decoding unit including the luminance coding and decoding block at position (x0, y0).

[0204] bdpcm_dir_flag[x0][y0] equal to 0 specifies that the prediction direction used in the bdpcm block is horizontal, otherwise it is vertical.

[0205] 2.8 Partition Structure

[0206] In HEVC, the CTU is partitioned into CUs by using a quadtree structure (referred to as the coding tree) to adapt to various local characteristics. At the leaf CU level, it is determined whether to use inter-picture (temporal) prediction or intra-picture (spatial) prediction to code and decode the picture region. According to the partition type of the PU, each leaf CU can be further partitioned into one, two, or four PUs. In a PU, the same prediction process is applied, and the relevant information is transmitted to the decoder on a PU basis. After obtaining the residual block by applying the prediction process based on the PU partition type, the leaf CU can be partitioned into transform units (TUs) according to another quadtree structure similar to the coding tree of the CU. An important feature of the HEVC structure is that it has multiple partition concepts, including CUs, PUs, and TUs.

[0207] In VVC, a quadtree of a nested multi-type tree using binary and ternary partition segmentation structures replaces the concept of multiple partition unit types, that is, it removes the separation of the CU, PU, and TU concepts, but CUs with sizes exceeding the maximum transform length need to be excluded and more flexibility in CU segmentation shapes is supported. In the coding tree structure, a CU can have a square or rectangular shape. First, the coding tree unit (CTU) is partitioned by a quadtree (also called a quadtree) structure. Then, the quadtree leaf nodes can be further partitioned by a multi-type tree structure. As Figure 7As shown, there are four partitioning types in the multi-type tree structure: vertical binary partitioning (SPLIT_BT_VER), horizontal binary partitioning (SPLIT_BT_HOR), vertical ternary partitioning (SPLIT_TT_VER), and horizontal ternary partitioning (SPLIT_TT_HOR). The leaf nodes of the multi-type tree are called coding units (CUs), and this segment can be used for prediction and transform processing without any further partitioning, unless the CU is too large for the maximum transform length. This means that, in most cases, in the quadtree of the coding block structure with a nested multi-type tree, the CUs, PUs, and TUs have the same block size. Exceptions occur when the maximum supported transform length is less than the width or height of the color component of the CU. In addition, the luminance and chrominance components have separate partitioning structures on the I slice. In addition, JVET-K0353 and JVET-K0354 establish signaling flags to determine whether to use a separate partitioning structure at the CTU / CU level.

[0208] 2.9 Deblocking scheme in VTM-4.0

[0209] Note that, in the following description, pNM represents the Nth sample from the left in the Mth row with respect to the vertical edge or the Nth sample from the top in the Mth column with respect to the vertical direction. qNM represents the Nth sample from the right in the Mth row with respect to the vertical edge, or the Nth sample from the bottom in the Mth column with respect to the horizontal edge. In Figure 8 Examples showing pNM and qNM are presented.

[0210] Note that, in the following description, pN represents the Nth sample from the left in the row with respect to the vertical edge or the Nth sample from the left in the column with respect to the horizontal edge. qN represents the Nth sample in the row with respect to the vertical edge, or the Nth sample in the column with respect to the horizontal edge.

[0211] The decision of filter on / off is completed in units of 4 rows. Figure 8 Pixels involved in the filter on / off decision are shown. Six pixels in the two red boxes of the first 4 rows are used to determine the filter on / off for the 4 rows. Six pixels in the two red boxes of the second 4 rows are used to determine the filter on / off for the second 4 rows.

[0212] In the current VTM, namely VTM-4.0, the deblocking scheme described in JVET-M0471 is used. First, the vertical edges in the picture are filtered. Then, the modified samples obtained by filtering the vertical edges in the picture are used as input to filter the horizontal edges. The vertical and horizontal edges in the CTB of each CTU are processed separately on a coding unit basis. Starting from the edge on the left hand side of the coding block, the vertical edges of the coding block in the coding unit are filtered in their geometric order towards the right hand side of the coding block. Starting from the edge at the top of the coding block, the horizontal edges of the coding block in the coding unit are filtered in their geometric order towards the bottom of the coding block.

[0213] 2.9.1 Boundary Decision

[0214] The filtering is applied to the 8x8 block boundaries. Additionally, it must be a transform block boundary or a coding sub-block boundary (e.g., due to the use of affine motion prediction, ATMVP). For those objects without such boundaries, the filter is disabled.

[0215] 2.9.2 Boundary Strength Calculation

[0216] For the transform block boundary / coding sub-block boundary, if it is located in the 8x8 grid, it can be filtered and the setting of bS[xD i [yD j (where [xD i [yD j represents the coordinates) is defined as follows:

[0217] – If the sample p0 or q0 is in the coding block of the coding unit coded in the intra prediction mode, then set bS[xD i [yD j equal to 2.

[0218] – Otherwise, if the block edge is also a transform block edge and the sample p0 or q0 is in the transform block containing one or more non-zero transform coefficient levels, then set bS[xD i [yD j equal to 1.

[0219] – Otherwise, if the prediction mode of the coding sub-block containing the sample p0 is different from the prediction mode of the coding sub-block containing the sample q0, then set bS[xD i [yD j equal to 1.

[0220] – Otherwise, if one or more of the following conditions are met, then set bS[xD i [yD j equal to 1:

[0221] – Both the coded block containing sample p0 and the coded block containing sample q0 are coded and decoded in the IBC prediction mode, and, in units of quarter luminance samples, the absolute difference between the horizontal or vertical components of the motion vectors used in the prediction of the two coded blocks is greater than or equal to 4.

[0222] – The coded block containing sample p0 is predicted using a different reference picture or a different number of motion vectors than the prediction of the coded block containing sample q0.

[0223] Note 1 – Determining whether the reference pictures used for the two coded blocks are the same or different depends only on which pictures are referenced, regardless of whether the index of reference picture list 0 or the index of reference picture list 1 is used to form the prediction, and it is not necessary to consider whether the index positions in the reference picture lists are different.

[0224] Note 2 – The number of motion vectors used to predict the coded block covering the top - left sample of (xSb, ySb) is equal to PredFlagL0[xSb][ySb]+PredFlagL1[xSb][ySb].

[0225] – One motion vector is used to predict the coded block containing sample p0, and one motion vector is used to predict the coded block containing sample q0, and, in units of quarter luminance samples, the absolute difference between the horizontal or vertical components of the motion vectors used is greater than or equal to 4.

[0226] – Two motion vectors and two different reference pictures are used to predict the coded block containing sample p0, and two motion vectors using two identical reference pictures are used to predict the coded block containing sample q0, and, in units of quarter luminance samples, the absolute difference between the horizontal or vertical components of the two motion vectors used for the prediction of the two coded blocks for the same reference picture is greater than or equal to 4.

[0227] – Two motion vectors using the same reference picture are used to predict the coded block containing sample p0, and two motion vectors using the same reference picture are used to predict the coded block containing sample q0, and both of the following conditions are true:

[0228] – In units of quarter luminance samples, the absolute difference between the horizontal or vertical components of the motion vectors in list 0 used for predicting the two coded blocks is greater than or equal to 4, or the absolute difference between the horizontal or vertical components of the motion vectors in list 1 used for predicting the two coded blocks is greater than or equal to 4.

[0229] – The absolute difference between the horizontal or vertical component of the motion vector for list 0 that is used to predict the coded block containing sample p0 and the horizontal or vertical component of the motion vector for list 1 that is used to predict the coded block containing sample q0 is greater than or equal to 4 in units of quarter luminance samples, or the absolute difference between the horizontal or vertical component of the motion vector for list 1 that is used to predict the coded block containing sample p0 and the horizontal or vertical component of the motion vector for list 0 that is used to predict the coded block containing sample q0 is greater than or equal to 4 in units of quarter luminance samples.

[0230] – Otherwise, set the variable bS[xD i [yD j equal to 0.

[0231] Tables 1 and 2 summarize the BS calculation rules.

[0232] Table 1. Boundary Strength (when SPS IBC is disabled)

[0233]

[0234] Table 2. Boundary Strength (when SPS IBC is enabled)

[0235]

[0236] 2.9.3 Deblocking Scheme for Luminance Component

[0237] This section describes the deblocking process.

[0238] The wider and stronger luminance filter is used only when Conditions 1, 2, and 3 are all TRUE.

[0239] Condition 1 is the "large block condition". This condition detects whether the samples on the P side and the Q side belong to large blocks, which are represented by the variables bSidePisLargeBlk and bSideQisLargeBlk respectively. The definitions of bSidePisLargeBlk and bSideQisLargeBlk are as follows.

[0240] bSidePisLargeBlk = ((the edge type is vertical and p0 belongs to a CU with a width >= 32) || (the edge type is horizontal and p0 belongs to a CU with a height >= 32))? TRUE : FALSE

[0241] bSideQisLargeBlk = ((the edge type is vertical and q0 belongs to a CU with a width >= 32) || (the edge type is horizontal and q0 belongs to a CU with a height >= 32))? TRUE : FALSE

[0242] Based on bSidePisLargeBlk and bSideQisLargeBlk, Condition 1 is defined as follows.

[0243] Condition1 = (bSidePisLargeBlk || bSidePisLargeBlk)? TRUE : FALSE

[0244] Next, if Condition 1 is true, Condition 2 will be further checked. First, the following variables are derived:

[0245] – First derive dp0, dp3, dq0, dq3 as in HEVC

[0246] – If the p side is greater than or equal to 32

[0247] dp0 = (dp0 + Abs(p50 - 2 * p40 + p30) + 1) >> 1

[0248] dp3 = (dp3 + Abs(p53 - 2 * p43 + p33) + 1) >> 1

[0249] – If the q side is greater than or equal to 32

[0250] dq0 = (dq0 + Abs(q50 - 2 * q40 + q30) + 1) >> 1

[0251] dq3 = (dq3 + Abs(q53 - 2 * q43 + q33) + 1) >> 1

[0252] Condition 2 = (d < β)? TRUE : FALSE

[0253] where d = dp0 + dq0 + dp3 + dq3, as shown in Section 2.2.4.

[0254] If Conditions 1 and 2 are valid, further check if any block uses sub - blocks:

[0255] If(bSidePisLargeBlk)

[0256] If(modeblockP == SUBBLOCKMODE)

[0257] Sp = 5

[0258] else

[0259] Sp = 7

[0260] else

[0261] Sp = 3

[0262] If (bSideQisLargeBlk)

[0263] If (modeblockQ == SUBBLOCKMODE)

[0264] Sq = 5

[0265] else

[0266] Sq = 7

[0267] else

[0268] Sq = 3

[0269] Finally, if both Condition 1 and Condition 2 are valid, the proposed de-blocking method will check Condition 3 (Large Block Strong Filtering Condition), which is defined as follows.

[0270] In Condition 3 StrongFilterCondition, the following variables are derived:

[0271] Derive dpq as in HEVC

[0272] As in HEVC, derive sp3 = Abs(p3 - p0),

[0273] If (the p side is greater than or equal to 32)

[0274] If (Sp == 5)

[0275] sp3 = (sp3 + Abs(p5 - p3) + 1) >> 1

[0276] else

[0277] sp3 = (sp3 + Abs(p7 - p3) + 1) >> 1

[0278] As in HEVC, derive sq3 = Abs(q0 - q3)

[0279] If (the q side is greater than or equal to 32)

[0280] If (Sq == 5)

[0281] sq3 = (sq3 + Abs(q5 - q3) + 1) >> 1

[0282] else

[0283] sq3 = (sq3 + Abs(q7 - q3) + 1) >> 1

[0284] In HEVC, StrongFilterCondition = (dpq is less than (β >> 2), sp3 + sq3 is less than (3 * β >> 5), and Abs(p0 - q0) is less than (5 * t C + 1) >> 1)? TRUE : FALSE.

[0285] 2.9.4 Stronger Deblocking Filter for Luma (Designed for Larger Blocks)

[0286] When the samples on either side of the boundary belong to a large block, a bilinear filter will be used. Define the samples belonging to a large block as: for a vertical edge, its width >= 32; for a horizontal edge, its height >= 32.

[0287] The bilinear filter is listed below.

[0288] The block boundary samples p from i = 0 to Sp - 1 i and the block boundary samples q from j = 0 to Sq - 1 (as described in the above HEVC deblocking, p i and q i are the i-th sample in the row for filtering the vertical edge or the i-th sample in the column for filtering the horizontal edge) will then be replaced with linear interpolation as follows:

[0289] — p i ′ = (f i * Middle s，t + (64 - f i ) * P s + 32) >> 6), clipped to p i ± tcPD i

[0290] — q j ′ = (g j * Middle s，t + (64 - g j ) * Q s + 32) >> 6), clipped to q j ± tcPD j

[0291] where the tcPD i and tcPD j terms are the location-dependent clipping described in Section 2.3.6, and g j , f i , Middle s，t , P s and Q s are as follows:

[0292]

[0293]

[0294] 2.9.5 Chroma Deblocking Control

[0295] Chroma strong filtering is used on both sides of the block boundary. Here, when both sides of the chroma edge are greater than or equal to 8 (chroma position), chroma filtering is selected and a decision is made that satisfies the following three conditions: The first is the boundary strength and the decision for large blocks. When the block width or height orthogonal to the block edge in the chroma samples is equal to or greater than 8, the proposed filter can be applied. The second and third are basically the same as the HEVC luma deblocking decisions, which are the on / off decision and the strong filter decision respectively.

[0296] In the first decision, the boundary strength (bS) is modified for chroma filtering (as shown in Table 2). Check the conditions in Table 2 in sequence. If the conditions are met, the remaining conditions with lower priority will be skipped.

[0297] When bS is equal to 2 or when a large block boundary is detected and bS is equal to 1, chroma deblocking is performed.

[0298] The second and third conditions are basically the same as the decisions for the HEVC luma strong filter as follows.

[0299] Under the second condition:

[0300] As in HEVC luma deblocking, then d is derived.

[0301] When d is less than β, the second condition will be TRUE.

[0302] Under the third condition, the derivation of StrongFilterCondition is as follows:

[0303] As in HEVC, dpq is derived.

[0304] As in HEVC, sp3 = Abs(p3 - p0)

[0305] As in HEVC, sq3 = Abs(q0 - q3)

[0306] As in the HEVC design, StrongFilterCondition = (dpq is less than (β >> 2), sp3 + sq3 is less than (β >> 3), and Abs(p0 - q0) is less than (5 * t C + 1) >> 1).

[0307] 2.9.6 Strong Deblocking Filter for Chroma

[0308] The following defines a strong deblocking filter for chrominance:

[0309] p2′ = (3*p3 + 2*p2 + p1 + p0 + q0 + 4) >> 3

[0310] p1′ = (2*p3 + p2 + 2*p1 + p0 + q0 + q1 + 4) >> 3

[0311] p0′ = (p3 + p2 + p1 + 2*p0 + q0 + q1 + q2 + 4) >> 3

[0312] The proposed chrominance filter performs deblocking on a 4x4 chrominance sample grid.

[0313] 2.9.7 Position-Dependent Clipping

[0314] The position-dependent clipping tcPD is applied to samples in the output of the luminance filtering process that includes strong and long filters, which modify 7, 5, and 3 samples at the boundaries. Assuming a quantization error distribution, it is proposed to increase the clipping value of samples that are expected to have higher quantization noise, so that the reconstructed sample values are expected to have a greater deviation from the true sample values.

[0315] For each P or Q boundary filtered with an asymmetric filter, according to the result of the decision process in Section 2.3.3, the decoder selects a position-dependent threshold table from the two provided tables (i.e., Tc7 and Tc3 listed in the following table) as side information:

[0316] Tc7 = {6, 5, 4, 3, 2, 1, 1};

[0317] Tc3 = {6, 4, 2};

[0318] tcPD = (Sp == 3)? Tc3 : Tc7;

[0319] tcQD = (Sq == 3)? Tc3 : Tc7;

[0320] For P or Q boundaries filtered with a short symmetric filter, a lower-magnitude threshold depending on the unknown is applied:

[0321] Tc3 = {3, 2, 1};

[0322] After defining the thresholds, the filtered p’ i and q’ i sample values are clipped according to the tcP and tcQ clipping values:

[0323] p” i = Clip3(p’ i + tcP i , p’ i – tcPi , p' i );

[0324] q'' j = Clip3(q' j + tcQ j , q' j – tcQ j , q' j );

[0325] where p' i and q' i are the filtered sample values, p'' i and q'' j are the sample values output after clipping, and tcP i tcP i is the clipping threshold derived from the VVC tc parameters and tcPD and tcQD. The function Clip3 is the clipping function defined in VVC.

[0326] 2.9.8 Sub - block Deblocking Adjustment

[0327] As shown in the luma control, to be able to use the long filter and sub - block deblocking in parallel friendly, the long filter is restricted to modify at most 5 samples on the side where sub - block deblocking (AFFINE or ATMVP or DMVR) is used. Additionally, the sub - block deblocking is adjusted such that the sub - block boundaries close to the CU or implicit TU boundaries on the 8x8 grid are restricted to modify at most two samples on each side.

[0328] The following applies to sub - block boundaries that are not aligned with the CU boundary.

[0329]

[0330] Among them, the edge equal to 0 corresponds to the CU boundary, and the edges equal to 2 or equal to the orthogonal length - 2 correspond to 8 samples of the sub - block boundary from the CU boundary, etc. If the implicit partitioning of the TU is used, the implicit TU is true.

[0331] 2.9.9 Limitations of the 4CTU / 2CTU Row Buffers for Luma / Chroma

[0332] When the horizontal edge is aligned with the CTU boundary, the filtering of the horizontal edge limits Sp = 3 for luma, and Sp = 1 and Sq = 1 for chroma.

[0333] 2.10 Palette Mode (PCIP) Combined with Intra - frame Prediction

[0334] In JVET-M0051, a palette mode combined with intra prediction was proposed. In this scheme, the decoder first derives a prediction block based on the intra prediction method. Then, the decoder decodes the palette and the index map. The decoder uses the decoded palette information to refine the prediction block and reconstruct the block.

[0335] Figure 9 An example of the combination of palette information and intra prediction information is shown. First, the decoder generates a prediction block (with pixel values A0 - A15). And, the decoder decodes the index map in the palette mode. To reconstruct this block, if the decoded sequence number is equal to "0", the corresponding pixel is marked as "P", as Figures 2 - 6 shown. The pixel marked as "P" is reconstructed by the pixel value in the intra prediction block. Otherwise, this pixel is reconstructed by the palette color (e.g., C0, C1).

[0336] 2.11 Composite Palette Mode (CPM)

[0337] As in JVET-N0259, when the current index is 0, the sample in the composite palette mode can be reconstructed by copying the co-located sample in the IBC prediction. Otherwise, the current sample is reconstructed by the palette color. Figure 10 The proposed palette mode is shown.

[0338] In the composite palette mode, on the luminance block, the IBC prediction is generated by using the first available Merge candidate, so there is no need to signal the motion information of the IBC prediction. On the chrominance block, the motion compensation in CPM is the same as that in VTM4.

[0339] In addition, the encoding and decoding of escape pixels are also modified. The quantized residual between the original sample and the co-located IBC prediction sample is signaled. On the decoder, each escape pixel in the proposed palette mode is reconstructed by combining the decoded residual and the IBC prediction.

[0340] 2.12 Intra Mode Encoding and Decoding with 67 Intra Prediction Modes in JVET-N0185

[0341] To capture any edge direction that appears in natural videos, the number of directional intra modes in VTM4 is extended from 33 used in HEVC to 65. The new directional modes not in HEVC are indicated by red dashed arrows in Figure 11 And the planar mode and the DC mode remain unchanged. These denser directional intra prediction modes are applicable to all block sizes and intra prediction of luminance and chrominance.

[0342] For intra blocks, a unified 6-MPM list is proposed regardless of whether the encoding and decoding tools of MRL and ISP are applied. AsFigure 12 As shown, an MPM list is constructed based on the intra prediction modes of the left and upper neighboring blocks in VTM4.0.

[0343] Assuming that the mode on the left is denoted as "left" and the mode of the upper block is denoted as "up", the unified MPM list is constructed according to the following sequential steps:

[0344] – When the intra prediction mode of the neighboring block is invalid, its intra prediction mode is set to "Planar" by default.

[0345] – If "left" and "up" are the same and both are angular:

[0346] MPM list → {Planar, left, left - 1, left + 1, DC, left - 2}

[0347] – If "left" and "up" are different and both are angular:

[0348] Set the "Max" mode to the larger mode among "left" and "up"

[0349] If the difference between the "left" and "up" modes is between 2 and 62 (inclusive of 2 and 62),

[0350] MPM list → {Planar, left, up, DC, Max - 1, Max + 1}

[0351] Otherwise

[0352] MPM list → {Planar, left, up, DC, Max - 2, Max + 2}

[0353] – If "left" and "up" are different and one of the "left" and "up" modes is an angular mode and the other is a non - angular mode:

[0354] Set the "Max" mode to the larger mode among "left" and "up"

[0355] MPM list → {Planar, left, up, DC, Max - 1, Max + 1}

[0356] – If both the "left" and "up" modes are non - angular modes:

[0357] MPM list → {Planar, DC, V, H, V - 4, V + 4}

[0358] Note that when the mode index (e.g., Max + 2) is out of the range [0, 6], it can be replaced by another valid mode index.

[0359] Signal the first MPM candidate, i.e., the Planar mode, separately from the rest of the MPM candidates.

[0360] A more detailed description of syntax, semantics, and decoding processing is as follows:

[0361] 7.3.6.5 Coding and Decoding Unit Syntax

[0362]

[0363]

[0364]

[0365]

[0366] The syntax elements intra_luma_mpm_flag[x0][y0], intra_luma_not_planar_flag[x0][y0], intra_luma_mpm_idx[x0][y0], and intra_luma_mpm_remainder[x0][y0] specify the intra prediction mode for luma samples. The array indices x0, y0 specify the position (x0, y0) of the top-left luma sample of the coding and decoding block under consideration relative to the top-left luma sample of the picture. When intra_luma_mpm_flag[x0][y0] is equal to 1, the intra prediction mode is inferred from adjacent intra prediction coding units according to Clause 8.4.2.

[0367] When intra_luma_mpm_flag[x0][y0] does not exist (e.g., when ISP is enabled or MRL is enabled (reference index > 0)), it is inferred to be equal to 1.

[0368] When intra_luma_not_planar_flag[x0][y0] does not exist (e.g., when MRL is enabled), it is inferred to be equal to 1.

[0369] 8.4.2 Derivation Process of Luma Intra Prediction Mode

[0370] The inputs to this process are:

[0371] – The luma position (xCb, yCb) specifies the top-left sample of the current luma coding block relative to the top-left luma sample of the current picture,

[0372] – The variable cbWidth specifies the width of the current coding block in luma samples,

[0373] – The variable cbHeight specifies the height of the current coding block in luma samples.

[0374] In this process, the luma intra prediction mode IntraPredModeY[xCb][yCb] is derived.

[0375] Table 8-1 specifies the values of IntraPredModeY[xCb][yCb] for in-frame prediction mode and their related names.

[0376] Table 8-1 - Specification of In-frame Prediction Mode and Related Names

[0377]

[0378] Note: The in-frame prediction modes INTRA_LT_CCLM, INTRA_L_CCLM, and INTRA_T_CCLM are only applicable to chrominance components.

[0379] The derivation of IntraPredModeY[xCb][yCb] is as follows:

[0380] - If intra_luma_not_planar_flag[xCb][yCb] is equal to 1, then the following ordered steps are taken:

[0381] 1. Set the adjacent positions (xNbA, yNbA) to be equal to (xCb - 1, yCb + cbHeight - 1) and (xNbB, yNbB) to be equal to (xCb + cbWidth - 1, yCb - 1), respectively.

[0382] 2. For X replaced by A or B, the derivation of the variable candIntraPredModeX is as follows:

[0383] - Invoke the availability check process specified in Clause 6.4.X[Ed.(BB): Adjacent Block Availability Check Process tbd], where the position (xCurr, yCurr) set to be equal to (xCb, yCb) and the adjacent positions (xNbY, yNbY) set to be equal to (xNbX, yNbX) are used as inputs, and the output is assigned to availableX.

[0384] - The derivation of the candidate in-frame prediction mode candIntraPredModeX is as follows:

[0385] - If one or more of the following conditions are true, then set candIntraPredModeX equal to INTRA_PLANAR.

[0386] - The variable availableX is equal to FALSE.

[0387] - CuPredMode[xNbX][yNbX] is not equal to MODE_INTRA, and ciip_flag[xNbX][yNbX] is not equal to 1.

[0388] – The pcm_flag[xNbX][yNbX] is equal to 1.

[0389] – X is equal to B and yCb - 1 is less than ((yCb >> CtbLog2SizeY) << CtbLog2SizeY).

[0390] – Otherwise, set candIntraPredModeX equal to IntraPredModeY[xNbX][yNbX].

[0391] 3. The derivation of candModeList[x] for x = 0..4 is as follows:

[0392] – If candIntraPredModeB is equal to candIntraPredModeA, and candIntraPredModeA is greater than INTRA_DC, then the derivation of candModeList[x] for x = 0..4 is as follows:

[0393] candModeList[0] = candIntraPredModeA (8 - 10)

[0394] candModeList[1] = 2 + ((candIntraPredModeA + 61) % 64) (8 - 12)

[0395] candModeList[2] = 2 + ((candIntraPredModeA - 1) % 64) (8 - 13)

[0396] candModeList[3] = INTRA_DC (8 - 11)

[0397] candModeList[4] = 2 + ((candIntraPredModeA + 60) % 64) (8 - 14)

[0398] – Otherwise, if candIntraPredModeB is not equal to candIntraPredModeA, and candIntraPredModeA or candIntraPredModeB is greater than INTRA_DC, then the following conditions apply:

[0399] – The derivation of variables minAB and maxAB is as follows:

[0400] minAB = Min(candIntraPredModeA, candIntraPredModeB) (8 - 24)

[0401] maxAB = Max(candIntraPredModeA, candIntraPredModeB) (8 - 25)

[0402] – If both candIntraPredModeA and candIntraPredModeB are greater than INTRA_DC, the derivation of candModeList[x] for x = 0..4 is as follows:

[0403] candModeList[0] = candIntraPredModeA (8 - 27)

[0404] candModeList[1] = candIntraPredModeB (8 - 29)

[0405] candModeList[2] = INTRA_DC (8 - 29)

[0406] – If the range of maxAB - minAB is from 2 to 62 (inclusive of 2 and 62), the following applies:

[0407] candModeList[3] = 2 + ((maxAB + 61) % 64) (8 - 30)

[0408] candModeList[4] = 2 + ((maxAB - 1) % 64) (8 - 31)

[0409] – Otherwise, the following applies:

[0410] candModeList[3] = 2 + ((maxAB + 60) % 64) (8 - 32)

[0411] candModeList[4] = 2 + ((maxAB) % 64) (8 - 33)

[0412] – Otherwise (candIntraPredModeA or candIntraPredModeB is greater than INTRA_DC), the derivation of candModeList[x] for x = 0..4 is as follows:

[0413] candModeList[0] = ma (8 - 65)

[0414] candModeList[1] = INTRA_DC (8 - 66)

[0415] candModeList[2] = 2 + ((maxAB + 61) % 64) (8 - 66)

[0416] candModeList[3] = 2 + ((maxAB - 1) % 64) (8 - 67)

[0417] candModeList[4] = 2 + ((maxAB + 60) % 64) (8 - 68)

[0418] – Otherwise, the following applies: candModeList[0] = INTRA_DC (8 - 71)

[0419] candModeList[1] = INTRA_ANGULAR50 (8 - 72)

[0420] candModeList[2] = INTRA_ANGULAR18 (8 - 73)

[0421] candModeList[3] = INTRA_ANGULAR46 (8 - 74)

[0422] candModeList[4] = INTRA_ANGULAR54 (8 - 75)

[0423] 4. IntraPredModeY[xCb][yCb] can be derived by applying the following steps:

[0424] – If intra_luma_mpm_flag[xCb][yCb] is equal to 1, then set IntraPredModeY[xCb][yCb] to be equal to candModeList[intra_luma_mpm_idx[xCb][yCb]].

[0425] – Otherwise, derive IntraPredModeY[xCb][yCb] by applying the following ordered steps:

[0426] 1. For i = 0..3 and for each i, j = (i + 1)..4, when candModeList[i] is greater than candModeList[j], the two values will be swapped as follows:

[0427] (candModeList[i], candModeList[j]) = Swap(candModeList[i], candModeList[j]) (8 - 94)

[0428] 2. Derive IntraPredModeY[xCb][yCb] through the following ordered steps:

[0429] i. Set IntraPredModeY[xCb][yCb] to be equal to intra_luma_mpm_remainder[xCb][yCb].

[0430] ii. Increment the value of IntraPredModeY[xCb][yCb] by 1.

[0431] iii. For i equal to 0 to 4, inclusive, when IntraPredModeY[xCb][yCb] is greater than or equal to candModeList[i], increment the value of IntraPredModeY[xCb][yCb] by 1.

[0432] – Otherwise (when intra_luma_not_planar_flag[xCb][yCb] is equal to 0), set IntraPredModeY[xCb][yCb] to be equal to INTRA_PLANAR.

[0433] Set the variable IntraPredModeY[x][y] to be equal to IntraPredModeY[xCb][yCb] (where x = xCb..xCb+cbWidth-1 and y = yCb..yCb+cbHeight-1).

[0434] 2.13 History-based Merge Candidate Derivation

[0435] After the spatial MVP and TMVP, add the history-based MVP (HMVP) Merge candidates to the Merge list. In this method, the motion information of previously encoded / decoded blocks is stored in a table and used as the MVP for the current CU. The table with multiple HMVP candidates will be retained during the encoding / decoding process. When a new CTU row is encountered, reset (empty) this table. As long as there is a CU with inter-frame encoding / decoding that is not a sub-block, the associated motion information will be added as a new HMVP candidate to the last entry of the table.

[0436] In VTM5, the size S of the HMVP table is set to 5, which means that up to 5 history-based MVP (HMVP) candidates can be added to the table. When a new motion candidate is inserted into this table, the constrained first-in-first-out (FIFO) rule is utilized, where a redundancy check is first applied to find out if the same HMVP exists in the table. If found, the same HMVP is deleted from the table, and then all HMVP candidates are moved forward. The HMVP candidate can be used in the Merge candidate list construction process. The latest few HMVP candidates in the table are checked in order and inserted into the candidate list after the TMVP candidate. Apply redundancy checks to HMVP candidates to spatial Merge candidates.

[0437] 2.14 Zoom List

[0438] Scaling lists (cf quantization matrix) can be applied in the (inverse) quantization process. In HEVC, user-defined scaling values ​​are signaled via PPS, and each possible TB size, color component and prediction type (intra / inter) can have its own scaling list, except for 32x32 chroma blocks, which are only used for 4:4:4RExt chroma formats. For 16x16 and 32x32 scaling lists, this list is specified by an 8x8 grid of values ​​that are repeated to the required size, as well as a value for the entry corresponding to the DC frequency position.

[0439] 3. Examples of technical limitations of traditional technologies

[0440] Some issues are observed on blocks encoded or decoded in palette mode or its variants such as PCIP or CPM.

[0441] 1. Palette modes may need special treatment in deblocking.

[0442] 2. For better coding efficiency, CPM and / or PCIP can be considered as intra prediction modes.

[0443] 3. The MV / BV of a block encoded and decoded using CPM can be used as the MV / BV prediction of subsequent blocks to improve encoding and decoding efficiency.

[0444] 4. The intra prediction direction of a block encoded and decoded with PCIP can be used as the intra mode prediction of subsequent blocks to improve encoding and decoding efficiency.

[0445] 5. As in VTM, chroma CPM requires motion vector derivation to be performed at the sub-block level, which increases the complexity.

[0446] 6. The signaling of Chroma PCIP does not take into account the availability of Chroma IBC mode.

[0447] 4. Example Embodiments

[0448] The following list of embodiments should be regarded as examples for explaining general concepts. These embodiments should not be interpreted in a narrow sense. In addition, the listed techniques can be combined in any way, for example, to improve the encoding and decoding efficiency and reduce the decoding or encoding complexity.

[0449] The methods described below can be applied to the current palette mode, PCIP, CPM, or other encoding and decoding methods that may rely on the palette for residual encoding and decoding.

[0450] 1. In deblocking, in addition to the existing encoding and decoding modes (e.g., intra, inter, IBC), the palette mode can also be regarded as a separate mode (e.g., MODE_PLT).

[0451] a. In one example, if the blocks on the P side and the Q side are both encoded and decoded in the palette mode, the boundary strength can be set to 0.

[0452] b. In one example, if block A on one side is encoded and decoded in the palette mode while block B on the other side is not encoded and decoded in the palette mode, the boundary strength may only depend on the information of block B.

[0453] i. In one example, block B is encoded and decoded in the intra mode. In this case, the boundary strength can be set to 2.

[0454] ii. In one example, block B is encoded and decoded in the IBC mode. In this case, if block B has at least one non-zero coefficient, the boundary strength can be set to 1.

[0455] iii. In one example, block B is encoded and decoded in the IBC mode. In this case, if all the coefficients in block B are 0, the boundary strength can be set to 0.

[0456] c. In one example, the above methods can also be applied in PCIP, CPM, or other palette-related methods.

[0457] 2. In deblocking (e.g., boundary strength derivation), CPM can be treated in the same way as the IBC mode.

[0458] a. Optionally, CPM can be treated in the same way as the palette mode in deblocking (e.g., boundary strength derivation).

[0459] b. Optionally, CPM can be treated in the same way as the intra mode in deblocking (e.g., boundary strength derivation).

[0460] c. Optionally, how to apply the deblocking filter can depend on the use of CPM.

[0461] 3. The PCIP can be treated in the same way as the intra mode in the deblocking process (e.g., boundary strength derivation).

[0462] a. Optionally, the PCIP can be treated in the same way as the palette mode in the deblocking process (e.g., boundary strength derivation).

[0463] b. Optionally, how the deblocking filter is applied can depend on the use of the PCIP.

[0464] 4. The coding block flag of the palette coding / decoding block / CPM / PCIP can be set to a fixed value or depend on whether there are escape pixels in the block.

[0465] a. In one example, the coding block flag can be set to 0.

[0466] b. In one example, the coding block flag can be set to 1.

[0467] c. In one example, if the current block has no escape pixels, the coding block flag can be set to 0.

[0468] d. In one example, if the current block has at least one escape pixel, the coding block flag can be set to 1.

[0469] 5. Deblocking may not be performed on the samples coded / decoded using the palette mode and / or CPM and / or PCIP.

[0470] a. In one example, if the blocks on the P side or Q side are coded / decoded using CPM and / or PCIP, deblocking may not be performed on the samples on the P side and / or Q side.

[0471] b. In one example, if the blocks on the P side are coded / decoded using CPM and / or PCIP and the blocks on the Q side are not coded / decoded using these modes, deblocking may not be performed on the samples on the P side, and vice versa.

[0472] c. In one example, for the CPM mode, deblocking filtering may be performed only on the samples copied from IBC prediction. Deblocking may not be performed on the other samples in the palette entry.

[0473] d. In one example, for the PCIP mode, deblocking filtering may be performed only on the samples copied from intra prediction. Deblocking may not be performed on the other samples in the palette entry.

[0474] 6. The indication of the use of CPM for the transform unit / prediction unit / coding block / region can be coded separately from the existing prediction modes, which may include the intra mode, inter mode, IBC mode, and palette mode.

[0475] a. In one example, syntax elements related to the prediction mode (e.g., cu_skip_flag, pred_mode_flag, pred_mode_ibc_flag, and cu_palette_flag) can be coded and decoded before the usage indication of CPM.

[0476] b. Optionally, in addition, the usage indication of CPM can be signaled conditionally based on the prediction mode.

[0477] 1. In one example, when the prediction mode is the intra block copy mode (i.e., MODE_IBC), the signaling of the usage indication of the CPM mode can be skipped. In addition, when the current prediction mode is MODE_IBC, the usage indication of CPM can be inferred as false.

[0478] 7. Whether to signal the CPM and / or PCIP related syntax of the color component can depend on the partitioning structure and / or the color format and / or the separate plane coding.

[0479] a. In one example, for the chrominance component, if a single tree is applied, the signaling of the CPM and / or PCIP related syntax can be skipped.

[0480] b. Optionally, for the chrominance component, if a dual tree is applied, the CPM and / or PCIP related syntax can be signaled.

[0481] c. Optionally, if the color format is 4:0:0, the signaling of the CPM and / or PCIP related syntax for the chrominance block can be skipped.

[0482] i. Optionally, in addition, they can still be signaled, but they are not used in the decoding process.

[0483] 8. The signaling of the usage indication of the CPM mode for the chrominance block can be based on the coding and decoding information (e.g., prediction mode) of one or more selected blocks in the corresponding luma block.

[0484] a. In one example, when at least one luma block among the selected luma blocks is coded and decoded in the IBC mode, the CPM indication for the chrominance block can be signaled.

[0485] i. Optionally, when all the selected luma blocks are coded and decoded in the IBC mode, the CPM indication for the chrominance block can be signaled.

[0486] ii. Optionally, if the selected blocks are not coded and decoded using the intra block copy mode (i.e., MODE_IBC), the usage indication of the chrominance CPM may not be signaled. In addition, when the prediction mode of the selected blocks is not MODE_IBC, the usage indication of CPM can be inferred as the default value (e.g., false).

[0487] iii. Optionally, if not all selected blocks are encoded / decoded using the Intra Block Copy mode (i.e., MODE_IBC), the use indication of the chrominance CPM may not be signaled. When the prediction mode of the selected blocks is not MODE_IBC, the use indication of the CPM may be inferred as the default value (e.g., false).

[0488] b. In one example, the size of the selected luma block can be the smallest CU / PU / TU size or the unit for motion / mode storage (e.g., 4×4).

[0489] c. In one example, the selected luma block can be a CU / PU / TU located at the center, top-left, top-right, bottom-left, or bottom-right position covering the corresponding luma region. An example of the corresponding luma region is Figure 4 - 1 as shown.

[0490] i. In one example, assume that the top-left coordinates of the current chroma block are (x0, y0), and the width and height of the current chroma block are w0 and h0 respectively. The coordinates of the top-left sample, the width, and the height in the corresponding luma region can be scaled according to the color format.

[0491] a) For the 4:2:0 color format, the top-left coordinates of the co-located luma region are (2*x0, 2*y0), and its width and height are 2*w0 and 2*h0 respectively.

[0492] b) For the 4:4:4 color format, the top-left coordinates of the co-located luma region are (x0, y0), and its width and height are w0 and h0 respectively.

[0493] ii. Assume that the top-left coordinates of the co-located luma region are (x, y) and its width and height are W and H respectively. Then the coordinates of the center position can be:

[0494] a) (x + W / 2, y + H / 2)

[0495] b) (x + W / 2 - 1, y + H / 2 - 1)

[0496] c) (x + W / 2, y + H / 2 - 1)

[0497] d) (x + W / 2 - 1, y + H / 2)

[0498] iii. Assume that the top-left coordinates of the co-located luma region are (x, y), and its width and height are W and H. Then the coordinates of the top-left position can be (x, y).

[0499] iv. Assume that the top-left coordinates of the co-located luma region are (x, y) and its width and height are W and H. Then the coordinates of the top-right position can be:

[0500] a) (x + W, y)

[0501] b) (x + W - 1, y)

[0502] v. Assuming the upper - left coordinates of the co - located luminance region are (x, y), and its width and height are W and H, the coordinates of the lower - left position can be:

[0503] a) (x, y + H)

[0504] b) (x, y + H - 1)

[0505] vi. Assuming the upper - left coordinates of the co - located luminance region are (x, y) and its width and height are W and H, the coordinates of the lower - right position can be:

[0506] a) (x + W, y + H)

[0507] b) (x + W, y + H - 1)

[0508] c) (x + W - 1, y + H)

[0509] d) (x + W - 1, y + H - 1)

[0510] Figure 13 Examples of the corresponding luminance regions are shown.

[0511] 9. The signaling of the use of the chroma PCIP mode can depend on the coding and decoding information of one or more selected luminance blocks.

[0512] a. In one example, if the selected blocks in the corresponding luminance block are coded and decoded in the intra - block copy mode (i.e., MODE_IBC), the indication of the use of chroma PCIP may not be signaled. Additionally, when the prediction mode of the selected blocks of the corresponding luminance block is MODE_IBC, it can be inferred that the indication of the use of PCIP is false.

[0513] i. Optionally, when all selected luminance blocks are coded and decoded in the IBC mode, the PCIP indication for the chroma blocks may not be signaled.

[0514] ii. Optionally, when all samples in the corresponding luminance region are coded and decoded in the IBC mode, the PCIP indication for the chroma blocks may not be signaled.

[0515] b. In one example, the size of the selected luminance block can be the smallest CU / PU / TU size or the unit for motion / mode storage (e.g., 4×4).

[0516] c. In one example, the selected luminance block may be a CU / PU / TU covering the center, upper left, upper right, lower left, or lower right position of the corresponding luminance region. An example of the corresponding luminance region is as shown in Figure 4 - 1 shown.

[0517] i. In one example, assume that the upper left coordinates of the current chrominance block are (x0, y0), and the width and height of the current chrominance block are w0 and h0 respectively. The coordinates of the upper left sample point, the width and height of the corresponding luminance region can be scaled according to the color format.

[0518] a) For the 4:2:0 color format, the upper left coordinates of the co-located luminance region are (2*x0, 2*y0), and its width and height are 2*w0 and 2*h0 respectively.

[0519] b) For the 4:4:4 color format, the upper left coordinates of the co-located luminance region are (x0, y0), and its width and height are w0 and h0 respectively.

[0520] ii. Assume that the upper left coordinates of the co-located luminance region are (x, y) and its width and height are W and H respectively. Then the coordinates of the center position can be:

[0521] a) (x + W / 2, y + H / 2)

[0522] b) (x + W / 2 - 1, y + H / 2 - 1)

[0523] c) (x + W / 2, y + H / 2 - 1)

[0524] d) (x + W / 2 - 1, y + H / 2)

[0525] iii. Assume that the upper left coordinates of the co-located luminance region are (x, y), and its width and height are W and H. Then the coordinates of the upper left position can be (x, y).

[0526] iv. Assume that the upper left coordinates of the co-located luminance region are (x, y) and its width and height are W and H. Then the coordinates of the upper right position can be:

[0527] a) (x + W, y)

[0528] b) (x + W - 1, y)

[0529] v. Assume that the upper left coordinates of the co-located luminance region are (x, y), and its width and height are W and H. Then the coordinates of the lower left position can be:

[0530] a) (x, y + H)

[0531] b) (x, y + H - 1)

[0532] vi. Assuming the top - left coordinates of the co - located luminance region are (x, y) and its width and height are W and H, then the coordinates of the bottom - right position can be:

[0533] a)(x + W, y + H)

[0534] b)(x + W, y + H - 1)

[0535] c)(x + W - 1, y + H)

[0536] d)(x + W - 1, y + H - 1)

[0537] 10. It is proposed to consider the CPM mode as a separate prediction mode (e.g., represented by MODE_PLT_IBC) in addition to the existing prediction modes.

[0538] a. In one example, the prediction mode may include intra - frame of intra - frame strip / I - picture / intra - frame slice group, intra - frame block copy, palette mode, and CPM mode.

[0539] b. Optionally, the prediction mode may include intra - frame of intra - frame strip / I - picture / intra - frame slice group, palette mode, and CPM mode.

[0540] c. In one example, the prediction mode may include intra - frame, inter - frame, intra - frame block copy, palette mode, and CPM mode of inter - frame strip / P and / or B - picture / intra - frame slice group.

[0541] d. Optionally, the prediction mode may include at least two of intra - frame, inter - frame, intra - frame block copy, palette mode, and CPM mode.

[0542] e. In one example, when the prediction mode is an inter - frame mode (i.e., MODE_INTER), the signaling of the use indication of the CPM mode can be skipped. Additionally, if the current prediction mode is MODE_INTER, it can be inferred that the use indication of the CPM mode is false.

[0543] 11. CPM can be regarded as a special case of the existing prediction mode. In this case, the syntax related to the CPM mode can be further signaled under certain usage conditions of the existing prediction mode.

[0544] a. In one example, when the prediction mode is an intra - frame mode (i.e., MODE_INTRA), the signaling of the syntax related to CPM can be skipped. Additionally, if the current prediction mode is MODE_INTRA, it can be inferred that the use indication of the CPM mode is false.

[0545] b. In one example, when the prediction mode is the skip mode (i.e., the skip flag is equal to 1), signaling notification of the syntax related to CPM can be skipped. Additionally, if the skip mode is adopted on the current CU, the usage indication of CPM can be inferred as false.

[0546] c. In one example, when the prediction mode is the palette mode (e.g., MODE_PLT), the usage indication of CPM and the syntax related to CPM can be signaled.

[0547] d. Optionally, when the prediction mode is the intra mode, inter mode, or intra block copy mode, the syntax related to CPM can be skipped.

[0548] e. Optionally, when the prediction mode is the intra mode rather than the pulse code modulation (PCM) mode, the syntax related to CPM can be signaled.

[0549] f. In one example, when the prediction mode is the palette mode (e.g., MODE_PLT), the usage indication of CPM and the syntax related to CPM can be signaled before or after the usage indication of the PCM mode. In one example, when the CPM mode is applied, the syntax related to CPM can be signaled.

[0550] g. When the prediction mode is the intra mode, inter mode, or intra block copy mode, signaling notification of the syntax related to CPM can be skipped.

[0551] h. Optionally, when the prediction mode is the intra mode rather than the pulse code modulation (PCM) mode, the syntax related to CPM can be signaled.

[0552] i. In one example, when the prediction mode is the intra block copy mode, the syntax related to CPM can be signaled.

[0553] j. Optionally, when the prediction mode is the palette mode, inter mode, or intra mode, signaling notification of the syntax related to CPM can be skipped.

[0554] k. The above method can also be applied to PCIP or other palette-related methods.

[0555] 12. The MV of the block decoded by CPM can also be used as the MV prediction for subsequent blocks.

[0556] a. In one example, the MV of the block decoded by CPM can be used as a Merge candidate for other blocks.

[0557] b. In one example, the MV of the block decoded by CPM can be used as an HMVP candidate for other blocks.

[0558] i. Optionally, in addition, the HMVP table may be updated after decoding / encoding a block decoded / encoded by CPM.

[0559] c. In one example, the MV of a block decoded / encoded by CPM can be used as an AMVP candidate for other blocks.

[0560] d. Optionally, after decoding / encoding a block decoded / encoded by CPM, the update process of the HMVP table is not allowed.

[0561] e. Optionally, it is not allowed to use the BV of a block decoded / encoded by CPM as a motion vector predictor for subsequent blocks.

[0562] 13. The intra prediction direction of a block decoded / encoded by PCIP can also be used as an intra mode predictor for subsequent blocks.

[0563] a. In one example, the intra prediction direction of a block decoded / encoded by PCIP can be used as the MPM mode for subsequent blocks.

[0564] i. In one example, if the block decoded / encoded by PCIP selects the horizontal direction, the intra mode can be set to VER_IDX and used as the MPM mode for subsequent blocks.

[0565] ii. In one example, if the block decoded / encoded by PCIP selects the vertical direction, the intra mode can be set to HOR_IDX and used as the MPM mode for subsequent blocks.

[0566] b. Optionally, it is not allowed to use the intra prediction direction of a block decoded / encoded by CPM as the MPM for subsequent blocks.

[0567] 14. The maximum number of entries to be used in the palette can vary from one video unit (e.g., CTB, CU, PU, TU) to another. The binarization can be changed according to the maximum number of entries in the palette to be used.

[0568] a. In one example, it can depend on the coding information.

[0569] i. In one example, it can depend on the block dimension.

[0570] a) In one example, the size of a larger block decoded / encoded using the palette mode may have a larger maximum number of entries, while the size of a smaller block decoded / encoded using the palette mode may have a smaller maximum number of entries.

[0571] b) In one example, for a 4x4 block, only N0 palette entries may be allowed.

[0572] c) In one example, for an 8×4 block and / or a 4×8 block, only N1 palette entries may be allowed.

[0573] d) In one example, for an 8x8 block, only N2 palette entries may be allowed.

[0574] e) In one example, for a 16x16 block, only N3 palette entries may be allowed.

[0575] f) In one example, N0, N1, N2, and N3 are fixed numbers (e.g., 3, 7, or 15), and they can be the same or different.

[0576] g) In one example, the relationship between the maximum number of palette entries and the block dimensions is shown in the following table, where W and H represent the current block width and block height, respectively, and Max is an operation that obtains the larger value between two inputs.

[0577] Table 4-1 Example of the relationship between the maximum palette entries and the block dimensions

[0578] Max(W,H) Maximum value of palette entries 4 3 8 7 16 15 32 31 64 31 128 31

[0579] h) In one example, the relationship between the maximum number of palette entries and the block dimensions is shown in the following table, where W and H represent the current block width and block height, respectively, and Max is an operation that obtains the larger value between two inputs.

[0580] Table 4-2 An example of the relationship between the maximum palette entries and the block dimensions

[0581] Max(W,H) Maximum value of palette entries 4 3 8 15 16 15 32 31 64 31 128 31

[0582] b. In one example, it may depend on the color components.

[0583] i. In one example, a luminance block encoded and decoded in the palette mode may have a larger maximum number of entries, while a chrominance block encoded and decoded in the palette mode may have a smaller maximum number of entries.

[0584] a) In one example, the maximum number of entries for a luminance block can be N, and thus the maximum number of entries for a chrominance block can be M.

[0585] i. In one example, M is a fixed number (e.g., 3, 7, or 15), and M can be less than N.

[0586] c. In one example, it may depend on the partitioning structure, such as a dual-tree or a single-tree.

[0587] 15. A scaling matrix (e.g., a user-defined scaling matrix) that prohibits the scaling of blocks encoded and decoded in the palette / CPM / PCIP mode is proposed.

[0588] a. Optionally, a scaling matrix may be applied to the blocks coded / decoded by palette / CPM / PCIP.

[0589] b. How to select a scaling matrix for the blocks coded / decoded by palette / CPM / PCIP may be performed in the same way as for the transform skip coded blocks.

[0590] 16. Whether the above method is applicable and / or how to adopt it may be based on:

[0591] a. Video content (such as screen content or natural content)

[0592] b. Messages signaled in DPS / SPS / VPS / PPS / APS / picture header / strip header / slice group header / largest coding unit (LCU) / coding unit (CU) / LCU row / group of LCUs / TU / PU block / video coding unit.

[0593] c. The position of CU / PU / TU / block / video coding unit.

[0594] d. The block dimensions of the current block and / or its neighboring blocks.

[0595] i. In one example, the above method may be applied only when both the width and height of the current block are less than T (e.g., 32 or 64).

[0596] e. The block shape of the current block and / or its neighboring blocks.

[0597] f. The prediction mode of the current block.

[0598] g. The intra mode of the current block and / or its neighboring blocks.

[0599] h. The motion / block vector of the current block and / or its neighboring blocks.

[0600] i. Indication of color format (e.g., 4:2:0, 4:4:4).

[0601] j. Coding tree structure.

[0602] k. Strip / slice group type and / or picture type.

[0603] l. Color component (e.g., can be applied only to chrominance component or luminance component).

[0604] m. Temporal layer ID.

[0605] n. Profile / level / tier of the standard.

[0606] 5. Embodiments on top of JVET-N1001-v6

[0607] The following changes made on top of the draft provided by JVET-N1001-v6 are highlighted in bold italics. Double brackets are placed before and after the deleted text.

[0608] Sequence parameter set RBSP syntax

[0609]

[0610] Coding unit syntax

[0611]

[0612]

[0613] Prediction mode syntax

[0614] It is proposed that all the text and tables in the following are new additions to the current version VVC.

[0615]

[0616]

[0617]

[0618] Palette syntax

[0619]

[0620]

[0621]

[0622]

[0623]

[0624]

[0625] The following description corresponds to the changes discussed in the draft.

[0626] Palette semantics

[0627] In the following semantics, the array indices x0, y0 specify the position (x0, y0) of the top-left luma sample of the coding block under consideration relative to the top-left luma sample of the picture.

[0628] cu_palette_flag specifies the use of the palette mode in the current coding unit. cu_palette_flag == 1 indicates that the palette mode is applied in the current coding unit. cu_palette_flag == 0 indicates that the palette mode does not apply to the current coding unit.

[0629] The palette_predictor_run is used to determine the number of zeros before a non-zero entry in the array PalettePredictorEntryReuseFlags.

[0630] Bitstream conformance requirement: The value of palette_predictor_run shall be in the range from 0 to (PredictorPaletteSize - predictorEntryIdx) (inclusive of 0 and (PredictorPaletteSize - predictorEntryIdx)), where predictorEntryIdx corresponds to the current position in the array PalettePredictorEntryReuseFlags. The variable NumPredictedPaletteEntries specifies the number of entries reused from the predictor palette in the current palette. The value of NumPredictedPaletteEntries shall be in the range from 0 to Palette_max_size (inclusive of 0 and Palette_max_size).

[0631] num_signalled_palette_entries[startComp] specifies the number of entries in the current palette that are explicitly signalled by the first color component of the current palette table startComp.

[0632] If num_signalled_palette_entries[startComp] does not exist, it is inferred to be equal to 0.

[0633] The variable CurrentPaletteSize specifies the size of the current palette and is derived as follows:

[0634] CurrentPaletteSize[startComp] = NumPredictedPaletteEntries[startComp] + num_signalled_palette_entries[startComp](XX)

[0635] The value of CurrentPaletteSize[startComp] shall be in the range from 0 to palette_max_size (inclusive of 0 and palette_max_size).

[0636] new_palette_entries[cIdx][i] specifies the value of the i-th signaling notified palette entry for color component cIdx.

[0637] The variable PredictorPaletteEntries[cIdx][i] specifies the i-th element of color component cIdx in the predictor palette.

[0638] The variable CurrentPaletteEntries[cIdx][i] specifies the i-th element of color component cIdx in the current palette, and its derivation is as follows:

[0639]

[0640]

[0641] When palette_escape_val_present_flag is equal to 1, it specifies that the current coding unit contains at least one escaped coded sample. When escape_val_present_flag is equal to 0, it specifies that there is no escaped coded sample in the current coding unit. If it does not exist, then it is inferred that the value of palette_escape_val_present_flag is equal to 1.

[0642] The variable MaxPaletteIndex specifies the maximum possible value of the palette index of the current coding unit. If cu_palette_ibc_mode is 0, then the value of MaxPaletteIndex is set to CurrentPaletteSize + Palette_escape_val_present_flag. Otherwise, if cu_palette_ibc_mode is 1, then MaxPaletteIndex is set to be equal to CurrentPaletteSize + Palette_escape_val_present_flag + 1

[0643] num_palette_indices_minus1 plus 1 is the number of explicitly signaled or inferred index palette indices of the current block.

[0644] If num_palette_indices_minus1 does not exist, then it is inferred that it is equal to 0.

[0645] The palette_index_idc is an indication of the array index represented by CurrentPaletteEntries. For the first index in the block, the value of palette_index_idc shall be in the range of 0 to MaxPaletteIndex (including the endpoints), and for the remaining indices in the block, the value of palette_index_idc shall be in the range of 0 to (MaxPaletteIndex - 1) (including the endpoints).

[0646] If palette_index_idc does not exist, it is inferred to be equal to 0.

[0647] The variable PaletteIndexIdc[i] stores the explicitly signaled or inferred i-th palette_index_idc. All elements of the array PaletteIndexIdc[i] are initialized to 0.

[0648] copy_above_indices_for_final_run_flag being equal to 1 specifies that if horizontal raster scanning is used, the index palette index at the last position in the coding / decoding unit is copied from the index palette index of the previous row. Or if vertical raster scanning is used, the value is copied from the index palette index of the left column. copy_above_indices_for_final_run_flag being equal to 0 specifies that the index palette index at the last position in the coding / decoding unit is copied from PaletteIndexIdc[num_palette_indices_minus1].

[0649] If copy_above_indices_for_final_run_flag does not exist, it is inferred to be equal to 0.

[0650] palette_transpose_flag being equal to 1 specifies that vertical raster scanning is used to scan the indices of the pixels in the current coding / decoding unit. palette_transpose_flag being equal to 0 specifies that horizontal raster scanning is used to scan the indices of the pixels in the current coding / decoding unit.

[0651] copy_above_palette_indices_flag being equal to 1 specifies that if horizontal raster scanning is used, the palette index is equal to the palette index at the same position in the previous row; if vertical raster scanning is used, it is at the same position in the left column. copy_above_palette_indices_flag being equal to 0 specifies that the indication of the palette index of the sample is coded or inferred in the bitstream.

[0652] The variable CopyAboveIndicesFlag[xC][yC] being equal to 1 specifies that the palette index is copied from the palette index in the previous row (horizontal scan) or left column (vertical scan). CopyAboveIndicesFlag[xC][yC] being equal to 0 specifies that the palette index is explicitly encoded / decoded or inferred in the bitstream. The array indices xC, yC specify the position (xC, yC) of the sample relative to the top-left luma sample of the picture.

[0653] The variable PaletteIndexMap[xC][yC] specifies the palette index, which is the index of the array represented by CurrentPaletteEntries. The array indices xC, yC specify the position (xC, yC) of the sample relative to the top-left luma sample of the picture. The value of PaletteIndexMap[xC][yC] shall be in the range from 0 to MaxPaletteIndex (including 0 and MaxPaletteIndex).

[0654] When CopyAboveIndicesFlag[xC][yC] is equal to 1, the variable PaletteRun specifies the number of consecutive positions with the same palette index in the previous row (horizontal scan) or left column (vertical scan) minus 1. Or when CopyAboveIndicesFlag[xC][yC] is equal to 0, it specifies the number of consecutive positions with the same palette index minus 1.

[0655] The variable PaletteMaxRun represents the maximum possible value of PaletteRun. Bitstream conformance requires that the value of PaletteMaxRun shall be greater than or equal to 0.

[0656] palette_run_prefix specifies the prefix part in the binarization of PaletteRun.

[0657] palette_run_suffix specifies the suffix part in the binarization of PaletteRun. If palette_run_suffix does not exist, it is inferred that the value of palette_run_suffix is equal to 0.

[0658] The value of PaletteRun is derived as follows:

[0659] – If palette_run_prefix is less than 2, the following applies:

[0660] PaletteRun = Palette_run_prefix(XX)

[0661] – Otherwise (palette_run_prefix is greater than or equal to 2), the following conditions apply:

[0662] PrefixOffset = 1 << (palette_run_prefix - 1)

[0663] PaletteRun = PrefixOffset + Palette_run_suffix(XX)

[0664] The palette_escape_val specifies the escape encoded / decoded sample value for the quantization of the component.

[0665] The variable PaletteEscapeVal[cIdx][xC][yC] specifies the escape value of the sample whose PaletteIndexMap[xC][yC] is equal to (MaxPaletteIndex – 1) and the palette_escape_val_present_flag is equal to 1. The array index cIdx specifies the color component. The array indices xC, yC specify the position (xC, yC) of the sample relative to the top-left luma sample of the picture.

[0666] Bitstream conformance requirement: The range of PaletteEscapeVal[cIdx][xC][yC] should be in the range from 0 to (1 << ((BitDepthY + 1)) - 1 (inclusive) for cIdx equal to 0, and in the range from 0 to (1 << (BitDepthC + 1)) - 1 for cIdx not equal to 0.

[0667] General decoding process of the coded / decoded unit decoded in the intra prediction mode

[0668] The inputs to this process are:

[0669] – The luma position (xCb, yCb), relative to the top-left luma sample of the current picture, specifying the top-left sample of the current coded / decoded block;

[0670] – The variable cbWidth, specifying the width of the current coded / decoded block in luma samples,

[0671] – The variable cbHeight, specifying the height of the current coded / decoded block in luma samples,

[0672] – The variable treeType, specifying whether to use a single tree or a dual tree, and if a dual tree is used, it specifies whether the current tree corresponds to the luma or chroma component.

[0673] The output of this process is the modified reconstructed picture before loop filtering.

[0674] Invoke the derivation process of quantization parameters specified in Clause 8.4.1, taking the luminance position (xCb, yCb), the width cbWidth of the current coded / decoded block in the luminance samples, the height cbHeight of the current coded / decoded block in the luminance samples, and the variable treeType as inputs.

[0675] When treeType is equal to SINGLE_TREE or treeType is equal to DUAL_TREE_LUMA, the decoding process of the luminance samples is specified as follows:

[0676] – If pcm_flag[xCb][yCb] is equal to 1, modify the reconstructed image as follows:

[0677] S L [xCb + i][yCb + j] = pcm_sample_luma[(cbHeight * j) + i] << (BitDepth Y - PcmBitDepth Y ),(8 - 1),

[0678] where i = 0..cbWidth - 1, j = 0..cbHeight – 1

[0679] – Otherwise, if cu_palette_flag[xCb][yCb] is equal to 1, the following applies:

[0680] – If treeType is equal to SINGLE_TREE, the following applies

[0681] 1. Invoke the general decoding process of the palette block specified in Clause XXX with the luminance position (xCb, yCb), where the variable startComp is set to be equal to 0, the variable cIdx is set to be equal to 0, the variable nTbW is set to be equal to cbWidth, and the variable nTbH is set to be equal to cbHeight.

[0682] 2. Invoke the general palette predictor update process of the palette block specified in Clause 8.X.X with the luminance position (xCb, yCb), where the variable startComp is set to be equal to 0 and the variable numComps is set to be equal to 1.

[0683] – Otherwise, the following applies:

[0684] 1. Invoke the derivation process of the luma intra prediction mode specified in Clause 8.2.2, where the luma position (xCb, yCb), the width cbWidth of the current coding / decoding block in luma samples, the height cbHeight of the current coding / decoding block, and the luma samples are used as inputs.

[0685] 2. Invoke the general decoding process of intra blocks specified in Clause 8.2.4.1, where the luma position (xCb, yCb), the tree type treeType, the variable nTbW set to be equal to cbWidth, the variable nTbH set to be equal to cbHeight, the variable predModeIntra set to be equal to IntraPredModeY[xCb][yCb], and the variable cIdx set to be equal to 0 are used as inputs, and its output is the modified reconstructed picture before loop filtering.

[0686] When treeType is equal to SINGLE_TREE or treeType is equal to DUAL_TREE_CHROMA, the decoding process of chroma samples is specified as follows:

[0687] – If pcm_flag[xCb][yCb] is equal to 1, then modify the reconstructed image as follows:

[0688] S Cb [xCb / SubWidthC+i][yCb / SubHeightC+j] =

[0689] pcm_sample_chroma[(cbHeight / SubWidthC*j)+i]<<(BitDepth C -PcmBitDepth C ),

[0690] where i = 0..cbWidth / SubWidthC – 1 and j = 0..cbHeight / SubHeightC-1 (8-2)

[0691] S Cr [xCb / SubWidthC+i][yCb / SubHeightC+j] =

[0692] pcm_sample_chroma[(cbHeight / SubWidthC*(j+cbHeight / SubHeightC))+i]<<

[0693] (BitDepth C -PcmBitDepth C ),

[0694] where i = 0..cbWidth / SubWidthC - 1 and j = 0..cbHeight / SubHeightC - 1 (8 - 3)

[0695] – Otherwise, if cu_palette_flag[xCb][yCb] equals 1, the following applies:

[0696] – If treeType equals SINGLE_TREE, the following applies:

[0697] 1. Call the general decoding process of the palette segment specified in clause XXX with the luma position (xCb, yCb), where the variable startComp is set to equal 0, the variable cIdx is set to 1, the variable nTbW is set to equal (cbWidth / 2), and the variable (cbHeight / 2) is set to equal cbHeight.

[0698] 2. Call the general decoding process of the palette segment specified in clause XXX with the luma position (xCb, yCb), where the variable startComp is set to equal 0, the variable cIdx is set to 2, the variable nTbW is set to equal (cbWidth / 2), and the variable (cbHeight / 2) is set to equal cbHeight.

[0699] 3. Call the general palette predictor update process of the palette segment specified in clause 8.X.X with the luma position (xCb, yCb), where the variable startComp is set to equal 0 and the variable numComps is set to 3.

[0700] – Otherwise, the following applies:

[0701] 1. Call the general decoding process of the palette segment specified in clause XXX with the luma position (xCb, yCb), where the variable startComp is set to equal 1, the variable cIdx is set to equal 1, the variable nTbW is set to equal (cbWidth / 2), and the variable (cbHeight / 2) is set to equal cbHeight.

[0702] 2. Call the general decoding process of the palette segment specified in clause XXX with the luma position (xCb, yCb), where the variable startComp is set to equal 1, the variable cIdx is set to 2, the variable nTbW is set to equal (cbWidth / 2), and the variable (cbHeight / 2) is set to equal cbHeight.

[0703] 3. Invoke the general palette predictor update process of the color palette specified in Clause XXX using the luminance position (xCb, yCb), where the variable startComp is set to be equal to 1 and the variable numComps is set to be equal to 2.

[0704] – Otherwise, the following applies:

[0705] 1. Invoke the derivation process of the chrominance intra prediction mode specified in Clause 8.2.3, where the luminance position (xCb, yCb), the width cbWidth and the height in the current coded block of the luminance samples, and the luminance samples cbHeight are used as inputs.

[0706] 2. Invoke the general decoding process of the intra block specified in Clause 8.2.4.1 using the chrominance position (xCb / 2, yCb / 2) and the tree type treeType, where the variable nTbW set to be equal to (cbWidth / 2), the variable nTbH set to be equal to (cbHeight / 2), the variable predModeIntra set to be equal to IntraPredModeC[xCb][yCb], the variable cIdx set to 1, and the output is the modified reconstructed picture before loop filtering.

[0707] 3. When invoking the general decoding process of the intra block specified in Clause 8.2.4.1 using the chrominance position (xCb / 2, yCb / 2) and the tree type treeType, where the variable nTbW set to be equal to (cbWidth / 2), the variable nTbH set to be equal to (cbHeight / 2), the variable predModeIntra set to be equal to IntraPredModeC[xCb][yCb], the variable cIdx set to 2, and the output is the modified reconstructed picture before loop filtering.

[0708] Decoding process of the palette mode

[0709] The inputs to this process are:

[0710] – The position (xCb, yCb), specifying the top-left luminance sample of the current block relative to the top-left luminance sample of the current picture

[0711] – The variable startComp specifies the first color component in the palette table

[0712] – A variable cIdx, specifying the color component of the current block

[0713] – Two variables nTbW and nTbH respectively specify the width and height of the current block.

[0714] The output of this process is an array recSamples[x][y], where x = 0..nTbW-1 and y = 0..nTbH-1, which specifies the reconstructed sample values for this block.

[0715] The variables nSubWidth and nSubHeight depend on the value of cIdx and are derived as follows:

[0716] – If cIdx is equal to 0, then nSubWidth is set to 1 and nSubHeight is set to 1.

[0717] – If startComp is equal to 1 and if cIdx > 1, then nSubWidth is set to 1 and nSubHeight is set to 1.

[0718] – Otherwise, nSubWidth is set to SubWidthC and nSubHeight is set to SubHeightC.

[0719] – If cu_palette_ibc_mode[xCb][yCb] is equal to 1, then the following applies:

[0720] – Call the motion vector derivation process specified in Clause 8.6.2, where the top-left sample of the current luma coding block is the luma position (xCb, yCb) relative to the top-left luma sample of the current picture, the variable cbWidth specifies the width of the current coding block in luma samples, and the variable cbHeight specifies the height of the current coding block in luma samples. Its output is the luma motion vector in 1 / 16 fractional sample precision mvL.

[0721] - Call the general IBC prediction process specified in Clause 8.6.3, where the luma position (xCb, yCb) specifies that the top-left sample of the current coding block is relative to the top-left sample of the current picture, the variable cbWidth specifies the width of the current coding block in luma samples, the variable cbHeight specifies the height of the current coding block in luma samples, the variables numSbX and numSbY specify the number of luma coding sub-blocks in the horizontal and vertical directions, the motion vectors mv[xSbIdx][ySbIdx] where xSbIdx = 0..numSbX–1 and ySbIdx = 0..numSbY–1, and the variable cIdx specifies the color component index of the current block. Its output is an array predSamples of predicted samples predSamples[x][y].

[0722] The (nTbW x nTbH) block of the reconstructed sample array recSamples at position (xCb, yCb) is represented by recSamples[x][y], where x = 0..nTbW-1 and y = 0..nTbH–1. And for each x in the range from 0 to nTbW-1 (including 0 and nTbW-1) and each y in the range from 0 to nTbH-1 (including 0 and nTbH-1), the value of recSamples[x][y] is derived as follows:

[0723] – The variables xL and yL are derived as follows:

[0724] xL = palette_transpose_flag? x * nSubHeight : x * nSubWidth (5-4)

[0725] yL = palette_transpose_flag? y * nSubWidth : y * nSubHeight (5-5)

[0726] – The variable bIsEscapeSample is derived as follows:

[0727] – If PaletteIndexMap[xCb+xL][yCb+yL] is equal to MaxPaletteIndex and palette_escape_val_present_flag is equal to 1, then set bIsEscapeSample to be equal to 1.

[0728] – Otherwise, set bIsEscapeSample to be equal to 0.

[0729] – If bIsEscapeSample is equal to 0, then the following applies:

[0730] – If cu_palette_ibc_mode[xCb][yCb] is equal to 1, then the following applies:

[0731] – If PaletteIndexMap[xCb+xL][yCb+yL] is equal to 0, then the following applies: recSamples[x][y] = predSamples[x][y]

[0732] – Otherwise, the following applies:

[0733] recSamples[x][y] = CurrentPaletteEntries[cIdx][PaletteIndexMap[xCb + xL][yCb + yL] - 1] (5-6)

[0734] – Otherwise, the following applies:

[0735] recSamples[x][y] = CurrentPaletteEntries[cIdx][PaletteIndexMap[xCb + xL][yCb + yL]] (5-7)

[0736] – Otherwise, if cu_transquant_bypass_flag is equal to 1, the following applies:

[0737] recSamples[x][y] = PaletteEscapeVal[cIdx][xCb + xL][yCb + yL](5-8)

[0738] – Otherwise (bIsEscapeSample is equal to 1 and cu_transquant_bypass_flag is equal to 0), the following ordered steps apply:

[0739] 1. Invoke the derivation process of quantization parameters, where the position (xCb, yCb) specifies the top-left sample of the current block relative to the top-left sample of the current picture.

[0740] 2. The quantization parameter qP is derived as follows:

[0741] – If cIdx is equal to 0,

[0742] qP = Max(0, Qp'Y) (5-9)

[0743] – Otherwise, if cIdx is equal to 1,

[0744] qP = Max(0, Qp'Cb) (5-10)

[0745] – Otherwise (cIdx is equal to 2),

[0746] qP = Max(0, Qp′Cr) (5-11)

[0747] 3. The variable bitDepth is derived as follows:

[0748] bitDepth = (cIdx == 0)? BitDepth Y : BitDepth C (5-12)

[0749] 4. Define the list levelScale[] as levelScale[k] = {40, 45, 51, 57, 64, 72}, where k = 0..5.

[0750] 5. The following applies:

[0751] tmpVal = (PaletteEscapeVal[cIdx][xCb + xL][yCb + yL] * levelScale[qP % 77]) << ((qP / 6) + 32) >> 6 (5 - 13) recSamples[x][y] = Clip3(0, (1 << bitDepth) - 1, tmpVal) (5 - 14)

[0752] Palette predictor update process in palette mode

[0753] The input to this process is:

[0754] – Position (xCb, yCb), which defines that the top - left luminance sample of the current block is relative to the top - left luminance sample of the current picture,

[0755] – Variable startComp, which defines the first color component in the palette table,

[0756] – Variable numComps, which defines the number of color components in the palette table,

[0757] The output of this process is the variable PredictorPaletteSize[startComp], which defines the size of the updated palette predictor variable, and the array PredictorPaletteEntries[cIdx][i], where cIdx = startComp,..., startComp + numComps - 1, i = 0..PredictorPaletteSize - 1, which defines the updated palette predictor for the next block.

[0758] The derivation or modification of the variables PredictorPaletteSize and the array PredictorPaletteEntries is as follows

[0759] The requirement for bit - stream consistency is that the value of PredictorPaletteSize should be in the range from 0 to PaletteMaxPredictorSize (including 0 and PaletteMaxPredictorSize).

[0760] Figure 14Ais a block diagram of a video processing apparatus 1400. The apparatus 1400 can be used to implement one or more methods described herein. The apparatus 1400 can be embodied in a smart phone, a tablet computer, a computer, an Internet of Things (IoT) receiver, etc. The apparatus 1400 can include one or more processors 1402, one or more memories 1404, and video processing hardware 1406. The processor 1402 can be configured to implement one or more methods described in this application document. The memory (or memories) 1404 can be used to store data and code for implementing the methods and techniques described herein. The video processing hardware 1406 can be used to implement some of the techniques described in this application document in hardware circuits. The video processing hardware 1406 can be partially or fully included within the processor 1402 in the form of dedicated hardware or a graphics processing unit (GPU) or a dedicated signal processing block.

[0761] Figure 14B is another example of a block diagram of a video processing system in which the disclosed techniques can be implemented. Figure 14B is a block diagram showing an example video processing system 1410 in which various techniques disclosed herein can be implemented. Various embodiments can include some or all components of the system 1410. The system 1410 can include an input 1412 for receiving video content. The video content can be received in a raw or uncompressed format, such as 8-bit or 10-bit multi-component pixel values, or can be received in a compressed or encoded format. The input 1412 can represent a network interface, a peripheral bus interface, or a storage interface. Examples of network interfaces include wired interfaces such as Ethernet, passive optical network (PON), etc. and wireless interfaces such as Wi-Fi or cellular interfaces.

[0762] The system 1410 can include a codec component 1414 that can implement various encoding or decoding methods described in this application document. The codec component 1414 can reduce the average bit rate of the video from the input 1412 to the output of the codec component 1414 to produce a coded or decoded representation of the video. Thus, codec techniques are sometimes referred to as video compression or video transcoding techniques. The output of the codec component 1414 can be stored or can be transmitted through a connected communication, as shown by component 1416. The stored or communicated bitstream (or coded) representation of the video received at the input 1412 can be used by the component 1418 to generate pixel values or a displayable video to be sent to the display interface 1420. The process of generating a user-viewable video from the bitstream representation is sometimes referred to as video decompression. Additionally, although certain video processing operations are referred to as "codec" operations or tools, it should be understood that codec tools or operations are used at the encoder, and the corresponding decoding tools or operations for reversing the encoding result will be performed by the decoder.

[0763] Examples of a peripheral bus interface or a display interface can include a Universal Serial Bus (USB), a High-Definition Multimedia Interface (HDMI), a DisplayPort, etc. Examples of a storage interface include SATA (Serial Advanced Technology Attachment), PCI, IDE interface, etc. The techniques described herein can be implemented in various electronic devices, such as mobile phones, laptop computers, smart phones, or other devices capable of performing digital data processing and / or video display.

[0764] Figure 15A An example method 1510 for video processing is shown. Method 1510 includes, at step 1512, for the conversion between a video unit of a video and an encoded / decoded representation of the video, determining, according to a rule, a maximum number of entries of a palette of representative values to be used during the conversion. Method 1510 further includes, at step 1514, performing the conversion using the palette. In some embodiments, the rule specifies the maximum number of entries according to characteristics of the video unit.

[0765] Figure 15B An example method 1520 for video processing is shown. Method 1520 includes, at step 1522, for the conversion between a current video block of a video and an encoded / decoded representation of the video, determining, according to a rule, one or more encoding / decoding parameters for deblocking processing of the current video block based on the use of a palette mode encoding / decoding tool for the current video block. Method 1520 further includes, at step 1524, performing the conversion using the one or more encoding / decoding parameters. In some embodiments, the rule specifies that one or more encoding / decoding parameters for a video block encoded / decoded using a palette mode encoding / decoding tool are derived differently from parameters for other encoding / decoding tools. In some embodiments, the palette mode encoding / decoding tool includes: during encoding, representing the current video block as the encoded / decoded representation using a palette of representative sample values; or during decoding, reconstructing the current video block from the encoded / decoded representation using a palette of representative sample values.

[0766] Figure 15CAn example method 1530 for video processing is shown. Method 1530 includes, at step 1532, for the conversion between a current video block of a video and the coded representation of the video, determining one or more parameters for deblocking processing to be applied to the current video block based on a rule based on the use of a Composite Palette Mode (CPM) coding tool for the current video block. Method 1530 further includes, at step 1534, performing the conversion using the one or more parameters. In some embodiments, the rule specifies that the coded representation includes an indication of one or more parameters using syntax elements that are the same as the syntax elements for an adjacent video block coded using another coding mode. In some embodiments, the CPM coding tool includes representing or reconstructing the current video block by combining the use of a palette of representative entries with one or more samples derived by an Intra Block Copy mode.

[0767] Figure 15D An example method 1540 for video processing is shown. Method 1540 includes, at step 1542, for the conversion between a current video block of a video and the coded representation of the video, determining parameters for deblocking processing to be applied to the current video block based on a rule based on the coding mode of an adjacent video block. Method 1540 further includes, at step 1544, performing the conversion based on the determination. In some embodiments, the rule specifies using the same parameters for the case of coding the adjacent video block using a Palette Mode Combined with Intra Prediction (PCIP) or another coding mode. In some embodiments, the PCIP includes using intra prediction to derive a prediction block of the current video block and using palette information associated with the current video block to refine the prediction block.

[0768] Figure 15EAn example method 1550 for video processing is shown. Method 1550 includes, at step 1552, performing a conversion between a video unit of a video and an encoded / decoded representation of the video. In some embodiments, the video unit corresponds to a current video block of the video. In some embodiments, the current video block is represented in the encoded / decoded representation according to an encoding / decoding mode, the encoding / decoding mode being a palette encoding / decoding mode, or a palette mode combined with an intra prediction mode (PCIP), or a composite palette mode (CPM), wherein the encoded / decoded representation includes a flag of the current video block, the flag indicating whether the current video block has non-zero coefficients according to a rule, wherein the rule provides that the flag has a value according to the encoding / decoding mode or the presence of escape pixels in the encoded / decoded representation of the current video block, wherein the palette encoding / decoding mode includes representing or reconstructing the current video block using a palette of representative sample values, wherein the CPM includes representing or reconstructing the current video block by combining the use of a palette of representative entries with one or more samples derived by an intra block copy mode, and wherein the PCIP includes using intra prediction to derive a predicted block of the current video block and using palette information to refine the predicted block.

[0769] In some embodiments, the video unit corresponds to a current video block and an encoded / decoded representation of the current video block that are encoded / decoded using a composite palette mode (CPM), wherein the encoded / decoded representation includes an indication of the CPM at a video region level, the indication of the CPM being separate from an indication of an intra mode, an inter mode, an intra block copy mode, or a palette mode applicable to the video region, and wherein the CPM allows reconstruction of samples in the current video block by combining the use of a palette of representative entries with one or more samples derived by an intra block copy mode. In some embodiments, the video unit corresponds to a current chrominance block encoded / decoded using an encoding / decoding mode, wherein the encoded / decoded representation selectively includes an indication of the encoding / decoding mode based on encoding / decoding information of one or more selected regions of a luminance block corresponding to the current chrominance block. In some embodiments, the video unit corresponds to a video picture including a plurality of video blocks, wherein the video blocks are encoded / decoded using an encoding / decoding mode that uses a palette of representative values to encode / decode the video blocks and signals a syntax element of the encoding / decoding mode based on values of one or more prediction modes applied to the current video block.

[0770] Figure 16AAn example method 1610 for video processing is shown. Method 1610 includes, in step 1612, for the conversion between a video including a plurality of video blocks and an encoded / decoded representation of the video, determining the applicability of deblocking processing to each of the plurality of video blocks based on the conditions of each video block in the plurality of video blocks related to an encoding / decoding mode, where the encoding / decoding mode uses a palette of representative values to encode / decod a corresponding video block, and where the encoding / decoding mode includes a palette mode, a palette mode combined with an intra prediction mode (PCIP), or a composite palette mode (CPM). Method 1610 further includes, in step 1614, performing the conversion based on the determination. In some example scenarios, where the palette mode is applicable to the corresponding video block to encode / decod the pixels of the corresponding video block using a palette of component values, and where the PCIP is applicable to the corresponding video block to derive a prediction block of the corresponding video block using intra prediction and to refine the prediction block using palette information, and where the CPM is applicable to the corresponding video block to represent or reconstruct samples within the corresponding video block by combining the use of a palette of representative entries with one or more samples derived by an intra block copy mode.

[0771] Figure 16B An example method 1620 for video processing is shown. Method 1620 includes, in step 1622, performing a conversion between a current video block of a color component of a video and an encoded / decoded representation of the current video block, where the current video block is encoded / decoded using an encoding / decoding mode including a palette mode combined with an intra prediction mode (PCIP), or a composite palette mode (CPM). In some embodiments, syntax elements related to the encoding / decoding mode are selectively included in the encoded / decoded representation based on characteristics of the color components of the video, or a segmentation structure or planar encoding used for the current video block. In some embodiments, the PCIP is applicable to the current video block to derive a prediction block of the corresponding video block using intra prediction and to refine the prediction block using palette information, and where the CPM is applicable to the current video block to represent or reconstruct samples within the corresponding video block by combining the use of a palette of representative entries with one or more samples derived by an intra block copy mode.

[0772] Figure 16CAn example method 1630 for video processing is shown. Method 1630 includes, in step 1632, performing a first conversion between a first coded representation of a current video block and the current video block coded using a Composite Palette Mode (CPM), where the CPM mode allows reconstruction of samples in the current video block by selectively using a palette of representative entries, and where the current video block has a motion vector associated with the current video block. Method 1630 further includes, in step 1634, performing a second conversion between a next video block and a second coded representation of the next video block, where during the second conversion, motion information associated with the current video block is used to predict motion information of the next video block.

[0773] Figure 16D An example method 1640 for video processing is shown. Method 1640 includes, in step 1642, performing a first conversion between a first coded representation of a current video block and the current video block coded using a Palette Mode in combination with Intra Prediction (PCIP), where the PCIP mode allows using intra prediction to derive a predicted block of the current video block and allows refining the current predicted block using palette information, and where the current video block has an intra prediction direction associated with the current video block. Method 1640 further includes, in step 1644, performing a second conversion between a next video block and a second coded representation of the next video block, where during the second conversion, the intra mode of the current video block is used as an intra mode predictor for the next video block.

[0774] Figure 16E An example method 1650 for video processing is shown. Method 1650 includes, in step 1652, determining a prohibited use of a scaling matrix during a conversion between a video picture including one or more video blocks and a coded representation of the video picture due to the use of a coding mode related to a palette. Method 1650 further includes, in step 1654, performing the conversion based on the determination of the prohibited use of the scaling matrix.

[0775] Figure 16F An example method 1660 for video processing is shown. Method 1660 includes, in step 1662, determining an allowed use of a scaling matrix during a conversion between a video picture including one or more video blocks and a coded representation of the video picture due to the use of a coding mode related to a palette. Method 1660 further includes, in step 1664, performing the conversion based on the determination of the allowed use of the scaling matrix.

[0776] Some embodiments of the disclosed technology include making a decision or determination to enable a video processing tool or mode. In one example, when a video processing tool or mode is enabled, the encoder will use or implement the tool or mode in the processing of video blocks, but not necessarily modify the produced bitstream based on the use of the tool or mode. That is, when a video processing tool or mode is enabled based on a decision or determination, the conversion from video blocks to the bitstream representation of the video will use the video processing tool or mode. In another example, when a video processing tool or mode is enabled, the decoder will process the bitstream knowing that the bitstream has been modified based on the video processing tool or mode. That is, the conversion from the bitstream representation of the video to video blocks will be performed using the video processing tool or mode enabled based on the decision or determination.

[0777] Some embodiments of the disclosed technology include making a decision or determination to disable a video processing tool or mode. In one example, when a video processing tool or mode is disabled, the encoder will not use the tool or mode in converting video blocks to the bitstream representation of the video. In another example, when a video processing tool or mode is disabled, the decoder will process the bitstream knowing that the bitstream has not been modified using the video processing tool or mode enabled based on the decision or determination.

[0778] In this application document, the terms "video processing" or "visual media processing" may represent video encoding, video decoding, video compression, or video decompression. For example, a video compression algorithm may be applied during the conversion from the pixel representation of a video to the corresponding bitstream representation, and vice versa. As defined by the syntax, the bitstream representation of the current video block may, for example, correspond to bits co-located within the bitstream or extended at different positions within the bitstream. For example, a macroblock may be encoded according to the transform and codec error residual values and may also use bits in the header and other fields in the bitstream.

[0779] Some embodiments may be described using the following clause-based description. The first set of clauses describes specific features and aspects of the disclosed technology in the previous sections.

[0780] 1. A video encoding and decoding method, comprising: determining parameters for deblocking processing of a video region during conversion between the video region of a video and the bitstream representation of the video region based on palette encoding and decoding mode conditions related to the video region; and performing deblocking processing using the selected parameters during the conversion.

[0781] 2. The method according to clause 1, wherein the parameters for the deblocking processing are different from the parameters used for conversion of another video region encoded using an intra-frame encoding mode or an inter-frame encoding mode or an intra-block copy encoding mode.

[0782] 3. The method according to any one of Clauses 1-2, wherein the parameter includes a boundary strength value, and the boundary strength value is determined based on whether the adjacent video blocks on the P side and the Q side of the current video block are encoded or decoded using the palette mode.

[0783] 4. The method according to Clause 3, wherein due to encoding or decoding the video blocks on the P side and the Q side using the palette mode, the boundary strength is equal to zero.

[0784] 5. The method according to any one of Clauses 1-4, wherein the palette mode corresponds to a palette mode combined with an intra prediction mode (PCIP) or a composite palette mode (CPM).

[0785] Other examples and embodiments of the above method are provided in Item 1 of Chapter 4.

[0786] 6. A video processing method, comprising: during the conversion between a video picture and a bitstream representation of the video picture, based on an assignment rule, selecting deblocking filter parameters for a video block to which deblocking processing is to be applied to the video picture based on the encoding or decoding mode of adjacent video blocks; and performing the conversion by applying the deblocking processing according to the assignment rule; wherein the assignment rule stipulates that in the case of encoding or decoding adjacent blocks using the composite palette mode or another encoding or decoding mode, the same parameters are used.

[0787] 7. The method according to Clause 6, wherein the another encoding or decoding mode corresponds to an intra block copy mode.

[0788] 8. The method according to Clause 6, wherein the another encoding or decoding mode corresponds to a palette encoding or decoding mode.

[0789] Other examples and embodiments of the above method are provided in Item 2 of Chapter 4.

[0790] 9. A video processing method, during the conversion between a video picture and a bitstream representation of the video picture, based on an assignment rule, selecting deblocking filter parameters for a video block to which deblocking processing is to be applied to the video picture based on the encoding or decoding mode of adjacent video blocks; and performing the conversion by applying the deblocking processing according to the assignment rule; wherein the assignment rule stipulates that in the case of encoding or decoding adjacent blocks using the palette mode combined with an intra prediction mode (PCIP) or another encoding or decoding mode, the same parameters are used.

[0791] 10. The method according to Clause 9, wherein the another encoding or decoding mode corresponds to an intra block copy mode.

[0792] 11. The method according to clause 9, wherein the other coding and decoding mode corresponds to a palette coding and decoding mode.

[0793] Other examples and embodiments of the above method are provided in item 3 of chapter 4.

[0794] 12. A video processing method, comprising: performing a conversion between a video block of a video picture and a bitstream representation of the video block using a palette coding and decoding mode, in which a palette of component values is used to code and decode pixels of the corresponding video block, wherein a flag in the bitstream representation provides an indication of the presence of non-zero coefficients in the corresponding video block, and the flag is used to signal a video block coded using the palette coding and decoding mode, an intra prediction (PCIP) mode in combination with the palette coding and decoding mode, or a composite palette mode (CPM).

[0795] 13. The method according to clause 12, wherein the value of the flag depends on whether the corresponding block has escape pixels.

[0796] Other examples and embodiments of the above method are provided in item 4 of chapter 4.

[0797] 14. A video processing method, comprising: during a conversion between a video picture including a plurality of video blocks and a bitstream representation of the video picture, determining the applicability of deblocking processing to each of the plurality of video blocks based on conditions of a palette coding and decoding mode related to each of the plurality of video blocks; and selectively using the deblocking processing on the plurality of video blocks during the conversion based on the applicability.

[0798] 15. The method according to clause 14, wherein determining the applicability determines that the deblocking processing is not applicable to blocks coded using the palette mode in combination with an intra prediction mode (PCIP) or the composite palette (CPM).

[0799] 16. The method according to clause 14, wherein determining the applicability includes: determining that P-side samples or Q-side samples are not applicable in cases where P-side blocks or Q-side blocks are coded using the palette mode or the PCIP mode or the CPM, respectively.

[0800] Other examples and embodiments of the above method are provided in item 5 of chapter 4.

[0801] 17. A method for video processing, comprising: performing a conversion between a video frame including a plurality of video blocks and a bitstream representation of the video frame encoded and decoded using a Composite Palette Mode (CPM); wherein the bitstream representation includes an indication of the CPM mode at a video region level, the indication being separate from an indication of an intra mode, an inter mode, an Intra Block Copy mode, or a palette mode applicable to the video region.

[0802] 18. The method according to clause 17, wherein the video region corresponds to a transform unit or a prediction unit or a codec block unit.

[0803] 19. The method according to any one of clauses 17 - 18, wherein the bitstream representation includes the CPM mode after an indication of an intra mode, an inter mode, an Intra Block Copy mode, or a palette mode applicable to the video region.

[0804] Other examples and embodiments of the above method are provided in item 6 of chapter 4.

[0805] 20. A video processing method, comprising: performing a conversion between a video block of a video picture and a bitstream representation of the video block using a codec mode, in which a palette of component values is used to encode and decode pixels of the corresponding video block, wherein, based on characteristics of a color component to which a given video block belongs, a syntax element related to the palette codec mode is selectively included in the bitstream representation of the given video block; wherein the codec mode includes a palette codec mode, a Palette Coding in combination with Intra Prediction mode (PCIP), or a Composite Palette Mode (CPM).

[0806] 21. According to the video processing method, wherein a syntax element related to the palette codec mode is also selectively included based on a segmentation structure or planar coding used for a given video block.

[0807] 22. The method according to any one of clauses 20 - 21, wherein the syntax element is included only for a dual-tree segmentation structure.

[0808] 23. The method according to any one of clauses 20 - 21, wherein, since the video picture is in 4:0:0 format and the given video block is a chrominance block, the syntax element is skipped.

[0809] Other examples and embodiments of the above method are provided in item 7 of chapter 4.

[0810] 24. The method according to clause 20, wherein the given video block corresponds to a chrominance component, and wherein syntax elements related to the palette coding mode are selectively included based on the coding and decoding information of one or more selected blocks of the luminance block corresponding to the given video block.

[0811] 25. The method according to clause 24, wherein the one or more selected blocks of the luminance block correspond to the smallest coding unit or the smallest prediction unit or the smallest transform unit size.

[0812] Other examples and embodiments of the above method are provided in item 8 of chapter 4.

[0813] 26. The method according to any one of clauses 20 - 25, wherein the coding mode is the PCIP mode.

[0814] Other examples and embodiments of the above method are provided in item 9 of chapter 4.

[0815] 27. A video processing method, comprising: performing a conversion between a video picture including a plurality of video blocks and a bitstream representation of the video picture, wherein at least some of the plurality of blocks are coded and decoded using a predictive coding mode, and wherein a video block is coded and decoded using a composite palette mode (CPM), and the composite palette mode is separately identified or treated as a special case from other predictive coding modes used for coding and decoding other video blocks of the video picture.

[0816] 28. The method according to clause 27, wherein the other predictive coding modes include an intra - stripe or an intra - mode of an I - picture or an intra - slice group, intra - block copy, a palette mode, and a CPM mode.

[0817] Other examples and embodiments of the above method are provided in item 10 of chapter 4.

[0818] 29. The method according to clause 27, wherein the CPM is identified as a special case of other predictive coding modes in the bitstream representation, and syntax elements of the CPM are selectively signaled based on the type of the other predictive mode for which the CPM is a special case.

[0819] 30A. The method according to clause 29, wherein the type of the other predictive mode is an intra - mode, and wherein signaling of the syntax elements of the CPM is skipped.

[0820] 30B. The method according to clause 29, wherein the type of the other predictive mode is a skip mode, and wherein signaling of the syntax elements of the CPM is skipped.

[0821] Other examples and embodiments of the above method are provided in Item 11 of Chapter 4.

[0822] 31. A video processing method, comprising: performing a first conversion between a current video block and a first bitstream representation of the current video block, wherein the current video block is encoded and decoded using a Composite Palette Mode (CPM), and the current video block has a motion vector associated with the current video block; and performing a second conversion between a next video block and a second bitstream representation of the next video block, wherein motion prediction using the motion vector associated with the current video block is utilized during the second conversion.

[0823] 32. The method according to clause 31, wherein the motion prediction is used as a Merge candidate for the second conversion.

[0824] 33. The method according to clause 31 or 32, wherein the motion prediction is used as a history-based motion vector predictor during the second conversion.

[0825] Other examples and embodiments of the above method are provided in Item 12 of Chapter 4.

[0826] 34. A video processing method, comprising: performing a first conversion between a current video block and a first bitstream representation of the current video block, wherein the current video block is encoded and decoded using a Pattern Combined with Intra Prediction mode (PCIP), and the current video block has an intra prediction direction associated with the current video block; and performing a second conversion between a next video block and a second bitstream representation of the next video block, wherein an intra mode predictor based on the intra prediction direction associated with the current video block is used during the second conversion.

[0827] 35. The method according to clause 34, wherein the intra prediction direction associated with the current video block is used as the most likely mode candidate during the second conversion.

[0828] Other examples and embodiments of the above method are provided in Item 13 of Chapter 4.

[0829] 36. A video processing method, comprising: determining a maximum number of entries of a palette used during a conversion between a current video block of a video and a bitstream representation of the video, and performing the conversion using the maximum number, wherein a field in the bitstream representation signals a change in the maximum number in a video unit.

[0830] 37. The method according to clause 36, wherein the video unit corresponds to a Coding Tree Block or a Coding Unit or a Prediction Unit or a Transform Unit.

[0831] 38. A method according to clauses 36 - 37, wherein the maximum number is based on the coding and decoding information of the current video block.

[0832] Other examples and embodiments of the above method are provided in item 14 of chapter 4.

[0833] 39. A video processing method, comprising: determining to prohibit the use of a scaling matrix due to using a palette coding and decoding mode during the conversion between a video picture including one or more video blocks and the bitstream representation of the video picture; and performing the conversion based on the determination to prohibit the use of the scaling matrix.

[0834] 40. A video processing method, comprising: determining to allow the use of a scaling matrix due to using a palette coding and decoding mode during the conversion between a video picture including one or more video blocks and the bitstream representation of the video picture; and performing the conversion based on the determination to allow the use of the scaling matrix.

[0835] Other examples and embodiments of the above method are provided in item 15 of chapter 4.

[0836] 41. A method according to any one of clauses 1 to 40, wherein the method is applied when determining the characteristics of the video for which the conversion is to be performed.

[0837] 42. A method according to clause 41, wherein the characteristics of the video include the content type of the video.

[0838] 43. A method according to clause 41, wherein the characteristics of the video include the block dimensions for the conversion.

[0839] 44. A method according to clause 41, wherein the characteristics of the video include the segmentation tree structure for the conversion.

[0840] Other examples and embodiments of the above method are provided in item 16 of chapter 4.

[0841] 45. A method according to any one of clauses 1 - 44, wherein the conversion includes generating pixel values of the video region from the bitstream representation.

[0842] 46. A method according to any one of clauses 1 - 44, wherein the conversion includes generating the bitstream representation from the pixel values of the video region.

[0843] 47. A video processing apparatus, comprising: a processor configured to implement the method according to any one or more of clauses 1 to 46.

[0844] 48. A computer-readable medium having program code stored thereon, the code when executed causing a processor to implement the method according to any one or more of clauses 1 to 46.

[0845] A second set of clauses describes specific features and aspects of the technologies disclosed in the previous sections, for example, Example Implementations 1 to 16.

[0846] 1. A video processing method, comprising: for the conversion between a video unit of a video and an encoded / decoded representation of the video, determining, according to a rule, a maximum number of entries of a palette of representative values to be used during the conversion; and performing the conversion using the palette, and wherein the rule specifies the maximum number according to characteristics of the video unit.

[0847] 2. The method according to clause 1, wherein the video unit corresponds to a block, a coded tree block, or a coding unit, or a prediction unit, or a transform unit.

[0848] 3. The method according to any one of clauses 1 - 2, wherein the characteristics include encoding / decoding information of the video unit.

[0849] 4. The method according to any one of clauses 1 - 2, wherein the maximum number of the video unit is based on the dimension of the video unit.

[0850] 5. The method according to any one of clauses 1 - 2, wherein the maximum number of the video unit is based on the color components of the video unit.

[0851] 6. The method according to clause 5, wherein the maximum number of the video unit corresponding to a luminance block is greater than the maximum number of another video unit corresponding to a chrominance block.

[0852] 7. The method according to clause 6, wherein M is a fixed number and M is less than N, where M is the maximum number determined for the another video unit and N is the maximum number determined for the video unit.

[0853] 8. The method according to any one of clauses 1 - 2, wherein the maximum number of the video unit is based on a segmentation structure for segmenting the video unit.

[0854] 9. A video processing method, comprising:

[0855] For the conversion between the current video block of a video and the codec representation of the video, one or more codec parameters for deblocking processing of the current video block are determined according to rules based on the use of the palette mode codec tool for the current video block; and the conversion is performed using the one or more codec parameters, wherein the rules stipulate that one or more codec parameters of a video block coded and decoded using the palette mode codec tool are derived differently from the parameters for other codec tools, wherein the palette mode codec tool includes: during encoding, representing the current video block as the codec representation using a palette of representative sample values; or during decoding, reconstructing the current video block from the codec representation using a palette of representative sample values.

[0856] 10. The method according to clause 9, wherein the other codec tools include an intra codec mode or an inter codec mode or an intra block copy codec mode.

[0857] 11. The method according to any one of clauses 9 - 10, wherein the one or more parameters for the current video block include a boundary strength value, and the boundary strength value is determined based on whether adjacent video blocks on the P side and the Q side of the current video block are coded and decoded using the palette mode.

[0858] 12. The method according to clause 11, wherein the boundary strength is equal to zero because both the P side and Q side video blocks are coded and decoded using the palette mode.

[0859] 13. The method according to clause 11, wherein the adjacent video blocks include a first video block coded and decoded using the palette mode and a second video block not coded and decoded using the palette mode, and wherein the boundary strength depends only on the information of the second video block.

[0860] 14. The method according to any one of clauses 9 - 13, wherein the palette mode corresponds to a palette mode combined with an intra prediction mode (PCIP), or a composite palette mode (CPM).

[0861] 15. A video processing method includes: for the conversion between a current video block of a video and an encoded / decoded representation of the video, determining, according to a rule, one or more parameters for deblocking processing to be applied to the current video block based on the use of a Composite Palette Mode (CPM) encoding / decoding tool for the current video block; and performing the conversion using the one or more parameters, and wherein the rule stipulates that the encoded / decoded representation includes an indication of the one or more parameters using syntax elements that are the same as the syntax elements of an adjacent video block encoded using another encoding mode, and wherein the CPM encoding / decoding tool includes representing or reconstructing the current video block by combining the use of a palette of representative entries with one or more samples derived by an Intra Block Copy mode.

[0862] 16. The method according to clause 15, wherein the another encoding mode corresponds to a palette mode.

[0863] 17. The method according to clause 15, wherein the another encoding mode corresponds to an Intra Block Copy mode.

[0864] 18. A method for a video processing method includes: for the conversion between a current video block of a video and an encoded / decoded representation of the video, determining, based on a rule and based on the encoding mode of an adjacent video block, parameters for deblocking processing to be applied to the current video block; and

[0865] performing the conversion based on the determination, and wherein the rule stipulates using the same parameters for the case of encoding the adjacent video block using a Palette Mode Combined with Intra Prediction (PCIP) or another encoding mode, and wherein the PCIP includes using intra prediction to derive a predicted block of the current video block, and using palette information associated with the current video block to refine the predicted block.

[0866] 19. The method according to clause 18, wherein the another encoding mode corresponds to a palette mode, and wherein the current video block is represented or reconstructed using a palette of representative sample values.

[0867] 20. The method according to clause 18, wherein the another encoding mode corresponds to an Intra Block Copy mode.

[0868] 21. A video processing method includes: performing a conversion between a current video block of a video and an encoded / decoded representation of the video, wherein the current video block is represented in the encoded / decoded representation according to an encoding / decoding mode, the encoding / decoding mode being a palette encoding mode, or a palette mode combined with an intra prediction mode (PCIP), or a composite palette mode (CPM); wherein the encoded / decoded representation includes a flag of the current video block, the flag indicating according to a rule whether the current video block has non-zero coefficients; wherein the rule stipulates that the flag has a value according to the encoding / decoding mode or the presence of escape pixels in the encoded / decoded representation of the current video block, wherein the palette encoding mode includes using a palette of representative sample values to represent or reconstruct the current video block, wherein the CPM includes representing or reconstructing the current video block by combining the use of a palette of representative entries with one or more samples derived by an intra block copy mode, and wherein the PCIP includes using intra prediction to derive a predicted block of the current video block and using palette information to refine the predicted block.

[0869] 22. The method according to clause 21, wherein the value of the flag is 0 or 1.

[0870] 23. The method according to clause 21, wherein the value of the flag depends on whether the corresponding video block has escape pixels.

[0871] 24. A video processing method includes: for a conversion between a video including a plurality of video blocks and an encoded / decoded representation of the video, determining the applicability of deblocking processing to each of the plurality of video blocks based on conditions of each video block among the plurality of video blocks related to an encoding / decoding mode, the encoding / decoding mode using a palette of representative values to encode / decode a corresponding video block, wherein the encoding / decoding mode includes a palette mode, a palette mode combined with an intra prediction mode (PCIP), or a composite palette mode (CPM); and performing the conversion based on the determination, and wherein the palette mode is applicable to the corresponding video block to encode / decode pixels of the corresponding video block using a palette of component values, and wherein the PCIP is applicable to the corresponding video block to derive a predicted block of the corresponding video block using intra prediction and use palette information to refine the predicted block, and wherein the CPM is applicable to the corresponding video block to represent or reconstruct samples within the corresponding video block by combining the use of a palette of representative entries with one or more samples derived by an intra block copy mode.

[0872] 25. The method according to clause 24, wherein determining the applicability determines that the deblocking process does not apply to P-side blocks or Q-side blocks encoded or decoded using the palette mode combined with intra prediction mode (PCIP) or the composite palette mode (CPM).

[0873] 26. The method according to clause 24, wherein determining the applicability includes: in a case where a Q-side block is not encoded or decoded using the palette mode combined with intra prediction mode (PCIP) or the composite palette mode (CPM), determining that the deblocking process does not apply to P-side blocks encoded or decoded using the PCIP and the CPM.

[0874] 27. The method according to clause 24, wherein determining the applicability includes: in a case where a P-side block is not encoded or decoded using the palette mode combined with intra prediction mode (PCIP) or the composite palette mode (CPM), determining that the deblocking process does not apply to Q-side blocks encoded or decoded using the PCIP or the CPM.

[0875] 28. The method according to clause 24, wherein determining the applicability includes: determining that the deblocking process applies to samples copied from an intra block copy (IBC) of a video block encoded using the composite palette mode (CPM).

[0876] 29. The method according to clause 24, wherein determining the applicability includes: determining that the deblocking process applies to samples copied from an intra prediction of a video block encoded using the palette mode combined with intra prediction mode (PCIP).

[0877] 30. A video processing method, comprising: performing a conversion between a current video block encoded using the composite palette mode (CPM) and an encoded representation of the current video block, wherein the encoded representation includes an indication of the CPM at a video region level, the indication of the CPM being separate from an indication of an intra mode, an inter mode, an intra block copy mode, or a palette mode applicable to the video region, and wherein the CPM allows reconstruction of samples in the current video block by combining the use of a palette of representative entries with one or more samples derived through an intra block copy mode.

[0878] 31. The method according to clause 30, wherein the video region corresponds to a transform unit or a prediction unit or a coding block unit.

[0879] 32. The method according to any one of clauses 30-31, wherein the encoded representation includes the indication of the CPM after an indication of an intra mode, an inter mode, an intra block copy mode, or a palette mode applicable to the video region.

[0880] 33. The method according to any one of clauses 30-31, wherein the prediction mode based on the current video block includes an indication of the CPM.

[0881] 34. The method according to clause 33, wherein for the prediction mode that is not the intra block copy mode, it includes an indication of the CPM.

[0882] 35. A video processing method, comprising: performing a conversion between a current video block of a color component of a video and an encoded / decoded representation of the current video block, wherein the current video block is encoded / decoded using an encoding / decoding mode including a palette mode combined with intra prediction mode (PCIP), or a composite palette mode (CPM), wherein, based on characteristics of the color component of the video, or a segmentation structure or planar encoding / decoding for the current video block, a syntax element related to the encoding / decoding mode is selectively included in the encoded / decoded representation, and wherein the PCIP is applicable to the current video block to derive a prediction block of the corresponding video block using intra prediction and refine the prediction block using palette information, and wherein the CPM is applicable to the current video block to represent or reconstruct samples within the corresponding video block by combining the use of a palette of representative entries with one or more samples derived by the intra block copy mode.

[0883] 36. The method according to clause 35, wherein in a case where the current video block corresponds to a chrominance component and the current video block has a single-tree segmentation structure, the syntax element is not included.

[0884] 37. The method according to clause 35, wherein in a case where the current video block corresponds to a chrominance component and the current video block has a double-tree segmentation structure, the syntax element is included.

[0885] 38. The method according to clause 35, wherein in a case where the current video block corresponds to a chrominance component and the current video block has a 4:0:0 color format, the syntax element is not included.

[0886] 39. A video processing method, comprising: performing a conversion between a current chrominance block of a video and an encoded / decoded representation of the video, wherein the current chrominance block is encoded / decoded using an encoding / decoding mode, and wherein, based on encoded / decoding information of one or more selected regions of a luminance block corresponding to the current chrominance block, the encoded / decoded representation selectively includes an indication of the encoding / decoding mode.

[0887] 40. The method according to clause 39, wherein the coding / decoding mode is a composite palette mode (CPM) that allows representing or reconstructing samples in the current chrominance block by combining the use of a palette of representative entries with one or more samples derived by an intra-block copy mode.

[0888] 41. The method according to clause 39, wherein the coding / decoding mode is a palette mode combined with an intra prediction mode (PCIP) that allows using intra prediction to derive a prediction block for the chrominance block and using palette information associated with the current chrominance block to refine the prediction block.

[0889] 42. The method according to clause 39, wherein due to coding / decoding at least one of the one or more selected regions using an intra-block copy mode, the coded representation includes an indication of the coding / decoding mode.

[0890] 43. The method according to clause 39, wherein due to coding / decoding all of the one or more selected regions using an intra-block copy mode, the coded representation includes an indication of the coding / decoding mode.

[0891] 44. The method according to clause 39, wherein due to not coding / decoding at least one of the one or more selected regions using an intra-block copy mode, the coded representation does not include an indication of the coding / decoding mode.

[0892] 45. The method according to clause 39, wherein due to not coding / decoding all of the one or more selected regions using an intra-block copy mode, the coded representation does not include an indication of the coding / decoding mode.

[0893] 46. The method according to clause 39, wherein the selected region has a size corresponding to a minimum size of a coding unit or a prediction unit or a transform unit or a storage unit for motion or mode.

[0894] 47. The method according to clause 39, wherein the selected region corresponds to a coding unit or a prediction unit or a transform unit that covers a specific position of the corresponding luma block.

[0895] 48. A video processing method includes performing a conversion between a video picture including a plurality of video blocks and a coded representation of the video picture, and wherein a coding / decoding mode is used to code / decode the video blocks, the coding / decoding mode using a palette of representative values to code / decode the video blocks and signaling a syntax element of the coding / decoding mode based on values of one or more prediction modes applied to the current video block.

[0896] 49. The method according to clause 48, wherein the encoding / decoding mode corresponds to a composite palette mode (CPM), a palette mode combined with an intra prediction mode (PCIP), or other palette-related methods.

[0897] 50. The method according to clause 48, wherein the one or more prediction modes include at least two of an intra mode, an inter mode, an intra block copy, a palette mode, and a CPM mode for an intra strip or an I picture or a P and / or B picture or an intra slice group.

[0898] 51. The method according to clause 48, wherein the type of the one or more prediction modes includes an inter mode, and wherein signaling of syntax elements related to the palette mode is skipped.

[0899] 52. The method according to clause 49, wherein the syntax elements of the CPM mode in the encoding / decoding representation are selectively signaled based on the value of the one or more prediction modes.

[0900] 53. The method according to clause 52, wherein the type of the one or more prediction modes includes an intra mode, and wherein signaling of the syntax elements is skipped.

[0901] 54. The method according to clause 52, wherein the type of the one or more prediction modes includes a skip mode, and wherein signaling of the syntax elements is skipped because the type of the one or more prediction modes is the skip mode.

[0902] 55. The method according to clause 52, wherein the type of the one or more prediction modes includes a palette mode, and wherein the syntax elements are signaled.

[0903] 56. The method according to clause 52, wherein the type of the one or more prediction modes includes an intra mode, an inter mode, or an intra block copy mode, and wherein the syntax elements are signaled.

[0904] 57. The method according to clause 52, wherein the type of the one or more prediction modes includes an intra mode, but does not include a pulse code modulation (PCM) mode, and wherein the syntax elements are signaled.

[0905] 58. The method according to clause 52, wherein the type of the one or more prediction modes includes a palette mode, and wherein the syntax elements are signaled before or after an indication of the use of a pulse code modulation (PCM) mode.

[0906] 59. The method according to clause 52, wherein the type of the one or more prediction modes includes an intra mode, and wherein the syntax element is signaled.

[0907] 60. The method according to clause 52, wherein the type of the one or more prediction modes includes a palette mode, an inter mode or an intra mode, and wherein signaling of the syntax element is skipped.

[0908] 61. A video processing method, comprising: performing a first conversion between a first codec representation of a current video block and the current video block encoded and decoded using a composite palette mode (CPM), the CPM mode allowing reconstruction of samples in the current video block by selectively using a palette of representative entries, wherein the current video block has a motion vector associated with the current video block; and performing a second conversion between a next video block and a second codec representation of the next video block, wherein during the second conversion, motion information associated with the current video block is used to predict motion information of the next video block.

[0909] 62. The method according to clause 61, wherein the motion prediction is used as a Merge candidate for the second conversion.

[0910] 63. The method according to clause 61, wherein the motion prediction is used as a history motion vector prediction (HMVP) during the second conversion.

[0911] 64. The method according to clause 61, wherein the motion prediction is used as an advanced motion vector prediction (AMVP) candidate for the second conversion.

[0912] 65. The method according to clause 61, wherein after the first conversion, an update process based on a history motion vector prediction (HMVP) table is not allowed, the HMVP table including one or more entries corresponding to motion information of one or more previously encoded and decoded blocks.

[0913] 66. The method according to clause 61, wherein the motion vector (bv) of the current video block is not allowed to be used as a motion prediction for the second conversion.

[0914] 67. A video processing method includes: performing a first transformation between a first codec representation of a current video block and the current video block encoded and decoded using a palette mode (PCIP) combined with an intra prediction mode, where the PCIP mode allows using intra prediction to derive a predicted block of the current video block and allows using palette information to refine the current predicted block, and where the current video block has an intra prediction direction associated with the current video block; and performing a second transformation between a next video block and a second codec representation of the next video block, where during the second transformation, the intra mode of the current video block is used as an intra mode predictor for the next video block.

[0915] 68. The method according to clause 67, where during the second transformation, the intra prediction direction associated with the current video block is used as the most likely mode candidate.

[0916] 69. The method according to clause 67, where during the second transformation, the intra prediction direction associated with the current video block is not allowed to be used as the most likely mode candidate.

[0917] 70. A video processing method includes: determining a prohibited use of a scaling matrix during a transformation between a video picture including one or more video blocks and a codec representation of the video picture due to the use of a codec mode related to a palette; and

[0918] performing the transformation based on the determination of the prohibited use of the scaling matrix.

[0919] 71. A video processing method includes: determining an allowed use of a scaling matrix during a transformation between a video picture including one or more video blocks and a codec representation of the video picture due to the use of a codec mode related to a palette; and

[0920] performing the transformation based on the determination of the allowed use of the scaling matrix.

[0921] 72. The method according to clause 1, where the codec mode related to the palette corresponds to a palette mode, a composite palette mode (CPM), or a palette mode combined with an intra prediction mode (PCIP).

[0922] 73. The method according to any one of clauses 1 to 72, where the method is applied when determining characteristics of the video for which the transformation is being performed.

[0923] 74. The method according to clause 73, where the characteristics of the video include the content type of the video.

[0924] 75. The method according to clause 73, wherein the characteristics of the video include a coding / decoding unit, a prediction unit, a transformation unit, or the position of the current video block.

[0925] 76. The method according to clause 73, wherein the characteristics of the video include the block dimension and / or block shape of the current video block or an adjacent block for the transformation.

[0926] 77. The method according to clause 73, wherein the characteristics of the video include the prediction mode and / or intra mode of the current video block or an adjacent block.

[0927] 78. The method according to clause 73, wherein the characteristics of the video include the motion vector and / or block vector of the current video block or an adjacent block.

[0928] 79. The method according to clause 73, wherein the characteristics of the video include the color format or color components of the video.

[0929] 80. The method according to clause 73, wherein the characteristics of the video include the coding tree structure applied to the current video block.

[0930] 81. The method according to any one of clauses 1 - 80, wherein the transformation includes generating pixel values of the video region from the coded representation.

[0931] 82. The method according to any one of clauses 1 - 80, wherein the transformation includes generating the bitstream representation from the pixel values of the video region.

[0932] 83. A video processing apparatus, including a processor configured to implement the method according to any one or more of clauses 1 to 82.

[0933] 84. A computer-readable medium storing program code, which when executed causes a processor to implement the method according to any one or more of clauses 1 to 82.

[0934] The disclosures and other solutions, examples, embodiments, modules, and functional operations described in this application document can be implemented in digital electronic circuits, or in computer software, firmware, or hardware, including the structures disclosed in this specification and their structural equivalents, or a combination of one or more of them. The content disclosed in this specification and other embodiments can be implemented as one or more computer program products, that is, modules of computer program instructions encoded on a tangible and non-transitory computer-readable medium for a data processing apparatus to execute or control the operation of the data processing apparatus. The computer-readable medium can be a machine-readable storage device, a machine-readable storage substrate, a storage device, a substance composition affecting a machine-readable propagated signal, or a combination of one or more of them. The term "data processing unit" or "data processing apparatus" includes all apparatuses, devices, and machines for processing data, including, for example, programmable processors, computers, or multi-processors or groups of computers. In addition to hardware, the apparatus can also include code for creating an execution environment for the computer program, such as code constituting processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them. A propagated signal is an artificially generated signal, for example, a machine-generated electrical, optical, or electromagnetic signal that is generated to encode information for transmission to a suitable receiver device.

[0935] A computer program (also referred to as a program, software, software application, script, or code) can be written in any form of programming language (including compiled or interpreted languages) and can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. The program can be stored in a part of a file that holds other programs or data (for example, one or more scripts in a markup language document), in a single file dedicated to the program, or in multiple coordinated files (for example, files that store one or more modules, subroutines, or portions of code). A computer program can be deployed to be executed on one or more computers, which are located at one site or distributed across multiple sites and interconnected by a communication network.

[0936] The processing and logic flows described in this specification can be executed by one or more programmable processors that execute one or more computer programs to perform functions by operating on input data and generating output. The processing and logic flows can also be executed by special-purpose logic circuits, and the apparatus can also be implemented as special-purpose logic circuits, such as FPGAs (Field Programmable Gate Arrays) or ASICs (Application-Specific Integrated Circuits).

[0937] For example, a processor suitable for executing a computer program includes general and special-purpose microprocessors, and any one or more of any type of digital computer. Generally, a processor will receive instructions and data from a read-only memory or a random access memory or both. The basic components of a computer are a processor that executes instructions and one or more storage devices that store instructions and data. Generally, a computer will also include one or more mass storage devices for storing data, such as, for example, magnetic disks, magneto-optical disks, or optical disks, or is operatively coupled to one or more mass storage devices to receive data therefrom or transfer data thereto, or both. However, a computer need not have such devices. Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, including, for example, semiconductor memory devices such as EPROM, EEPROM, and flash memory devices; magnetic disks, such as internal hard disks or removable hard disks; magneto-optical disks; and CD-ROM and DVD-ROM disks. The processor and the memory may be supplemented by, or incorporated in, special-purpose logic circuitry.

[0938] Although this patent document contains many details, it should not be construed as limiting any invention or the scope of any claims, but rather as describing features of particular embodiments of a particular invention. Certain features described in the context of separate embodiments may also be implemented in combination in a single embodiment. Conversely, the various functions described in the context of a single embodiment may also be implemented separately in multiple embodiments, or in any suitable sub-combination. Additionally, although the above features may be described as acting in certain combinations and even initially claimed as such, in some cases, one or more features from a claimed combination may be removed from the combination, and the claimed combination may be directed to a sub-combination or a variation of a sub-combination.

[0939] Similarly, although operations are described in the figures in a particular order, this should not be understood to mean that such operations must be performed in the particular order shown or in sequential order to obtain a desired result, or that all illustrated operations must be performed. Additionally, the separation of various system components in the embodiments described in this patent document should not be understood to mean that such separation is required in all embodiments.

[0940] Only some implementations and examples are described, and other implementations, enhancements, and variations may be made based on what is described and illustrated in this patent document.

Claims

1. A method for processing video data, comprising: For the conversion between the blocks of a video and the bitstream of the video, determine the maximum number of entries of the palette of the block to be used in a prediction mode based on the segmentation structure of the block; and perform the conversion based on the determination, wherein, in the prediction mode, the reconstructed samples are represented by a set of representative color values, and the set of representative color values includes at least one of 1) a palette predictor, 2) escape samples, or 3) palette information included in the bitstream, wherein the maximum number of entries of the palette is further determined based on the slice type of the block, the color component of the block, and the segmentation structure of the block, and wherein, when the block meets the conditions, the maximum number of entries of the palette of the block is greater than the maximum number of entries of the palette of the luminance block corresponding to the block, and the conditions include a first condition based on the slice type, a second condition based on the color component, and a third condition based on the segmentation structure, the first condition being that the block has a first slice type, the second condition being that the block is a luminance block, and the third condition being that the block has a dual-tree segmentation structure.

2. The method according to claim 1, wherein, The maximum number of entries of the palette is also determined based on the coding / decoding information of the block.

3. The method according to claim 1, wherein, The maximum number of entries of the above-mentioned palette is also determined based on the size of the block.

4. The method according to claim 1, wherein, The maximum number of entries of the palette for the luminance block is greater than the maximum number of entries of the palette for the chrominance block.

5. The method according to claim 4, wherein, When the segmentation structure is a local dual-tree segmentation structure, the maximum number of entries of the palette for the luminance block is greater than the maximum number of entries of the palette for the chrominance block.

6. The method according to claim 5, wherein, In the local dual-tree segmentation structure, the coding / decoding tree unit is divided using a luminance segmentation scheme and a chrominance segmentation scheme, and wherein the luminance segmentation scheme has the same recursive segmentation tree type as the chrominance segmentation scheme until a size threshold is reached.

7. The method according to claim 1, wherein, The maximum number of entries of the palette having a single-tree segmentation structure is different from the maximum number of entries of the palette having a dual-tree segmentation structure, and wherein, in the single-tree segmentation structure, the same segmentation structure is applied to the luminance component and the chrominance component, and in the dual-tree segmentation structure, different segmentation structures are applied to the luminance component and the chrominance component.

8. The method according to claim 1, wherein, The first slice type is not an I slice.

9. The method according to claim 1, wherein, The conditions further include that the block has a first color format, and the block has a first size.

10. The method according to claim 9, wherein, The first color format is 4:2:

0.

11. The method according to claim 9, wherein, The conditions further include that the block is obtained from a first partitioning type.

12. The method according to claim 9, wherein, When the block meets the conditions, the prediction mode is disabled for the chrominance block corresponding to the block.

13. The method according to claim 1, wherein the conversion includes encoding and decoding the block into the bitstream.

14. The method according to claim 1, wherein the conversion includes decoding the block from the bitstream.

15. An apparatus for processing video data, comprising a processor and a non-transitory memory having instructions thereon, wherein when the processor executes the instructions, the processor is caused to: For the conversion between a block of video and the bitstream of the video, determine the maximum number of entries of the palette of the block to be used in the prediction mode based on the segmentation structure of the block; and Perform the conversion based on the determination, wherein, In the prediction mode, the reconstructed samples are represented by a set of representative color values, and the set of representative color values includes at least one of 1) a palette predictor, 2) escape samples, or 3) palette information included in the bitstream, wherein the maximum number of entries of the palette is further determined based on the slice type of the block, the color component of the block, and the segmentation structure of the block, and Among them, when the block meets the conditions, the maximum number of entries in the palette of the block is greater than the maximum number of entries in the palette of the luminance block corresponding to the block. The conditions include a first condition based on the stripe type, a second condition based on the color component, and a third condition based on the segmentation structure. The first condition is that the block has a first stripe type, the second condition is that the block is a luminance block, and the third condition is that the block has a dual-tree segmentation structure.

16. The apparatus according to claim 15, wherein, The maximum number of entries in the palette is also determined based on the encoding and decoding information of the block.

17. The apparatus according to claim 15, wherein, The maximum number of entries in the palette is also determined based on the size of the block.

18. The apparatus according to claim 15, wherein, The maximum number of entries in the palette for the luminance block is greater than the maximum number of entries in the palette for the chrominance block.

19. The apparatus according to claim 15, wherein, The maximum number of entries in the palette with a single-tree segmentation structure is different from the maximum number of entries in the palette with a dual-tree segmentation structure. And among them, in the single-tree segmentation structure, the luminance component and the chrominance component apply the same segmentation structure, and in the dual-tree segmentation structure, the luminance component and the chrominance component apply different segmentation structures.

20. A non - transitory computer - readable storage medium for storing instructions that cause a processor to perform the following operations: For the conversion between a block of a video and the bitstream of the video, determining a maximum number of entries of a palette of the block to be used in a prediction mode based on a segmentation structure of the block; and Performing the conversion based on the determination, wherein, In the prediction mode, the reconstructed samples are represented by a set of representative color values, and this set of representative color values includes at least one of 1) a palette predictor, 2) escape samples, or 3) the palette information included in the bitstream. Among them, the maximum number of entries in the palette is also determined based on the stripe type of the block, the color component of the block, and the segmentation structure of the block, and Among them, when the block meets the conditions, the maximum number of entries in the palette of the block is greater than the maximum number of entries in the palette of the luminance block corresponding to the block. The conditions include a first condition based on the stripe type, a second condition based on the color component, and a third condition based on the segmentation structure. The first condition is that the block has a first stripe type, the second condition is that the block is a luminance block, and the third condition is that the block has a dual-tree segmentation structure.

21. A non - transitory computer - readable recording medium that stores a bitstream of a video generated by a method executed by a video processing apparatus, wherein the method includes: Determine the maximum number of entries in the palette of the block to be used in the prediction mode based on the segmentation structure of the block; And Generate the bitstream based on the determination. Among them, in the prediction mode, the reconstructed samples are represented by a set of representative color values, and this set of representative color values includes at least one of 1) a palette predictor, 2) escape samples, or 3) the palette information included in the bitstream. Among them, the maximum number of entries in the palette is also determined based on the stripe type of the block, the color component of the block, and the segmentation structure of the block, and Among them, when the block meets the conditions, the maximum number of entries in the palette of the block is greater than the maximum number of entries in the palette of the luminance block corresponding to the block. The conditions include a first condition based on the stripe type, a second condition based on the color component, and a third condition based on the segmentation structure. The first condition is that the block has a first stripe type, the second condition is that the block is a luminance block, and the third condition is that the block has a dual-tree segmentation structure.

22. A method for storing a bitstream, including: Determine the maximum number of entries in the palette of the block to be used in the prediction mode based on the segmentation structure of the block; Generate the bitstream based on the determination; Store the bitstream in a non-transitory computer-readable storage medium, wherein, in the prediction mode, the reconstructed sample is represented by a set of representative color values, and the set of representative color values includes at least one of 1) a palette predictor, 2) an escape sample, or 3) palette information included in the bitstream, wherein the maximum number of entries of the palette is further determined based on the stripe type of the block, the color component of the block, and the segmentation structure of the block, and wherein when the block satisfies the conditions, the maximum number of entries of the palette of the block is greater than the maximum number of entries of the palette of the luminance block corresponding to the block, and the conditions include a first condition based on the stripe type, a second condition based on the color component, and a third condition based on the segmentation structure. The first condition is that the block has a first stripe type, the second condition is that the block is a luminance block, and the third condition is that the block has a dual-tree segmentation structure.

23. A video processing apparatus, including a processor configured to implement the method according to any one of claims 5 - 6 and 8 to 14.

24. A non - transitory computer - readable medium that stores program code which, when executed, causes a processor to implement the method according to any one of claims 2 to 14.

Images