Encoding of palette modes in prediction processing

Abstract

A device, system, and method for video processing are described. An exemplary method for video processing determines the number of neighboring blocks of a current block for a joint inter- and intra-prediction mode according to rules that specify how to treat blocks coded using a palette coding mode when counting the number of neighboring blocks that are intra-coded for the joint inter- and intra-prediction mode for conversion between the current block of the video and a bitstream representation of the video. The method also includes performing the conversion based on the determination.

Description

[Technical Field]

[0001] (Cross-reference of related applications) This application is the national phase of International Patent Application PCT / CN2020 / 105405, filed on 29 July 2020, claiming priority and benefits of International Patent Application PCT / CN2019 / 098204, filed on 29 July 2019. The entire disclosure of the above application is incorporated by reference as part of the disclosure herein.

[0002] This specification relates to video and image coding and decoding technologies. [Background technology]

[0003] Digital video accounts for the largest amount of bandwidth usage on the Internet and other digital communication networks. As the number of connected user devices capable of receiving and displaying video increases, the bandwidth demand for digital video usage is expected to continue to grow. [Overview of the Initiative]

[0004] The disclosed technology may be used by embodiments of video or image decoders or encoders in which palette-mode coding is used.

[0005] In one exemplary embodiment, a method for processing video is disclosed. This method includes performing a conversion between a block of video regions of video and a bitstream representation of video. This bitstream representation is processed according to a first formatting rule that specifies whether a signal is issued for the block to indicate a first use of palette mode, and a second formatting rule that specifies the position of this first indication for a second signal to indicate a predictive mode for the block.

[0006] In another exemplary embodiment, a method for processing video is disclosed. This method includes determining a prediction mode based on one or more acceptable prediction modes, which include at least one palette mode of a block of video region in video, for a conversion between a block of video region in video and a bitstream representation of the video. The method determines an instruction to use a palette mode according to the prediction mode. This method also includes performing this conversion based on one or more acceptable prediction modes.

[0007] In another exemplary embodiment, a method of video processing is disclosed, which includes performing a conversion between blocks of video and a bitstream representation of video. The bitstream representation is processed according to a format rule that specifies that it is to signal, in a dependent manner, an instruction to use a first palette mode and an instruction to use a second intra-block copy (IBC) mode.

[0008] In another exemplary embodiment, a method for processing video is disclosed. This method includes determining, based on the dimensions of a block of video, the presence of an instruction to use a palette mode in the bitstream representation, and performing the conversion between a block of video and a bitstream representation of the video, and performing the conversion based on this determination.

[0009] In another exemplary embodiment, a method for processing video is disclosed. This method includes determining, based on the dimensions of the block, whether there is an instruction to use intra-block copy (IBC) mode in the bitstream representation for conversion between a block of video and a bitstream representation of the video, and performing the conversion based on this determination.

[0010] In another exemplary embodiment, a method for processing video is disclosed. This method includes determining, based on a second display of the video region containing the block, whether a palette mode is permitted for the block, and performing the conversion between the block and a bitstream representation of the block.

[0011] In another exemplary embodiment, a method for processing video is disclosed. This method includes determining, based on a second display of the video region containing the block, whether intra-block copy (IBC) mode is permitted for the block, and performing the conversion between the block and the bitstream representation of the block.

[0012] In another exemplary embodiment, a method for processing video is disclosed. The method includes determining a first bit depth of a first sample associated with a palette entry in palette mode for conversion between a block of video and a bitstream representation of the video. The first bit depth is different from a second bit depth associated with the block. The method also includes performing a conversion based on the determination.

[0013] In another exemplary embodiment, a method for processing video is disclosed. This method includes determining, for the purpose of converting the current block of video to a bitstream representation of the video, that neighboring blocks encoded in palette mode to the current block be treated as intra-encoded blocks having a default mode, while constructing a list of maximum probability mode (MPM) candidates for the current block, if the neighboring blocks are located above or to the left of the current block. The method also includes performing the conversion based on the determination.

[0014] In another exemplary embodiment, a method for processing video is disclosed. This method includes determining parameters for deblocking filtering according to a rule for blocks of video encoded as palette-mode coded blocks in a bitstream representation of video. This method also includes performing a conversion between the blocks and the bitstream representation of video using the parameters for deblocking filtering.

[0015] In another exemplary embodiment, a method for processing video is disclosed. This method includes determining to process neighboring blocks of the current block encoded in palette mode as non-intra encoded blocks while constructing a list of maximum probability mode (MPM) candidates for the current block for conversion between the current block of video and the bitstream representation of the video. The method also includes performing the conversion based on the determination.

[0016] In another exemplary embodiment, a method for processing video is disclosed. This method includes: determining quantization parameters associated with a block of video; encoding the block of video into a bitstream representation of the video as a partially palette-encoded block based on modified values ​​of the quantization parameters; and signaling the encoding information related to the quantization parameters in the bitstream representation.

[0017] In another exemplary embodiment, a method for processing video is disclosed. This method includes deriving quantization parameters based on a bitstream representation of video, and partially decoding a palette coding block based on modified quantization parameters determined by modifying the quantization parameters.

[0018] In another exemplary embodiment, a method for processing video is disclosed. This method includes determining the representation of escape samples of a block in a bitstream representation of video, regardless of whether bypass mode is enabled for the block, for a block of video encoded as a palette-encoded block in the bitstream representation of the video. Based on the determination, this method also includes performing a conversion between the block and the bitstream representation.

[0019] In another exemplary embodiment, a method for processing video is disclosed. This method includes determining a first quantization process for blocks of video encoded as palette-encoded blocks in a bitstream representation of the video. The first quantization process is different from a second quantization process applicable to non-palette-mode encoded blocks. The method also includes performing a conversion between the blocks and the bitstream representation based on the determination.

[0020] In another exemplary embodiment, a method for processing video is disclosed. This method includes transforming a video containing luminance blocks and corresponding chrominance blocks to a bitstream representation of the video, according to a rule. The rule specifies that when the current luminance block is encoded using a palette coding mode and the corresponding current chrominance block is encoded in a derivation mode, the current luminance block is treated as having a default intra-prediction mode, and the current chrominance block is encoded in a default intra-prediction mode. This palette coding mode includes encoding the current luminance block using a palette of representative sample values.

[0021] In another exemplary embodiment, a method for processing video is disclosed. This method includes performing a conversion between a video containing one or more blocks and a bitstream representation of the video. For this conversion, a list of motion candidates is constructed for each block according to a rule. This rule stipulates that motion information of a block encoded using a palette encoding mode is treated as unavailable or invalid for encoding subsequent blocks.

[0022] In another exemplary embodiment, a method for processing video is disclosed. This method includes determining, based on a rule, the number of context coding bins of a block of video that is encoded as a palette-mode coded block in a bitstream representation of the video. Based on the determination, this method also includes performing a transformation between the block of video and the bitstream representation of the video.

[0023] In another exemplary embodiment, a method for processing video is disclosed. This method includes transforming a video containing luminance blocks and corresponding chrominance blocks to a bitstream representation of the video, in accordance with rules that, when the current luminance block is encoded using a palette coding mode and the corresponding current chrominance block is encoded using a derivation mode, the current luminance block is treated as having a default intra-predictive mode and the current chrominance block is encoded using a default intra-predictive mode. This palette coding mode includes encoding the current luminance block using a palette of representative sample values.

[0024] In another exemplary embodiment, a method for processing video is disclosed. This method includes performing a transformation between a video containing one or more blocks and a bitstream representation of the video, for which a list of motion candidates is constructed for each block based on rules that stipulate that motion information of a block encoded using a palette encoding mode is treated as unavailable or invalid for encoding of consecutive blocks.

[0025] In another exemplary embodiment, a method for processing video is disclosed. This method includes determining, based on a rule, the number of context coding bins of a block of video that is encoded as a palette-mode coded block in a bitstream representation of the video. Based on the determination, this method also includes performing a transformation between the block of video and the bitstream representation of the video.

[0026] In another exemplary embodiment, a method for processing video is disclosed. This method includes determining, based on a rule, the number of intra-encoded neighboring blocks of the current block for inter- and intra-coupled prediction modes, for the purpose of converting the current block of video to a bitstream representation of the video. The rule specifies a method for handling blocks encoded using a palette encoding mode when counting the number of intra-encoded neighboring blocks for inter- and intra-coupled prediction modes, and using a palette encoding mode includes encoding the block using a palette of representative sample values. The method also includes performing a conversion based on a determination.

[0027] In another exemplary embodiment, a method for processing video is disclosed. This method includes, in a filtering process, deciding to skip an operation on a sample of the current block of video for a conversion between the current block of video and a bitstream representation of the video. This sample is encoded using a palette encoding mode, which includes encoding the block using a palette of representative sample values. The method also includes performing a conversion based on the decision.

[0028] In another exemplary embodiment, a method for processing video is disclosed. This method includes determining a scan order selected from three or more scan orders for converting a block of video to a bitstream representation of the video. The block is encoded in palette mode using a palette of representative sample values. The method also includes performing the conversion based on a determination.

[0029] In another exemplary embodiment, a method for processing video is disclosed. This method includes determining one or more scan sequences for scanning the coefficients of a block based on the shape of the block, for the purpose of converting a block of video to a bitstream representation of the video. The method also includes performing the conversion based on the determination.

[0030] In another exemplary embodiment, a method for image processing is disclosed. This method determines to apply only one scan order to the blocks in which the coefficients of the blocks are scanned for block-based quantization residual domain difference pulse coding modulation (BDPCM) processing, which represents the difference between the quantization residual of the block's intra-prediction and the prediction of the quantization residual in the bitstream representation of the block using difference pulse coding modulation (DPCM), for the conversion of blocks of images to a bitstream representation of the image of the image. The method also includes performing the conversion based on the determination.

[0031] In another exemplary embodiment, a method for processing images is disclosed. This method includes determining to process a transform unit, encoded block, or region of a palette mode encoded separately from a prediction mode using a palette mode, and performing further processing on this transform unit, encoded block, or region using the palette mode.

[0032] In another exemplary embodiment, a method for processing video is disclosed. This method includes determining, with respect to the current video block, that a sample associated with one palette entry of a palette mode has a first bit depth different from a second bit depth associated with the current video block, and performing further processing of the current video block based on this at least one palette entry.

[0033] In another exemplary embodiment, another method of image processing is disclosed. This method involves performing a transformation between a current image block of a picture of an image and a bitstream representation of the image, wherein in the bitstream representation, information is signaled or derived based on the encoding conditions of the current image block regarding whether an intrablock copy mode is used in the transformation, and the intrablock copy mode includes encoding the current image block from another image block in the picture.

[0034] In yet another exemplary embodiment, another method of image processing is disclosed. This method includes determining whether to apply a deblocking filter while transforming the current image block of a picture of an image, wherein the current image block is encoded using palette-mode coding, which represents the current image block using representative sample values ​​smaller than the total number of pixels in the current image block; and performing a transformation such that, if it is determined that a deblocking filter should be applied, the deblocking filter is applied.

[0035] In yet another exemplary embodiment, another method of image processing is disclosed. This method includes determining a quantization or dequantization process for use in a conversion between a current image block of a picture of a certain image and a bitstream representation of the image, wherein the current image block is encoded using palette-mode coding, which represents the current image block using fewer representative sample values ​​than the total number of pixels in the current image block; and performing the conversion based on this determination of quantization or dequantization.

[0036] In yet another exemplary embodiment, another method of image processing is disclosed. This method includes determining that the current image block is a palette-encoded block for a conversion between the current image block of an image containing multiple image blocks and a bitstream representation of the image; performing a maximum probability mode list construction process by considering the current image block as an intra-encoded block based on this determination; and performing the conversion based on the result of the list construction process, wherein the palette-encoded block is encoded or decoded using palette or representation sample values.

[0037] In yet another exemplary embodiment, another method of image processing is disclosed. This method includes determining that the current image block is a palette-encoded block for a conversion between the current image block of an image containing multiple image blocks and a bitstream representation of the image, performing a maximum probability mode list construction process by considering the current image block to be a non-intra-encoded block based on this determination, and performing the conversion based on the result of this list construction process, wherein the palette-encoded block is encoded or decoded using palette or representation sample values.

[0038] In yet another exemplary embodiment, another method of image processing is disclosed. This method includes, for a conversion between the current image block of an image containing multiple image blocks and a bitstream representation of the image, determining that the current image block is a palette-encoded block, performing a list-building process by considering the current image block to be an unavailable block based on this determination, and performing the conversion based on the result of the list-building process, wherein the palette-encoded block is encoded or decoded using palette or representation sample values.

[0039] In yet another exemplary embodiment, another method of image processing is disclosed. This method includes determining that the current image block is a palette-encoded block during a conversion between the current image block and its bitstream representation, determining the range of context coding bins to be used for this conversion based on the current image block being a palette-encoded block, and performing this conversion based on the range of context coding bins.

[0040] In yet another exemplary embodiment, the method described above may be implemented by a video encoder including a processing unit.

[0041] In yet another exemplary embodiment, these methods may be carried out in the form of processing unit executable instructions and stored on a computer-readable program medium.

[0042] These and other embodiments are further described herein. [Brief explanation of the drawing]

[0043] [Figure 1] Example of intrablock copy [Figure 2] Example of a block encoded in palette mode [Figure 3] Example of using a palette predictor to signal palette entries. [Figure 4] Examples of horizontal and vertical transverse scanning [Figure 5] Example of palette index coding [Figure 6] Block diagram showing an example of an image processing device. [Figure 7] Block diagram showing an example of a video encoder implementation. [Figure 8] Flowchart showing an example of an image processing method [Figure 9] Examples of pixels involved in determining whether a filter is on or off and selecting a strong or weak filter. [Figure 10] Examples of binarization in four modes [Figure 11] Examples of binarization in four modes [Figure 12] 67 examples of intramode prediction directions [Figure 13] Examples of nearby video blocks [Figure 14] Examples of ALF filter shapes (Saturation: 5x5 diamond, Luminance: 7x7 diamond) [Figure 15A] Example of subsampled Laplacian calculation of vertical gradient [Figure 15B] Example of subsampled Laplacian calculation for horizontal gradient [Figure 15C] Example of subsampled Laplacian calculation for diagonal gradients [Figure 15D] Example of subsampled Laplacian calculation for diagonal gradients [Figure 16] Examples of modified divisions in virtual boundaries [Figure 17] Example of modified ALF filtering for luminance components at a virtual boundary [Figure 18] Examples of four 1-D3 pixel patterns for pixel classification in EO [Figure 19] An example of grouping four bands and representing them by their starting band position. [Figure 20] Example of an upper-left neighborhood block used in CIIP weight derivation [Figure 21] Example of luminance mapping using a saturation scaling architecture [Figure 22] Example of a traversal order for a 4x4 block [Figure 23] Another example of a traversal order for a 4x4 block [Figure 24] Block diagram showing an exemplary image processing system 2400 in which various technologies disclosed herein may be implemented. [Figure 25] A flowchart illustrating another video processing method based on this technology. [Figure 26] Another flowchart showing a different video processing method based on this technology. [Figure 27] Another flowchart showing a different video processing method based on this technology. [Figure 28] Another flowchart showing a different video processing method based on this technology. [Figure 29] Another flowchart showing a different video processing method based on this technology. [Figure 30] Another flowchart showing a different video processing method based on this technology. [Figure 31] Another flowchart showing a different video processing method based on this technology. [Figure 32] Another flowchart showing a different video processing method based on this technology. [Figure 33]Another flowchart showing a different video processing method based on this technology. [Figure 34] Another flowchart showing a different video processing method based on this technology. [Figure 35] Another flowchart showing a different video processing method based on this technology. [Figure 36A] Another flowchart showing a different video processing method based on this technology. [Figure 36B] Another flowchart showing a different video processing method based on this technology. [Figure 37] Another flowchart showing a different video processing method based on this technology. [Figure 38] Another flowchart showing a different video processing method based on this technology. [Figure 39] Another flowchart showing a different video processing method based on this technology. [Figure 40] Another flowchart showing a different video processing method based on this technology. [Figure 41] Another flowchart showing a different video processing method based on this technology. [Figure 42] Another flowchart showing a different video processing method based on this technology. [Figure 43] Another flowchart showing a different video processing method based on this technology. [Figure 44] Another flowchart showing a different video processing method based on this technology. [Figure 45] Another flowchart showing a different video processing method based on this technology. [Figure 46] Another flowchart showing yet another video processing method based on this technology [Modes for carrying out the invention]

[0044] This specification provides various techniques that can be used by decoders of image or video bitstreams to improve the quality of decompressed or decoded digital video or images. For brevity, the term “video” is used herein to include both a series of pictures (conventionally called video) and individual images. Furthermore, video encoders may implement these techniques during the encoding process to reconstruct decoded frames for use in further encoding.

[0045] Chapter headings are used in this specification for ease of understanding, and embodiments disclosed in one chapter are not limited to that chapter alone. Thus, embodiments in one chapter can be combined with embodiments in other chapters.

[0046] 1. Outline of the Invention This specification relates to video coding technology. Specifically, the present invention relates to palette coding, which uses a representation based on basic colors in video coding. It may be applied to existing video coding standards such as HEVC, or it may be applied to establish a standard (Versatile Video Coding). The present invention is also applicable to future video coding standards or video codecs.

[0047] 2. Initial consultations Video coding standards have primarily evolved through the development of well-known ITU-T and ISO / IEC standards. ITU-T created H.261 and H.263, while ISO / IEC created MPEG-1 and MPEG-4 Visual. The two organizations jointly developed the H.262 / MPEG-2 Video, H.264 / MPEG-4 AVC (Advanced Video Coding), and H.265 / HEVC standards. Since H.262, video coding standards have been based on hybrid video coding structures that utilize time prediction and transformation coding. To explore future video coding technologies beyond HEVC, VCEG and MPEG jointly established JVET (Joint Video Exploration Team) in 2015. Since then, many new methods have been adopted by JVET and incorporated into reference software called JEM (Joint Exploration Mode). In April 2018, the Joint Video Expert Team (JVET) was established between VCEG (Q6 / 16) and ISO / IEC JTC1 SC29 / WG11 (MPEG), and is working on developing the VVC standard with the goal of reducing the bitrate by 50% compared to HEVC.

[0048] Figure 7 is a block diagram of an example implementation of a video encoder. Figure 7 shows that the encoder implementation incorporates a feedback path in which the video encoder also performs video decoding (reconstructing a compressed representation of the video data for use in encoding the next video data).

[0049] 2.1 Intrablock Copy Intra-block copying (IBC), also known as Current Picture Referencing (CPR), is employed in HEVC Screen Content Coding extensions (HEVC-SCC) and the current VVC test model (VTM-4.0). IBC extends the concept of motion compensation from inter-frame coding to intra-frame coding. As shown in Figure 1, the current block is predicted by one reference block within the same picture when IBC is applied. Before encoding or decoding the current block, the samples in the reference block must already be reconstructed. While IBC is not very efficient for most sequences captured by a camera, it shows a significant coding gain for screen content. This is because screen content pictures often contain many repeating patterns such as icons and characters. IBC can effectively remove redundancy between these repeating patterns. In HEVC-SCC, an inter-coding unit (CU) can apply IBC if it selects the current picture as its reference picture. In this case, MV is renamed Block Vector (BV), and BV always has integer pixel precision. To conform to the main profile HEVC, the current picture is marked as a "long-running" reference picture in the Decoded Picture Buffer (DPB). Similarly, in multiple view / 3D video encoding standards, reference pictures between views are also marked as "long-running" reference pictures.

[0050] After the BV finds its reference block, it can generate a prediction by copying this reference block. The residual can be obtained by subtracting the reference pixel from the original signal. Then, as with other coding modes, transformations and quantization can be applied.

[0051] However, if the reference block lies outside the picture, overlaps with the current block, lies outside the reconstructed region, or lies outside the valid region limited by some constraint, some or all of the pixel values ​​will be undefined. Essentially, there are two solutions to this problem. One is to disallow such situations, for example, bitstream compatibility. The other is to apply padding to these undefined pixel values. The following sub-sessions will detail these solutions.

[0052] 2.2 IBC in HEVC Screen Content Encoding Extensions In the HEVC screen content encoding extension, when a block uses the current picture as a reference, it should be ensured that the entire referenced block is within the available reconfigured area, as shown in the text of the following specification. The variables offsetX and offsetY are derived as follows: offsetX=(ChromaArrayType==0)?0:(mvCLX[0]&0×7 ?2:0) (8-106) offsetY=(ChromaArrayType==0)?0:(mvCLX[1]&0×7 ?2:0) (8-107) If the reference picture is the current picture, the bitstream compatibility requirement is that the luminance motion vector mvLX should adhere to the following constraints: The derivation process for the availability of z-scan sequence blocks, as defined in Section -6.4.1, is called with input (xCb, yCb) set to (xCurr, yCurr) and a neighboring luminance position (xNbY, yNbY) set to (xPb + (mvLX[0]>>2)-offsetX, yPb + (mvLX[1]>>2)-offsetY), and the output is TRUE. The derivation process for the availability of z-scan sequence blocks, as defined in Section -6.4.1, is called with inputs (xCurr, yCurr) set to equal (xCb, yCb) and a neighboring luminance position (xNbY, yNbY) set to equal (xPb+(mvLX[0]>>2)+nPbW-1+offsetX, yPb+(mvLX[1]>>2)+nPbH-1+offsetY), and the output is TRUE. - One or both of the following conditions must be TRUE. - The value of (mvLX[0]>>2)+nPbW+xB1+offsetX is less than or equal to 0. - The value of (mvLX[1]>>2)+nPbH+yB1+offsetY is less than or equal to 0. - The following conditions must be true. (xPb+(mvLX[0]>>2)+nPbSw-1+offsetX) / CtbSizeY-xCurr / CtbSizeY<=yCurr / CtbSizeY-(yPb+(mvLX[1]>>2)+nPbSh-1+offsetY) / CtbSizeY (8-108)

[0053] Thus, cases where a referenced block overlaps with the current block, or where a referenced block is outside the picture, do not occur. There is no need to fill in the referenced block or the predicted block.

[0054] 2.3. IBC in the VVC Test Model In current VVC test models, such as the VTM-4.0 design, the entire reference block should have the current Coding Tree Unit (CTU) and should not overlap with the current block. Therefore, there is no need to pad the reference or prediction block. The IBC flag is encoded as the prediction mode of the current CU. Thus, for each CU, there are a total of three prediction modes: MODE_INTRA, MODE_INTER, and MODE_IBC.

[0055] 2.3.1 IBC Merge Mode In IBC merge mode, the index pointing to the entry in the IBC merge candidate list is parsed from the bitstream. The construction of this IBC merge list can be put together according to the following sequence of steps.

[0056] Step 1: Derivation of spatial candidates Step 2: Insertion of HMVP candidates Step 3: Inserting the average candidate for each pair

[0057] In the derivation of spatial merge candidates, up to four merge candidates are selected from the candidates located at the positions shown in the diagram. The derivation order is A1, B1, B0, A0, B2. Position B2 is considered only if any of the Prediction Units (PUs) at positions A1, B1, B0, or A0 are unavailable (e.g., belonging to a different slice or tile) or are not encoded in IBC mode. After adding the candidate at position A1, the remaining candidates are inserted and subjected to a redundancy check, which ensures that candidates with the same motion information are reliably excluded from the list, thereby improving encoding efficiency. To reduce computational complexity, the aforementioned redundancy check does not consider all possible candidate pairs. Instead, only pairs linked by the arrows shown in the diagram are considered, and a candidate is added to the list only if the corresponding candidate used in the redundancy check does not have the same motion information.

[0058] If, after inserting spatial candidates, the IBC merge list size is still smaller than the maximum IBC merge list size, IBC candidates from the HMVP table can be inserted. A redundancy check is performed when inserting HMVP candidates.

[0059] Finally, insert the pair's average candidate into the IBC merge list.

[0060] If the referenced block identified by the merge candidate lies outside the picture, overlaps with the current block, lies outside the reconfigured area, or lies outside the valid area restricted by some constraint, the merge candidate is called an invalid merge candidate.

[0061] Note that you may insert invalid merge candidates into the IBC merge list.

[0062] 2.3.2 IBC AMVP Mode In IBC AMVP mode, AMVP indices pointing to entries in the IBC AMVP list are parsed from the bitstream. The construction of this IBC AMVP list can be summarized in the following sequence of steps.

[0063] Step 1: Derivation of spatial candidates

[0064] Check A0 and A1 until a suitable candidate is found.

[0065] Check B0, B1, and B2 until a suitable candidate is found.

[0066] Step 2: Insertion of HMVP candidates

[0067] Step 3: Inserting a zero candidate

[0068] If, after inserting spatial candidates, the IBC AMVP list size is still smaller than the maximum IBC AMVP list size, then IBC candidates from the HMVP table can be inserted.

[0069] Finally, insert zero candidates into the IBC AMVP list.

[0070] 2.4 Palette Mode The fundamental idea behind palette mode is to represent samples in CU with a small set of representative color values. This set is called the palette. Samples outside the palette can also be indicated by signaling (sometimes quantized) component values ​​after an escape symbol. This is shown in Figure 2.

[0071] 2.5 Palette Mode in HEVC Screen Content Encoding Extensions (HEVC-SCC) In HEVC-SCC's palette mode, a prediction scheme is used to encode the palette and index map.

[0072] 2.5.1 Encoding of Palette Entries Palette predictors are maintained to encode palette entries. The maximum palette size and palette predictors are signaled in the Sequence Parameter Set (SPS). In HEVC-SCC, palette_predictor_initializer_present_flag is introduced into the PPS. If this flag is 1, an entry for initializing the palette predictor is signaled in the bitstream. Palette predictors are initialized at the beginning of each CTU row, each slice, and each tile. Depending on the value of palette_predictor_initializer_present_flag, the palette predictor is either reset to 0 or initialized using the palette predictor initialization entry signaled to the PPS. HEVC-SCC enables a palette predictor initialization module of size 0 to allow explicit disabling of palette predictor initialization at the PPS level.

[0073] For each entry in the palette predictor, a reuse flag is signaled to indicate whether it is part of the current palette. This is shown in Figure 3. The reuse flag is transmitted using zero run-length coding. After this, the number of new palette entries is signaled using exponential Golomb coding of order zero. Finally, the component value for the new palette entry is signaled.

[0074] 2.5.2 Encoding of Palette Indexes The palette index is encoded using horizontal and vertical transverse scanning, as shown in Figure 4. The palette_transpose_flag is used to explicitly signal the scanning order in the bitstream. The following subsections assume that the scanning is horizontal.

[0075] Palette indices are encoded using two main palette sample modes: "INDEX" and "COPY_ABOVE". As previously mentioned, escape symbols are also signaled as "INDEX" mode, and an index equal to the maximum palette size is assigned. This mode is signaled using a flag to exclude the top row, or if the previous mode was "COPY_ABOVE". In "COPY_ABOVE" mode, the palette index of the sample in the row above is copied. In "INDEX" mode, the palette index is explicitly signaled. In both "INDEX" and "COPY_ABOVE" modes, an execution value is signaled that specifies the number of subsequent samples to be encoded using the same mode. If an escape symbol is part of an execution in "INDEX" or "COPY_ABOVE" mode, an escape component value is signaled for each escape symbol. The encoding of the palette index is shown in Figure 5.

[0076] This syntax sequence is executed as follows: First, the number of index values ​​for the CU is signaled. This is followed by the signaling of the actual index values ​​for the entire CU using truncated binary coding. In bypass mode, both the number of indices and the index values ​​are coded. This groups the index-related bypass bins. Next, palette sample mode (if necessary) and execution are signaled in an interleaved manner. Finally, the component escape values ​​corresponding to the escape samples for the entire CU are grouped and coded in bypass mode.

[0077] After signaling the index value, an additional syntax element, last_run_type_flag, is signaled. This syntax element eliminates the need to signal the execution value corresponding to the last execution in the block, in conjunction with the index number.

[0078] In HEVC-SCC, palette mode is also available for 4:2:2, 4:2:0, and monochrome saturation formats. Signaling of palette entries and palette indices is largely the same for all saturation formats. For non-monochrome formats, each palette entry consists of three components. For monochrome formats, each palette entry consists of a single component. For subsampled luminance directions, saturation samples are associated with a saturation sample index divisible by 2. After reconstructing the palette index of the CU, if only a single component is associated with a sample, only the first component of the palette entry is used. The only difference in signaling is the escape component value. For each escape sample, the number of escape component values ​​signaled may vary depending on the number of components associated with that sample.

[0079] In VVC, a dual-tree coding structure is used for encoding the intra-slice, so the luminance component and the two saturation components may have different palettes and palette indices. Alternatively, the two saturation components may share the same palette and palette indices.

[0080] 2.6 Deblocking Schemes in VVC In the following explanation, pN M qN represents the Nth sample from the left in the Mth row relative to a vertical edge, or the Nth sample from the top in the Mth column relative to a horizontal edge. M pN represents the Nth sample from the right in the Mth row for a vertical edge, or the Nth sample from the bottom in the Mth column for a horizontal edge. M and qN M An example is shown in Figure 9.

[0081] In the following explanation, p N represents the Nth sample from the left of a row relative to a vertical edge, or the Nth sample from the top of a column relative to a horizontal edge, and q N This represents the Nth sample to the right of a row relative to a vertical edge, or the Nth sample from the bottom of a column relative to a horizontal edge.

[0082] The filter is turned on or off in units of four rows. Figure 9 shows the pixels involved in determining whether the filter is on or off. The six pixels within the two red boxes for the first four rows are used to determine whether the filter is on or off for those four rows. The six pixels within the two red boxes for the second four rows are used to determine whether the filter is on or off for the second four rows.

[0083] In some embodiments, first, the vertical edges of the picture are selected. Then, using the samples modified by the vertical edge filtering process as input, the horizontal edges of the picture are filtered. The vertical and horizontal edges in the CTB of each CTU are processed separately for each coding unit. The vertical edges of the coding blocks in a coding unit are filtered starting from the left edge of the coding block and proceeding in their geometric order towards the right edge of the coding block. The horizontal edges of the coding blocks in a coding unit are filtered starting from the upper edge of the coding block and proceeding in their geometric order through the edges towards the lower edge of the coding block.

[0084] 2.6.1 Determination of Boundaries Filtering is applied to the 8×8 block boundaries. Further, it must be the boundary of the transform block or the coding sub-block (for example, because affine motion prediction, ATMVP is used). If it is not such a boundary, the filter is disabled.

[0085] 2.6.2 Boundary Strength Calculation Regarding the boundary of the transform block / coding sub-block, if it is located on the 8×8 grid, it may be filtered, and the setting of bS[xD i [yD j ([xD i [yD j represents coordinates) is defined as follows.

[0086] - If sample p0 or q0 is in the coding block of a coding unit encoded in the intra prediction mode, bS[xD i [yD j is set equal to 2.

[0087] - Alternatively, if the block edge is also a transform block edge and sample p0 or q0 is in a transform intra block containing one or more non-zero transform coefficient levels, bS[xD i][yD j ] is set to equal to 1.

[0088] -Alternatively, if the prediction mode of the coded subblock containing sample p0 is different from the prediction mode of the coded subblock containing sample q0, then bS[xD i ][yD j ] is set to equal to 1.

[0089] -Alternatively, if one or more of the following conditions are true, then bS[xD i ][yD j ] is set to equal to 1.

[0090] -The coded subblock containing sample p0 and the coded subblock containing sample q0 are both coded in IBC prediction mode, and the absolute difference between the horizontal or vertical components of the motion vectors used to predict the two coded subblocks is 4 or greater per quarter luminance sample.

[0091] - For the prediction of the coded subblock containing sample p0, a different reference picture or a different number of motion vectors is used compared to the prediction of the coded subblock containing sample q0.

[0092] Note 1 - Whether the reference pictures used for the two encoding subblocks are the same or different is determined solely by which picture is referenced, regardless of whether the prediction is formed using the index to reference picture list 0 or the index to reference picture list 1, and regardless of whether the index positions within the reference picture lists are different.

[0093] Note 2 - The number of motion vectors used to predict the coded subblock containing the top-left sample (xSb, ySb) is equal to PredFlagL0[xSb][ySb] + PredFlagL1[xSb][ySb].

[0094] - One motion vector is used to predict the coded subblock containing sample p0, and another motion vector is used to predict the coded subblock containing sample q0, and the absolute difference between the horizontal or vertical components of the motion vectors used is 4 or greater per quarter luminance sample unit.

[0095] - Predict the coded subblock containing sample p0 using two motion vectors and two different reference pictures, and predict the coded subblock containing sample q0 using two motion vectors from the same two reference pictures, and the absolute difference between the horizontal or vertical components of the two motion vectors used to predict the two coded subblocks from the same reference picture is 4 or greater in 1 / 4 luminance samples.

[0096] - Predict the coded subblock containing sample p0 using two motion vectors from the same reference picture, and predict the coded subblock containing sample q0 using two motion vectors from the same reference picture, provided that both of the following conditions are met:

[0097] - The absolute difference between the horizontal or vertical components of the motion vectors in List 0 used to predict the two coded subblocks is 4 or greater in a quarter luminance sample, or the absolute difference between the horizontal or vertical components of the motion vectors in List 1 used to predict the two coded subblocks is 4 or greater in a quarter luminance sample unit.

[0098] - The absolute difference between the horizontal or vertical component of the List 0 motion vector used to predict the coded subblock containing sample p0 and the List 1 motion vector used to predict the coded subblock containing sample q0 is 4 or greater per quarter luminance sample unit, or the absolute difference between the horizontal or vertical component of the List 1 motion vector used to predict the coded subblock containing sample p0 and the List 0 motion vector used to predict the coded subblock containing sample q0 is 4 or greater per quarter luminance sample unit.

[0099] - Alternatively, the variable bS[xD i ][yD j Set ] to 0.

[0100] Tables 2-1 and 2-2 summarize the rules for calculating balance sheets.

[0101] [Table 1]

[0102] [Table 2]

[0103] 2.6.3 Determination of Deblocking of Luminance Components The deblocking decision process is described in this subsection.

[0104] [ka]

[0105] Condition 1 is the "large block condition". This condition detects whether the samples on the P side and Q side belong to a large block represented by the variables bSidePisLargeBlk and bSideQisLargeBlk, respectively. bSidePisLargeBlk and bSideQisLargeBlk are defined as follows:

[0106] bSidePisLargeBlk=((edge ​​type is vertical and p0belongs to CU with width>=32) | | (edge ​​type is horizontal and p0belongs to CU with height>=32))?TRUE:FALSE

[0107] bSideQisLargeBlk=((edge ​​type is vertical and q0belongs to CU with width>=32) | | (edge ​​type is horizontal and q0belongs to CU with height>=32))?TRUE:FALSE

[0108] Based on bSidePisLargeBlk and bSideQisLargeBlk, condition 1 is defined as follows:

[0109] [ka]

[0110] [ka]

[0111] - dp0, dp3, dq0, dq3 are first derived as HEVC.

[0112] - if(p side is greater than or equal to 32)

[0113] dp0=(dp0+Abs(p50-2*p40+p30)+1)>>1

[0114] dp3=(dp3+Abs(p53-2*p43+p33)+1)>>1

[0115] - if(q side is greater than or equal to 32)

[0116] dq0=(dq0+Abs(q50-2*q40+q30)+1)>>1

[0117] dq3=(dq3+Abs(q53-2*q43+q33)+1)>>1

[0118] [ka]

[0119] As shown in Chapter 2.2.4, d = dp0 + dq0 + dp3 + dq3.

[0120] If conditions 1 and 2 are valid, further check whether any of the blocks use subblocks.

[0121] If (bSidePisLargeBlk)

[0122] If (mode block P == SUBBLOCKMODE)

[0123] Sp=5

[0124] Else

[0125] Sp=7

[0126] Else

[0127] Sp=3

[0128] If(bSideQisLargeBlk)

[0129] If(mode block Q == SUBBLOCKMODE)

[0130] Sq=5

[0131] Else

[0132] Sq=7

[0133] Else

[0134] Sq=3

[0135] [ka]

[0136] [ka]

[0137] dpq is derived in the same way as HEVC.

[0138] sp3 = Abs(p3 - p0), which can be derived in the same way as HEVC.

[0139] if(p side is greater than or equal to 32)

[0140] if(Sp==5)

[0141] sp3=(sp3+Abs(p5-p3)+1)>>1

[0142] Else

[0143] sp3=(sp3+Abs(p7-p3)+1)>>1

[0144] sq3 = Abs(q0 - q3) can be derived in the same way as HEVC.

[0145] if(q side is greater than or equal to 32)

[0146] If (Sq == 5)

[0147] sq3=(sq3+Abs(q5-q3)+1)>>1

[0148] Else

[0149] sq3=(sq3+Abs(q7-q3)+1)>>1

[0150] Similar to HEVC, StrongFilterCondition=(dpq is less than (β>>2),sp3+sq3is less than(3*β>>5),and Abs(p0-q0)is less than(5*t C +1)>>1)?TRUE:FALSE

[0151] 2.6.4 Stronger non-blocking filter for brightness (designed for larger blocks) Bilinear filters are used when samples on both sides of a boundary belong to a single large block. A single sample belonging to a large block is defined as having a width ≥ 32 for vertical edges and a height ≥ 32 for horizontal edges.

[0152] The bilinear filter is shown below.

[0153] Next, q for i=0 to Sp-1 and j=0 to Sq-1 i The block boundary sample p is (where pi, qi is the i-th sample in the row filtering vertical edges, or the i-th sample in the column filtering horizontal edges). i Replace it using linear interpolation as follows.

[0154]

number

[0155] tcPD i and tcPD j The term is position-dependent clipping as described in Chapter 2.3.6, and g j ,f i Middle s,t ,P s and Q s This is shown in Table 2-3.

[0156] [Table 3] [Table 4]

[0157] 2.6.5 Controlling the unblocking of saturation A highly saturated filter is applied to both sides of the block boundary. Here, the saturation filter is selected when both sides of the saturation edge have a value of 8 (saturation position) or greater, and three conditions are met (the first condition is for determining boundary strength, similar to large blocks). The proposed filter is applicable when the width or height of blocks orthogonal to the block edges in the saturation sample domain is 8 or greater. The second and third conditions are essentially the same as those for determining the deblocking of HEVC luminance, and are for determining on / off and strong filtering, respectively.

[0158] In the first decision, the boundary intensity (bS) is modified for saturation filtering, as shown in Table 2-2. The conditions in Table 2-2 are checked sequentially. If a condition is met, the remaining lower-priority conditions are skipped.

[0159] If a large block boundary is detected, saturation deblocking is performed if bS is equal to 2 or bS is equal to 1.

[0160] The second and third conditions are essentially the same as determining the strong HEVC luminance filter, as follows:

[0161] Under the second condition, d is derived in the same way as in HEVC luminance deblocking.

[0162] The second condition is TRUE if d is less than β.

[0163] In the third condition, StrongFilterCondition is derived as follows:

[0164] dpq is derived in the same way as HEVC.

[0165] sp3 = Abs(p3 - p0) can be derived in the same way as HEVC.

[0166] The equation sq3 = Abs(q0 - q3) can be derived in the same way as HEVC.

[0167] In HEVC design, StrongFilterCondition=(dpq is less than(β>>2),sp3+sq3is less than(β>>3),and Abs(p0-q0)is less than(5*t C +1)>>1).

[0168] 2.6.6 Strong unblocking filter for saturation A strong non-blocking filter for saturation is defined as follows:

[0169] p2´=(3*p3+2*p2+p1+p0+q0+4)>>3

[0170] p1´=(2*p3+p2+2*p1+p0+q0+q1+4)>>3

[0171] p0´=(p3+p2+p1+2*p0+q0+q1+q2+4)>>3

[0172] The proposed saturation filter performs deblocking on a 4x4 saturation sample grid.

[0173] 2.6.7 Position-dependent clipping Position-dependent clipping (tcPD) is applied to the output samples of a luminance filtering process that includes strong, long filters that correct 7, 5, and 3 samples at the boundary. Assuming a quantization error distribution, it is proposed to increase the clipping value for samples that are expected to have higher quantization noise, and thus are expected to have a higher deviation of the reconstructed sample value from the true sample value.

[0174] For each P or Q boundary filtered by the asymmetric filter, a position-dependent threshold table is selected from two tables (i.e., Tc7 and Tc3 shown below) provided to the decoder as supplementary information, based on the results of the decision-making process in Chapter 2.3.3. Tc7={6,5,4,3,2,1,1}; Tc3={6,4,2}; tcPD=(Sp==3)?Tc3:Tc7; tcQD=(Sq==3)?Tc3:Tc7;

[0175] For P or Q boundaries filtered by a short symmetric filter, a smaller position-dependent threshold is applied. Tc3={3,2,1};

[0176] After defining the threshold, the filtered p' is determined according to the tcP and tcQ clipping values. i and q' i Clip the sample values. p´´ i =Clip3(p' i +tcP i ,p' i -tcP i ,p' i ); q´´ j =Clip3(q' j +tcQ j ,q´ j -tcQ j ,q´ j );

[0177] Here, p' i and q' i This is a filtered sample value, p'' i and q´´ j This is the output sample value after clipping, and tcP i tcP i is the clipping threshold derived from the VVC tc parameters, tcPD and tcQD. The function Clip3 is a clipping function as defined in VVC.

[0178] 2.6.8 Adjustment for deblocking subblocks To enable parallel-friendly deblocking and subblock deblocking using both long filters, the long filters limit sample correction on the side using subblock deblocking (AFFINE, ATMVP, or DMVR) to a maximum of 5, as shown in the luminance control for the long filters. Furthermore, subblock deblocking is adjusted so that subblock boundaries near CU or implicit TU boundaries on an 8x8 grid are limited to a maximum of 2 samples corrected on each side.

[0179] The following applies to CU boundaries and unaligned subblock boundaries.

[0180] If (mode block Q==SUBBLOCKMODE && edge !=0){

[0181] if(!(implicitTU && (edge==(64 / 4))))

[0182] if(edge==2 || edge==(orthogonalLength-2) || edge==(56 / 4) || edge==(72 / 4))

[0183] Sp=Sq=2;

[0184] Else

[0185] Sp=Sq=3;

[0186] Else

[0187] Sp=Sq=bSideQisLargeBlk ?5:3

[0188] }

[0189] In this case, edges equal to 0 correspond to CU boundaries, and edges equal to 2 or orthogonalLength-2 correspond to 8 samples of the subblock boundary from the CU boundary. Here, if implicit partitioning of TU is used, the implicit TU is true.

[0190] 2.6.9 Limitation to 4CTU / 2CTU row buffers for brightness / saturation If the horizontal edge aligns with the CTU boundary, horizontal edge filtering is limited to Sp=3 for luminance and Sp=1 and Sq=1 for saturation.

[0191] 2.7 Intra-mode coding in VVC To capture arbitrary edge directions appearing in natural footage, the number of directional intra-modes in VTM5 is expanded from 33 used in HEVC to 65. New directional modes not present in HEVC are indicated by red dotted arrows in Figure 12, while the planar and DC modes remain the same. These higher-density directional intra-predictive modes apply to all block sizes and to both luminance and saturation intra-predictive modes.

[0192] In VTM5, several conventional angle intra-prediction modes are adaptively replaced by wide-angle intra-prediction modes for non-square blocks. For details on wide-angle intra-prediction, please refer to Chapter 3.3.1.2.

[0193] In HEVC, all intra-encoded blocks have a square shape, and the length of each side is a power of 2. Thus, no division operation is required to generate intra-predictors using DC mode. In VTM5, blocks may be rectangular, and in general, division operations are required for each block. To avoid division operations for DC prediction, only the longest side is used to calculate the average for non-square blocks.

[0194] To keep the complexity of generating the Most Probable Mode (MPM) list low, an intra-mode coding method with six MPMs is used by considering the intra-modes of two available neighbors. To construct the MPM list, the following three aspects are considered.

[0195] 1. Default intra-mode 2. Neighboring intra-mode 3. Derived intra-mode

[0196] Regardless of whether the MRL and ISP coding tools are applied, a unified 6-MPM list is used for the intra-block. The MPM list is constructed based on the intra-mode of the upper-left neighboring block. Here, if the mode of the left block is Left and the mode of the above block is Above, the integrated MPM list is constructed as follows (the left and upper blocks are shown in Figure 13).

[0197] - If the neighboring block is not available, its intra-mode is set to Planar by default.

[0198] - If both the Left and Above modes are non-angular modes,

[0199] *MPM list → {Planar, DC, V, H, V-4, V+4}

[0200] - If one of the Left and Above modes is an angular mode and the other is a non-angular mode, the following occurs.

[0201] * Set the mode Max in Left and Above to the larger mode.

[0202] * MPM list → {Planar, Max, DC, Max-1, Max+1, Max-2}

[0203] - If both Left and Above have angles and they are different, then the following occurs:

[0204] *Set Mode Max to a larger mode in Left and Above.

[0205] *When the difference between Left and Above modes is within the range of 2 to 62

[0206] MPM list → {Planar, Left, Above, DC, Max-1, Max+1}

[0207] *Otherwise,

[0208] MPM list → {Planar, Left, Above, DC, Max-2, Max+2}

[0209] -If both Left and Above have angles and are the same, then the following applies:

[0210] *MPM list → {Planar, Left, Left-1, Left+1, DC, Left-2}

[0211] Additionally, the first bin of the mpm index codeword is CABAC context-encoded. A total of three contexts are used, corresponding to whether the current intrablock is MRL enabled, ISP enabled, or a regular intrablock.

[0212] During the generation process of the six MPM lists, pruning is used to remove duplicate modes, ensuring that only the unique mode is included in each MPM list. Truncated Binary Code (TBC) is used for entropy coding of the 61 non-MPM modes.

[0213] In the case of chroma intra-mode coding, a total of eight intra-modes are permitted for chroma intra-mode coding. These modes include five traditional intra-modes and three component-common linear model modes (CCLM, LM_A, and LM_L). The chroma mode signal notification and derivation processes are shown in Table 2-4. Chroma mode coding directly depends on the intra-prediction mode of the corresponding luminance block. In the I-slice, the separation of the block partition structure for luminance and chroma components is enabled, so one chroma block may correspond to multiple luminance blocks. Therefore, in the case of chroma DM mode, the intra-prediction mode of the corresponding luminance block is directly inherited, including the center position of the current chroma block.

[0214] [Table 5]

[0215] 2.8 Quantized Residual Block Differential Pulse Code Modulation (QR-BDPCM) In some embodiments, quantized residual block differential pulse-code modulation (QR-BDPCM) is proposed to efficiently encode screen content.

[0216] The prediction direction used in QR-BDPCM can be vertical or horizontal prediction mode. Intra prediction, like intra prediction, predicts the entire block by sample copying in the prediction direction (horizontal or vertical prediction). The residuals are quantized, and the delta between the quantized residuals and their predictor (horizontal or vertical) quantized values ​​is encoded. This can be explained as follows: For a block of size M (rows) × N (columns), r i,j, for \(0\leq i\leq M - 1\) and \(0\leq j\leq N - 1\), after performing intra prediction horizontally (copying the pixel values of the left adjacent pixels of the prediction block line by line) or vertically (copying the upper adjacent line for each line in the prediction block) using samples that have not been filtered from the boundary samples of the upper or left block, it is the prediction residual. \(Q(r i , j ), for \(0\leq i\leq M - 1\) and \(0\leq j\leq N - 1\) represents the quantized version of the residual \(r i,j \), where in this case, the residual is the difference between the original block and the prediction block values. Next, block DPCM is applied to the quantized residual samples, and as a result, a modified \(M\times N\) array \(R ~ i , j \) with elements \(r ~ \) is obtained. When vertical BDPCM is signaled, it is as follows.

[0217]

Number

[0218] In the case of horizontal prediction, similar rules apply, and the residual quantization samples are obtained by the following formula.

[0219]

Number

[0220] The residual quantization sample \(r ~ i,j \) is sent to the decoder.

[0221] <l On the decoder side, reversing the above calculations, \(Q(r i,j ), for \(0\leq i\leq M - 1\) and \(0\leq j\leq N - 1\) is generated. In the case of vertical prediction, ​​​​​​​​​​In the horizontal direction,

[0224]

number

[0225] Inversely quantized residual Q -1 (Q(r i,j Add the )) to the intrablock predicted value to generate the reconstructed sample value.

[0226] The main advantage of this scheme is that reverse DPCM can be performed on the fly during coefficient parsing, requiring only the addition of predictors during coefficient parsing, or even after parsing.

[0227] 2.9 Adaptive Loop Filters In VTM5, an Adaptive Loop Filter (ALF) with block-based filter adaptation is applied. For the luminance component, one of 25 filters is selected for every 4x4 block based on the direction and function of the local gradient.

[0228] 2.9.1 Filter Shape In VTM5, two diamond filter shapes (shown in Figure 14) are used. A 7x7 diamond is applied to the luminance component, and a 5x5 diamond is applied to the saturation component.

[0229] 2.9.2 Block divisions For the luminance component, each 4x4 block is classified into one of 25 classes. The classification index C is derived based on its directionality D and the quantized value of activity A^ as follows:

[0230]

number

[0231] To calculate D and A^, we first use the 1-D Laplacian to calculate the slopes in the horizontal, vertical, and two diagonal directions.

[0232]

number

[0233] In this case, i and j represent the coordinates of the top-left sample in the 4x4 intrablock, and R(i,j) represents the reconstructed sample at coordinates (i,j).

[0234] To reduce the complexity of the block divisions, a subsampled 1-D Laplacian calculation is applied. As shown in Figures 15A to 15D, the same subsampling position is used for gradient calculations in all directions.

[0235] Then, the maximum and minimum values ​​of the gradient D in the horizontal and vertical directions are set as follows.

[0236]

number

[0237] The maximum and minimum values ​​of the two diagonal gradients are set as follows:

[0238]

number

[0239] To derive the value of directivity D, these values ​​are compared with each other and with two thresholds t1 and t2.

[0240]

number

[0241] If both are TRUE, D is set to 0.

[0242]

number

[0243] In that case, continue from step 3, or continue from step 4.

[0244]

number

[0245] If so, D is set to 2, or D is set to 1.

[0246]

number

[0247] If so, D is set to 4, or D is set to 3.

[0248] Activity value A is calculated as follows:

[0249]

number

[0250] A is further quantized to a range of 0 to 4, and the quantized value is denoted as A^.

[0251] No classification method is applied to the saturation components in a picture; rather, a single set of ALF coefficients is applied to each saturation component.

[0252] 2.9.3. Geometric transformation of filter coefficients and clipping values Before filtering each 4x4 luminance block, geometric transformations such as rotation or diagonal and vertical inversion are applied to the filter coefficients f(k,l) and corresponding filter clipping values ​​c(k,l) based on the gradient values ​​calculated for that block. This is equivalent to applying these transformations to samples within the filter support region. The idea is to make different blocks to which ALF is applied more similar by aligning their orientations.

[0253] We introduce three geometric transformations, including diagonal, vertical inversion, and rotation.

[0254]

number

[0255] Here, K is the size of the filter, 0 ≤ k, l ≤ K-1 are the coefficient coordinates, position (0,0) is in the upper left corner, and position (K-1,K-1) is in the lower right corner. This transformation is applied to the filter coefficients f(k,l) and clipping value c(k,l) based on the gradient values ​​calculated for that block. The relationship between the transformation and the four gradients in the four directions is summarized in the table below.

[0256] [Table 6]

[0257] 2.9.4 Filter Parameter Signal Notification In VTM5, ALF filter parameters are signaled in an Adaptive Parameter Set (APS). A single APS can signal up to 25 sets of luminance filter coefficients and clipping value indices, and up to 1 set of saturation filter coefficients and clipping value indices. Different classifications of filter coefficients can be merged to reduce bit overhead. The APS index used for the current slice is signaled in the slice header.

[0258] The clipping value index decoded from APS allows the clipping value to be determined using a luminance table and a chroma table of the clipping value. These clipping values ​​depend on the internal bit depth. Specifically, the luminance table and chroma table of the clipping value are obtained by the following formulas.

[0259]

number

[0260] Here, B is equal to the internal bit depth, and N is equal to 4, which is the number of allowed clipping values ​​in VTM5.0.

[0261] Filtering may be controlled at the CTB level. One flag is always signaled to indicate whether ALF is applied to the luminance CTB. A single luminance CTB can select one filter set from 16 fixed filter sets and one filter set from multiple APSs. The luminance CTB is signaled with a filter set index indicating which filter set is applied. Both the encoder and decoder pre-define and hardcode the 16 fixed filter sets.

[0262] The filter coefficients are quantized with a norm equal to 128. To reduce the complexity of multiplication, the coefficient values ​​for non-center positions are set to -2. 7 ~2 7 Bitstream conformance is applied so that it falls within the range of -1. The center position coefficient is not signaled in the bitstream and is assumed to be equal to 128.

[0263] 2.9.5 Filtering Process On the decoder side, when ALF is enabled for CTB, each sample R(i,j) in CU is filtered, and as a result, the sample value R'(i,j) is obtained as shown below.

[0264]

number

[0265] In this case, f(k,l) represents the decoded filter coefficients, K(x,y) is the clipping function, and c(k,l) represents the decoded clipping parameters. The variables k and l vary between -L / 2 and L / 2, where L represents the filter length. The clipping function K(x,y) = min(y,max(-y,x)), which corresponds to the function Clip3(-y,y,x).

[0266] 2.9.6 Virtual boundary filtering process for row buffer reduction In VTM5, modified block division and filtering are used for samples near horizontal CTU boundaries to reduce the row buffer requirements of the ALF. To achieve this, virtual boundaries are defined as rows by shifting the horizontal CTU boundary by "N" samples, as shown in Figure 16, where N is equal to 4 for the luminance component and 2 for the chroma component.

[0267] As shown in Figure 2-11, the modified block divisions are applied to the luminance components. Only samples above the virtual boundary are used for the 1D Laplacian gradient calculation of the 4x4 blocks above the virtual boundary. Similarly, only samples below the virtual boundary are used for the 1D Laplacian gradient calculation of the 4x4 blocks below the virtual boundary. Thus, the quantization of the activity value A is scaled up or down by taking into account the reduced number of samples used in the 1D Laplacian gradient calculation.

[0268] For the filtering process, symmetric padding operations at the virtual boundary are used for both the luminance and chroma components. As shown in Figure 17, if the sample to be filtered is located below the virtual boundary, padding is applied to neighboring samples located above the virtual boundary. Conversely, the corresponding samples on the other side are also symmetrically padded.

[0269] 2.10 Sample Adaptive Offset (SAO) The encoder applies a sample-adaptive offset (SAO) to the reconstructed signal after unblocking filtering, using an offset defined for each CTB by the encoder. The HM encoder first determines whether or not to perform SAO processing on the current slice. If SAO is applied to the slice, each CTB is classified into one of five SAO types, as shown in Table 2-6. The concept of SAO is to reduce distortion by classifying pixels into categories and adding an offset to the pixels in each category. The SAO operation includes an edge offset (EO) that uses edge characteristics for pixel classification in SAO types 1-4 and a band offset (BO) that uses pixel intensity for pixel classification in SAO type 5. Each applicable CTB has SAO parameters, including sao_merge_left_flag, sao_merge_up_flag, SAO type, and four offsets. If sao_merge_left_flag is equal to 1, the current CTB reuses the SAO type and CTB offset to the left. If sao_merge_up_flag is equal to 1, the current CTB reuses the SAO type and the offset of the above CTB.

[0270] [Table 7]

[0271] 2.10.1 Operation of each SAO type As shown in Figure 18, the edge offset classifies the current pixel p by considering the edge orientation information using four 1-D3 pixel patterns. From left to right, these are 0°, 90°, 135°, 45°, and so on.

[0272] According to Table 2-7, each CTB is classified into five categories.

[0273] [Table 8]

[0274] Band offset (BO) uses the top 5 bits of the pixel value as a band index to classify all pixels in a single CTB region into 32 uniform bands. In other words, the pixel intensity range is divided into 32 equally divided segments from zero to the maximum intensity value (for example, 255 for an 8-bit pixel). As shown in Figure 19, four adjacent bands are grouped together, and each group is indicated by its leftmost position. The encoder searches all positions by compensating for the offset of each band to obtain the group with the greatest reduction in distortion.

[0275] 2.11 Integrated Inter-Prediction and Intra-Prediction (CIIP) In VTM5, when a CU is encoded in merge mode, an additional flag is signaled to indicate whether the Combined Inter / Intra Prediction (CIIP) combined mode is applied to the current CU if the CU contains at least 64 luminance samples (i.e., CU width × CU height is 64 or greater), and if both the CU width and CU height are less than 128 luminance samples. As the name suggests, CIIP prediction combines the inter-prediction signal and the intra-prediction signal. Using the same inter-prediction processing applied to normal merge mode, CIIP mode P inter The interpretation signal is derived, and after normal intraprediction processing in planar mode, the intraprediction signal P is obtained. intraThe following is derived. Then, the intra-prediction signal and the inter-prediction signal are combined using a weighted average, and in this case, the weight values ​​are calculated as follows, depending on the encoding mode of the upper left neighboring block (shown in Figure 20).

[0276] - Set isIntraTop to 1 if the adjacent element is available and intra-encoded, otherwise set isIntraTop to 0.

[0277] - Set isIntraLeft to 1 if the leftmost column is available and intra-encoded; otherwise, set isIntraLeft to 0.

[0278] - (isIntra Top If +isIntraLeft) is equal to 2, then wt is set to 3.

[0279] - Or, (isIntra Top If (+isIntraLeft) is equal to 1, wt is set to 2.

[0280] - Alternatively, set wt to 1.

[0281] The CIIP prediction is formed as follows:

[0282]

number

[0283] 2.12 Luminance Mapping with Saturation Scaling (LMCS) In VTM5, a new encoding tool called Luma Mapping with Chroma Scaling (LMCS) is added as a new processing block before the loop filter. LMCS has two main components: 1) in-loop mapping of the luminance component based on an adaptive piecewise linear model, and 2) application of luminance-dependent chroma residual scaling for the chroma component. Figure 21 shows the LMCS architecture from the decoder's perspective. Blocks shaded in light blue in Figure 21 indicate where processing is applied in the mapped domain, and these include inverse quantization, inverse transform, luminance intra-prediction, and addition of luminance prediction and luminance residual. Blocks not shaded in Figure 21 indicate where processing is applied in the original (i.e., unmapped) domain, and these include deblocking, loop filters such as ALF and SAO, motion compensation prediction, chroma intra-prediction, addition of chroma prediction with chroma residual, and storing the decoded picture as a reference picture. The light yellow shaded block in Figure 21 is a new LMCS function block that includes forward and backward mapping of luminance signals as well as luminance-dependent saturation scaling. Like most other tools in VVC, LMCS can be enabled / disabled at the sequence level using the SPS flag.

[0284] 3. Examples of problems that the embodiment aims to solve A single palette flag is typically used to indicate whether a palette mode is employed in the current CU, which can have different restrictions and variances in its entropy coding. However, how to better encode the palette flag in previous video coding standards has not yet been sufficiently explored.

[0285] Palette samples may contain visual artifacts when processed by post-loop filtering.

[0286] For non-square blocks, the pallet scanning order can be improved.

[0287] 4. Examples of Embodiments The following detailed inventions should be considered examples illustrating general concepts. These inventions should not be interpreted in a narrow sense. Furthermore, these inventions can be combined in any way. 1. Instructions for using palette mode for transformation units / prediction units / encoded blocks / regions may be encoded separately from the prediction mode. a. In one example, the prediction mode may be encoded before the instruction to use the palette. i. Alternatively, instructions for using the pallet may be conditionally signaled based on the predictive mode. 1. In one example, if the prediction mode is intrablock copy mode (i.e., MODE_IBC), the signal notification for the instruction to use palette mode may be skipped. Alternatively, if the current prediction mode is MODE_IBC, the instruction to use palette may be inferred to be false. 2. In one example, if the prediction mode is intermode (i.e., MODE_INTER), the signal notification for the instruction to use pallet mode may be skipped. Alternatively, if the current prediction mode is MODE_INTER, the instruction to use pallet mode may be inferred to be false. 3. In one example, if the prediction mode is intra mode (i.e., MODE_INTRA), the signal notification for the instruction to use pallet mode may be skipped. Alternatively, if the current prediction mode is MODE_INTRA, the instruction to use pallet mode may be inferred to be false. 4. In one example, if the prediction mode is skip mode (i.e., the skip flag is equal to 1), the signal notification for the instruction to use palette mode may be skipped. Alternatively, if skip mode is employed in the current CU, the instruction to use palette mode may be inferred to be false. 5. In one example, if the prediction mode is intra-mode (e.g., MODE_INTRA), the signal to use pallet mode may be signaled. Alternatively, if the prediction mode is inter-mode or intra-block copy mode, the signal to use pallet mode may be skipped. a) Alternatively, if the prediction mode is intra-mode and not pulse-code modulation (PCM) mode, a signal may be given to indicate the use of pallet mode. b) Alternatively, if the prediction mode is an intra-mode, the instruction to use palette mode may be signaled before the instruction to use PCM mode. In one example, if palette mode is applied, the signal for using PCM mode may be skipped. c) Alternatively, if the prediction mode is intermode or intrablock copy mode, the signal notification for the use of palette mode may be skipped. 6. In one example, if the prediction mode is intermode (e.g., MODE_INTER), the instruction to use palette mode may be signaled. a) Alternatively, if the predictive mode is intra mode, the signal notification instructing the use of pallet mode may be skipped. 7. In one example, if the prediction mode is intrablock copy mode, the instruction to use pallet mode may be signaled. Alternatively, if the prediction mode is intermode or intramode, the signaling instruction to use pallet mode may be skipped. ii. Alternatively, the use of palette mode may be conditionally signaled based on the type of picture / slice / tile group. b. In one example, the prediction mode may be encoded after the instruction to use the palette mode. c. In one example, if the prediction mode is INTRA mode or INTER_MODE, an instruction to use pallet mode may be signaled. i. In one example, the instruction to use palette mode may be encoded after the skip flag, the prediction mode, and the PCM mode flag. ii. In one example, the instruction to use palette mode may be encoded after the skip flag, after the prediction mode, and before the PCM mode flag. iii. In one example, if the current block is encoded in intra mode, the display of palette mode and IBC mode may be further signaled. 1. In one example, one bit flag may be signaled to indicate whether palette or IBC mode is signaled. 2. In one example, the notification of a bit flag may be skipped under certain conditions, such as the block dimensions or whether IBC or palette mode is enabled for a single tile / tile group / slice / picture / sequence. d. In one example, the prediction mode (e.g., whether it is intra-mode or inter-mode) is first encoded, followed by a conditional signal notification of whether it is palette mode. i. In one example, if the prediction mode is intra mode, another flag indicating whether or not it is palette mode may be further signaled. 1. In one example, if palette mode is enabled for a single video data unit (e.g., sequence / picture / tile group / tile), an "other flag" may be signaled. 2. In one example, “another flag” may be signaled under the conditions of block dimensions. 3. Alternatively, if it is not in palette mode, a flag indicating whether or not it is in PCM mode may be further signaled. 4. In one example, “another flag” may be context-encoded according to the information of neighboring blocks. Alternatively, “another flag” may be a context encoded with only one context. Alternatively, “another flag” may be bypass-encoded, i.e., without context. ii. Alternatively, if the prediction mode is intermode, another flag indicating whether or not it is IBC mode may be further signaled. 1. In one example, when IBC mode is enabled for a single video data unit (e.g., sequence / picture / tile group / tile), an "other flag" may be signaled. 2. In one example, “another flag” may be signaled under the conditions of block dimensions. 2. It is proposed to add palette mode as an additional candidate for prediction mode. Therefore, there is no need to signal the use of palette mode separately from the prediction mode. a. In one example, the prediction mode may include intra, intrablock copy, and palette modes for intraslice / Ipicture / intratile group. b. Alternatively, the prediction mode may include an intra-palette mode for intra-slice / I-picture / intra-tile groups. c. In one example, the prediction mode may include intra, intrablock copy, and palette modes for a 4x4 block. d. In one example, the prediction mode may include intra, inter, intra-block copy, and palette modes for inter-slice and / or B-picture / tile groups. e. In one example, the prediction mode may include intra, inter, and intra-block copy modes for inter-slice / P and / or B-picture / tile groups. f. Alternatively, the prediction mode may include at least two of the intra, inter, intrablock copy, and palette modes. g. In one example, the intermode does not have to be included in the 4x4 block prediction mode. h. In one example, if a block is not encoded as skip mode (which is a special case of intermode), the predictive mode index may be signaled. i. In one example, the binarization of the four modes is defined as intra(1), inter(00), IBC(010), and palette(011). ii. In one example, the binarization of the four modes is defined as intra(10), inter(00), IBC(01), and palette(11), as shown in Figure 10. iii. In one example, if the current slice is an intra-slice and IBC is not enabled in SPS, the binarization of palette and intra-mode is defined as palette(1) and intra(0). iv. In one example, if the current slice is not an intra-slice and IBC is not enabled in SPS, the binarization of palette, inter, and intra modes is defined as intra(1), inter(00), and palette(01). v. In one example, if the current slice is an intra-slice and IBC is enabled in SPS, the binarization of palette and intra-mode is defined as IBC(1), palette(01), intra(00). vi. In one example, the binarization of the four modes is defined as inter(1), intra(01), IBC(001), and palette(000). vii. In one example, the binarization of the four modes is defined as intra(1), inter(01), IBC(001), and palette(000). viii. In one example, the binarization of the four modes is defined as inter(0), intra(10), IBC(111), and palette(110), as shown in Figure 11. 3. Signal notification for the use of Palette / IBC mode may depend on information from other modes. a. In one example, if the current prediction mode is intra mode and not IBC mode, an instruction to use pallet mode may be signaled. b. In one example, if the current prediction mode is intra mode and not pallet mode, the instruction to use IBC mode may be signaled. 4. How mode information is signaled may depend on the type of slice / picture / tile group. a. In one example, if it is an I-slice / intratile group, one flag may be signaled indicating whether or not it is in IBC mode. If it is not in IBC mode, another flag may be further signaled indicating whether it is in palette mode or intra mode. b. In one example, if it is an I-slice / intratile group, one flag may be signaled to indicate whether it is in intra-mode or not. If it is not in intra-mode, another flag may be further signaled to indicate whether it is in pallet mode or IBC mode. 5. Instructions for using palette mode may be signaled and / or derived based on the following conditions: a. Current block dimensions i. In one example, the instruction to use palette mode may be signaled only for blocks whose width * height is less than or equal to a threshold (e.g., 64 * 64). ii. In one example, the instruction to use palette mode may be signaled only for blocks where both the width and height are greater than or equal to a threshold (e.g., 64). iii. In one example, the instruction to use palette mode may be signaled only for blocks where all of the following conditions are true: 1. The width and / or height are greater than or equal to a threshold, e.g., 16. 2. The width and / or height are below a threshold (e.g., 32 or 64). iv. In one example, the instruction to use palette mode may be signaled only for blocks whose width is equal to their height (i.e., square blocks). b. Prediction mode for the current block c. Quantization parameters of the current block d. Palette flags of neighboring blocks e. Intrablock copy flag for neighboring blocks f. Display color format (e.g., 4:2:0, 4:4:4) g. Separation / Dual Encoding Tree Structure h. Slice / tile group type and / or picture type 6. Instructions for using IBC mode may be signaled and / or derived based on the following conditions: a. Current block dimensions i. In one example, the instruction to use IBC mode may be signaled only for blocks where both the width and height are less than 128. b. Prediction mode for the current block c. Quantization parameters of the current block d. Palette flags of neighboring blocks e. Intrablock copy flag for neighboring blocks f. Display color format (e.g., 4:2:0, 4:4:4) g. Separation / Dual Encoding Tree Structure h. Slice / tile group type and / or picture type 7. Palette mode may be treated as intra mode (e.g., MODE_INTRA) in the deblocking decision process. a. In one example, if the p-side or q-side sample is encoded in palette mode, the boundary intensity is set to 2. b. In one example, if both the p-side and q-side samples are encoded in palette mode, the boundary intensity is set to 2. c. Alternatively, the palette mode may be treated as an inter-mode (e.g., MODE_INTER) in the deblocking decision process. 8. The palette mode may be treated as a separate mode (e.g., MODE_PLT) in the deblocking decision process. a. In one example, if the p-side and q-side samples are encoded in palette mode, the boundary intensity is set to 0. i. Alternatively, if one side of the sample is encoded in palette mode, the boundary intensity is set to 0. b. In one example, if the p-side sample is encoded in IBC mode and the q-side sample is encoded in palette mode, the boundary intensity is set to 1, and vice versa. c. In one example, if the p-side sample is encoded in intra-mode and the q-side sample is encoded in palette mode, the boundary intensity is set to 2, and vice versa. 9. Palette mode may be treated as a conversion skip block in the deblocking process. a. Alternatively, the palette mode may be treated as a BDPCM block during the deblocking process. 10. Based on slice / tile group / picture level flags, instructions for using palette mode for a single block may be signaled and / or derived. a. In one example, this flag indicates the Motion Vector Difference (MMVD: Merge With Motion Vector Difference, also known as UMVE) and / or Adaptive Motion Vector Resolution (AMVR) modes.

[0288] [ka]

[0289] This indicates whether a slight motion vector difference (MVD) is allowed during the merge. Alternatively, further,

[0290] [ka]

[0291] The signal notification instructing the use of pallet mode was skipped, suggesting that pallet mode is disabled. b. In one example, this flag indicates whether palette mode is enabled for a slice / tile group / picture. Alternatively, if such a flag indicates that palette mode is disabled, the signaling for the use of palette mode for a single block is skipped, and it is inferred that palette mode is disabled. 11. Based on slice / tile group / picture level flags, an instruction to use intra-block copy mode (IBC) for a single block may be signaled and / or derived. a. In one example, this flag indicates the Motion Vector Difference (MMVD, also known as UMVE) and / or Adaptive Motion Vector Resolution (AMVR) modes.

[0292] [ka]

[0293] This indicates whether a small motion vector difference (MVD) is allowed during the merge. Alternatively, further,

[0294] [ka]

[0295] The signal notification instructing the use of IBC mode is skipped, and it is presumed that IBC mode is disabled. b. In one example, this flag indicates whether IBC mode is enabled for a slice / tile group / picture. Alternatively, if such a flag indicates that IBC mode is disabled, the signal notification to use IBC mode for a single block is skipped, and it is inferred that IBC mode is disabled. 12. A sample associated with a single palette entry may have a different bit depth than the original / reconstructed sample's internal bit depth and / or bit depth. a. In one example, if we show that a sample associated with 1 may have a bit depth equal to N, then the following may apply: i. In one example, N may be an integer (for example, 8). ii. In one example, N may be greater than the internal bit depth and / or bit depth of the original / reconstructed sample. iii. In one example, N may be smaller than the internal bit depth and / or bit depth of the original / reconstructed sample. iv. In one example, N may depend on the following: 1. Current block dimensions 2. Quantization parameters of the current block 3. Display of color format (e.g., 4:2:0, 4:4:4) 4. Separation / Dual Encoding Tree Structure 5. Slice / tile group type and / or picture type 6. Number of palette entries 7. Number of prediction palette entries 8. Index of color components b. In one example, samples associated with multiple palette entries may have different bit depths. i. In one example, C0 and C1 are two palette entries in the current palette, which may have bit depths equal to b0 and b1, respectively. b0 does not have to be equal to b1. 1. In one example, b0 may be greater than or less than the internal bit depth and / or bit depth of the original / reconstructed sample, and / or b1 may be greater than or less than the internal bit depth and / or bit depth of the original / reconstructed sample. c. In one example, in palette mode, the samples may be reconstructed according to the shifted values ​​of the samples associated with the palette entries. i. In one example, the sample may be reconstructed by left-shifting the sample in the palette entry by M bits. ii. In one example, the reconstructed value is (C << M) + (1 << (M - 1)), where C is a palette entry. iii. In one example, the sample may be reconstructed by right-shifting the sample in the palette entry by M bits. iv. In one example, the reconstructed value may be clip((C + (1 << (M - 1))) >> M, 0, (1 << N) - 1), where C is a palette entry and N is the bit depth of the reconstruction. v. Alternatively, further, in one example, M may depend on the difference in bit depth between samples associated with the palette entry and the internal bit depth of the reconstructed sample / original sample. 1. In one example, M may be equal to the internal bit depth minus the bit depth of the sample in the palette entry. 2. In one example, M may be equal to the bit depth of the sample in the palette entry minus the internal bit depth. 3. In one example, M may be equal to the bit depth of the original sample minus the bit depth of the sample in the palette entry. 4. In one example, M may be equal to the bit depth of the sample in the palette entry minus the bit depth of the original sample. 5. In one example, M may be equal to the bit depth of the reconstructed sample minus the bit depth of the sample in the palette entry. 6. In one example, M may be equal to the bit depth of the sample in the palette entry minus the bit depth of the reconstructed sample. vi. In one example, M may be an integer (e.g., 2). vii. Alternatively, further, in one example, M may depend on the following. 1. The block size of the current block 2. The quantization parameter of the current block 3. The display of the color format (e.g., 4:2:0, 4:4:4) 4. Separation / Dual Encoding Tree Structure 5. Slice / tile group type and / or picture type 6. Number of palette entries 7. Number of prediction palette entries 8. Sample position in Block / Picture / Slice / Tile 9. Index of color components viii. In one example, a sample-based lookup operation in a palette entry may be used during sample reconstruction. 1. In one example, the values ​​in the lookup table may be signaled in the LCU's Sequence Parameter Set (SPS), Video Parameter Set (VPS), Picture Parameter Set (PPS), Picture Header, Slice Header, Tile Group Header, LCU Row, and LCU Group. 2. In one example, the values ​​in the lookup table may be inferred from the SPS / VPS / PPS / Picture Header / Slice Header / Tile Group Header / LCU Row / LCU Group of the LCU. 13. The signal-notified / derived quantization parameter (QP) of a palette coding block may be modified first, such as by clipping, before being used to derive escape pixels / samples. a. In one example, the QP range applied to the palette coding block may be treated similarly to the conversion skip mode and / or BDPCM mode. b. In one example, the QP applied to the Palette coding block may be modified to max(Qp, 4+T), where T is an integer and Qp is a signal-notified or derived quantization parameter for this block. i. In one example, T may be a predetermined threshold.

[0296] [ka]

[0297] 14. How escape samples / symbols are encoded may be unified, regardless of whether quantum exchange bypass is enabled or not. a. In one example, the escape sample may be signaled with a fixed length. b. In one example, the escape sample may be signaled with a fixed length using N bits. i. In one example, N may be an integer (e.g., 8 or 10) and may depend on the following: 1. Messages signaled in the SPS / VPS / PPS / Picture Header / Slice Header / Tile Group Header / LCU Row / LCU Group of the LCU. 2. Internal bit depth 3. Input bit depth 4. Current block dimensions 5. Quantization parameters of the current block 6. Display of color format (e.g., 4:2:0, 4:4:4) 7. Separation / Dual Encoding Tree Structure 8. Slice / tile group type and / or picture type c. In one example, the code length for signaling one escape pixel / sample may depend on the internal bit depth. i. Alternatively, the code length for signaling a single escape pixel / sample may depend on the input bit depth. d. In one example, the code length for signaling one escape pixel / sample may depend on the quantization parameters. i. In one example, the code length for signaling one escape pixel / sample may be f(Qp). 1. In one example, the function f may be defined as (internal bit depth - (Qp - 4) / 6). 15. Quantization and / or dequantization processes for palette-coded blocks and non-pallet-coded blocks may be defined in different ways. a. In one example, instead of using a quantization process for conversion coefficients or residuals, escape samples may be quantized using a right bit shift. b. In one example, instead of using inverse quantization for the conversion coefficients or residuals, the escape samples may be inversely quantized using a left bit shift. c. The following may be applied on the encoder side: i. In one example, the escape pixel / sample value may be signaled as f(p,Qp), in which case p is the pixel / sample value. ii. In one example, the function f may be defined as p >> (((Qp-4) / 6), where p is the pixel / sample value and Qp is the quantization parameter. iii. In one example, the escape pixel / sample value may be signaled as p>>N, in which case p is the pixel / sample value. 1. In one example, N may be an integer (for example, 2) and may depend on the following: a) Messages that are signaled in the SPS / VPS / PPS / Picture Header / Slice Header / Tile Group Header / LCU Row / LCU Group of the LCU b) Internal bit depth c) Input bit depth d) Current block dimensions e) Quantization parameters of the current block f) Display of color format (e.g., 4:2:0, 4:4:4) g) Separation / dual coding tree structure h) Slice / tile group type and / or picture type d. The following may be applied on the decoder side: i. In one example, the escape pixel / sample value may be signaled as f(bd,p,Qp). 1. In one example, the function f may be defined as clip(0, (1 << (bd - (Qp - 4) / 6)) - 1, (p + (1 << (bd - 1))) >> ((Qp - 4) / 6)). ii. In one example, the escape pixel / sample value may be reconstructed as f(p, Qp), where p is the decoded escape pixel / sample value. 1. In one example, f may be defined as p << ((Qp - 4) / 6). iii. In one example, the escape pixel / sample value may be reconstructed as f(bd, p, Qp), where p is the decoded escape pixel / sample value. 1. In one example, the clip function may be defined as clip(0, (1 << bd) - 1, p << ((Qp - 4) / 6)). iv. In the above example, the clip function clip(a, i, b) is (i <a ? a:(i>b ? b:i)) may be defined as v. In the above example, the clipping function clip(a,i,b) may also be defined as (i<=a ? a:(i>=b ? b:i)). vi. In the above example, p is the pixel / sample value, bd is the internal bit depth or input bit depth, and Qp is the quantization parameter. 16. Palette coding blocks may be treated as a single intra block (e.g., MODE_INTRA) during the process of building the list of Most Probable Modes (MPMs). a. In one example, when fetching the intra-mode of an adjacent (adjacent or non-adjacent) block during the construction of an MPM list, if a neighboring block (e.g., left and / or above) is encoded in palette mode, it may be treated as a conventional intra-encoded block in default mode (e.g., MODE_INTRA). i. In one example, the default mode may be DC / PLANAR / VER / HOR mode. ii. In one example, the default mode may be a single intra-prediction mode. iii. In one example, the default mode may be signaled in the Dependency Parameter Set (DPS) / SPS / VPS / PPS / APS / Picture Header / Slice Header / Tile Group Header / Largest Coding Unit (LCU) / Coding Unit (CU) / LCU Row / LCU Group / TU / PU Block / Video Coding Unit. 17. Palette coding blocks may be treated as non-intra blocks during the process of building the list of Most Probable Modes (MPMs) (for example, they may be treated as blocks with a prediction mode equal to MODE_PLT). a. In one example, when fetching the intra-mode of a neighboring block during the construction of an MPM list, if one neighboring block (e.g., left and / or above) is encoded in palette mode, it may be treated in the same or similar manner as if it were encoded in inter-mode. b. In one example, when fetching the intra-mode of a neighboring block during the construction of an MPM list, if one neighboring block (e.g., left and / or above) is encoded in palette mode, it may be treated in the same or similar manner as if it were encoded in IBC mode. 18. A saturation block encoded in palette mode that corresponds to a saturation block encoded in DM mode may be interpreted as having a default intra-prediction mode. a. In one example, a corresponding saturation block encoded in palette mode may be treated as an intra block (e.g., MODE_INTRA) or a palette block (e.g., MODE_PLT) if the saturation block is encoded in DM mode. b. In one example, the default prediction mode may be DC / PLANAR / VER / HOR mode. c. In one example, the default prediction mode may be a single intra-prediction mode. d. In one example, the default prediction mode may be signaled in the DPS / SPS / VPS / PPS / APS / Picture Header / Slice Header / Tile Group Header / Maximum Encoded Unit (LCU) / Encoded Unit (CU) / LCU Row / LCU Group / TU / PU Block / Video Encoded Unit. 19. Palette coding blocks may be treated as unavailable blocks during list construction for History-Based Motion Vector Prediction (HMVP), Merge, and / or Advanced Motion Vector Prediction (AMVP) modes. a. In one example, an unavailable block is a block that does not have motion information, or a block whose motion information cannot be used to predict other blocks. b. In one example, a block encoded in palette mode may be treated as an intra block (e.g., MODE_INTRA) or a palette block (e.g., MODE_PLT) during the list construction process in HMVP, MERGE, and / or AMVP modes. i. Alternatively, in one example, when fetching motion information of neighboring blocks during the construction of an HMVP, MERGE, and / or AMVP list, neighboring blocks encoded in palette mode may be treated as blocks with invalid reference indices. ii. Alternatively, in one example, when fetching motion information of neighboring blocks during the construction of an HMVP, MERGE, and / or AMVP list, neighboring blocks encoded in palette mode may be treated as interblocks with a reference index of 0. iii. Alternatively, in one example, when fetching motion information of neighboring blocks during list construction in HMVP, MERGE, and / or AMVP modes, neighboring blocks encoded in palette mode may be treated as interblocks with zero motion vectors. 20. How to handle blocks encoded in palette mode (for example, whether or not to apply the methods described above, and / or how to apply them) may be based on the following: a. Video content (e.g., screen content or nature content) b. In one example, the default mode may be signaled in DPS / SPS / VPS / PPS / APS / Picture Header / Slice Header / Tile Group Header / Maximum Encoded Unit (LCU) / Encoded Unit (CU) / LCU Row / LCU Group / TU / PU Block / Video Encoded Unit. c. Location of CU / PU / TU / block / video encoding unit d. Block dimensions of the current block and / or nearby blocks e. Block shape of the current block and / or nearby blocks f. Display of color format (e.g., 4:2:0, 4:4:4, RGB, YUV, etc.) g. Encoding tree structure (e.g., dual tree or single tree) h. Slice / tile group type and / or picture type i. Color components (for example, this may apply only to the luminance component and / or saturation component). j. Temporary Layer ID k. Standard profile / level / layer 21. Context coding bins for palette coding blocks may be restricted to fall within a certain range. a. In one example, the counter is allocated to a single block to record the number of context-encoded bins. If the counter exceeds a threshold, bypass encoding is applied instead of context encoding. i. Alternatively, a NumColorComp counter may be assigned to record the number of context-encoded bins for each color component. NumColorComp is the number of color components encoded in one block (for example, for one CU in YUV format, NumColorComp would be set to 3). ii. Alternatively, the counter may be initialized to zero, and then incremented by 1 after encoding one bin in the context. b. Alternatively, the counter may be initialized with some value greater than zero (e.g., W*H*K), and after encoding one bin in the context, the counter is decremented by 1. If the counter is less than or equal to T, bypass encoding is applied instead of using context encoding. i. In one example, T is set to 0 or 1. ii. In one example, T is set according to the number of decoded information or coding paths, etc. c. In one example, a palette-encoded block may have the same or different thresholds as a TS-encoded block or a non-TS-encoded block in terms of context-encoded bins. i. In one example, the number of context coding bins for a palette coding block may be set to (W*H*T), where W and H are the width and height of one block, respectively, and T is an integer. In one example, T is set to be the same as that used for a TS coding block, for example, 1.75 or 2. ii. In one example, the number of context coding bins in a palette coding block may be set to (W*H*NumColorComp*T), where W and H are the width and height of one block, respectively, NumColorComp is the number of color components coded in one block (e.g., one CU in the YUV format, where NumColorComp is set to 3), and T is an integer. In one example, T is set to be the same as that used for a TS coding block, e.g., 1.75 or 2. d. In one example, the threshold for a palette-encoded block may be smaller than that of a TS-encoded block or a non-TS-encoded block in terms of context-encoded bins. e. In one example, the threshold for a palette-encoded block may be greater than that of a TS-encoded block or a non-TS-encoded block in terms of context-encoded bins. 22. Palette coded blocks are treated as non-intra blocks in the process of counting neighboring intra blocks in CIIP mode (for example, they are treated as blocks with a prediction mode equal to MODE_PLT). a. In one example, when counting neighboring intrablocks in CIIP mode and fetching the intramode of a neighboring block, if one neighboring block (e.g., left and / or above) is encoded in palette mode, it may be treated in the same or similar manner as if it were encoded in intermode. b. In one example, when counting neighboring intrablocks in CIIP mode and fetching the intramode of a neighboring block, if one neighboring block (e.g., left and / or above) is encoded in palette mode, it may be treated in the same or similar manner as if it were encoded in IBC mode. c. Alternatively, a palette-encoded block may be treated as an intrablock in the process of statistically analyzing neighboring intrablocks in CIIP mode. 23. It is proposed to skip the pre- and / or post-filtering process for palette-encoded samples. a. In one example, palette-encoded samples do not need to be deblocked. b. In one example, the palette-encoded samples do not need to be offset-compensated in the SAO process. c. In one example, palette-encoded samples do not need to be filtered during ALF processing. i. In one example, classification in ALF processing may skip palette-encoded samples. d. In one example, LMCS may be disabled for palette-encoded samples. 24. In palette mode, it is proposed to add more scan sequences. a. In one example, an inverse horizontal traverse scan sequence defined as follows may be used: i. In one example, the scanning direction for odd-numbered rows may be from left to right. ii. In one example, the scanning direction for even-numbered rows may be from right to left. iii. In one example, the scanning order of the 4x4 blocks may be as shown in Figure 22. b. In one example, an inverse vertical traverse scan sequence defined as follows may be used: i. In one example, the scanning direction for odd-numbered rows may be from top to bottom. ii. In one example, the scanning direction for even-numbered rows may be from bottom to top. iii. In one example, the scanning order of the 4x4 blocks may be as shown in Figure 23. 25. The acceptable combinations of scan order may depend on the shape of the block. a. In one example, if the ratio between the width and height of a block is greater than a threshold, only the horizontal traverse scan order and the inverse horizontal traverse scan order may be applied. i. In one example, the threshold is equal to 1. ii. In one example, the threshold is equal to 4. b. In one example, if the ratio between the height and width of a block is greater than a threshold, only vertical traverse and inverse vertical traverse scan sequences may be applied. i. In one example, the threshold is equal to 1. ii. In one example, the threshold is equal to 4. 26. In QR-BDPCM processing, it is proposed to allow only one intra-prediction direction and / or one scanning direction. a. In one example, for a block with a width greater than its height, only the vertical direction is permitted. b. In one example, for blocks where the width is less than the height, only the horizontal direction is permitted. c. In one example, the orientation indication of the QR-BDPCM may be inferred for non-square blocks. i. In one example, the orientation of the QR-BDPCM may also be inferred vertically if the width is greater than the height of the block. ii. In one example, the orientation of the QR-BDPCM may also be inferred horizontally if the width is less than the height of the block. 27. The methods in Bullets 24, 25, and 26 may be applied only to blocks having w*Th≧h or h*Th≧w, where w and h are the width and height of the block, respectively, and Th is the threshold. a. In one example, Th is an integer (e.g., 4 or 8) and may be based on the following: i. Visual content (e.g., screen content or natural content) ii. In one example, the default mode may be signaled in DPS / SPS / VPS / PPS / APS / Picture Header / Slice Header / Tile Group Header / Large Encoding Unit (LCU) / Encoding Unit (CU) / LCU Row / LCU Group / TU / PU Block / Video Encoding Unit. iii. Location of CU / PU / TU / block / video encoding unit iv. Block dimensions of the current block and / or nearby blocks v. Block shape of the current block and / or nearby blocks vi. Display color format (e.g., 4:2:0, 4:4:4, RGB, YUV, etc.) vii. Encoding tree structure (e.g., dual tree or single tree) viii. Slice / tile group type and / or picture type ix. Color components (for example, this may apply only to the luminance component and / or saturation component). x.Temporary Layer ID xi. Standard profile / level / layer

[0298] 5. Additional Embodiments In the following examples, newly added text is shown in bold italics, and deleted text is marked with "[]".

[0299] 5.1 Embodiment #1 This chapter presents exemplary embodiments in which the bitstream representation of video may be modified compared to the baseline bitstream syntax.

[0300] [Table 9]

[0301] [ka]

[0302] Table 10

[0303]

change

[0304] Table 11

[0305] Table 12

[0306]

change

[0307] Table 13

[0308]

change

[0309] Table 14

[0310] Table 15

[0311] Table 16

[0312] Table 17

[0313] [Table 18]

[0314] [ka]

[0315] [Table 19]

[0316] [Table 20]

[0317] 5.2 Embodiment #2 In this embodiment, modeType will be described.

[0318] For coding units within a coding tree node, [ka] Specify the variable modeType, which defines whether or not a particular encoding mode is available (MODE_TYPE_INTRA) and whether or not each of the inter encoding modes is available (MODE_TYPE_INTER).

[0319] 5.3 Embodiment #3 In this embodiment, the syntax of the coding unit is described. In this embodiment, pred_mode_plt_flag is signaled after pred_mode_ibc_flag.

[0320] [Table 21] [Table 22]

[0321] 5.4 Embodiment #4 In this embodiment, the syntax of the coding unit is described. In this embodiment, pred_mode_plt_flag is signaled after pred_mode_ibc_flag, and pred_mode_plt_flag is signaled only when the current prediction mode is MODE_INTRA.

[0322] [Table 23] [Table 24]

[0323] 5.5 Embodiment #5 In this embodiment, the syntax of the coding unit is described. In this embodiment, pred_mode_ibc_flag is signaled after pred_mode_plt_flag.

[0324] [Table 25] [Table 26]

[0325] 5.6 Embodiment #6 In this embodiment, the syntax of the coding unit is described. In this embodiment, pred_mode_ibc_flag is signaled after pred_mode_plt_flag, and pred_mode_plt_flag is signaled only when the current prediction mode is MODE_INTRA.

[0326] [Table 27] [Table 28]

[0327] 5.7 Embodiment #7 In this embodiment, the syntax of the coding unit will be described. In this embodiment, when the prediction mode is MODE_INTRA, pred_mode_plt_flag and pred_mode_ibc_flag are signaled.

[0328] [Table 29] [Table 30]

[0329] 5.8 Embodiment #8 In this embodiment, the syntax of the coding unit is described. In this embodiment, pred_mode_plt_flag and pred_mode_ibc_flag are signaled when the prediction mode is not MODE_INTRA.

[0330] [Table 31] [Table 32]

[0331] 5.9 Embodiment #9 In this embodiment, the syntax of the coding unit will be described. In this embodiment, when the prediction mode is MODE_INTER, pred_mode_plt_flag and pred_mode_ibc_flag are signaled.

[0332] [Table 33] [Table 34]

[0333] 5.10 Embodiment #10 This embodiment explains the meaning of pred_mode_plt_flag. [ka]

[0334] 5.11 Embodiment #11 This embodiment explains the meaning of pred_mode_plt_flag. [ka]

[0335] 5.12 Embodiment #12 In this embodiment, the derivation of boundary strength will be explained. 8.8.3.5 Derivation process for boundary filtering strength The input for this process is as follows: One picture sample sequence recPicture, The position (xCb, yCb) that defines the top-left sample of the current encoding block relative to the top-left sample of the current picture. The variable nCbW defines the width of the current coded block. The variable nCbH defines the height of the current coding block. The `edgeType` variable specifies whether to filter vertical (EDGE_VER) edges or horizontal (EDGE_HOR) edges. The variable cIdx defines the color components of the current coding block. 2D (nCbW)×(nCbH) array edgeFlags. The output of this process is a two-dimensional (nCbW) × (nCbH) array bS that defines the boundary filtering strength. ... Variable bS[xD i ][yD j ] is derived as follows: If cIdx is equal to 0 and both samples p0 and q0 are included in an encoded block where intra_bdpcm_flag is equal to 1, then bS[xD i ][yD j ] is set to equal to 0. Alternatively, if sample p0 or q0 is in the coding block of the coding unit coded in intra-predictive mode, then bS[xD i ][yD j ] is set to equal to 2. Alternatively, if a block edge is also a transform block edge, and sample p0 or q0 is in an encoded block where ciip_flag is equal to 1, then bS[xD i ][yD j ] is set to equal to 2. Alternatively, if the block edge is also a transformation block edge and sample p0 or q0 is in a transformation intrablock containing one or more non-zero transformation coefficient levels, then bS[xD i ][yD j ] is set to equal to 1.

[0336] [ka]

[0337] Alternatively, if the prediction mode of the coded subblock containing sample p0 is different from the prediction mode of the coded subblock containing sample q0, then bS[xD i ][yD j ] is set to equal to 1. Alternatively, if cIdx is 0 and one or more of the following conditions are true, then bS[xD i ][yD j ] is set to equal to 1. Both the coded subblock containing sample p0 and the coded subblock containing sample q0 are coded in IBC prediction mode, and the absolute difference of the horizontal or vertical components of the motion vectors used to predict the two coded subblocks is 4 or greater per quarter luminance sample. For the prediction of the coded subblock containing sample p0, a different reference picture or a different number of motion vectors is used compared to the prediction of the coded subblock containing sample q0. Note 1 - Whether the reference pictures used for the two coding subblocks are the same or different is determined solely by which pictures are referenced, regardless of whether the prediction is formed using an index to reference picture list 0 or an index to reference picture list 1, and regardless of whether the index positions within the reference picture lists are different. Note 2 - The number of motion vectors used to predict the coded subblock containing the top-left sample (xSb, ySb) is equal to PredFlagL0[xSb][ySb] + PredFlagL1[xSb][ySb]. One motion vector is used to predict the coded subblock containing sample p0, and another motion vector is used to predict the coded subblock containing sample q0, and the absolute difference of the horizontal or vertical components of the motion vectors used is 4 or greater per quarter luminance sample unit. Using two motion vectors and two different reference pictures, predict the coded subblock containing sample p0, and using two motion vectors from the same two reference pictures, predict the coded subblock containing sample q0, and the absolute difference of the horizontal or vertical components of the two motion vectors used to predict the two coded subblocks from the same reference picture is 4 or greater per quarter luminance sample unit. Using two motion vectors from the same reference picture, predict the coded subblock containing sample p0, and using two motion vectors from the same reference picture, predict the coded subblock containing sample q0, provided that both of the following conditions are met. The absolute difference between the horizontal or vertical components of the motion vectors in List 0 used to predict the two coded subblocks is 4 or greater in a quarter luminance sample, or the absolute difference between the horizontal or vertical components of the motion vectors in List 1 used to predict the two coded subblocks is 4 or greater in a quarter luminance sample unit. The absolute difference between the horizontal or vertical component of the List 0 motion vector used to predict the coded subblock containing sample p0 and the List 1 motion vector used to predict the coded subblock containing sample q0 is 4 or greater per quarter luminance sample unit, or the absolute difference between the horizontal or vertical component of the List 1 motion vector used to predict the coded subblock containing sample p0 and the List 0 motion vector used to predict the coded subblock containing sample q0 is 4 or greater per quarter luminance sample unit. Alternatively, the variable bS[xD i ][yD j Set ] to 0.

[0338] 5.13a Embodiment #13a In this embodiment, the derivation of boundary strength will be explained. 8.8.3.5 Derivation process for boundary filtering strength The input for this process is as follows: One picture sample sequence recPicture, The position (xCb, yCb) that defines the top-left sample of the current encoding block relative to the top-left sample of the current picture. The variable nCbW defines the width of the current coded block. The variable nCbH defines the height of the current coding block. The `edgeType` variable specifies whether to filter vertical (EDGE_VER) edges or horizontal (EDGE_HOR) edges. The variable cIdx defines the color components of the current coding block. 2D (nCbW)×(nCbH) array edgeFlags. The output of this process is a two-dimensional (nCbW) × (nCbH) array bS that defines the boundary filtering strength. ... Variable bS[xD i ][yD j ] is derived as follows: If cIdx is equal to 0 and both samples p0 and q0 are included in a coded block where intra_bdpcm_flag is equal to 1, then bS[xD i [yD j is set equal to 0. Alternatively, if sample p0 or q0 is in the coded block of a coded unit coded in an intra prediction mode, then bS[xD i [yD j is set equal to 2. Alternatively, if the block edge is also a transform block edge and sample p0 or q0 is in a coded block where ciip_flag is equal to 1, then bS[xD i [yD j is set equal to 2. Alternatively, if the block edge is also a transform block edge and sample p0 or q0 is in a transform intra block that contains one or more non-zero transform coefficient levels, then bS[xD i [yD j is set equal to 1.

[0339]

Chemical transformation

[0340] Alternatively, if the prediction mode of the coded sub-block containing sample p0 is different from the prediction mode of the coded sub-block containing sample q0, then bS[xD i [yD j is set equal to 1. Alternatively, if cIdx is 0 and one or more of the following conditions are true, then bS[xD i [yD j is set equal to 1. The coded sub-block containing sample p0 and the coded sub-block containing sample q0 are both coded in the IBC prediction mode, and the absolute difference between the horizontal or vertical components of the motion vectors used for prediction of the two coded sub-blocks is 4 or more in units of 1 / 4 luminance samples. For the prediction of the coded subblock containing sample p0, a different reference picture or a different number of motion vectors is used compared to the prediction of the coded subblock containing sample q0. Note 1 - Whether the reference pictures used for the two coding subblocks are the same or different is determined solely by which pictures are referenced, regardless of whether the prediction is formed using an index to reference picture list 0 or an index to reference picture list 1, and regardless of whether the index positions within the reference picture lists are different. Note 2 - The number of motion vectors used to predict the coded subblock containing the top-left sample (xSb, ySb) is equal to PredFlagL0[xSb][ySb] + PredFlagL1[xSb][ySb]. One motion vector is used to predict the coded subblock containing sample p0, and another motion vector is used to predict the coded subblock containing sample q0, and the absolute difference of the horizontal or vertical components of the motion vectors used is 4 or greater per quarter luminance sample unit. Using two motion vectors and two different reference pictures, predict the coded subblock containing sample p0, and using two motion vectors from the same two reference pictures, predict the coded subblock containing sample q0, and the absolute difference of the horizontal or vertical components of the two motion vectors used to predict the two coded subblocks from the same reference picture is 4 or greater per quarter luminance sample unit. Using two motion vectors from the same reference picture, predict the coded subblock containing sample p0, and using two motion vectors from the same reference picture, predict the coded subblock containing sample q0, provided that both of the following conditions are met. The absolute difference between the horizontal or vertical components of the motion vectors in List 0 used to predict the two coded subblocks is 4 or greater in a quarter luminance sample, or the absolute difference between the horizontal or vertical components of the motion vectors in List 1 used to predict the two coded subblocks is 4 or greater in a quarter luminance sample unit. The absolute difference between the horizontal or vertical component of the List 0 motion vector used to predict the coded subblock containing sample p0 and the List 1 motion vector used to predict the coded subblock containing sample q0 is 4 or greater per quarter luminance sample unit, or the absolute difference between the horizontal or vertical component of the List 1 motion vector used to predict the coded subblock containing sample p0 and the List 0 motion vector used to predict the coded subblock containing sample q0 is 4 or greater per quarter luminance sample unit. Alternatively, the variable bS[xD i ][yD j Set ] to 0.

[0341] 5.13b Embodiment #13b In this embodiment, the encoding and reconstruction of escape samples will be described.

[0342] [Table 35]

[0343] [ka] [ka]

[0344] [[The list levelScale[] is defined as levelScale[k]={40,45,51,57,64,72} when k=0..5. ]] The following applies: [[tmpVal=(PaletteEscapeVal[cIdx][xCb+xL][yCb+yL]* levelScale[qP%6])<<(qP / 6)+32)>>6 (8-77) recSamples[x][y]=Clip3(0,(1< <bitDepth)-1,tmpVal) (8-78)]]

[0345] [ka]

[0346] 5.14 Embodiment #14 8.4.5.3 Decoding process in palette mode The input for this process is as follows: - The position (xCb, yCb) that defines the top-left sample of the current block relative to the top-left luminance sample of the current picture. -The variable startComp defines the first color component of the palette table. -The variable cIdx defines the color components of the current block. - Two variables nCbW and nCbH that define the width and height of the current block, respectively. The output of this process is the array recSamples[x][y], where x=0..nCbW-1 and y=0..nCbH-1 define the reconstructed sample values ​​for the block. Based on the value of cIdx, the variables nSubWidth and nSubHeight are derived as follows: -If cIdx is equal to 0, nSubWidth is set to 1 and nSubHeight is set to 1. Alternatively, set nSubWidth to SubWidthC and nSubHeight to SubHeightC. The (nCbW × nCbH) block of the reconstructed sample sequence recSamples at position (xCb, yCb) is represented by recSamples[x][y] where x = 0...nCTbW-1 and y = 0...nCbH-1, and the values ​​of recSamples[x][y] for each x in the range 0 to nCbW-1 and each y in the range 0 to nCbH-1 are derived as follows. -The variables xL and yL are derived as follows: xL=palette_transpose_flag ? x * nSubHeight:x * nSubWidth (8-234) yL=palette_transpose_flag ? y * nSubWidth:y * nSubHeight (8-235) The variable bIsEscapeSample is derived as follows: -If PaletteIndexMap[xCb+xL][yCb+yL] is equal to MaxPaletteIndex and palette_escape_val_present_flag is equal to 1, then bIsEscapeSample is set to 1. Alternatively, set bIsEscapeSample to 0. If -bIsEscapeSample is equal to 0, the following applies: recSamples[x][y]=CurrentPaletteEntries[cIdx][PaletteIndexMap[xCb+xL][yCb+yL]] (8-236) -Alternatively, if cu_transquant_bypass_flag is equal to 1, the following applies: recSamples[x][y]=PaletteEscapeVal[cIdx][xCb+xL][yCb+yL] (8-237) -Alternatively, (if bIsEscapeSample is equal to 1 and cu_transquant_bypass_flag is equal to 0), the following ordered steps apply: 1. The quantization parameter derivation process specified in Section 8.7.1 is invoked using the position (xCb, yCb) that defines the top-left sample of the current block relative to the top-left sample of the current picture. [Ed.(BB): Since QP is already derived at the beginning of the intraCU decoding process, there is no need to derive it again within this dependent term. This seems to be present in HEVC v4 SCC, but such redundancy can be eliminated. Please verify.]

[0347] [ka]

[0348] QpPrimeTsMin=4+min_qp_prime_ts_minus4 3. The variable bitDepth is derived as follows: bitDepth=(cIdx==0)?BitDepth Y :BitDepth C (8-241) 4. The list levelScale[] is defined as levelScale[k]={40,45,51,57,64,72} when k=0..5. [Ed.(BB): For non-palletized CUs, levelScale depends on rectNonTsFlag. Does this also apply here?] 5. The following applies: tmpVal=(PaletteEscapeVal[cIdx][xCb+xL][yCb+yL]* levelScale[qP%6])<<(qP / 6)+32)>>6 (8-242) recSamples[x][y]=Clip3(0,(1< <bitDepth)-1,tmpVal) (8-243) When one of the following conditions is true, -cIdx is equal to 0 and numComps is equal to 1 -cIdx is equal to 3 The variable PredictorPaletteSize[startComp] and the array PredictorPaletteEntries are derived or modified as follows: for(i=0;i <CurrentPaletteSize[startComp];i++) for(cIdx=startComp;cIdx<(startComp+numComps);cIdx++) newPredictorPaletteEntries[cIdx][i]=CurrentPaletteEntries[cIdx][i] newPredictorPaletteSize=CurrentPaletteSize[startComp] for(i=0;i <PredictorPaletteSize && newPredictorPaletteSize<PaletteMaxPredictorSize;i++) if(!PalettePredictorEntryReuseFlags[i]){ for(cIdx=startComp;cIdx<(startComp+numComps);cIdx++) (8-244) newPredictorPaletteEntries[cIdx][newPredictorPaletteSize]= PredictorPaletteEntries[cIdx][i] newPredictorPaletteSize++ } for(cIdx=startComp;cIdx<(startComp+numComps);cIdx++) for(i=0;i <newPredictorPaletteSize;i++) PredictorPaletteEntries[cIdx][i]=newPredictorPaletteEntries[cIdx][i] PredictorPaletteSize[startComp]=newPredictorPaletteSize A requirement for bitstream compatibility is that the value of PredictorPaletteSize[startComp] falls within the range of 0 to PaletteMaxPredictorSize.

[0349] 5.15 Embodiment #15 8.4.2 Derivation process for luminance intra-prediction mode ... - Or, (when both skip_intra_flag[xPb][yPb] and DimFlag[xPb][yPb] are equal to 0), IntraPredModeY[xPb][yPb] is derived by the following order steps. 1. The neighboring positions (xNbA, yNbA) and (xNbB, yNbB) are respectively set equal to (xPb - 1, yPb) and (xPb, yPb - 1). 2. When replacing X with either A or B, the variable candIntraPredModeX is derived as follows. ● The availability derivation process of the block in the z-scan order defined in section 6.4.1 is called with the position (xCurr, yCurr) set equal to (xPb, yPb) and the neighboring position (xNbY, yNbY) set equal to (xNbX, yNbX) as inputs, and its output is assigned to availableX. ● The candidate intra prediction mode candIntraPredModeX is derived as follows. ● When availableX is equal to FALSE, candIntraPredModeX is set equal to INTRA_DC. ● - Or, [[- or CuPredMode[xNbX][yNbX] is MODE_INTRA or pcm_flag[xNbX][yNbX] is equal to 1, or candIntraPredModeX is set equal to INTRA_DC]]

Chemical formula

[0350] 5.16 Embodiment #16 8.4.2 Derivation Process for Luminance Intra Prediction Mode The input to this process is as follows. - The luminance position (xCb, yCb) that defines the top-left sample of the current luminance coding block with respect to the top-left luminance sample of the current picture - The variable cbWidth that defines the width of the current coding block in the luminance samples, - The variable cbHeight that defines the height of the current coding block in the luminance samples. In this process, the luminance intra prediction mode IntraPredModeY[xCb][yCb] is derived. 1. When replacing X with either A or B, the variable candIntraPredModeX is derived as follows.

[0351]

Chemical formula

[0352]

Chemical formula

[0353] 5.17 Embodiment #17 8.4.3 Derivation process for luminance intra-prediction mode The input for this process is as follows: - The luminance position (xCb, yCb) that defines the top-left sample of the current luminance encoding block relative to the top-left luminance sample of the current picture. - The variable cbWidth defines the width of the current coded block in the luminance sample. - A variable cbHeight that defines the height of the current encoded block in the luminance sample. In this process, the luminance intra-prediction mode IntraPredModeY[xCb][yCb] is derived. 2. If X is replaced with either A or B, the variable candIntraPredModeX is derived as follows: The block availability derivation process defined in section -6.4.X [Ed.(BB): Neighboring block availability derivation process tbd] is called with inputs (xCb, yCb) set to a position (xCurr, yCurr) and a neighboring position (xNbX, yNbX) set to a neighboring position (xNbY, yNbY), and assigns the output to availableX. - The candidate intra-prediction mode candIntraPredModeX is derived as follows: -If one or more of the following conditions are true, then candIntraPredModeX [ka] - The variable availableX is equal to FALSE. - CuPredMode[xNbX][yNbX] is not equal to MODE_INTRA. - intra_mip_flag[xNbX][yNbX] is equal to 1. - X is equal to B and yCb - 1 is less than ((yCb >> CtbLog2SizeY) << CtbLog2SizeY). - Alternatively, set candIntraPredModeX equal to IntraPredModeY[xNbX][yNbX]. ··· When x = xCb..xCb + cbWidth - 1 and y = yCb..yCb + cbHeight - 1, the variable IntraPredModeY[x][y] is set equal to IntraPredModeY[xCb][yCb].

[0354] 5.18 Embodiment #18 8.4.3 Derivation Process for Luminance Intra Prediction Mode The input to this process is as follows: - The luminance position (xCb, yCb) that defines the top - left sample of the current luminance coding block with respect to the top - left luminance sample of the current picture - The variable cbWidth that defines the width of the current coding block in the luminance samples, - The variable cbHeight that defines the height of the current coding block in the luminance samples. In this process, the luminance intra prediction mode IntraPredModeY[xCb][yCb] is derived. 3.When replacing X with either A or B, the variable candIntraPredModeX is derived as follows:

[0355]

Chemical formula

Chemical formula

Chemical formula

[0359] 5.20 Embodiment #20

[0360] [Table 38] [Table 39]

[0361] 5.21 Embodiment #21

[0362] [Table 40] [Table 41]

[0363] 5.22 Embodiment #22 In this embodiment, the syntax of the coding unit is described. In this embodiment, pred_mode_plt_flag is signaled after pred_mode_ibc_flag.

[0364] [Table 42] [Table 43]

[0365] 5.23 Embodiment #23

[0366] [Table 44] [Table 45]

[0367] 5.24 Embodiment #24 In this embodiment, the syntax of the coding unit is described. In this embodiment, pred_mode_plt_flag is signaled after pred_mode_ibc_flag.

[0368] [Table 46] [Table 47]

[0369] 5.25 Embodiment #25 In this embodiment, the syntax of the coding unit will be described. In this embodiment, when the current prediction mode is MODE_PLT, the palette syntax is signaled.

[0370] [Table 48] [Table 49] [Table 50]

[0371] 5.26 Embodiment #26 In this embodiment, the process for deriving the saturation intra-prediction mode will be described. [ka] The input for this process is as follows: - The luminance position (xCb, yCb) that defines the top-left sample of the current saturation encoding block relative to the top-left luminance sample of the current picture. - The variable cbWidth defines the width of the current coded block in the luminance sample. - A variable cbHeight that defines the height of the current encoded block in the luminance sample. In this process, the saturation intra-prediction mode IntraPredModeC[xCb][yCb] is derived. The corresponding intra-prediction mode, lumaIntraPredMode, is derived as follows: -If intra_mip_flag[xCb][yCb] is equal to 1, lumaIntraPredMode is set to be equal to INTRA_PLANAR. - Alternatively, if CuPredMode[0][xCb][yCb] is MODE_IBC [ka] lumaIntraPredMode is set to be equal to INTRA_DC. Alternatively, lumaIntraPredMode is set to be equal to IntraPredModeY[xCb+cbWidth / 2][yCb+cbHeight / 2]. ...

[0372] 5.27 Embodiment #27 This embodiment describes a picture reconstruction process that performs a mapping process for luminance samples. Picture reconstruction with mapping processing for luminance samples. The inputs to this process are as follows: - The position of the top-left sample of the current block relative to the top-left sample of the current picture (xCurr, yCurr) - The variable nCurrSw defines the block width. - The variable nCurrSh defines the height of the block. - PredSamples, an (nCurrSw) × (nCurrSh) array that defines the brightness prediction samples for the current block. - ResSamples, an (nCurrSw) × (nCurrSh) array that defines the luminance residual samples of the current block. The output of this process is the reconstructed luminance picture sample array, recSamples. The (nCurrSw) × (nCurrSh) array of the mapped predicted brightness samples, predMapSamples, is derived as follows: -If any of the following conditions are true, predMapSamples[i][j] will be set to equal to predSamples[i][j] when i=0..nCurrSw-1 and j=0..nCurrSh-1. -CuPredMode[0][xCurr][yCurr] is equal to MODE_INTRA. -CuPredMode[0][xCurr][yCurr] is equal to MODE_IBC. [ka] -CuPredMode[0][xCurr][yCurr] is equal to MODE_INTER, and ciip_flag[xCurr][yCurr] is equal to 1. -Alternatively, (if CuPredMode[0][xCurr][yCurr] is equal to MODE_INTER and ciip_flag[xCurr][yCurr] is equal to 0), the following applies: ...

[0373] 5.28 Embodiment #28 In this embodiment, the scanning sequence corresponding to bullet point 24 in Chapter 4 will be described. This process takes the block width blkWidth and block height blkHeight as input. The output of this process is the arrays hReverScan[sPos][sComp] and vReverScan[sPos][sComp]. The array hReverScan represents the horizontal reverse scan order, and the array vReverScan represents the vertical scan order. The array index sPos defines the scan position in the range from 0 to (blkWidth*blkHeight)-1. If the array index sComp is 0, the horizontal component is defined, and if the array index sComp is 1, the vertical component is defined. Based on the values ​​of blkWidth and blkHeight, the arrays hTravScan fd and vTravScan are derived as follows. i=0 for(y=0;y <blkHeight;y++) { if(y%2 !=0){ for(x=0;x <blkWidth;x++){ hReverScan[i][0]=x hReverScan[i][1]=y i++ } } else { for(x=blkWidth-1;x>=0;x--){ hReverScan[i][0]=x hReverScan[i][1]=y i++ } } } i=0 for(x=0;x <blkWidth;x++) { if(x%2 !=0) { for(y=0;y <blkHeight;y++){ vReverScan[i][0]=x vReverScan[i][1]=y i++ } } else { for(y=blkHeight-1;y>=0;y--){ vReverScan[i][0]=x vReverScan[i][1]=y i++ } } }

[0374] Figure 6 is a block diagram of the video processing device 600. The device 600 may be used to implement one or more of the methods described herein. The device 600 may be implemented by a smartphone, tablet, computer, IoT (Internet of Things) receiver, etc. The device 600 may include one or more processing units 602, one or more memories 604, and video processing hardware 606. The processing unit(s) 602 may be configured to implement one or more of the methods described herein. The memories(s) 604 may be used to store data and code used to implement the methods and techniques described herein. The video processing hardware 606 may be used to implement the techniques described herein in hardware circuitry. In some embodiments, the hardware 606 may be at least partially incorporated into the processing unit 602 (e.g., a graphics coprocessor).

[0375] Figure 8 is a flowchart of method 800 for processing video. Method 800 includes determining (805) to process a transformation unit, encoded block, or region of a palette mode encoded separately from the prediction mode using the palette mode, and performing further processing on this transformation unit, encoded block, or region using the palette mode (810).

[0376] Refer to Method 800, and several examples of palette-mode coding and their use are described in Chapter 4 of this specification.

[0377] Referring to Method 800, the video blocks may be encoded in a video bitstream in which bit efficiency can be achieved by using bitstream generation rules relating to palette mode encoding.

[0378] This method may include encoding the prediction mode before displaying the use of palette mode.

[0379] This method may include conditionally signaling the use of palette mode based on predictive mode.

[0380] This method may include skipping signal notifications for the use of palette mode when the prediction mode is intrablock copy mode.

[0381] This method may include determining that the indication of using palette mode is false based on the current prediction mode being intrablock copy mode.

[0382] This method may include the fact that the prediction mode is intermode and that signal notifications for the use of palette mode are skipped.

[0383] This method may include determining that the indication of using palette mode is false based on the current prediction mode being intermode.

[0384] This method may include the fact that the prediction mode is intra-mode and the signal notification for the use of palette mode is skipped.

[0385] This method may include determining that the indication of using palette mode is false based on the current prediction mode being intra mode.

[0386] This method may include the fact that the prediction mode is intra-mode and the signal notification for the use of palette mode is skipped.

[0387] This method may include signaling that the prediction mode is intrablock copy mode and that palette mode is in use.

[0388] This method may include signaling the use of palette mode based on the type of picture, slice, or tile group.

[0389] This method could include adding palette mode as a candidate for prediction mode.

[0390] This method allows the prediction mode to include one or more of the following: intra mode, intra block copy mode, intra slice palette mode, or inter slice, I picture, P picture, B picture, or intra tile group.

[0391] This method may include the prediction mode being two or more of the following: intra mode, inter mode, intra block copy mode, or palette mode.

[0392] This method may include indicating the use of palette mode by signal notification or deriving it based on conditions.

[0393] This method may include conditions such as the block dimensions of the current block, the prediction mode of the current block, the quantization parameter (QP) of the current block, the palette flag of the neighboring block, the intra-block copy flag of the neighboring block, the display of the color format, separate or dual coding tree structures, or one or more of the slice type, group type, or picture type.

[0394] This method may include signaling or deriving the use of palette mode based on slice level flags, tile group level flags, or picture level flags.

[0395] This method may include signaling or deriving instructions to use intrablock copy mode based on slice level flags, tile group level flags, or picture level flags.

[0396] Referring to items 6-9 disclosed in the previous chapter, in some embodiments, it is preferable to use the following solutions.

[0397] One solution may include a video processing method that performs a conversion between the current video block of a picture of video and a bitstream representation of this video, wherein in the bitstream representation, information is signaled or derived based on the encoding conditions of this current video block regarding whether an intra-block copy mode is used in this conversion, and this intra-block copy mode includes encoding this current video block from another video block in this picture. Various embodiments may implement the following features:

[0398] - The encoding conditions include the block dimensions of the current video block.

[0399] - The coding conditions include the prediction mode of the current video block or the quantization parameters used to transform the current video block.

[0400] Referring to items 13-15 disclosed in the previous chapter, it is preferable in some embodiments to implement the following solutions.

[0401] One solution may include determining whether to apply a deblocking filter while transforming the current picture block of the picture, wherein the current picture block is encoded using palette-mode coding, which represents the current picture block using representative sample values ​​smaller than the total number of pixels in the current picture block, and performing the transformation such that a deblocking filter is applied if it is determined that a deblocking filter should be applied.

[0402] Another solution may include a video processing method that includes determining a quantization or dequantization process to use during the conversion between the current video block of a picture of a video and its bitstream representation, wherein the current video block is encoded using palette-mode coding, which represents the current video block using fewer representative sample values ​​than the total number of pixels in the current video block; and performing the conversion based on this determination of the quantization or dequantization process. Additional features may include:

[0403] - The quantization or dequantization process determined for the current video block will differ from any other quantization or dequantization process applied to a different video block encoded in a different way than the palette encoding mode.

[0404] -This conversion involves encoding the current video block into a bitstream representation.

[0405] -This conversion involves decoding the bitstream representation to generate the current video block of the video.

[0406] -This determination uses the same determination process as another determination process used for converting other intra-encoded video blocks.

[0407] It is understood that the disclosed technology may be implemented in a video encoder or decoder to improve compression efficiency using an improved coding tree structure.

[0408] Some solutions, referring to items 16-21 in the previous chapter, are as follows:

[0409] A video processing method comprising: determining that the current video block is a palette-encoded block for the purpose of converting between the current video block of a video containing multiple video blocks and a bitstream representation of this video; performing a maximum probability mode list construction process by considering the current video block as an intra-encoded block based on this determination; and performing the conversion based on the result of this list construction process, wherein the palette-encoded block is encoded or decoded using palette or representation sample values.

[0410] In the method described above, the list construction process treats neighboring palette-encoded blocks as intra-blocks in default mode.

[0411] A video processing method comprising: determining that the current video block is a palette-encoded block for a conversion between the current video block of a video containing multiple video blocks and a bitstream representation of this video; performing a maximum probability mode list construction process based on this determination, assuming that the current video block is a non-intra-encoded block, and performing the conversion based on the result of this list construction process, wherein the palette-encoded block is encoded or decoded using palette or representation sample values.

[0412] In the above method, when the list construction process fetches the intra-mode of a neighboring palette coding block, it treats the neighboring palette coding block as an inter-coding block.

[0413] A video processing method comprising: determining that the current video block is a palette-encoded block for a conversion between the current video block of a video containing multiple video blocks and a bitstream representation of this video; performing a list construction process by considering the current video block as an unavailable block based on this determination; and performing the conversion based on the result of this list construction process, wherein the palette-encoded block is encoded or decoded using palette or representation sample values.

[0414] In the above method, the list construction process is a motion vector prediction based on history.

[0415] In the above method, the list construction process is either MERGE or advanced motion vector prediction mode.

[0416] In the above method, the determination further includes making a determination based on the content of the video.

[0417] In the above method, the determination corresponds to a field in the bitstream representation.

[0418] A video processing method comprising: determining that the current video block is a palette-encoded block during a conversion between the current video block and its bitstream representation; determining the range of context encoding bins to be used for this conversion based on the current video block being a palette-encoded block; and performing this conversion based on the range of context encoding bins.

[0419] In the above method, bins of the current video block that are outside the range are encoded using bypass coding techniques, or decoded using bypass decoding techniques during the conversion.

[0420] The conversion is the method described above, which includes encoding the video into the bitstream representation.

[0421] In the above method, the conversion includes decoding the bitstream representation to generate the video.

[0422] Figure 24 is a block diagram showing an exemplary video processing system 2400 in which various technologies disclosed herein may be implemented. Various implementations may include some or all of the modules of system 2400. System 2400 may include an input unit 2402 for receiving video content. The video content may be received in raw or uncompressed format, for example, with 8 or 10-bit multimodule pixel values, or in a compressed or encoded format. Input unit 1902 may represent a network interface, a peripheral bus interface, or a storage interface. Examples of network interfaces include wired interfaces such as Ethernet® and passive optical networking (PON), and wireless interfaces such as Wi-Fi® or cellular interfaces.

[0423] System 2400 may include an encoding module 2404 capable of implementing various encoding or encoding methods described herein. The encoding module 2404 may reduce the average bitrate of video from input unit 2402 to its output, generating an encoded representation of the video. This encoding technique may therefore be referred to as video compression or video code conversion. The output of the encoding module 2404 may be stored or transmitted via connected communication, as represented by module 2406. The bitstream (or encoded) representation of the video received, stored, or transmitted in input unit 2402 may be used by module 2408 to generate pixel values ​​or a displayable image transmitted to display interface unit 2410. The process of generating a user-viewable image from the bitstream representation may be referred to as video decompression (video unpacking). Furthermore, specific video processing operations are referred to as “encoding” operations or tools, and it will be understood that encoding tools or operations are performed by encoders and their corresponding decoding tools or operations, which reverse the decoding result, are performed by decoders.

[0424] Examples of peripheral bus interface units or display interface units may include Universal Serial Bus (USB), High Definition Multimedia Interface (HDMI), or DisplayPort. Examples of storage interfaces include Serial Advanced Technology Attachment (SATA), PCI, and IDE interfaces. The technologies described herein may be implemented in various electronic devices such as mobile phones, laptop computers, smartphones, or other devices capable of performing digital data processing and / or video display.

[0425] Figure 25 is a flowchart of a video processing method 2500 according to the present technology. This method 2500 includes, in operation 2510, a conversion between a block of video region of video and a bitstream representation of video. This bitstream representation is processed according to a first format rule that specifies whether a signal is issued for the first use of palette mode for this block, and a second format rule that specifies the position of this first display in response to a second signal for the use of predictive mode for this block.

[0426] In some embodiments, the video region includes a video conversion unit, encoding unit, prediction unit, or region. In some embodiments, an instruction for the second use of the prediction mode is positioned before an instruction for the first use of the palette mode in the bitstream representation.

[0427] In some embodiments, a first instruction to use palette mode is conditionally included in the bitstream representation based on a second instruction to use prediction mode. In some embodiments, if the second instruction to use prediction mode indicates intrablock copy (IBC) prediction mode, the first instruction to use palette mode is skipped in the bitstream representation. In some embodiments, if the second instruction to use prediction mode indicates interprediction mode, the first instruction to use palette mode is skipped in the bitstream representation. In some embodiments, if the second instruction to use prediction mode indicates intraprediction mode, the first instruction to use palette mode is skipped in the bitstream representation. In some embodiments, if the second instruction to use prediction mode indicates skip mode, the first instruction to use palette mode is skipped in the bitstream representation. In some embodiments, skipping the first instruction to use palette mode in the bitstream representation indicates that palette mode is not being used.

[0428] In some embodiments, the instruction to use the first palette mode is encoded in the bitstream if the instruction to use the second predictive mode indicates IBC predictive mode. In some embodiments, the instruction to use the first palette mode is encoded in the bitstream if the instruction to use the second predictive mode indicates intra predictive mode. In some embodiments, the predictive mode is not a pulse-code modulation (PCM) mode. In some embodiments, the instruction to use the first palette mode is encoded before the instruction to use the PCM mode in the bitstream representation. In some embodiments, the instruction to use the PCM mode is skipped in the bitstream representation. In some embodiments, the indication of IBC mode is encoded in the bitstream representation. In some embodiments, when intra predictive mode is used, a flag in the bitstream representation indicates whether palette mode or IBC mode is signaled in the bitstream representation. In some embodiments, the flag is skipped based on the state of the block, which includes the block dimensions, whether IBC mode is enabled for the region associated with the block, or whether palette mode is enabled for the region associated with the block.

[0429] In some embodiments, an instruction to use a first palette mode is encoded in the bitstream if an instruction to use a second prediction mode indicates an interprediction mode. In some embodiments, an instruction to use a first palette mode is encoded after at least one of the following: an indication of skip mode, an indication of prediction mode, or an instruction to use PCM mode. In some embodiments, an instruction to use a first palette mode is encoded after an indication of skip mode or prediction mode and before an instruction to use PCM mode.

[0430] In some embodiments, a first indication for the use of palette mode is positioned before a second indication for the use of prediction mode in the bitstream representation. In some embodiments, a first indication for the use of palette mode is positioned after a second indication for the use of prediction mode, and the second indication for the use of prediction mode indicates intra-prediction mode or inter-prediction mode in the bitstream representation. In some embodiments, a first indication for the use of palette mode is signaled based on the type of picture, slice, or tile group. In some embodiments, a first indication for the use of palette mode may include a first flag indicating that the palette mode is enabled for a block. In some embodiments, a first indication for the use of palette mode is conditionally included in the bitstream representation based on a first flag indicating that the palette mode is enabled at the sequence level, picture level, tile group level, or tile level. In some embodiments, when the palette mode is disabled for a block, another flag indicating the PCM mode of the block is included in the bitstream representation. In some embodiments, the first flag is context-encoded based on information from one or more neighboring blocks of the current block. In some embodiments, the first flag is encoded without contextual information from one or more neighboring blocks of the current block.

[0431] In some embodiments, the instruction for the second use of the prediction mode includes a second flag indicating the prediction mode. In some embodiments, if the second flag in the bitstream representation indicates that the prediction mode is inter-mode, the bitstream representation further includes a third flag indicating whether intra-block copy mode is enabled. In some embodiments, if the second flag in the bitstream representation indicates that the prediction mode is intra-mode, the bitstream representation further includes a third flag indicating whether intra-block copy mode is enabled. In some embodiments, the third flag is conditionally included in the bitstream representation based on the block dimensions.

[0432] In some embodiments, a block is an encoded unit, and a second flag in the bitstream representation indicates that the prediction mode is intra-mode. In some embodiments, a first flag is conditionally included in the bitstream representation based on the block dimensions.

[0433] Figure 26 is a flowchart of a video processing method 2600 according to the present technology. In operation 2610, the method 2600 includes determining a prediction mode based on one or more acceptable prediction modes, which include at least one palette mode of a block of video region in the video, for conversion between the block and the bitstream representation of the video. According to the prediction mode, an instruction to use a palette mode is determined. In operation 2620, the method 2600 includes performing the conversion based on the determination.

[0434] In some embodiments, one or more tolerance prediction modes include intra modes. In some embodiments, one or more tolerance prediction modes include one intrablock copy (IBC) mode. In some embodiments, one or more tolerance prediction modes include inter modes.

[0435] In some embodiments, the video region includes an intra-slice, intra-picture, or intra-tile group. In some embodiments, one or more acceptable prediction modes include an intra-mode, an intra-block copy mode, and a palette mode.

[0436] In some embodiments, the image region includes interslice, interpicture, intertile group, P-slice, B-slice, P-picture, or B-picture. In some embodiments, one or more allowable prediction modes include intra-mode, intra-block copy mode, palette mode, and inter-mode.

[0437] In some embodiments, the block dimensions may be 4x4. In some embodiments, one or more allowable prediction modes exclude an intermode if the block dimensions are 4x4.

[0438] In some embodiments, the bitstream representation includes at least one prediction mode index representing one or more acceptable prediction modes if the block is not encoded in skip mode, and the prediction mode index is represented using one or more binary bins.

[0439] In some embodiments, the prediction mode index is represented using three binary bins, where the first bin value "1" indicates intra-mode, the first bin value "0" and the second bin value "0" indicate inter-mode, the first bin value "0", the second bin value "1", and the third bin value "0" indicate IBC mode, and the first bin value "0", the second bin value "1", and the third bin value "1" indicate pallet mode.

[0440] In some embodiments, the prediction mode index is represented using two binary bins, where the first bin value "1" and the second bin value "0" represent intra-mode, the first bin value "0" and the second bin value "0" represent inter-mode, the first bin value "0" and the second bin value "1" represent IBC mode, and the first bin value "1" and the second bin value "1" represent pallet mode.

[0441] In some embodiments, the prediction mode index is represented using one binary bin when the current slice of the image is an intra-slice and IBC mode is disabled, where the first bin value "0" represents intra-mode and the second bin value "1" represents palette mode.

[0442] In some embodiments, the predictive mode index is represented using two binary bins when the current slice of video is not an intra-slice and IBC mode is disabled, with the first bin value "1" representing intra-mode, the first bin value "0" and the second bin value "0" representing inter-mode, and the first bin value "0" and the second bin value "1" representing palette mode. In some embodiments, the predictive mode index is represented using two binary bins when the current slice of video is an intra-slice and IBC mode is enabled, with the first bin value "1" representing IBC mode, the first bin value "0" and the second bin value "1" representing palette mode, and the first bin value "0" and the second bin value "0" representing intra-mode. In some embodiments, the indication of IBC mode usage is signaled in the Sequence Parameter Set (SPS) of the bitstream representation.

[0443] In some embodiments, the prediction mode index is represented using three binary bins.

[0444] Here, the first bin value "1" represents intermode, the first bin value "0" and the second bin value "1" represent intramode, the first bin value "0", the second bin value "0", and the third bin value "1" represent IBC mode, and the first bin value "0", the second bin value "0", and the third bin value "0" represent palette mode.

[0445] In some embodiments, the prediction mode index is represented using three binary bins.

[0446] Here, the first bin value "1" represents intra-mode, the first bin value "0" and the second bin value "1" represent inter-mode, the first bin value "0", the second bin value "0", and the third bin value "1" represent IBC mode, and the first bin value "0", the second bin value "0", and the third bin value "0" represent palette mode.

[0447] In some embodiments, the prediction mode index is represented using three binary bins, where the first bin value "0" indicates intermode, the first bin value "1" and the second bin value "0" indicate intramode, the first bin value "1", the second bin value "1", and the third bin value "1" indicate IBC mode, and the first bin value "1", the second bin value "1", and the third bin value "0" indicate pallet mode.

[0448] In some embodiments, if one condition is met, the signal notifications of one or more binary bins are skipped in the bitstream representation. In some embodiments, the condition includes block dimensions. In some embodiments, this condition includes a disabled prediction mode, and in the bitstream representation, the binary bins corresponding to this prediction mode are skipped.

[0449] Figure 27 is a flowchart of a video processing method 2700 according to this technology. This method 2700 includes, in operation 2710, a conversion between a block of video and a bitstream representation of video. The bitstream representation is processed according to a format rule that specifies the signaling of a first use of palette mode and a second use of intra-block copy (IBC) mode in a mutually dependent manner.

[0450] In some embodiments, the formatting rules stipulate that if the prediction mode of a block is equal to a first prediction mode that is not IBC mode, a first display is signaled in a bitstream representation. In some embodiments, the formatting rules stipulate that if the prediction mode of a block is equal to a first prediction mode that is not palette mode, a second display is signaled in a bitstream representation. In some embodiments, the first prediction mode is intra mode.

[0451] Figure 28 is a flowchart of a video processing method 2800 according to this technology. Method 2800 includes, in operation 2810, determining, based on the dimensions of the block, whether there is an instruction to use palette mode in the bitstream representation for conversion between the block of video and the bitstream representation of the video. Method 2800 includes, in operation 2820, performing the conversion based on that determination.

[0452] Figure 29 is a flowchart of a video processing method 2900 according to the present technology. Method 2900 includes, in operation 2910, determining, based on the dimensions of the block, whether there is an instruction to use intra-block copy (IBC) mode in the bitstream representation for conversion between the block of video and the bitstream representation of the video. Method 2900 includes, in operation 2920, performing the conversion based on that determination. In some embodiments, the block dimensions may include at least one of the number of samples in the block, the width of the block, or the height of the block.

[0453] In some embodiments, if the width of a block is less than or equal to a threshold, this indication is signaled in a bitstream representation. In some embodiments, if the height of a block is less than or equal to a threshold, this indication is signaled in a bitstream representation. In some embodiments, the threshold may be 64.

[0454] In some embodiments, if the width and height of a block are greater than a threshold, this indication is signaled in a bitstream representation. In some embodiments, the threshold may be 4. In some embodiments, if the number of samples in a block is greater than a threshold, this indication is signaled in a bitstream representation. In some embodiments, the threshold may be 16. In some embodiments, if the width of a block is equal to the height of a block, this indication is signaled in a bitstream representation.

[0455] In some embodiments, the representation is not included in the bitstream representation if (1) the width of the block is greater than a first threshold, (2) the height of the block is greater than a second threshold, or (3) the number of samples in the block is less than or equal to a third threshold. In some embodiments, the first and second thresholds are 64. In some embodiments, the third threshold may be 16.

[0456] In some embodiments, this determination is made based further on features associated with the block. In some embodiments, the features include the block's prediction mode. In some embodiments, the features include the block's quantization parameters. In some embodiments, the features include the palette flags of the block's neighboring blocks. In some embodiments, the features include the IBC flags of the block's neighboring blocks. In some embodiments, the features include the display of the block's color format. In some embodiments, the features include the block's coding tree structure. In some embodiments, the features include the block's slice group type, tile group type, or picture type.

[0457] Figure 30 is a flowchart of a video processing method 3000 according to the present technology. Method 3000 includes, in operation 3010, determining whether palette mode is permitted for a block of video based on a second display of the video region containing the block, for the purpose of converting between the block of video and its bitstream representation. Method 3000 also includes, in operation 3020, performing the conversion based on that determination.

[0458] In some embodiments, the image region may include slices, tile groups, or pictures. In some embodiments, the bitstream representation omits an explicit indication of whether palette mode is allowed if a second indication indicates that slight motion vector differences are enabled. In some embodiments, the second indication is represented as a flag present in the bitstream representation. In some embodiments, the second indication indicates whether palette mode is enabled for the image region. In some embodiments, the bitstream representation does not include an explicit indication of whether palette mode is allowed if a second indication indicates that palette mode is disabled for the image region. In some embodiments, if the bitstream representation does not include an explicit indication of whether palette mode is allowed, palette mode is not allowed for the block.

[0459] Figure 31 is a flowchart of a video processing method 3100 according to the present technology. Method 3100 includes, in operation 3110, determining whether intra-block copy (IBC) mode is permitted for a block of video based on a second display of the video region containing the block, for the purpose of converting between the block of video and the bitstream representation of the block of video. Method 3100 also includes, in operation 3120, performing the conversion based on that determination.

[0460] In some embodiments, the image region may include slices, tile groups, or pictures. In some embodiments, the bitstream representation omits an explicit indication of whether IBC mode is permitted if a second indication indicates that a slight motion vector difference is enabled. In some embodiments, the second indication is represented as a flag present in the bitstream representation. In some embodiments, the second indication indicates whether IBC mode is enabled for the image region. In some embodiments, the bitstream representation does not include an explicit indication of whether IBC mode is permitted if a second indication indicates that IBC mode is disabled for the image region. In some embodiments, if the bitstream representation does not include an explicit indication of whether IBC mode is permitted, then IBC mode is not permitted for the block.

[0461] Figure 32 is a flowchart of a video processing method 3200 according to the present technology. Method 3200 includes, in operation 3210, determining a first bit depth of a first sample associated with a palette entry in palette mode for conversion between a block of video and a bitstream representation of the video. The first bit depth is different from a second bit depth associated with the block. Method 3200 also includes, in operation 3220, performing the conversion based on the determination.

[0462] In some embodiments, the second bit depth includes an internal bit depth for the block. In some embodiments, the second bit depth includes a bit depth associated with the original sample of the block. In some embodiments, the second bit depth includes a bit depth associated with the reconstructed sample of the block. In some embodiments, the first bit depth is a positive integer. In some embodiments, the first bit depth is equal to 8. In some embodiments, the first bit depth is greater than the second bit depth. In some embodiments, the first bit depth is less than the second bit depth. In some embodiments, the first bit depth is determined based on the dimensions of the block. In some embodiments, the first bit depth is determined based on the quantization parameters of the block. In some embodiments, the first bit depth is determined based on the color format indication of the block. In some embodiments, the first bit depth is determined based on the coding tree structure of the block. In some embodiments, the first bit depth is determined based on the slice group type, tile group type, or picture type of the block.

[0463] In some embodiments, the first bit depth is determined based on the number of palette entries associated with the block. In some embodiments, the first bit depth is determined based on the number of entries in the palette predictor associated with the block. In some embodiments, the first bit depth is determined based on one or more indices of the color components of the block.

[0464] In some embodiments, the second sample is associated with another palette entry in palette mode, and the second sample has a third bit depth different from the first bit depth. In some embodiments, the third bit depth is greater than the second bit depth. In some embodiments, the third bit depth is less than the second bit depth.

[0465] In some embodiments, the third sample of the block is reconstructed based on shifting the value of the first sample by M bits (M is a positive integer). In some embodiments, shifting the value of the first sample includes shifting the first sample M bits to the left. In some embodiments, the first sample has a value of C, and the reconstructed second sample has a value of (C << M)+(1 << (M - 1)). In some embodiments, shifting the value of the first sample includes shifting the first sample M bits to the right. In some embodiments, the first sample has a value of C, and the reconstructed second sample has a value determined based on (C+(1 << (M - 1))) >> M, and this value is limited by a minimum value of 0 and a maximum value of (1 << N)-1. In some embodiments, M is determined based on the difference between the first bit depth and the second bit depth. In some embodiments, M is equal to the second bit depth minus the first bit depth. In some embodiments, M is equal to the first bit depth minus the second bit depth. In some examples, M is equal to 2. In some embodiments, M is determined based on the dimension of the block. In some embodiments, M is determined based on the quantization parameter of the block. In some embodiments, M is determined based on an indication of the color format of the block. In some embodiments, M is determined based on the coding tree structure of the block. In some embodiments, M is determined based on the slice group type, tile group type, or picture type of the block. In some embodiments, M is determined based on the number of palette entries associated with the block. In some embodiments, M is determined based on the number of predicted palette entries associated with the block. In some embodiments, M is determined based on the positions of the first and third samples in the block. In some embodiments, M is determined based on the index of the color component of the block.

[0466] In some embodiments, this method includes determining a first sample associated with a palette entry based on a lookup operation performed on a sample table. In some embodiments, the values ​​in the sample table are signaled in the sequence parameter set (SPS), video parameter set (VPS), picture parameter set (PPS), picture header, slice header, tile group header, row of the maximum coding unit (LCU), or group of LCUs in the bitstream representation. In some embodiments, the values ​​in the sample table are derived based on information in the sequence parameter set (SPS), video parameter set (VPS), picture parameter set (PPS), picture header, slice header, tile group header, row of the maximum coding unit (LCU), or group of LCUs in the bitstream representation.

[0467] Figure 33 is a flowchart of a video processing method 3300 according to the present technology. Method 3300 includes, in operation 3310, deciding to process neighboring blocks encoded in palette mode of the current block as intra-encoded blocks having a default mode, while constructing a list of maximum probability mode (MPM) candidates for the current block, if the neighboring blocks are located above or to the left of the current block, for the purpose of converting the current block of video to a bitstream representation of the video. Method 3300 includes, in operation 3320, performing the conversion based on that decision.

[0468] In some embodiments, the default mode includes a planar mode. In some embodiments, the default mode may include a DC mode, a vertical mode, or a horizontal mode. In some embodiments, the default mode is signaled in a dependency parameter set, a sequence parameter set, a video parameter set, a picture parameter set (PPS), a picture header, a slice header, a tile group header, a maximum coding unit (LCU), a coding unit (CU), an LCU row, an LCU group, a transform unit (TU), a prediction unit (PU) block, or a video coding unit in a bitstream representation.

[0469] Figure 34 is a flowchart of a video processing method 3400 according to this technology. Method 3400 includes, in operation 3410, determining parameters for deblocking filtering according to rules for blocks of video encoded as palette-mode encoded blocks in the bitstream representation of the video. Method 3400 also includes, in operation 3420, performing a conversion between the blocks and the bitstream representation of the video using the parameters for deblocking filtering.

[0470] In some embodiments, the rules stipulate that blocks be treated as intra-encoded blocks of video in order to determine parameters for non-blocking filtering.

[0471] In some embodiments, the boundary strength for deblocking filtering is 2 when the first side of the block boundary or the second side of the block boundary is encoded in palette mode. In some embodiments, the boundary strength for deblocking filtering is determined to be 2 when the first side of the block boundary and the second side of the block boundary are encoded in palette mode. In some embodiments, the rule stipulates that the block is treated as an intra-encoded block of the video in order to determine the parameters for deblocking filtering.

[0472] In some embodiments, the rules stipulate that palette mode be treated separately from other modes in unblocking filtering in order to determine the parameters of unblocking filtering. In some embodiments, if the first side of the block boundary or the second side of the block boundary is encoded in palette mode, the boundary strength for unblocking filtering is determined to be 0. In some embodiments, if the first side of the block boundary is encoded in palette mode and the second side of the block boundary is encoded in intra-block copy (IBC) mode, the boundary strength for unblocking filtering is 1. In some embodiments, if the first side of the block boundary is encoded in palette mode and the second side of the block boundary is encoded in intra mode, the boundary strength for unblocking filtering is 2.

[0473] In some embodiments, the rule specifies that the palette mode is treated as a transform skip mode in unblocking filtering in order to determine the parameters of unblocking filtering. In some embodiments, the rule specifies that the palette mode is treated as block-based delta pulse code modulation (BDPCM) in unblocking filtering in order to determine the parameters of unblocking filtering.

[0474] Figure 35 is a flowchart of a video processing method 3500 according to the present technology. Method 3500 includes, in operation 3410, deciding to process neighboring blocks of the current block encoded in palette mode as non-intra encoded blocks while constructing a list of maximum probability mode (MPM) candidates for the current block for conversion between the current block of video and the bitstream representation of the video. Method 3500 also includes, in operation 3520, performing the conversion based on that decision.

[0475] In some embodiments, if a neighboring block is located above or to the left of the current block, the neighboring block is treated as an inter-coded block. In some embodiments, if a neighboring block is located above or to the left of the current block, the neighboring block is treated as an intra-coded block (IBC).

[0476] Figure 36A is a flowchart of a video processing method 3600 according to the present technology. Method 3600 includes, in operation 3610, determining the quantization parameters associated with a block of video for a given block. Method 3600 includes, in operation 3620, encoding the block of video as a partially palette-encoded block in a bitstream representation of the video based on the modified values ​​of the quantization parameters. The method further includes, in operation 3630, signaling the encoding information related to the quantization parameters in the bitstream representation.

[0477] Figure 36B is a flowchart of a video processing method 3650 according to the present technology. Method 3650 includes deriving quantization parameters based on a bitstream representation of the video in operation 3660. Method 3650 further includes partially decoding a palette coding block based on modified quantization parameters determined by modifying the quantization parameters in operation 3670.

[0478] In some embodiments, the quantization parameters are modified based on setting an upper limit for the quantization parameters. In some embodiments, the quantization parameters are modified based on setting a lower limit for the quantization parameters. In some embodiments, the quantization parameters are modified in the same way as a second quantization parameter associated with a block encoded in transform-skip mode. In some embodiments, the quantization parameters are modified in the same way as a third quantization parameter associated with a block encoded in block-based delta pulse code modulation (BDPCM) mode.

[0479] In some embodiments, the quantization parameter is denoted as Qp, and modifying the quantization parameter involves correcting the value of Qp to max(Qp, 4+T), where T is a non-negative integer. In some embodiments, T is based on a predefined threshold. In some embodiments, T is 4+Ts, and Ts is signaled in the bitstream representation. In some embodiments, Ts is signaled in the syntactic element min_qp_prime_ts_minus4 in the bitstream representation.

[0480] Figure 37 is a flowchart of a video processing method 3700 according to this technology. This method 3700 includes, in operation 3710, determining the representation of escape samples of blocks in a bitstream representation of video, regardless of whether bypass mode is enabled for the blocks, for blocks of video that are encoded as palette-encoded blocks in the bitstream representation of video. This method 3700 also includes, in operation 3720, performing a conversion between the blocks and the bitstream representation based on the determination.

[0481] In some embodiments, escape samples are represented in bitstream format using a fixed length. In some embodiments, the fixed length contains N bits, where N is a positive integer. In some embodiments, N is 8 or 10.

[0482] In some embodiments, escape samples are represented in bitstream format using a length determined based on the internal bit depth of the block. In some embodiments, escape samples are represented in bitstream format using a length determined based on the input bit depth of the block. In some embodiments, escape samples are represented in bitstream format using a length determined based on the quantization parameter of the block. In some embodiments, the length is defined as a function of the quantization parameter represented by f(Qp). In some embodiments, the internal bit depth of the block is d, and f(Qp) = (d - (Qp - 4) / 6).

[0483] Figure 38 is a flowchart of a video processing method 3800 according to this technology. This method 3800 includes determining a first quantization process for blocks of video encoded as palette-encoded blocks in the bitstream representation of the video in operation 3810. The first quantization process is different from a second quantization process applicable to non-palette-mode encoded blocks. This method 3800 also includes performing a conversion between the blocks and the bitstream representation based on the determination in operation 3820.

[0484] In some embodiments, the first quantization process includes bit-shifting the escape samples of a block to the right in order to quantize the escape samples. In some embodiments, the first quantization process includes bit-shifting the escape samples of a block to the left in order to dequantize the escape samples. In some embodiments, the sample is denoted as p, the quantized sample as Qp, and the value of the escape sample is encoded as a function of p and Qp represented by f(p,Qp). In some embodiments, f(p,Qp) = p >> ((Qp-4) / 6). In some embodiments, the sample is denoted as p, and the value of the escape sample is encoded as p >> N (where N is an integer). In some embodiments, N is 2. In some embodiments, N is determined based on a feature associated with the block.

[0485] In some embodiments, the feature includes a value signaled in a sequence parameter set, a video parameter set, a picture parameter set, a picture header, a slice header, a tile group header, a row of maximum coding units (LCUs), or a group of LCUs. In some embodiments, the feature includes the internal bit depth of the block. In some embodiments, the feature includes the input bit depth of the block. In some embodiments, the feature includes the dimension of the block. In some embodiments, the feature includes the quantization parameter of the block. In some embodiments, the feature includes the display of the color format of the block. In some embodiments, the feature includes the coding tree structure of the block. In some embodiments, the feature includes the slice type, tile group type, or picture type associated with the block.

[0486] In some embodiments, taking the sample as p, the bit depth associated with the block as bd, the quantized sample as Qp, the value of the escape sample is signaled as a function of bd, p, and Qp represented by f(bd, p, Qp). In some embodiments, f(bd, p, Qp) = clip(0, (1 << (bd - (Qp - 4) / 6)) - 1, (p + (1 << (bd - 1))) >> ((Qp - 4) / 6)). In some embodiments, f(bd, p, Qp) = clip(0, (1 << bd) - 1, p << ((Qp - 4) / 6)). In some embodiments, clip is clip(a, i, b) = (i <a ? a:(i>b ? b:i)) is defined. In some embodiments, the clip is defined as clip(a,i,b)=(i<=a ? a:(i>=b ? b:i)). In some embodiments, the bit depth includes the internal bit depth or the input bit depth.

[0487] Figure 39 is a flowchart of a video processing method 3900 according to this technology. This method 3900 includes, in the operation 3910, converting a video containing luminance blocks and corresponding chrominance blocks to a bitstream representation of the video based on a rule. This rule specifies that when the current luminance block is encoded using a palette encoding mode and the corresponding current chrominance block is encoded in a derivation mode, the current luminance block is treated as having a default intra-prediction mode, and the current chrominance block is encoded in a default intra-prediction mode. This palette encoding mode includes encoding the current luminance block using a palette of representative sample values.

[0488] In some embodiments, the current luminance block encoded using a palette coding mode is treated as an intra block. In some embodiments, the current luminance block encoded using a palette coding mode is treated as a palette block. In some embodiments, the default intra prediction mode includes DC mode, planar mode, vertical mode, or horizontal mode. In some embodiments, the default intra prediction mode includes any intra prediction mode. In some embodiments, the default intra prediction mode is signaled in a dependency parameter set, sequence parameter set, video parameter set, picture parameter set, adaptive parameter set, picture header, slice header, tile group header, maximum coding unit (LCU), coding unit (CU), LCU row, LCU group, conversion unit (TU), prediction unit (PU) block, or video coding unit.

[0489] Figure 40 is a flowchart of a video processing method 4000 according to this technology. Method 4000 includes, in operation 4010, performing a conversion between a video having one or more blocks and a bitstream representation of the video. For this conversion, a list of motion candidates is constructed for each block according to a rule. This rule specifies that motion information of a block encoded using palette encoding mode is to be treated as unavailable or invalid for encoding subsequent blocks.

[0490] In some embodiments, the list of motion candidates includes a list of motion vector prediction candidates based on history. In some embodiments, the list of motion candidates includes a list of merge candidates. In some embodiments, the list of motion candidates includes motion vector prediction candidates.

[0491] In some embodiments, a block is treated as having an invalid reference index. In some embodiments, a block is treated as having a reference index of 0. In some embodiments, a block is treated as having a zero motion vector.

[0492] In some embodiments, the applicability of a rule depends on a feature associated with the block. In some embodiments, the feature includes video content within the block. In some embodiments, the feature includes a dependency parameter set, a sequence parameter set, a video parameter set, a picture parameter set, an adaptive parameter set, a picture header, a slice header, a tile group header, a maximum coding unit (LCU), a coding unit (CU), an LCU row, an LCU group, a transform unit (TU), a predictive unit (PU) block, or a message signaled by a video coding unit associated with the block. In some embodiments, the feature includes the location of a video region associated with the block, where the video region includes a CU, PU, ​​TU, or video coding unit. In some embodiments, the feature includes the dimensions of the block or neighboring blocks. In some embodiments, the feature includes the shape of the block or neighboring blocks. In some embodiments, the feature includes a representation of the color format of the block. In some embodiments, the feature includes the coding tree structure of the block. In some embodiments, the feature includes the slice type, tile group type, or picture type of the block. In some embodiments, the feature includes the color components of the block. In some embodiments, the feature includes a time layer identifier of the block.

[0493] Figure 41 is a flowchart of a video processing method 4100 according to the present technology. Method 4100 includes, in operation 4110, determining the number of context coding bins for a block of video that is encoded as a palette-mode coded block in the bitstream representation of the video, based on a rule. Method 4100 further includes, in operation 4120, performing a conversion between the block of video and the bitstream representation of the video based on the determination.

[0494] In some embodiments, determining the number of context-coded bins involves assigning a counter to a block to track the number of context-coded bins. In some embodiments, the counter includes a counter for tracking the number of components to be coded in the block. In some embodiments, the counter is initialized to 0 and incremented by 1 for each bin coded using context coding. In some embodiments, the counter is initialized to N, where N is a positive integer. For each bin coded using context coding, the counter is decremented by 1.

[0495] In some embodiments, the rule identifies a threshold, and the method further includes applying bypass coding to any additional bins that exceed or fall below the threshold. In some embodiments, the threshold may be 0 or 1. In some embodiments, the threshold is determined based on the number of coding passes in a block or the decoded information in a block.

[0496] In some embodiments, the threshold is determined based on a second threshold related to the number of context coding bins for a transport stream (TS) coded block or a non-TS coded block. In some embodiments, the second threshold is denoted by T, where T is a positive number. This rule specifies that the threshold is W × H × T, where W is the width of the block and H is the height of the block. In some embodiments, the second threshold is denoted by T, where T is a positive number. This rule specifies that W × H × C × T, W, where W is the width of the block, H is the height of the block and C is the number of color components to be coded in the block. In some embodiments, T is 1.75 or 2. In some embodiments, the threshold is less than the second threshold. In some embodiments, the threshold is greater than the second threshold.

[0497] Figure 42 is a flowchart of a video processing method 4200 according to the present technology. Method 4200 includes, in operation 4210, determining, based on a rule, the number of intra-encoded neighboring blocks of the current block for inter- and intra-coupled prediction modes, for the conversion of the current block of video to a bitstream representation of the video. This rule specifies a method for handling blocks encoded using palette encoding modes when counting the number of intra-encoded neighboring blocks for inter- and intra-coupled prediction modes, and using palette encoding modes includes encoding blocks using a palette of representative sample values. Method 4200 also includes, in operation 4220, performing a conversion based on the determination.

[0498] In some embodiments, the scheme specifies that blocks encoded using the palette coding mode are treated as non-intra-encoded blocks that are excluded from the counting of the number of intra-encoded neighboring blocks. In some embodiments, blocks encoded in palette coding mode are treated as blocks having a prediction mode of MODE_PLT. In some embodiments, blocks encoded in palette coding mode are treated as inter-encoded blocks. In some embodiments, blocks encoded using palette coding mode are treated as blocks encoded in intra-block copy (IBC) coding mode, which encodes the current block with a sample from the current picture of the current block. In some embodiments, blocks encoded using palette coding mode are neighboring blocks above or to the left of the current block.

[0499] In some embodiments, the scheme specifies that blocks encoded using the palette coding mode should be treated as intra-encoded blocks to be included in the counting of intra-encoded neighboring blocks.

[0500] Figure 43 is a flowchart of a video processing method 4300 according to this technology. Method 4300 includes, in operation 4310, deciding to skip operations on the samples of the current block in the filtering process for conversion between the current block of video and the bitstream representation of the video. These samples are encoded using palette encoding mode, which includes encoding the block using a palette of representative sample values. Method 4300 also includes, in operation 4320, performing the conversion based on that decision.

[0501] In some embodiments, the operation may include non-blocking operations that smooth one or more boundaries of the current block. In some embodiments, the operation may include operations that reduce distortion by compensating for offsets in a sample-adaptive offset process that classifies samples. In some embodiments, the operation may include filtering operations in an adaptive loop filtering process. In some embodiments, the operation may include classification operations in an adaptive loop filtering process. In some embodiments, the operation may include luminance mapping by a chroma scaling operation in which luminance samples are mapped using an adaptive piecewise linear model and chroma samples undergo a luminance-dependent chroma residual scaling operation.

[0502] Figure 44 is a flowchart of a video processing method 4400 according to the present technology. Method 4400 includes, in operation 4410, determining one scan order selected from three or more scan orders for the conversion of one block of a video to one bitstream representation of this video. This block is encoded in palette mode using a palette of representative sample values. Method 4400 includes, in operation 4420, performing the conversion based on the determination.

[0503] In some embodiments, three or more scan sequences include a reverse horizontal scan sequence. In some embodiments, the reverse horizontal scan sequence includes scanning a block from left to right for rows with an odd number of indices, with the first row of the block assigned index 0. In some embodiments, the reverse horizontal scan sequence includes scanning a block from right to left for rows with an even number of indices, with the first row of the block assigned index 0.

[0504] In some embodiments, three or more scan sequences may include an inverse vertical scan sequence. In some embodiments, the inverse vertical scan sequence involves scanning a block from top to bottom for columns with an odd number of indices, with the first column of the block assigned index 0. In some embodiments, the inverse vertical scan sequence involves scanning a block from bottom to top for columns with an even number of indices, with the first column of the block assigned index 0.

[0505] In some embodiments, three or more scan sequences include a horizontal scan sequence that scans the coefficients of a block from left to right for rows with even indices and from right to left for rows with odd indices. These three or more scan sequences further include a vertical scan sequence that scans the coefficients of a block from top to bottom for rows with even indices and from bottom to top for rows with odd indices, with the first row of the block assigned index 0 and the first column of the block assigned index 0.

[0506] Figure 45 is a flowchart of a video processing method 4500 according to this technology. Method 4500 includes, in operation 4510, determining one or more scan sequences for scanning the coefficients of a block based on the shape of the block in order to convert between blocks of video and a bitstream representation of the video. Method 4500 also includes, in operation 4520, performing the conversion based on the determination.

[0507] In some embodiments, one or more scan sequences are applied when the ratio of the width to the height of a block is greater than a threshold. In some embodiments, one or more scan sequences are applied when the ratio of the height to the width of a block is greater than a threshold. In some embodiments, the threshold is equal to 1. In some embodiments, the threshold is equal to 4.

[0508] Figure 46 is a flowchart of a video processing method 4600 according to the present technology. Method 4600, in operation 4610, decides to apply only one scan order to scan the coefficients of a block for conversion between a block of video and a bitstream representation of the video, performs a block-based quantization residual domain difference pulse coding modulation (BDPCM) process, and uses difference pulse coding modulation (DPCM) to represent the difference between the intra-predicted quantization residual of this block and the predicted quantization residual of this block in the bitstream representation of this block. Method 4600 also includes, in operation 4620, performing a conversion based on that decision.

[0509] In some embodiments, the single scanning sequence includes scanning the block vertically if the block's width is greater than its height. In some embodiments, the single scanning sequence includes scanning the block horizontally if its width is less than its height. In some embodiments, the single scanning sequence is derived based on the block's shape.

[0510] In some embodiments, at least one scan sequence can be applied if the width of a block, denoted by W, and the height of a block, denoted by H, satisfy a condition. In some embodiments, the condition includes W × Th ≥ H or H × Th ≥ W, where Th is an integer threshold. In some embodiments, T is 4 or 8. In some embodiments, Th is determined based on the features of the video. In some embodiments, the features include the content of the video. In some embodiments, the features include a decoder parameter set, slice parameter set, video parameter set, picture parameter set, adaptive parameter set, picture header, slice header, tile group header, maximum coding unit (LCU), coding unit (CU), LCU row, LCU group, tree unit (TU), picture unit (PU) block, or information signaled in the video coding unit of the bitstream representation. In some embodiments, the features include the position of a block in the video image of the video. In some embodiments, this feature includes the dimensions of this block or the dimensions of neighboring blocks. In some embodiments, this feature includes the shape of this block or the shapes of neighboring blocks. In some embodiments, the feature includes the color format of the block. In some embodiments, the feature includes the encoded tree structure of the image. In some embodiments, the feature includes the slice type, tile group type, or picture type of the image. In some embodiments, the feature includes the color components of the block. In some embodiments, the feature includes the time layer identity of the image. In some embodiments, the feature may include the profile, level, or tier of standards for the image.

[0511] In some embodiments, the transformation generates the current block from the bitstream representation. In some embodiments, the transformation generates the bitstream representation from the current block.

[0512] Some embodiments of the disclosed technology include deciding or determining to enable a video processing tool or mode. In one example, if a video processing tool or mode is enabled, the encoder uses or implements this tool or mode when processing a block of video, but does not necessarily have to modify the resulting bitstream based on the use of this tool or mode. That is, the conversion from a block of video to a bitstream representation of video uses this video processing tool or mode if it is enabled based on the decision or determination. In another example, if a video processing tool or mode is enabled, the decoder processes the bitstream knowing that the bitstream has been modified based on the video processing tool or mode. That is, it performs the conversion from a bitstream representation of video to a block of video using the video processing tool or mode that was enabled based on the decision or determination.

[0513] Some embodiments of the disclosed technology include deciding or determining to disable a video processing tool or mode. In one example, if a video processing tool or mode is disabled, the encoder does not use this tool or mode when converting blocks of video into a bitstream representation of the video. In another example, if a video processing tool or mode is disabled, the decoder processes the bitstream knowing that the bitstream has not been modified using a video processing tool or mode that has been enabled based on the decision or determination.

[0514] The disclosed and other solutions, examples, embodiments, modules, and functional operation implementations described herein may be implemented in digital electronic circuits, or in computer software, firmware, or hardware, or in one or more combinations thereof, including the structures disclosed herein and their structural equivalents. The disclosed and other embodiments may be implemented as one or more modules of computer program instructions encoded on a computer-readable medium for implementation by one or more computer program products, i.e., data processing devices, or for controlling the operation of data processing devices. This computer-readable medium may be a machine-readable storage device, a machine-readable storage substrate, a memory device, a composition of a material that provides a machine-readable propagating signal, or one or more combinations thereof. The term “data processing device” includes, for example, a programmable processing device, a computer, or a plurality of processing devices, or all devices, devices, and machines for processing data, including computers. In addition to hardware, the device may include code that makes up the execution environment of the computer program, such as processing device firmware, a protocol stack, a database management system, an operating system, or code that constitutes one or more combinations thereof. A propagated signal is an artificially generated signal, such as a machine-generated electrical, optical, or electromagnetic signal, produced to encode information for transmission to a suitable receiving device.

[0515] A computer program (also called a program, software, software application, script, or code) can be written in any form of programming language, including compiled or interpreted languages, and can be deployed as a standalone program or in any form, including modules, components, subroutines, or other units suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. A program may be recorded in part of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), stored in a single file dedicated to that program, or stored in multiple coordinating files (e.g., a file containing one or more modules, subprograms, or parts of code). A computer program can also be deployed to run on one computer located at one site, or on multiple computers distributed across multiple sites and interconnected by a communication network.

[0516] The processing and logic flows described herein can be carried out by one or more programmable processing units that run one or more computer programs on input data to perform functions by generating outputs. The processing and logic flows can also be carried out by application-specific logic circuits, such as FPGAs (Field-Programmable Gate Arrays) or ASICs (Application-Specific Integrated Circuits), and the devices can also be implemented as special-purpose logic circuits.

[0517] Processing units suitable for executing computer programs include, for example, both general-purpose and dedicated microprocessors, as well as any one or more processing units of any type of digital computer. Generally, processing units receive instructions and data from read-only memory or random-access memory or both. Essential elements of a computer are processing units for executing instructions and one or more memory devices for storing instructions and data. Generally, a computer may include one or more mass storage devices for storing data, such as magnetic, magneto-optical disks, or optical disks, or may be operablely coupled to receive data from or transfer data to these mass storage devices. However, a computer is not required to have such devices. Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, such as EPROMs, EEPROMs, flash memory, magnetic disks, such as internal hard disks or removable disks, magneto-optical disks, and semiconductor memory devices such as CD-ROMs and DVD-ROM disks. Processing units and memory may be complemented by or incorporated into application-specific logic circuits.

[0518] Although this patent specification contains many details, these should not be interpreted as limiting the scope of any subject matter or claims, but rather as descriptions of features that may be specific to particular embodiments of a particular technology. Certain features described in this patent document in the context of separate embodiments may be implemented in combination in one example. Conversely, various features described in the context of one example may be implemented separately or in any suitable subcombination in multiple embodiments. Furthermore, features may be described above as acting in a particular combination and initially asserted as such, but one or more features from an asserted combination may, in some cases, be extracted from the combination, and the asserted combination may be directed towards a subcombination or a variation of a subcombination.

[0519] Similarly, although the operations are shown in a specific order in the drawings, this should not be understood as requiring that such operations be performed in a specific order or sequentially, or that all of the operations shown be performed, in order to achieve the desired result. Furthermore, the separation of the various system components in the examples described in this patent specification should not be understood as requiring such separation in all embodiments.

[0520] Only a few implementation forms and examples are described, and other embodiments, extensions, and modifications are possible based on the content described and illustrated in this patent document.

Claims

1. For the conversion between the current block of the video and the bitstream of the video, the number of intra-coded neighboring blocks of the current block is determined according to a rule for the combined inter-intra prediction mode, A method for processing video data, which includes performing the conversion based on the aforementioned decision, In the combined inter-intra prediction mode, the prediction signal for the current block is generated based on at least an intra-prediction signal, an inter-prediction signal, and weights for the intra-prediction signal and the inter-prediction signal. The weight is determined based on the count of the number of intracoded neighboring blocks of the current block. The intracoded neighborhood block is composed of an intracoded left neighbor block and an intracoded upper neighbor block. The aforementioned rule stipulates that a first block coded using the first prediction mode is treated as a non-intra block that is excluded from counting the number of intracoded neighboring blocks, In the first prediction mode, the reconstructed sample is represented by at least one of 1) a palette predictor, 2) an escape sample, or 3) palette information included in the bitstream. If the width and height of the first block are each less than or equal to a first threshold, a first instruction indicating whether or not to enable the first prediction mode for the first block is conditionally signaled in the bitstream, and in response to the width or height of the first block being greater than the first threshold, the first instruction is omitted from the bitstream. If the first block is a luminance block and the first prediction mode is enabled for the first block, the coding tool is not applied to the first block. The coding tool includes mapping and scaling processes based on a piecewise linear model. A method for processing video data.

2. The first block is treated as a block having the prediction mode MODE_PLT. The method according to claim 1.

3. The rule stipulates that, in the counting of the number of intracoded neighboring blocks, the first block is treated the same as the second block coded in the second prediction mode, In the second prediction mode described above, the prediction sample is derived from a block of sample values ​​in the same video region as determined by the block vector. The method according to claim 1.

4. The second block described above is treated as a non-intra block that is excluded from the counting of the number of intracoded neighboring blocks. The method according to claim 3.

5. The deblocking filtering process is skipped for the sample in the first block. The method according to any one of claims 1 to 4.

6. The first threshold is 64. The method according to claim 1.

7. The conversion includes encoding the video into the bitstream. The method according to any one of claims 1 to 6.

8. The conversion includes decoding the video from the bitstream. The method according to any one of claims 1 to 6.

9. A device for processing video data, comprising a processing unit and a non-temporary memory having instructions, When executed by the processing unit, the instruction is directed to the processing unit. For the conversion between the current block of the video and the bitstream of the video, the number of intra-coded neighboring blocks of the current block is determined according to a rule for the combined inter-intra prediction mode, This includes performing the conversion based on the aforementioned decision, In the combined inter-intra prediction mode, the prediction signal for the current block is generated based on at least an intra-prediction signal, an inter-prediction signal, and weights for the intra-prediction signal and the inter-prediction signal. The weight is determined based on the count of the number of intracoded neighboring blocks of the current block. The intracoded neighborhood block is composed of an intracoded left neighbor block and an intracoded upper neighbor block. The aforementioned rule stipulates that a first block coded using the first prediction mode is treated as a non-intra block that is excluded from counting the number of intracoded neighboring blocks, In the first prediction mode, the reconstructed sample is represented by at least one of 1) a palette predictor, 2) an escape sample, or 3) palette information included in the bitstream. If the width and height of the first block are each less than or equal to a first threshold, a first instruction indicating whether or not to enable the first prediction mode for the first block is conditionally signaled in the bitstream, and in response to the width or height of the first block being greater than the first threshold, the first instruction is omitted from the bitstream. If the first block is a luminance block and the first prediction mode is enabled for the first block, the coding tool is not applied to the first block. The coding tool includes mapping and scaling processes based on a piecewise linear model. A device for processing video data.

10. A non-temporary computer-readable storage medium for storing instructions, wherein the instructions are stored in a processing unit. For the conversion between the current block of the video and the bitstream of the video, the number of intra-coded neighboring blocks of the current block is determined according to a rule for the combined inter-intra prediction mode, This includes performing the conversion based on the aforementioned decision, In the combined inter-intra prediction mode, the prediction signal for the current block is generated based on at least an intra-prediction signal, an inter-prediction signal, and weights for the intra-prediction signal and the inter-prediction signal. The weight is determined based on the count of the number of intracoded neighboring blocks of the current block. The intracoded neighborhood block is composed of an intracoded left neighbor block and an intracoded upper neighbor block. The aforementioned rule stipulates that a first block coded using the first prediction mode is treated as a non-intra block that is excluded from counting the number of intracoded neighboring blocks, In the first prediction mode, the reconstructed sample is represented by at least one of 1) a palette predictor, 2) an escape sample, or 3) palette information included in the bitstream. If the width and height of the first block are each less than or equal to a first threshold, a first instruction indicating whether or not to enable the first prediction mode for the first block is conditionally signaled in the bitstream, and in response to the width or height of the first block being greater than the first threshold, the first instruction is omitted from the bitstream. If the first block is a luminance block and the first prediction mode is enabled for the first block, the coding tool is not applied to the first block. The coding tool includes mapping and scaling processes based on a piecewise linear model. A non-temporary computer-readable storage medium.

11. A method for storing a video bitstream, The aforementioned method, In accordance with the rules, determine the number of intra-coded neighboring blocks of the current block of the video in the combined inter-intra prediction mode, Based on the above determination, the bitstream is generated, This includes storing the bitstream in a non-transient, computer-readable recording medium, In the combined inter-intra prediction mode, the prediction signal for the current block is generated based on at least an intra-prediction signal, an inter-prediction signal, and weights for the intra-prediction signal and the inter-prediction signal. The weight is determined based on the count of the number of intracoded neighboring blocks of the current block. The intracoded neighborhood block is composed of an intracoded left neighbor block and an intracoded upper neighbor block. The aforementioned rule stipulates that a first block coded using the first prediction mode is treated as a non-intra block that is excluded from counting the number of intracoded neighboring blocks, In the first prediction mode, the reconstructed sample is represented by at least one of 1) a palette predictor, 2) an escape sample, or 3) palette information included in the bitstream. If the width and height of the first block are each less than or equal to a first threshold, a first instruction indicating whether or not to enable the first prediction mode for the first block is conditionally signaled in the bitstream, and in response to the width or height of the first block being greater than the first threshold, the first instruction is omitted from the bitstream. If the first block is a luminance block and the first prediction mode is enabled for the first block, the coding tool is not applied to the first block. The coding tool includes mapping and scaling processes based on a piecewise linear model. method.

Images