Video processing methods, devices and media

Abstract

An exemplary method for visual media processing includes: for encoding a current video block in a video region of visual media data into a bitstream representation of the visual media data, making a decision regarding the application of a cross-component linear model (CCLM) prediction mode based at least on a codec tree structure associated with the video region or at least inferred from the codec tree structure associated with the video region, wherein, in the CCLM prediction mode, a linear model is used to predict chroma samples of the current video block on reconstructed luminance samples of the current video block; and selectively including a syntax element in the bitstream representation, wherein the syntax element indicates the application of a CCLM prediction mode based at least on or at least inferred from the codec tree structure.

Description

[0001] Cross-references to related applications

[0002] In accordance with applicable patent law and / or the rules of the Paris Convention, this application aims to promptly claim priority and benefit from International Patent Application No. PCT / CN2019 / 089250, filed May 30, 2019. For all purposes of that law, the entire disclosure of the foregoing application is incorporated by reference as part of the disclosure of this patent. Technical Field

[0003] This document covers video and image encoding / decoding technologies. Background Technology

[0004] Digital video accounts for the largest share of bandwidth usage on the Internet and other digital communication networks. As the number of connected user devices capable of receiving and displaying video increases, the bandwidth demand for digital video is expected to continue to grow. Summary of the Invention

[0005] This paper describes devices, systems, and methods related to digital video encoding / decoding, and specifically, coefficient encoding / decoding in transform skipping modes for video encoding / decoding. The described methods can be applied to existing video codec standards (e.g., High Efficiency Video Coding (HEVC)) and future video codec standards (e.g., Versatile Video Coding (VVC)) or codecs.

[0006] In one exemplary aspect, a method for visual media encoding is disclosed. The method includes: for encoding a current video block in a video region of visual media data into a bitstream representation of the visual media data, making a decision regarding the application of a cross-component linear model (CCLM) prediction mode, at least based on or inferred from a codec tree structure associated with the video region, wherein, in the CCLM prediction mode, a linear model is used to predict chroma samples of the current video block on reconstructed luminance samples; and selectively including a syntax element in the bitstream representation, wherein the syntax element indicates at least based on or inferred from the codec tree structure.

[0007] In one exemplary aspect, a method for decoding visual media is disclosed. The method includes: determining a codec tree structure associated with the current video block from a bitstream representation comprising visual media data including a video region containing a current video block; determining, at least based on the codec tree structure, whether syntax elements are included in the bitstream representation, wherein the syntax elements indicate the application of a cross-component linear model (CCLM) prediction mode to the current video region, wherein, under the CCLM prediction mode, a linear model is used to predict chroma samples of the current video block on reconstructed luminance samples of the current video block; and generating the current video block from the bitstream representation.

[0008] In one exemplary aspect, a method for encoding visual media is disclosed. The method includes: for encoding a current video block in a video region of visual media data into a bitstream representation of the visual media data, making a decision regarding the application of a Luminosity Mapping and Chroma Scaling (LMCS) processing step to the video region, at least based on or inferred from a codec tree structure associated with the video region, wherein, in the LMCS processing step, luminosity samples in the video region are mapped using an adaptive piecewise linear model and / or luminosity-dependent chroma residual scaling operation is performed on chroma samples in the video region; and selectively including a syntax element in the bitstream representation, wherein the syntax element indicates the application of the LMCS processing step to the video region.

[0009] In one exemplary aspect, a method for visual media encoding is disclosed. The method includes: determining a codec tree structure associated with the current video block from a bitstream representation comprising visual media data including a video region containing a current video block; determining, at least based on the codec tree structure, whether syntax elements are included in the bitstream representation, wherein the syntax elements indicate the application of a Luminosity Mapping and Chroma Scaling (LMCS) processing step to the video region, wherein in the LMCS processing step, luminosity samples are mapped using an adaptive piecewise linear model, and luminosity-dependent chroma residual scaling operation is performed on the chroma samples; and generating the current video block from the bitstream representation.

[0010] In yet another example, the above method can be implemented by a video encoder device that includes a processor.

[0011] In yet another example, the above method can be implemented by a video decoder device that includes a processor.

[0012] In yet another example, these methods can be embodied in processor-executable instructions and stored on a computer-readable program medium.

[0013] These and other aspects are also described in this document. Attached Figure Description

[0014] Figure 1 An example of intra-frame block copying is shown.

[0015] Figure 2 An example of a block encoded and decoded in palette mode is shown.

[0016] Figure 3 An example of using palette prediction values ​​to signal palette entries is shown.

[0017] Figure 4 Examples of horizontal and vertical transverse scans are shown.

[0018] Figure 5 An example of encoding and decoding a palette index is shown.

[0019] Figure 6 An example of a multi-type tree partitioning pattern is shown.

[0020] Figure 7 An example of sample points used to derive parameters in a cross-component linear model (CCLM) prediction mode is shown.

[0021] Figure 8 An exemplary architecture for luminance mapping utilizing chroma scaling is shown.

[0022] Figures 9A-9E This is a flowchart illustrating an example of a video processing method.

[0023] Figure 10 This is a block diagram of an example hardware platform used to implement the visual media decoding or visual media encoding technologies described in this document.

[0024] Figure 11 This is a block diagram of an example video processing system in which the disclosed technology can be implemented.

[0025] Figure 12 This is a flowchart illustrating an example of a visual media encoding method.

[0026] Figure 13 This is a flowchart of an example of a visual media decoding method.

[0027] Figure 14 This is a flowchart illustrating an example of a visual media encoding method.

[0028] Figure 15 This is a flowchart of an example of a visual media decoding method. Detailed Implementation

[0029] This document provides a variety of techniques that decoders of image or video bitstreams can use to improve the quality of decompressed or decoded digital video or images. For the sake of brevity, the term "video" used in this document includes both sequences of pictures (traditionally referred to as video) and single images. Furthermore, video encoders can implement these techniques during the encoding process to reconstruct decoded frames for further encoding.

[0030] For ease of understanding, chapter headings are used in this document, but the embodiments and techniques are not limited to the respective chapters. Thus, embodiments from one chapter can be combined with embodiments from other chapters.

[0031] 1. Overview

[0032] This document relates to video codec technology. Specifically, it relates to coefficient encoding and decoding in transform skip mode in video codec. It can be applied to existing video codec standards (such as HEVC) or upcoming standards (e.g., Multifunction Video Coding (VCC)). It can also be applied to future video codec standards or video codecs.

[0033] 2. Preliminary Discussion

[0034] Video codec standards have primarily evolved through the development of well-known ITU-T and ISO / IEC standards. ITU-T developed H.261 and H.263, while ISO / IEC developed MPEG-1 and MPEG-4 Visualization. These two organizations jointly developed the H.262 / MPEG-2 video and H.264 / MPEG-4 Advanced Video Coding (AVC) standards, as well as the H.265 / HEVC standard [1,2]. Since H.262, video codec standards have been based on a hybrid video codec architecture, utilizing temporal prediction plus transform coding. To explore future video codec technologies beyond HEVC, VCEG and MPEG jointly established the Joint Video Exploration Team (JVET) in 2015. Since then, JVET has adopted many new methods and incorporated them into reference software called the Joint Exploration Model (JEM). In April 2018, the Joint Video Experts Group (JVET) between VCEG (Q6 / 16) and ISO / IEC JTC1 SC29 / WG11 (MPEG) was established to work on the VVC standard, with the goal of reducing the bit rate by 50% compared to HEVC.

[0035] The latest version of the VVC draft, namely Multi-Functional Video Codec (Draft 4), can be found at: http: / / phenix.it-sudparis.eu / jvet / doc_end_user / current_document.php?id=5755

[0036] The latest reference software for VVC (named VTM) can be found at the following URL: https: / / vcgit.hhi.fraunhofer.de / jvet / VVCSoftware_VTM / tags / VTM-4.0

[0037] 2.1 Intra-frame block copying

[0038] Intra-Block Copy (IBC), also known as Current Picture Reference, is used in HEVC Screen Content Coding Extension (HEVC-SCC) and the current VVC test model (VTM-4.0). IBC extends the concept of motion compensation from inter-frame coding to intra-frame coding. Figure 1 As shown, when IBC is applied, the current block is predicted from a reference block in the same image. Samples in the reference block must be reconstructed before the current block is encoded or decoded. Although IBC is inefficient for most camera-captured sequences, it shows significant encoding / decoding gains for screen content. This is because screen content images contain many repeating patterns, such as icons and text characters. IBC effectively eliminates redundancy between these repeating patterns. In HEVC-SCC, IBC can be applied if the inter-frame coding unit (CU) selects the current image as its reference image. In this case, the MV is renamed to a block vector (BV), and the BV always has integer pixel precision. For compatibility with the master profile HEVC, the current image is marked as the "long-term" reference image in the Decoded Picture Buffer (DPB). It should be noted that, similarly, in multi-view / 3D video codec standards, inter-view reference images are also marked as "long-term" reference images.

[0039] After finding the reference block following the BV, predictions can be generated by copying the reference block. The residual can be obtained by subtracting the reference pixel from the original signal. Then, transform and quantization can be applied as in other codec modes.

[0040] However, some or all pixel values ​​may be undefined when the reference block is outside the image, overlaps with the current block, is outside the reconstructed region, or is outside the valid region subject to certain constraints. Essentially, there are two ways to address this issue. One is to disallow this situation, for example, in bitstream consistency. The other is to apply padding to those undefined pixel values. The following subsections describe the solutions in detail.

[0041] 2.2 IBC in HEVC Screen Content Codec Extension

[0042] In HEVC's screen content encoding and decoding extensions, when a block uses the current image as a reference, it should ensure that the entire reference block is within the available reconstruction area, as indicated by the following specification text:

[0043] The variables offsetX and offsetY are derived as follows:

[0044] offsetX=(ChromaArrayType==0)? 0:(mvCLX[0]&0x7?2:0) (8-106)

[0045] offsetY=(ChromaArrayType==0)? 0:(mvCLX[1]&0x7?2:0) (8-107)

[0046] The requirement for bitstream consistency is that, when the reference image is the current image, the luminance motion vector mvLX should adhere to the following constraints:

[0047] – When the derivation of the z-scan sequence block availability specified in Clause 6.4.1 is invoked with (xCurr,yCurr) set to equal (xCb,yCb) and the neighboring luminance position (xNbY,yNbY) set to equal (xPb+(mvLX[0]>>2)-offsetX,yPb+(mvLX[1]>>2)-offsetY) as input, the output shall be equal to TRUE.

[0048] – When the derivation of the z-scan sequence block availability specified in Clause 6.4.1 is invoked with (xCurr,yCurr) set to equal (xCb,yCb) and the neighboring luminance position (xNbY,yNbY) set to equal (xPb+(mvLX[0]>>2)+nPbW-1+offsetX,yPb+(mvLX[1]>>2)+nPbH-1+offsetY) as input, the output shall be TRUE.

[0049] – One or two of the following conditions should be true:

[0050] The value of –(mvLX[0]>>2)+nPbW+xB1+offsetX is less than or equal to 0.

[0051] The value of –(mvLX[1]>>2)+nPbH+yB1+offsetY is less than or equal to 0.

[0052] The following conditions should be true:

[0053] (xPb+(mvLX[0]>>2)+nPbSw-1+offsetX) / CtbSizeY-xCurr / CtbSizeY<=yCurr / CtbSizeY-(yPb+(mvLX[1]>>2)+nPbSh-1+offsetY) / CtbSizeY(8-108)

[0054] Therefore, situations where the reference block overlaps with the current block or is outside the image will not occur. No need to fill the reference or prediction blocks.

[0055] 2.3 IBC in the VVC Test Model

[0056] In the current VVC test model, specifically the VTM-4.0 design, the entire reference block should be consistent with the current Coding Tree Unit (CTU) and should not overlap with the current block. Therefore, there is no need to pad the reference or prediction block. The IBC flag is encoded and decoded into the prediction mode of the current CU. Thus, for each CU, there are a total of three prediction modes: MODE_INTRA, MODE_INTER, and MODE_IBC.

[0057] 2.3.1 IBC Merge Mode

[0058] In IBC Merge mode, indices pointing to entries in the IBC Merge candidate list are parsed from the bitstream. The construction of the IBC Merge list can be summarized according to the following sequence of steps:

[0059] Step 1: Derive the candidate airspace

[0060] Step 2: Insert HMVP candidate

[0061] Step 3: Insert pairwise average candidates

[0062] In the derivation of spatial merge candidates, up to four merge candidates are selected from those located at positions A1, B1, B0, A0, and B2. The derivation order is A1, B1, B0, A0, and B2. Position B2 is considered only if any PU at positions A1, B1, B0, or A0 is unavailable (e.g., because it belongs to another slice or tile) or is not encoded / decoded in IBC mode. After adding the candidate at position A1, a redundancy check is performed on the addition of the remaining candidates, ensuring that candidates with the same motion information are excluded from the list, thereby improving encoding / decoding efficiency.

[0063] After inserting an empty domain candidate, if the IBC Merge list size is still smaller than the maximum IBC Merge list size, an IBC candidate from the HMVP table can be inserted. When inserting an HMVP candidate, a redundancy check is performed.

[0064] Finally, the pairwise average candidates are inserted into the IBC Merge list.

[0065] A Merge candidate is considered invalid when the reference block identified by the Merge candidate is outside the image, overlaps with the current block, is outside the reconstructed region, or is outside the valid region subject to certain constraints.

[0066] Please note that invalid Merge candidates may be inserted into the IBC Merge list.

[0067] 2.3.2 IBC AMVP Mode

[0068] In IBC AMVP mode, the AMVP index pointing to an entry in the IBC AMVP list is parsed from the bitstream. The construction of the IBC AMVP list can be summarized by the following steps:

[0069] Step 1: Derive the candidate airspace

[0070] Check A0 and A1 until a usable candidate is found.

[0071] Check B0, B1, and B2 until a usable candidate is found.

[0072] Step 2: Insert HMVP candidate

[0073] Step 3: Insert zero candidate

[0074] After inserting a spatial candidate, if the size of the IBC AMVP list is still smaller than the maximum size of the IBC AMVP list, then an IBC candidate from the HMVP table can be inserted.

[0075] Finally, the zero candidate was inserted into the IBC AMVP list.

[0076] 2.4 Adaptive Motion Vector Resolution (AMVR)

[0077] In HEVC, when `use_integer_mv_flag` in the strip header is equal to 0, signaling is used to notify the Motion Vector Difference (MVD) (between the motion vector of the CU and the predicted motion vector) in quarter-luminance sample units. In VVC, CU-level Adaptive Motion Vector Resolution (AMVR) is introduced. AMVR allows encoding and decoding of the CU's MVD with different precisions. Depending on the current control mode of the CU (normal AMVP mode or affine AVMP mode), the current CU's MVD can be adaptively selected as follows:

[0078] - Normal AMVP mode: quarter brightness sample, integer brightness sample, or four brightness sample.

[0079] - Affine AMVP mode: quarter brightness sample, integer brightness sample, or 1 / 16 brightness sample.

[0080] If the current CU has at least one non-zero MVD component, a conditional signaling instruction for the CU-level MVD resolution is provided. If all MVD components (i.e., both the horizontal and vertical MVD of reference list L0 and reference list L1) are zero, the quarter-spot luminance sample MVD resolution is inferred.

[0081] For a CU with at least one non-zero MVD component, signaling informs a first flag to indicate whether quarter-luminance sample MVD precision is used for the CU. If the first flag is 0, no further signaling is required, and quarter-luminance sample MVD precision is used for the current CU. Otherwise, signaling informs a second flag to indicate whether integer or quadruple luminance sample MVD precision is used for the normal AMVP CU. The same second flag is used to indicate whether integer or 1 / 16 luminance sample MVD precision is used for the affine AMVP CU. To ensure that the reconstructed MV has the expected precision (quarter-luminance sample, integer luminance sample, or quadruple luminance sample), the motion vector prediction value of the CU is rounded to the same precision as the MVD before being added to it. The motion vector prediction value is rounded to zero (i.e., negative motion vector prediction values ​​are rounded to positive infinity, and positive motion vector prediction values ​​are rounded to negative infinity).

[0082] The encoder uses RD checks to determine the motion vector resolution of the current CU. To avoid always performing CU-level RD checks three times for each MVD resolution, in VTM4, RD checks for MVD precision are only conditionally invoked, except for quarter-lumen samples. For normal AVMP mode, the RD costs for quarter-lumen sample MVD precision and integer lumen sample MVD precision are first calculated. Then, the RD costs for integer lumen sample MVD precision are compared with those for quarter-lumen sample MVD precision to determine if further checks for the RD costs of four-lumen sample MVD precision are necessary. When the RD cost for quarter-lumen sample MVD precision is significantly less than that for integer lumen sample MVD precision, the RD check for four-lumen sample MVD precision is skipped. For affine AMVP mode, if an affine inter-frame mode is not selected after checking the rate-distortion costs of affine Merge / Skip mode, Merge / Skip mode, quarter-lumen sample MVD precision normal AMVP mode, and quarter-lumen sample MVD precision affine AMVP mode, then 1 / 16 lumen sample MVD precision and 1-pixel MVD precision affine inter-frame mode are not checked. Furthermore, the affine parameters obtained in the 1 / 4 luminance sample MV precision affine inter-frame mode were used as the starting search points in the 1 / 16 luminance sample and 1 / 4 luminance sample MV precision affine inter-frame modes.

[0083] 2.5 Palette Mode

[0084] The basic idea behind the palette mode is that samples in a CU are represented by a small, representative set of color values. This set is called the palette. Samples outside the palette can also be indicated by signaling followed by an escape symbol (possibly quantized) of the component values. This is as follows: Figure 2 As shown.

[0085] 2.6 Palette Mode in HEVC Screen Content Codec Extension (HEVC-SCC)

[0086] In the palette mode of HEVC-SCC, prediction is used to encode and decode the palette and index mapping.

[0087] 2.6.1 Encoding and Decoding of Palette Entries

[0088] For encoding and decoding of palette entries, palette prediction values ​​are maintained. The maximum size of the palette and the palette prediction values ​​are signaled in the SPS. In HEVC-SCC, the `palette_predictor_initializer_present_flag` is introduced in the PPS. When this flag is 1, the entries used to initialize palette prediction values ​​are signaled in the bitstream. Palette prediction values ​​are initialized at the beginning of each CTU line, each stripe, and each slice. Depending on the value of `palette_predictor_initializer_present_flag`, the palette prediction values ​​are either reset to 0 or initialized using the palette prediction value initializer entry signaled in the PPS. In HEVC-SCC, a palette prediction value initializer of size 0 is enabled to allow explicit disabling of palette prediction value initialization at the PPS level.

[0089] For each entry in the palette prediction values, a reuse flag is signaled to indicate whether it is part of the current palette. This is as follows: Figure 3 As shown, a zero-run-length encoding / decoding is used to send the reuse flag. Following this, the number of new palette entries is signaled using zero-order exponential Golomb code. Finally, the component values ​​of the new palette entries are signaled.

[0090] 2.6.2 Palette Index Encoding and Decoding

[0091] The palette index is encoded and decoded using horizontal and vertical traversal scans, such as... Figure 5 As shown. The scan order is explicitly signaled in the bitstream using the palette_transpose_flag. For the remainder of the subsection, it is assumed that the scan is horizontal.

[0092] The palette index is encoded and decoded using two primary palette sample modes: "INDEX" and "COPY_ABOVE". As explained earlier, escape symbols are also signaled as being in "INDEX" mode and are assigned an index equal to the maximum palette size. A flag is used to signal the mode except in the top row or when the previous mode is "COPY_ABOVE". In "COPY_ABOVE" mode, the palette index of the sample in the previous row is copied. In "INDEX" mode, the palette index is explicitly signaled. For both "INDEX" and "COPY_ABOVE" modes, a run value is signaled, specifying the number of subsequent samples encoded and decoded using the same mode. When an escape symbol is part of a run in either "INDEX" or "COPY_ABOVE" mode, an escape component value is signaled for each escape symbol. The encoding and decoding of the palette index is as follows: Figure 5 As shown.

[0093] This syntactic sequence is performed as follows: First, the number of index values ​​for the CU is signaled. Next, the actual index values ​​for the entire CU are signaled using truncated binary encoding / decoding. Both the index number and index values ​​are encoded / decoded in bypass mode. This groups the bypass bits associated with the indexes together. Then, the palette sample mode (if necessary) and run length are signaled in an interleaved manner. Finally, the component escape values ​​corresponding to the escape samples of the entire CU are grouped together and encoded / decoded in bypass mode.

[0094] The signaling notification follows the index value and includes an additional syntax element, `last_run_type_flag`. This syntax element, combined with the number of indices, eliminates the need for the signaling notification to specify the run value corresponding to the last run in the block.

[0095] In HEVC-SCC, palette mode can also be enabled for 4:2:2, 4:2:0, and monochrome chroma formats. For all chroma formats, the signaling for palette entries and palette indices is almost identical. In non-monochrome formats, each palette entry includes three components. For monochrome formats, each palette entry includes a single component. For downsampled chroma directions, chroma samples are associated with a luminance sample index that is divisible by 2. After reconstructing the palette index for the CU, if a sample has only a single associated component, only the first component of the palette entry is used. The only difference in signaling notification is the number of escaped component values. For each escaped sample, the number of escaped component values ​​signaled may vary depending on the number of components associated with that sample.

[0096] 2.7 Coefficient Encoding and Decoding in Transform Skip Mode

[0097] In JVET-M0464 and JVET-N0280, several modifications were proposed to the coefficient encoding and decoding in the Transform Skip (TS) mode in order to adapt the residual encoding and decoding to the statistical and signal characteristics of the transform skip level.

[0098] The proposed modifications are listed below.

[0099] No last significant scan location: Since the residual signal reflects the spatial residual after prediction, and no energy compaction is performed on the TS through the transform, a higher probability of zero or invalid levels at the tail of the lower right corner of the transform block is no longer given. Therefore, the last valid scan position signaling is omitted in this case.

[0100] Sub-block CBF:The missing signaling at the last valid scan position requires the sub-block CBF signaling with the coded_sub_block_flag of the TS to be modified as follows:

[0101] Due to quantization, the aforementioned invalid sequences may still appear locally within the transform block. Therefore, as mentioned earlier, the last valid scan position is removed, and coded_sub_block_flag is encoded and decoded for all sub-blocks.

[0102] There is a special case regarding the `coded_sub_block_flag` of a subblock covering the DC frequency position (top-left subblock). In VVC Draft 3, this subblock's `coded_sub_block_flag` is never signaled and is always inferred to be equal to 1. When the last valid scan position is located in another subblock, this means there is at least one valid level outside the DC subblock. Therefore, although the subblock's `coded_sub_block_flag` is inferred to be equal to 1, the DC subblock may contain only zero / invalid levels. In the absence of last scan position information in the TS, the `coded_sub_block_flag` of each subblock is signaled. This also includes the DC subblock's `coded_sub_block_flag`, except when all other `coded_sub_block_flag` syntax elements are already equal to 0. In this case, the DC `coded_sub_block_flag` is inferred to be equal to 1 (inferDcSbCbf = 1). Since there must be at least one valid level in this DC subblock, if all other sig_coeff_flag syntax elements in this DC subblock are equal to 0, then the sig_coeff_flag syntax element at the first position (0,0) is not signaled and is deduced to be equal to 1 (inferSbDcSigCoeffFlag=1).

[0103] ○ Modify the context modeling of coded_sub_block_flag. The context model index is calculated as the sum of the coded_sub_block_flag to the left of the current sub-block and the coded_sub_block_flag above the current sub-block, instead of a logical disjunction between the two.

[0104] sig_coeff_flag context modeling:The local template in the sig_coeff_flag context modeling is modified to include only the neighbor to the left of the current scan position (NB0) and the neighbor above the current scan position (NB1). The context model offset is simply the number of valid neighbor positions: sig_coeff_flag[NB0] + sig_coeff_flag[NB1]. Therefore, the selection of different context sets based on the diagonal d within the current transform block is removed. This results in three context models and a single set of context models for encoding and decoding the sig_coeff_flag flag.

[0105] abs_level_gt1_flag and par_level_flag context modeling: A single context model is used for abs_level_gt1_flag and par_level_flag.

[0106] abs_remainder encoding / decoding: Although the empirical distribution of the residuals skipped by the transformation generally still conforms to a Laplace or geometric distribution, there are greater instationarities than at the absolute level of the transformation coefficients. In particular, for the absolute level of the residuals, the variance within the continuous realization window is higher. This inspired the following modifications to the binarization of the abs_remainder syntax and context modeling:

[0107] ○ A higher cutoff value is used in binarization, i.e., the transition point from the encoding using sig_coeff_flag, abs_level_gt1_flag, par_level_flag, and abs_level_gt3_flag to the Rice code of abs_remainder, and a dedicated context model for each binary bit position results in higher compression efficiency. Increasing the cutoff value will lead to more "greater than X" flags, for example, by introducing abs_level_gt5_flag, abs_level_gt7_flag, etc., until the cutoff value is reached. The cutoff value itself is fixed at 5 (numGtFlags = 5).

[0108] ○ Modify the template for rice parameter derivation, so that only the neighbors to the left of the current scan position and the neighbors above the current scan position are considered similar to the local template for sig_coeff_flag context modeling.

[0109] coeff_sign_flag context modeling:Due to the instabilities within the sign sequence and the fact that prediction residuals are often biased, a context model can be used to encode and decode signs even when the global empirical distribution is nearly uniform. A single dedicated context model is used for sign encoding and decoding, and the sign is parsed after sig_coeff_flag to keep all the binary bits from the context encoding and decoding together.

[0110] 2.8 Quantized Residual Block Differential Pulse Code Modulation (QR-BDPCM)

[0111] In JVET-M0413, a quantized residual block differential pulse-code modulation (QR-BDPCM) method is proposed to effectively encode and decode screen content.

[0112] The prediction direction used in QR-BDPCM can be either vertical or horizontal prediction mode. Similar to intra-frame prediction, intra-frame prediction is performed on the entire block by copying samples along the prediction direction (horizontal or vertical prediction). The residual is quantized, and the difference between the quantized residual and its predicted (horizontal or vertical) quantized value is encoded and decoded. This can be described as follows: For a block of size M (rows) × N (columns), let r i,j Let Q(r) be the prediction residual after performing intra-frame prediction using unfiltered samples from the upper or left block boundaries, either horizontally (copying the left neighbor pixel value row by row across the prediction block) or vertically (copying the top neighbor row to each row in the prediction block). i,j ), 0≤i≤M-1, 0≤j≤N-1, representing the residual r i,j The quantized version is then used, where the residual is the difference between the original block and the predicted block values. The block DPCM is then applied to the quantized residual samples to obtain a result with element-wise... Modified M×N array When signaling is sent to the vertical BDPCM:

[0113]

[0114] For horizontal prediction, similar rules are applied, and residual quantization samples are obtained in the following manner.

[0115]

[0116] Residual Quantization Samples It is sent to the decoder.

[0117] On the decoder side, the above calculation is reversed to produce: Q(r i,j), 0≤i≤M-1, 0≤j≤N-1. For the vertical prediction case,

[0118]

[0119] Regarding the horizontal situation

[0120]

[0121] Inverse quantization residual Q -1 (Q(r i,j The values ​​are added to the intra-block prediction values ​​to produce reconstructed sample values.

[0122] The main advantage of this approach is that DPCM can be performed dynamically during coefficient resolution, either by adding predicted values ​​as the coefficients are resolved or by performing inverse DPCM after resolution.

[0123] The draft text changes for QR-BDPCM are shown below.

[0124] 7.3.6.5 Encoding / Decoding Unit Syntax

[0125]

[0126]

[0127]

[0128] The value of bdpcm_flag[x0][y0] equal to 1 indicates that bdpcm_dir_flag exists in the codec unit of the luma codec block at position (x0, y0).

[0129] bdpcm_dir_flag[x0][y0] equal to 0 indicates that the prediction direction to be used in the bdpcm block is horizontal, otherwise it is vertical.

[0130] 2.9 Segmentation Structure

[0131] 2.9.1 Using a tree structure to segment CTU

[0132] In HEVC, the CTU is divided into CUs using a quadtree structure represented as a codec tree to accommodate various local characteristics. The decision of whether to use inter-frame picture (temporal) prediction or intra-frame picture (spatial) prediction to encode and decode picture regions is made at the leaf CU level. Depending on the PU partitioning type, each leaf CU can be further divided into one, two, or four PUs. Within a PU, the same prediction process is applied, and relevant information is sent to the decoder based on the PU. After obtaining residual blocks by applying the prediction process based on the PU partitioning type, the leaf CU can be partitioned into transform units (TUs) according to another quadtree structure similar to the codec tree used for CUs. A key feature of the HEVC structure is that it has multiple partitioning concepts, including CUs, PUs, and TUs.

[0133] In VVC, a quadtree using nested multi-type trees with binary and ternary partitioning structures replaces the concept of multiple partitioning unit types. That is, it eliminates the separation of CU, PU, ​​and TU concepts, except for CUs that are too large for the maximum transform length, and supports greater flexibility in CU partitioning shapes. In the codec tree structure, a CU can be square or rectangular. A codec tree unit (CTU) is first partitioned by a quadtree (also called a quadtree) structure. Then, the leaf nodes of the quadtree can be further partitioned using a multi-type tree structure. For example... Figure 6 As shown, there are four partitioning types in the multi-type tree structure: vertical binary partitioning (SPLIT_BT_VER), horizontal binary partitioning (SPLIT_BT_HOR), vertical ternary partitioning (SPLIT_TT_VER), and horizontal ternary partitioning (SPLIT_TT_HOR). The leaf nodes of the multi-type tree are called codec units (CUs). Unless the CU is too large for the maximum transform length, this segmentation is used for prediction and transform processing without any further partitioning. This means that in most cases, in a quadtree with a nested multi-type tree codec block structure, the CU, PU, ​​and TU have the same block size. Exceptions occur when the maximum supported transform length is less than the width or height of the CU color component. Additionally, the luma and chroma components have independent partitioning structures on the I-strip.

[0134] 2.10 Prediction using a cross-component linear model

[0135] To reduce cross-component redundancy, a cross-component linear model (CCLM) prediction mode is used in VTM4. This is achieved by predicting chromaticity samples based on reconstructed luminance samples from the same CU using the following linear model:

[0136] pred C (i,j)=α·rec L ′(i,j)+β

[0137] Among them, pred C (i,j) represents the predicted chromaticity sample points in the CU, and rec L (i,j) represents the downsampled reconstructed luminance sample points of the same CU. The linear model parameters α and β are derived from the relationship between the luminance and chrominance values ​​of two samples, which are the luminance samples with the minimum and maximum sample values ​​within the set of downsampled neighboring luminance samples, and their corresponding chrominance samples. The linear model parameters α and β are obtained according to the following equation.

[0138]

[0139] β=Y b -α·X b

[0140] Among them, Y a and X a This represents the luminance and chromaticity values ​​of the luminance sample point with the maximum luminance sample value. And X b and Y b These represent the luminance value and chromaticity value of the luminance sample point with the smallest luminance sample point, respectively. Figure 7 This shows an example of the positions of the left and top samples, as well as the sample points of the current block, involved in CCLM mode.

[0141] 2.11 Luminance Mapping and Chroma Scaling (LMCS)

[0142] In VTM4, a codec tool called Luminance Mapping and Chroma Scaling (LMCS) was added as a new processing block before the loop filter. LMCS has two main parts: 1) loop mapping of the luminance component based on an adaptive piecewise linear model; 2) for the chrominance component, applying luminance-dependent chrominance residual scaling. Figure 8 The LMCS architecture is shown from the decoder's perspective. Figure 8 The shadow block indication process is applied in the mapping domain; these processes include inverse quantization, inverse transform, intra-frame luminance prediction, and the addition of luminance prediction and luminance residual. Figure 8 The shadowless block indication process is applied in the original (i.e., unmapped) domain; and these processes include loop filters such as deblocking, ALF and SAO, motion compensation prediction, chroma intra-frame prediction, addition of chroma prediction and chroma residual, and storage of the decoded image as a reference image. Figure 8 The shading block in VVC is a new LMCS function block that includes forward and inverse mapping of the luminance signal and luminance-dependent chroma scaling. Like most other tools in VVC, LMCS can be enabled / disabled at the sequence level using the SPS flag.

[0143] 3. Examples of problems solved by the implementation examples

[0144] While coefficient encoding and decoding in JVET-N0280 can achieve encoding and decoding advantages over screen content encoding and decoding, coefficient encoding and TS mode may still have some drawbacks.

[0145] (1) The maximum allowed width or height of TS mode is controlled by a common value in PPS, which may limit flexibility.

[0146] (2) Each coding group (CG) needs to signal the cbf flag of TS mode, which may increase overhead costs.

[0147] (3) The coefficient scanning order does not consider the intra-frame prediction mode.

[0148] (4) Symbol encoding and decoding uses only one context.

[0149] (5) Skipping transformations of chroma components is not supported.

[0150] (6) The transform skip flag is applied to all prediction modes, which increases overhead and coding complexity.

[0151] 4. Examples of Implementation Methods

[0152] The detailed inventions described below should be considered as examples for explaining general concepts. These inventions should not be interpreted narrowly. Furthermore, these inventions can be combined in any way.

[0153] 1. The maximum allowed width and height of the transformation skip can be indicated by signaling in SPS / VPS / PPS / image header / strip header / group header / LCU line / LCU group.

[0154] a. In one example, the maximum allowed width and height of the transformation skip can be indicated by different messages in the signaling notification in SPS / VPS / PPS / Picture Header / Strip Header / Piece Group Header / LCU Line / LCU Group.

[0155] b. In one example, the maximum allowed width and / or height can first be signaled in the SPS / PPS and then updated in the image header / strip header / piece group header / LCU line / LCU group.

[0156] 2. The TS codec block can be divided into several coefficient groups (CGs) and signaling notifications of the codec block flag (Cbf) flag of at least one CG can be skipped.

[0157] a. In one example, signaling notifications for skipping the Cbf flag of all CGs can be given for TS codec blocks, for instance.

[0158] b. In one example, for TS mode, the skipped cbf flag for CG can be inferred to be 1.

[0159] c. In one example, whether to skip part or all of the CG flags can depend on the encoding / decoding mode.

[0160] i. In one example, for a TS codec intra-block, signaling notifications for all Cbf flags of the CG are skipped.

[0161] d. In one example, the skipped Cbf flag of CG can be inferred based on the following

[0162] i. Signaling notification messages in SPS / VPS / PPS / Image Header / Strip Header / Group Header / LCU Line / LCU Group / LCU / CU

[0163] ii. CG location

[0164] iii. Block dimensions of the current block and / or its neighboring blocks

[0165] iv. Block shape of the current block and / or its neighboring blocks

[0166] v. The most likely pattern of the current block and / or its neighboring blocks

[0167] vi. Prediction modes of neighboring blocks of the current block (intra-frame / inter-frame)

[0168] vii. Intra-prediction mode of neighboring blocks of the current block

[0169] viii. Motion vectors of neighboring blocks of the current block

[0170] ix. Indication of the QR-BDPCM mode of the current block's neighboring blocks

[0171] x. Current quantization parameters of the current block and / or its neighboring blocks.

[0172] xi. Color format specifications (such as 4:2:0, 4:4:4)

[0173] xii. Single / Dual Encoding / Decoding Tree Structure

[0174] xiii. Strip / Piece Type and / or Image Type

[0175] 3. The coefficient scanning order in the TS codec block can depend on the signaling notification messages in SPS / VPS / PPS / Picture Header / Strip Header / Slice Group Header / LCU Line / LCU Group / LCU / CU.

[0176] a. Alternatively, when using TS, the CG and / or coefficient scan order can depend on the intra-frame prediction mode.

[0177] i. In one example, if the intra-frame prediction mode is horizontally dominant, the scan order can be vertical.

[0178] 1. In one example, if the range of the intra-frame prediction mode index is from 2 to 34, the scan order can be vertical.

[0179] 2. In one example, if the range of the intra-frame prediction mode index is from 2 to 33, the scan order can be vertical.

[0180] ii. In one example, if the intra-frame prediction mode is vertically dominant, the scan order can be vertical.

[0181] 1. In one example, if the range of the intra-frame prediction mode index is from 34 to 66, the scan order can be vertical.

[0182] 2. In one example, if the range of the intra-frame prediction mode index is from 35 to 66, the scan order can be vertical.

[0183] iii. In one example, if the intra-frame prediction mode is vertically dominant, the scan order can be horizontal.

[0184] 1. In one example, if the range of the intra-frame prediction mode index is from 34 to 66, the scan order can be vertical.

[0185] 2. In one example, if the range of the intra-frame prediction mode index is from 35 to 66, the scan order can be vertical.

[0186] iv. In one example, if the intra-frame prediction mode is horizontally dominant, the scan order can be horizontal.

[0187] 1. In one example, if the range of the intra-frame prediction mode index is from 2 to 34, the scan order can be vertical.

[0188] 2. In one example, if the range of the intra-frame prediction mode index is from 2 to 33, the scan order can be vertical.

[0189] 4. It is proposed that the context of symbolic encoding and decoding can depend on the proximity information in the coefficient block of the TS mode.

[0190] a. In one example, the context for encoding / decoding the current symbol flag can depend on the values ​​of neighboring symbol flags in the TS mode.

[0191] i. In one example, the context for encoding and decoding the current symbol flag can depend on the values ​​of the symbol flags of the left and / or upper neighbors.

[0192] 1. In one example, the context of the current symbol can be deduced as C = (L + A), where C is the context id, L is the symbol of its left neighbor, and A is the symbol of its upper neighbor.

[0193] 2. In one example, the context of the current symbol can be deduced as C = (L + A * 2), where C is the context id, L is the symbol of its left neighbor, and A is the symbol of its upper neighbor.

[0194] 3. In one example, the context of the current symbol can be deduced as C = (L * 2 + A), where C is the context id, L is the symbol of its left neighbor, and A is the symbol of its upper neighbor.

[0195] ii. In one example, the context for encoding and decoding the current symbol flag can depend on the values ​​of the symbol flags of the left, top, and top-left neighbors.

[0196] iii. In one example, the context for encoding and decoding the current symbol flag can depend on the values ​​of the symbol flags of the left, top, top-left, and top-right neighbors.

[0197] b. In one example, the context for encoding and decoding the current symbol flag can depend on the position of the coefficients.

[0198] i. In one example, the context of a symbol can differ in different locations.

[0199] ii. In one example, the context of a symbol may depend on x+y, where x and y are the horizontal and vertical positions of the location.

[0200] iii. In one example, the context of the symbol can depend on min(x,y), where x and y are the horizontal and vertical positions of the location.

[0201] iv. In one example, the context of the symbol can depend on max(x,y), where x and y are the horizontal and vertical positions of the location.

[0202] 5. A mode that supports chroma transformation skipping is proposed.

[0203] a. In one example, the use of the chroma transformation skip mode can be based on a signaling notification message in the SPS / VPS / PPS / Picture Header / Strip Header / Piece Group Header / LCU Line / LCU Group / LCU / CU / Video Data Unit.

[0204] b. Alternatively, the use of a chroma transformation skip mode may be based on decoding information from one or more representative previous codec blocks in the same color component or other color components.

[0205] i. In one example, if the chroma TS flag of the representative block indicates false, then the chroma TS flag indication can be inferred as false. Alternatively, if the chroma TS flag of the representative block indicates true, then the chroma TS flag indication can be inferred as true.

[0206] ii. In one example, the representative block could be a luminance block or a chrominance block.

[0207] iii. In one example, the representative block can be any block within the juxtaposed brightness block.

[0208] iv. In one example, the representative block can be one of the neighboring chroma blocks of the current chroma block.

[0209] v. In one example, a representative block could be a block that covers the corresponding luminance sample of the center chroma sample within the current chroma block.

[0210] vi. In one example, the representative block can be the block that covers the corresponding luminance sample of the lower right chroma sample within the current chroma block.

[0211] 6. Whether and / or how to apply the transformation skip mode may depend on the signaling notification message in the SPS / VPS / PPS / Picture Header / Strip Header / Piece Group Header / LCU Line / LCU Group / LCU / CU / Video Data Unit.

[0212] a. In one example, the indication of when and / or how to apply the transform skip mode can depend on

[0213] i. Block dimensions of the current block and / or its neighboring blocks

[0214] ii. The block shape of the current block and / or its neighboring blocks

[0215] iii. The most likely pattern of the current block and / or its neighboring blocks

[0216] iv. Prediction modes of neighboring blocks of the current block (intra-frame / inter-frame)

[0217] v. Intra-prediction mode of neighboring blocks of the current block

[0218] vi. Motion vectors of neighboring blocks of the current block

[0219] vii. Indication of the QR-BDPCM mode of the current block's neighboring blocks

[0220] viii. Current quantization parameters of the current block and / or its neighboring blocks.

[0221] ix. Indication of color format (such as 4:2:0, 4:4:4)

[0222] x. Single / Dual codec tree structure

[0223] xi. Strip / Patch Type and / or Image Type

[0224] xii. Temporal layer ID

[0225] b. In one example, when the prediction mode is IBC mode and the block width and / or height are less than / greater than / equal to the threshold, a transform skip mode can be applied.

[0226] i. In one example, the threshold can be 4, 8, 16, or 32.

[0227] ii. In one example, the threshold can be signaled in the bitstream.

[0228] iii. In one example, the threshold can be based on

[0229] 1. Signaling notification messages in SPS / VPS / PPS / Image Header / Strip Header / Group Header / LCU Line / LCU Group / LCU / CU

[0230] 2. Block dimensions of the current block and / or its neighboring blocks.

[0231] 3. The block shape of the current block and / or its neighboring blocks.

[0232] 4. The most likely pattern of the current block and / or its neighboring blocks.

[0233] 5. Prediction modes of neighboring blocks of the current block (intra-frame / inter-frame)

[0234] 6. Intra-prediction mode of neighboring blocks of the current block

[0235] 7. Motion vectors of neighboring blocks of the current block

[0236] 8. Indication of the QR-BDPCM mode of the current block's neighboring blocks.

[0237] 9. Current quantization parameters of the current block and / or its neighboring blocks.

[0238] 10. Indications for color format (such as 4:2:0, 4:4:4)

[0239] 11. Single / Dual Encoding / Decoding Tree Structure

[0240] 12. Strip / Patch Type and / or Image Type

[0241] 13. Time-domain layer ID

[0242] 7. Whether the signaling notification indicates the TS mode can depend on the intra-prediction mode of the decoding / derived frame.

[0243] a. Alternatively, it may also depend on the permitted intra-prediction mode / direction used in the QR-BDPCM codec block and the use of QR-BDPCM.

[0244] b. For decoded or deduced intra-prediction modes, if it is part of the allowed set of intra-prediction modes / directions used in a QR-BDPCM codec block, the signaling notification of the TS flag can be skipped.

[0245] i. In one example, if QR-BDPCM is allowed for encoding and decoding a strip / picture / piece / brick, then vertical and horizontal modes are two allowed modes in the QR-BDPCM process, and if the intra-frame mode of the decoding / derivation is vertical or horizontal, then no signaling is sent to indicate the TS mode.

[0246] c. In one example, when the indicator of the QR-BDPCM mode (e.g., bdpcm_flag) is 1, it can be inferred that the transformation skip mode is enabled.

[0247] d. The above methods can be applied based on the following:

[0248] i. Signaling notification messages in SPS / VPS / PPS / Image Header / Strip Header / Group Header / LCU Line / LCU Group / LCU / CU

[0249] ii. Block dimensions of the current block and / or its neighboring blocks

[0250] iii. The block shape of the current block and / or its neighboring blocks

[0251] iv. The most likely pattern of the current block and / or its neighboring blocks

[0252] v. Prediction patterns of neighboring blocks of the current block (intra-frame / inter-frame)

[0253] vi. Intra-prediction mode of neighboring blocks of the current block

[0254] vii. Motion vectors of neighboring blocks of the current block

[0255] viii. Indication of the QR-BDPCM mode of the current block's neighboring blocks.

[0256] ix. Current quantization parameters of the current block and / or its neighboring blocks.

[0257] x. Indication of color format (such as 4:2:0, 4:4:4)

[0258] xi. Single / Dual Encoding / Decoding Tree Structure

[0259] xii. Strip / Piece Type and / or Image Type

[0260] xiii. Temporal layer ID

[0261] 8. Whether and / or how to apply QR-BDPCM may depend on the instructions of the TS mode.

[0262] a. In one example, the indication of whether to apply QR-BDPCM can be signaled at the Transformer Unit (TU) level, rather than at the CU level.

[0263] i. In one example, the indication of whether to apply QR-BDPCM can be signaled after the indication in TS mode is applied to TU.

[0264] b. In one example, QR-BDPCM is considered a special case of TS mode.

[0265] i. When a block is encoded or decoded in TS mode, another flag can be signaled to indicate whether QR-BDPCM or traditional TS mode is applied. If it is encoded or decoded in QR-BDPCM, the prediction direction used in QR-BDPCM can be further signaled.

[0266] ii. Alternatively, when a block is encoded or decoded in TS mode, a further signaling can be used to notify another flag to indicate which QR-BDPCM (e.g., QR-BDPCM based on horizontal / vertical prediction direction) or conventional TS mode is applied.

[0267] c. In one example, whether the QR-BDPCM indication is applied can be inferred based on the indication of the TS mode.

[0268] i. In one example, if the indication of whether to apply the transform skip flag on the same block is true, then the indication of whether to apply QR-BDPCM on the luma and / or chroma blocks can be inferred to be true. Alternatively, if the indication of whether to apply the transform skip flag on the luma and / or chroma blocks is true, then the indication of whether to apply QR-BDPCM on the same block can be inferred to be true.

[0269] ii. In one example, if the indication of whether to apply the transform skip flag on the same block is false, then the indication of whether to apply QR-BDPCM on the luma and / or chroma blocks can be inferred as false. Alternatively, if the indication of whether to apply the transform skip flag on the luma and / or chroma blocks is false, then the indication of whether to apply QR-BDPCM on the same block can be inferred as false.

[0270] 9. Whether and / or how to apply a single / dual tree may depend on the signaling notification message in the SPS / VPS / PPS / image header / strip header / piece group header / LCU line / LCU group / LCU / CU / video data unit.

[0271] a. In one example, whether to apply the single / dual tree indication may depend on whether the current strip / piece / LCU / LCU row / LCU group / video data unit is determined to be screen content.

[0272] i. Furthermore, in one example, whether a strip / piece / LCU / LCU line / LCU group / video data unit is determined as screen content can depend on

[0273] 1. Signaling notification messages / flags in SPS / VPS / PPS / Image Header / Strip Header / Slice Group Header / LCU Line / LCU Group / LCU / CU / Video Data Unit.

[0274] 2. Block dimensions of the current CTU and / or its neighboring CTUs.

[0275] 3. Block shape of the current CTU and / or its neighboring CTUs

[0276] 4. Current quantization parameters of the current CTU and / or its neighboring CTUs.

[0277] 5. Color format specifications (such as 4:2:0, 4:4:4)

[0278] 6. Previous strip / slice / LCU / LCU row / LCU group / video data unit individual / dual codec tree structure types

[0279] 7. Strip / Patch Type and / or Image Type

[0280] 8. Time-domain layer ID

[0281] b. In one example, it can be inferred whether to apply the single / double tree indication, which may depend on

[0282] i. Signaling notification messages in SPS / VPS / PPS / Image Header / Strip Header / Piece Group Header / LCU Line / LCU Group / LCU / CU / Video Data Unit.

[0283] ii. Hash hit ratio of IBC / inter-frame modes in previous encoded / decoded images / slices / reconstructed regions.

[0284] iii. Block dimensions of the current CTU and / or its neighboring CTUs

[0285] iv. Block shape of the current CTU and / or its neighboring CTUs

[0286] v. Current quantization parameters of the current CTU and / or its neighboring CTUs

[0287] vi. Indications of color format (such as 4:2:0, 4:4:4)

[0288] vii. Previous strip / slice / LCU / LCU row / LCU group / video data unit individual / dual codec tree structure types

[0289] viii. Strip / Piece Type and / or Image Type

[0290] ix. Time-domain layer ID

[0291] c. In one example, the indication of whether to apply CCLM and / or LMCS can depend on the single / dual codec tree structure type.

[0292] i. In one example, when using a single tree, the indications of CCLM and / or LMCS can be inferred as false.

[0293] d. The above method can also be applied to single-tree segmentation, or single / dual codec tree structure types.

[0294] 10. Whether to enable IBC can depend on the codec tree structure type.

[0295] a. In one example, for a given codec tree structure type (e.g., dual tree), the indication of the IBC mode, the block vector used in the IBC mode, and / or signaling notifications of other syntaxes related to the IBC mode can be skipped and inferred.

[0296] b. In one example, when a dual codec tree structure type is applied, the indication of the IBC mode can be inferred as false.

[0297] c. In one example, when a dual codec tree structure type is applied, the indication of the IBC mode of the luma block can be inferred as false.

[0298] d. In one example, when a dual codec tree structure type is applied, the indication of the IBC mode of the chroma block can be inferred as false.

[0299] e. In one example, the indication of the IBC mode can be inferred based on the following

[0300] i. Signaling notification messages in SPS / VPS / PPS / Image Header / Strip Header / Piece Group Header / LCU Line / LCU Group / LCU / CU / Video Data Unit.

[0301] ii. Hash hit rate of IBC / inter-frame modes in previous encoded / decoded images / slices / reconstructed regions

[0302] iii. Block dimensions of the current CTU and / or its neighboring CTUs

[0303] iv. Block shape of the current CTU and / or its neighboring CTUs

[0304] v. Current quantization parameters of the current CTU and / or its neighboring CTUs

[0305] vi. Indications of color format (such as 4:2:0, 4:4:4)

[0306] vii. Previous codec tree structure type for strips / slices / LCUs / LCU rows / LCU groups / video data units

[0307] viii. Strip / Piece Type and / or Image Type

[0308] ix. Time-domain layer ID

[0309] 11. Whether CCLM is enabled can depend on the codec tree structure type.

[0310] a. In one example, for a given codec tree structure type (e.g., dual tree), the indication of the CCLM mode and / or signaling notifications of other syntaxes associated with the CCLM mode can be skipped and inferred.

[0311] b. In one example, when a dual codec tree structure type is applied, the indication of CCLM mode can be inferred as false.

[0312] c. In one example, when applying a dual codec tree structure type, the indication of the CCLM mode can be inferred based on the following

[0313] i. Signaling notification messages in SPS / VPS / PPS / Image Header / Strip Header / Piece Group Header / LCU Line / LCU Group / LCU / CU / Video Data Unit.

[0314] ii. Hash hit rate of IBC / inter-frame modes in previous encoded / decoded images / slices / reconstructed regions

[0315] iii. Block dimensions of the current CTU and / or its neighboring CTUs

[0316] iv. Block shape of the current CTU and / or its neighboring CTUs

[0317] v. Current quantization parameters of the current CTU and / or its neighboring CTUs

[0318] vi. Indications of color format (such as 4:2:0, 4:4:4)

[0319] vii. Previous codec tree structure type for strips / slices / LCUs / LCU rows / LCU groups / video data units

[0320] viii. Strip / Piece Type and / or Image Type

[0321] ix. Time-domain layer ID

[0322] 12. Whether to enable LMCS for chroma components can depend on the codec tree structure type.

[0323] a. In one example, for a given codec tree structure type (e.g., dual tree), the LMCS indications for the chroma components and / or signaling notifications for other syntaxes related to the LMCS mode can be skipped and inferred.

[0324] b. In one example, when a dual codec tree structure type is applied, the LMCS indication of the chroma component can be inferred as false.

[0325] c. In one example, when applying a dual codec tree structure type, the indication of the LMCS of the chroma components can be inferred based on the following

[0326] i. Signaling notification messages in SPS / VPS / PPS / Image Header / Strip Header / Piece Group Header / LCU Line / LCU Group / LCU / CU / Video Data Unit.

[0327] ii. Hash hit rate of IBC / inter-frame modes in previous encoded / decoded images / slices / reconstructed regions

[0328] iii. Block dimensions of the current CTU and / or its neighboring CTUs

[0329] iv. Block shape of the current CTU and / or its neighboring CTUs

[0330] v. Current quantization parameters of the current CTU and / or its neighboring CTUs

[0331] vi. Indications of color format (such as 4:2:0, 4:4:4)

[0332] vii. Previous codec tree structure type for strips / slices / LCUs / LCU rows / LCU groups / video data units

[0333] viii. Strip / Piece Type and / or Image Type

[0334] ix. Time-domain layer ID

[0335] 13. The codec tree structure can depend on whether IBC is used.

[0336] a. In one example, the dual-tree structure and the IBC method can be enabled simultaneously at the sequence / picture / piece / brick / CTU / VPDU / 32x32 block / 64x32 block / 32x64 block level.

[0337] b. Alternatively, in one example, if the IBC method is enabled, the dual-tree structure can be disabled at the sequence / picture / piece / brick / CTU / VPDU / 32x32 block / 64x32 block / 32x64 block level.

[0338] c. In one example, when using IBC in a region, the chroma codec tree structure can be aligned with the luma codec tree structure.

[0339] i. In one example, the area could be a sequence / picture / piece / brick / CTU / VPDU / 32x32 block / 64x32 block / 32x64 block.

[0340] ii. In one example, when a juxtaposition of a luminance block is divided into sub-blocks, if the division of a chrominance block is allowed, then the chrominance block can be divided into sub-blocks.

[0341] iii. In one example, whether and how chroma blocks are divided can be inferred from the encoding / decoding structure of their juxtaposed luma blocks.

[0342] iv. In one example, when the chroma codec tree structure is inferred from the luma codec tree structure, signals that encode and decode the chroma codec tree structure can be skipped.

[0343] v. In one example, a flag could be used to indicate whether the chroma codec structure can be inferred from the luma codec structure. The signaling notification of the flag can depend on...

[0344] 1. Signaling notification messages in SPS / VPS / PPS / Image Header / Strip Header / Group Header / LCU Line / LCU Group / LCU / CU / Video Data Unit.

[0345] 2. Hash hit rate of IBC / inter-frame modes in previous encoded / decoded images / slices / reconstructed regions.

[0346] 3. Block dimensions of the current CTU and / or its neighboring CTUs.

[0347] 4. Block shape of the current CTU and / or its neighboring CTUs

[0348] 5. Current quantization parameters of the current CTU and / or its neighboring CTUs.

[0349] 6. Color format specifications (such as 4:2:0, 4:4:4)

[0350] 7. Previous codec tree structure types for strips / slices / LCUs / LCU rows / LCU groups / video data units

[0351] 8. Strip / Patch Type and / or Image Type

[0352] 9. Time-domain layer ID

[0353] 14. Whether to enable palette codec mode depends on the codec tree structure type.

[0354] a. In one example, for a given codec tree structure type (e.g., dual tree), signaling notifications indicating the palette codec mode can be skipped and inferred.

[0355] b. In one example, when a dual codec tree structure type is applied, the indication of the palette codec mode can be inferred as false.

[0356] c. In one example, when a dual codec tree structure type is applied, the indication of the palette codec mode for the luma block can be inferred as false.

[0357] d. In one example, when a dual codec tree structure type is applied, the indication of the palette codec mode for a chroma block can be inferred as false.

[0358] e. In one example, the inferred indication of the palette encoding / decoding mode could be based on

[0359] i. Signaling notification messages in SPS / VPS / PPS / Image Header / Strip Header / Piece Group Header / LCU Line / LCU Group / LCU / CU / Video Data Unit.

[0360] ii. Hash hit rate of IBC / inter-frame modes in previous encoded / decoded images / slices / reconstructed regions

[0361] iii. Block dimensions of the current CTU and / or its neighboring CTUs

[0362] iv. Block shape of the current CTU and / or its neighboring CTUs

[0363] v. Current quantization parameters of the current CTU and / or its neighboring CTUs

[0364] vi. Indications of color format (such as 4:2:0, 4:4:4)

[0365] vii. Previous codec tree structure type for strips / slices / LCUs / LCU rows / LCU groups / video data units

[0366] viii. Strip / Piece Type and / or Image Type

[0367] ix. Time-domain layer ID

[0368] 15. The codec tree structure can depend on whether a palette codec mode is used.

[0369] a. In one example, when using the palette codec mode in a region, the chroma codec tree structure can be aligned with the luminance codec tree structure.

[0370] i. In one example, the area could be a sequence / picture / piece / tile / CTU / VPDU / 32x32 block / 64x32 block

[0371] ii. In one example, when a juxtaposed luma block is divided into sub-blocks, a chroma block can be divided into sub-blocks if the chroma block is allowed to be divided.

[0372] iii. In one example, whether and how chroma blocks are divided can be inferred from the encoding / decoding structure of their juxtaposed luma blocks.

[0373] iv. In one example, when the chroma codec tree structure is inferred from the luma codec tree structure, signals that encode and decode the chroma codec tree structure can be skipped.

[0374] v. In one example, a flag could be used to indicate whether the chroma codec structure can be inferred from the luma codec structure. The signaling notification of the flag can depend on...

[0375] 1. Signaling notification messages in SPS / VPS / PPS / Image Header / Strip Header / Group Header / LCU Line / LCU Group / LCU / CU / Video Data Unit.

[0376] 2. Hash hit rate of IBC / inter-frame modes in previous encoded / decoded images / slices / reconstructed regions.

[0377] 3. Block dimensions of the current CTU and / or its neighboring CTUs.

[0378] 4. Block shape of the current CTU and / or its neighboring CTUs

[0379] 5. Current quantization parameters of the current CTU and / or its neighboring CTUs.

[0380] 6. Color format specifications (such as 4:2:0, 4:4:4)

[0381] 7. Previous codec tree structure types for strips / slices / LCUs / LCU rows / LCU groups / video data units

[0382] 8. Strip / Patch Type and / or Image Type

[0383] 9. Time-domain layer ID

[0384] 16. The motion / block vector of a sub-block / sample in a chroma IBC codec block can be derived from the first available IBC codec sub-region within the juxtaposed luma block.

[0385] a. In one example, the scanning order of sub-regions within a juxtaposed brightness block can be defined, such as the raster scan order.

[0386] b. In one example, a sub-region can be defined as the smallest decoding unit / minimum transform unit.

[0387] c. In one example, the motion / block vector of the entire sample in chroma IBC mode can be derived based on the motion vector of the top left sample in the juxtaposed luma block that is encoded or decoded in IBC or inter-frame mode.

[0388] 17. Motion / block vectors can be signaled in chroma IBC mode.

[0389] a. In one example, signaling can be used to notify the difference between the motion vector and the predicted motion vector value.

[0390] i. In one example, the motion vector prediction can be derived based on the motion vectors of the juxtaposed luma block, the neighboring luma blocks of the juxtaposed luma block, and the neighboring chroma blocks of the current chroma block.

[0391] 1. In one example, the motion / block vector prediction can be derived based on the motion vector of the top-left sample in the juxtaposed brightness block.

[0392] 2. In one example, the motion / block vector prediction can be derived based on the motion vector of a sample point with a center position in the juxtaposed brightness block.

[0393] 3. In one example, the motion / block vector prediction can be derived based on the motion vector of the top left sample in the juxtaposed luma block, encoded or decoded in IBC or inter-frame mode.

[0394] ii. In one example, the motion vector prediction associated with a sub-region of the luminance component can be scaled before being used as the prediction value.

[0395] iii. In one example, the block vector can be derived from the motion vector / block vector of neighboring (adjacent or non-adjacent) chroma blocks.

[0396] b. In one example, a block vector candidate list can be constructed, and the index of that list can be signaled.

[0397] i. In one example, the candidate list may include motion vectors / block vectors from juxtaposed luma blocks, neighboring luma blocks of juxtaposed luma blocks, and neighboring chroma blocks.

[0398] c. In one example, the indication of the AMVR flag can be inferred.

[0399] i. In one example, in a block encoded and decoded in chroma IBC mode, the indication of the AMVR flag can be inferred as false (0).

[0400] ii. In one example, within a block encoded and decoded in chroma IBC mode, the indication of motion vector difference can be inferred to integer precision.

[0401] d. In one example, a separate HMVP table can be used on the chroma IBC mode.

[0402] i. In one example, the chromaticity HMVP table and the luminance HMVP table can have different sizes.

[0403] e. In one example, can signaling notification blocks / motion vectors be based on chroma IBC mode?

[0404] i. Whether to encode and decode all sub-regions within the juxtaposed luma block in IBC mode.

[0405] 1. If so, signaling is not required to notify the block vector of the chroma block. Otherwise, signaling can be used to notify the block vector of the chroma block.

[0406] ii. Whether all sub-regions within the juxtaposed luma block are encoded and decoded in IBC mode, and whether all associated block vectors are valid.

[0407] 1. If so, signaling is not required to notify the block vector of the chroma block. Otherwise, signaling can be used to notify the block vector of the chroma block.

[0408] iii. Signaling notification messages in SPS / VPS / PPS / Image Header / Strip Header / Slice Group Header / LCU Line / LCU Group / LCU / CU / Video Data Unit.

[0409] iv. Hash hit rate of IBC / inter-frame modes in previous encoded / decoded images / slices / reconstructed regions

[0410] v. Block dimensions of the current CTU and / or its neighboring CTUs

[0411] vi. Block shape of the current CTU and / or its neighboring CTUs

[0412] vii. Current quantization parameters of the current CTU and / or its neighboring CTUs

[0413] viii. Indication of color format (such as 4:2:0, 4:4:4)

[0414] ix. Previous codec tree structure type for strips / slices / LCUs / LCU rows / LCU groups / video data units

[0415] x. Strip / group type and / or image type

[0416] xi. Time-domain layer ID

[0417] The examples described above can be used in the context of the methods described below, such as methods 900, 910, 920, 930, and 940, which can be implemented at a video decoder or video encoder.

[0418] An exemplary method for video processing includes performing a conversion between a current video block and a bitstream representation of the video including the current video block, wherein the conversion selectively uses a transform skip mode for the conversion based on an indicator included in the bitstream representation, and wherein, using the transform skip mode, the residual of the prediction error of the current video block is represented in the bitstream representation without applying a transform.

[0419] In some embodiments, the indicators are the maximum allowed width and maximum allowed height of the transformation skip mode.

[0420] In some embodiments, the maximum allowed width and height are signaled in the Sequence Parameter Set (SPS), Video Parameter Set (VPS), Picture Parameter Set (PPS), Picture Header, Strip Header, Slice Header, Maximum Codec Unit (LCU) Line, or LCU Group.

[0421] In some embodiments, the maximum allowed width and height are signaled in different messages.

[0422] In some embodiments, the maximum permissible width and height are signaled in the Sequence Parameter Set (SPS) or Picture Parameter Set (PPS), and the updated values ​​of the maximum permissible width and height are signaled in the picture header, strip header, slice header, maximum codec unit (LCU) line, or LCU group.

[0423] Figure 9A A flowchart of another exemplary method for video processing is shown. Method 900 includes, in step 902, determining that the current video block is encoded and decoded using a transform skip mode.

[0424] Method 900 includes, in step 904, performing a conversion between the current video block and a bitstream representation of the video including the current video block, based on the determination.

[0425] In some embodiments, the current video block is divided into multiple coefficient groups, and the bitstream represents signaling with the codec block flag of at least one of the multiple coefficient groups omitted. In the example, the bitstream represents signaling with the codec block flag of each of the multiple coefficient groups omitted.

[0426] In some embodiments, the codec block flags omitted in the signaling in the bitstream representation are inferred based on one or more of the following: (1) the message notified by signaling in the Sequence Parameter Set (SPS), Video Parameter Set (VPS), Picture Parameter Set (PPS), Picture Header, Strip Header, Slice Header, Maximum Codec Unit (LCU), LCU Line, LCU Group, or Codec Unit (CU); (2) the position of at least one coefficient group among a plurality of coefficient groups; (3) the block dimension of the current video block or at least one neighboring block of the current video block; (4) the block shape of the current video block or at least one neighboring block; and (5) the current video block. (6) The most likely mode of a frequency block or at least one neighboring block, (7) The prediction mode of at least one neighboring block, (8) One or more motion vectors of at least one neighboring block, (9) An indication of the quantization residual block differential pulse code modulation (QR-BDPCM) mode of at least one neighboring block, (10) The current quantization parameter (QP) of the current video block or at least one neighboring block, (11) An indication of the color format of the current video block, (12) The single or dual codec tree structure associated with the current video block, or (13) The stripe type, slice type or picture type of the current video block.

[0427] In some embodiments, the current video block is divided into multiple coefficient groups, and method 900 further includes the step of determining the coefficient scan order of the multiple coefficient groups. In an example, the coefficient scan order is based on a signaling notification message in a Sequence Parameter Set (SPS), Video Parameter Set (VPS), Picture Parameter Set (PPS), Picture Header, Strip Header, Slice Header, Maximum Codec Unit (LCU), LCU Line, LCU Group, or Codec Unit (CU).

[0428] In some embodiments, multiple coefficient groups or coefficient scan order are based on the intra-prediction mode of the current video block. In one example, the coefficient scan order is vertical, and the intra-prediction mode is horizontally dominant. In another example, the coefficient scan order is horizontal, and the intra-prediction mode is horizontally dominant. For example, the index of the intra-prediction mode ranges from 2 to 33 or from 2 to 34.

[0429] In some embodiments, multiple coefficient groups or coefficient scan order are based on the intra-prediction mode of the current video block. In one example, the coefficient scan order is vertical, and the intra-prediction mode is vertically dominant. In another example, the coefficient scan order is horizontal, and the intra-prediction mode is vertically dominant. For example, the index of the intra-prediction mode ranges from 34 to 66 or from 35 to 66.

[0430] In some embodiments, the context of a symbol flag is based on proximity information within coefficient blocks associated with the current video block. In one example, the context of a symbol flag is also based on the position of the coefficients in the coefficient block. In another example, the context of a symbol flag is based on (x+y), min(x,y), or max(x,y), where x and y are the horizontal and vertical values ​​of the coefficient's position, respectively.

[0431] Figure 9B A flowchart of yet another exemplary method for video processing is shown. Method 910 includes, in step 912, determining whether a chroma transformation skip mode is applicable for the current video block.

[0432] Method 910 includes, in step 914, performing a conversion between the current video block and a bitstream representation of the video including the current video block, based on the determination.

[0433] In some embodiments, the determination is based on a signaling notification message in a Sequence Parameter Set (SPS), Video Parameter Set (VPS), Picture Parameter Set (PPS), Picture Header, Strip Header, Slice Header, Maximum Codec Unit (LCU), LCU Line, LCU Group, Codec Unit (CU), or Video Data Unit.

[0434] In some embodiments, the determination is based on decoding information from one or more representative video blocks decoded prior to the conversion, and wherein the samples in each of the one or more representative video blocks and the current video block are based on common color information. In one example, the one or more representative video blocks include luma blocks or chroma blocks. In another example, the one or more representative video blocks include blocks juxtaposed within luma blocks.

[0435] Figure 9C A flowchart of yet another exemplary method for video processing is shown. Method 920 includes, in step 922, making a decision regarding the selective application of a conditional transform skip mode to the current video block during the conversion between the current video block and a bitstream representation of the video including the current video block.

[0436] Method 920 includes performing a conversion based on the decision in step 924.

[0437] In some embodiments, the condition is based on a signaling notification message in a Sequence Parameter Set (SPS), Video Parameter Set (VPS), Picture Parameter Set (PPS), Picture Header, Strip Header, Slice Header, Maximum Codec Unit (LCU), LCU Line, LCU Group, Codec Unit (CU), or Video Data Unit.

[0438] In some embodiments, the condition is based on one or more of the following: (1) the block dimension of the current video block or at least one neighboring block of the current video block, (2) the block shape of the current video block or at least one neighboring block, (3) the most likely pattern of the current video block or at least one neighboring block, (4) the prediction pattern of at least one neighboring block, (5) the intra-prediction pattern of at least one neighboring block, (6) one or more motion vectors of at least one neighboring block, (7) an indication of the quantization residual block differential pulse code modulation (QR-BDPCM) mode of at least one neighboring block, (8) the current quantization parameter (QP) of the current video block or at least one neighboring block, (9) an indication of the color format of the current video block, (10) the single or dual codec tree structure associated with the current video block, (11) the stripe type, slice type or picture type of the current video block, or (12) the temporal layer identifier (ID).

[0439] In some embodiments, a transform skip mode is applied, the prediction mode for the current video block is inter-block copy (IBC) mode, and the width or height of the current video block is compared with a threshold. In the example, the threshold is signaled in the bitstream representation. In another example, the threshold is 4, 8, 16, or 32.

[0440] In another example, the threshold is based on one or more of the following: (1) the message signaled in the Sequence Parameter Set (SPS), Video Parameter Set (VPS), Picture Parameter Set (PPS), Picture Header, Strip Header, Slice Header, Maximum Codec Unit (LCU), LCU Line, LCU Group, or Codec Unit (CU); (2) Temporal Layer Identifier (ID); (3) the block dimension of the current video block or at least one neighboring block of the current video block; (4) the block shape of the current video block or at least one neighboring block; and (5) the most probable modulus of the current video block or at least one neighboring block. (6) Prediction mode of at least one neighboring block, (7) Intra-prediction mode of at least one neighboring block, (8) One or more motion vectors of at least one neighboring block, (9) Indication of Quantization Residual Block Differential Pulse Code Modulation (QR-BDPCM) mode of at least one neighboring block, (10) Current Quantization Parameter (QP) of the current video block or at least one neighboring block, (11) Indication of color format of the current video block, (12) Single or dual codec tree structure associated with the current video block, or (13) Strip type, slice type or picture type of the current video block.

[0441] Figure 9D A flowchart of yet another exemplary method for video processing is shown. Method 930 includes, in step 932, making a decision regarding the selective application of quantized residual block differential pulse code modulation (QR-BDPCM) based on an indication of a transform skip mode in the bitstream representation during a conversion between a current video block and a bitstream representation of the video including the current video block.

[0442] Method 930 includes performing a conversion based on the decision in step 934.

[0443] In some embodiments, the indication of a transform skip mode is signaled at the transform unit (TU) level.

[0444] Figure 9E A flowchart of yet another exemplary method for video processing is shown. Method 940 includes, in step 942, making a decision regarding the selective application of a condition-based single tree or dual tree during the conversion between the current video block and a bitstream representation of the video including the current video block.

[0445] Method 940 includes performing a conversion based on the decision in step 944.

[0446] In some embodiments, the condition is based on a signaling notification message in a Sequence Parameter Set (SPS), Video Parameter Set (VPS), Picture Parameter Set (PPS), Picture Header, Strip Header, Slice Header, Maximum Codec Unit (LCU), LCU Line, LCU Group, Codec Unit (CU), or Video Data Unit.

[0447] In some embodiments, the condition is based on determining whether a strip, slice, maximum codec unit (LCU), LCU line, LCU group, or video data unit including the current video block is screen content. In an example, the determination is based on one or more of the following: (1) a signaling notification message in a sequence parameter set (SPS), video parameter set (VPS), picture parameter set (PPS), picture header, strip header, slice group header, LCU, LCU line, LCU group, codec unit (CU), or video data unit; (2) the block dimension of the current video block or at least one neighboring block of the current video block; (3) the block shape of the current video block or at least one neighboring block; (4) the current quantization parameter (QP) of the current video block or at least one neighboring block; (5) an indication of the color format of the current video block; (6) the single or dual codec tree structure associated with the current video block; (7) the stripe type, slice group type, or picture type of the current video block; or (8) the temporal layer identifier (ID).

[0448] Figure 10This is a block diagram of a video processing device 1000. The device 1000 can be used to implement one or more methods described herein. The device 1000 can be embodied in a smartphone, tablet, computer, Internet of Things (IoT) receiver, etc. The device 1000 may include one or more processors 1002, one or more memories 1004, and video processing hardware 1006. The processors (multiple) 1002 can be configured to implement one or more methods described in this document (including, but not limited to, methods 900, 910, 920, 930, and 940). The memories (multiple memories) 1004 can be used to store data and code for implementing the methods and techniques described herein. The video processing hardware 1006 can be used to implement some of the techniques described in this document in hardware circuitry.

[0449] In some embodiments, the video encoding / decoding method can be used as described in reference 1. Figure 10 The device is implemented on the aforementioned hardware platform.

[0450] In some embodiments, such as as described in items 5 and 10 above and below, the video processing method includes: determining, based on the type of codec tree structure corresponding to the current video block, whether an intra-block copy mode is applicable to the conversion between the current video block and the bitstream representation of the video; and performing the conversion based on the determination.

[0451] In the above method, the bitstream indicates the exclusion of intra-block copy mode. In other words, the bitstream does not carry explicit signaling for IBC mode.

[0452] In the above method, the codec tree structure is a dual codec tree structure, and this determination is not applicable to intra-block copy mode.

[0453] Figure 11 This is a block diagram of an example video processing system in which the disclosed techniques can be implemented. Various implementations may include some or all of the components of system 1100. System 1100 may include an input 1102 for receiving video content. The video content may be received in a raw or uncompressed format, such as 8 or 10-bit multi-component pixel values, or it may be in a compressed or encoded format. Input 1102 may identify a network interface, a peripheral bus interface, or a storage interface. Examples of network interfaces include wired interfaces such as Ethernet, Passive Optical Network (PON), etc., and wireless interfaces such as Wi-Fi or cellular interfaces.

[0454] System 1100 may include codec component 1104, which may implement the various codec or encoding methods described in this document. Codec component 1104 may reduce the average bit rate of the video from input 1102 to the output of codec component 1104 to produce a codec representation of the video. Therefore, codec techniques are sometimes referred to as video compression or video codec techniques. The output of codec component 1104 may be stored or transmitted via connected communication, as represented by component 1106. Component 1108 may use the stored or transmitted bitstream (or codec) representation of the video received at input 1102 to generate pixel values ​​or displayable video sent to display interface 1110. The process of generating user-visible video from the bitstream representation is sometimes referred to as video decompression. Furthermore, although some video processing operations are referred to as “codec” operations or tools, it should be understood that codec tools or operations are used at the encoder, and the corresponding decoding tools or operations that reverse the encoded result will be performed by the decoder.

[0455] Examples of peripheral bus interfaces or display interfaces may include Universal Serial Bus (USB), High Definition Multimedia Interface (HDMI), or display ports. Examples of storage interfaces include SATA (Serial Advanced Technology Attachment), PCI, IDE, etc. The technologies described in this document can be found in a variety of electronic devices, such as mobile phones, laptops, smartphones, or other devices capable of performing digital data processing and / or video display.

[0456] Figure 12 This is a flowchart illustrating an example of a visual media encoding method. The steps of this flowchart will be discussed in conjunction with Example 11 in Section 4 of this document. In step 1202, the process, for encoding a current video block in a video region of visual media data into a bitstream representation of the visual media data, makes a decision regarding the application of a cross-component linear model (CCLM) prediction mode, at least based on or inferred from the codec tree structure associated with the video region, wherein, in the CCLM prediction mode, the chroma samples of the current video block are predicted using a linear model on the reconstructed luma samples of the current video block. In step 1204, the process selectively includes a syntax element in the bitstream representation, wherein the syntax element indicates the application of a CCLM prediction mode, at least based on or inferred from the codec tree structure.

[0457] Figure 13This is a flowchart illustrating an example of a visual media decoding method. The steps of this flowchart will be discussed in conjunction with Example 11 in Section 4 of this document. In step 1302, the process determines the codec tree structure associated with the current video block from a bitstream representation comprising visual media data of a video region containing the current video block. In step 1304, the process determines, at least based on the codec tree structure, whether syntax elements are included in the bitstream representation, wherein the syntax elements indicate the application of a cross-component linear model (CCLM) prediction mode to the current video region, wherein, under the CCLM prediction mode, a linear model is used to predict the chroma samples of the current video block on the reconstructed luma samples. In step 1306, the process generates the current video block from the bitstream representation.

[0458] Figure 14 This is a flowchart illustrating an example of a visual media encoding method. The steps of this flowchart will be discussed in conjunction with Example 12 in Section 4 of this document. In step 1402, the process makes a decision regarding the application of a Luminosity Mapping and Chroma Scaling (LMCS) processing step to the video region, at least based on or inferred from the codec tree structure associated with the video region, to encode the current video block in the video region of the visual media data into a bitstream representation of the visual media data. In the LMCS processing step, luminosity samples in the video region are mapped using an adaptive piecewise linear model, and / or luminosity-dependent chroma residual scaling is performed on the chroma samples in the video region. In step 1404, the process selectively includes a syntax element in the bitstream representation that indicates the application of the LMCS processing step to the video region.

[0459] Figure 15 This is a flowchart illustrating an example of a visual media decoding method. The steps of this flowchart will be discussed in conjunction with Example 12 in Section 4 of this document. In step 1502, the process determines the codec tree structure associated with the current video block from a bitstream representation comprising visual media data of a video region containing the current video block. In step 1504, the process determines, at least based on the codec tree structure, whether syntax elements are included in the bitstream representation, wherein the syntax elements indicate the application of a Luminosity Mapping and Chroma Scaling (LMCS) processing step to the video region, wherein in the LMCS processing step, luminosity samples are mapped using an adaptive piecewise linear model, and luminosity-dependent chroma residual scaling is performed on the chroma samples. In step 1506, the process generates the current video block from the bitstream representation.

[0460] Some of the embodiments in this document are now presented in a terms-based format.

[0461] 1. A method for visual media encoding, comprising:

[0462] To encode a current video block within a video region of visual media data into a bitstream representation of the visual media data, a decision is made regarding the application of a cross-component linear model (CCLM) prediction mode, based at least on or inferred from the codec tree structure associated with the video region, wherein, under the CCLM prediction mode, a linear model is used to predict the chroma samples of the current video block on the reconstructed luminance samples; and

[0463] The bitstream representation may selectively include syntax elements that indicate the application of a CCLM prediction mode that is at least based on or at least inferred from the codec tree structure.

[0464] 2. A method for decoding visual media, comprising:

[0465] The codec tree structure associated with the current video block is determined from the bitstream representation of the visual media data, which includes the video region containing the current video block.

[0466] At least based on the codec tree structure, determine whether a syntax element is included in the bitstream representation, wherein the syntax element indicates the application of a cross-component linear model (CCLM) prediction mode to the current video region, wherein, under CCLM prediction mode, a linear model is used to predict the chroma samples of the current video block on the reconstructed luma samples of the current video block; and

[0467] Generate the current video block from the bitstream representation.

[0468] 3. The method according to any one or more of clauses 1-2, wherein, when syntactic elements are excluded in the bitstream representation, the syntactic elements indicating the application of the CCLM prediction mode are inferred.

[0469] 4. The method described in any one or more of Clauses 1-3, wherein, in the case that the codec tree structure is a dual codec tree structure type, the syntax element is inferred as a Boolean false value.

[0470] 5. The method according to any one or more of clauses 1-4, wherein, in the case that the codec tree structure is a dual codec tree structure type, syntax elements are excluded from the bitstream representation and inferred based on the block dimension of the current video block or at least one of the neighboring blocks of the current video block.

[0471] 6. The method according to any one or more of clauses 1-4, wherein, in the case that the codec tree structure is a dual codec tree structure type, syntax elements are excluded from the bitstream representation and inferred based on one or more of the following:

[0472] (1) Messages notified by signaling in the Sequence Parameter Set (SPS), Video Parameter Set (VPS), Picture Parameter Set (PPS), Picture Header, Strip Header, Slice Header, Maximum Codec Unit (LCU), LCU Line, LCU Group, or Codec Unit (CU).

[0473] (2) Hash hit rate of IBC mode and / or inter-frame mode in codec images / slices / strips / reconstruction regions associated with other video blocks.

[0474] (3) The block shape of the current video block or at least one neighboring block.

[0475] (4) The current quantization parameter (QP) of the current video block or at least one neighboring block.

[0476] (5) Indication of the color format of the current video block.

[0477] (6) A single or dual codec tree structure associated with the current video block,

[0478] (7) The current video block's strip type, segment type, or image type, or

[0479] (8) Time-domain layer ID.

[0480] 7. A method for visual media encoding, comprising:

[0481] Making a decision regarding the application of a Luminosity Mapping and Chroma Scaling (LMCS) processing step to a video region, based at least on or inferred from the codec tree structure associated with the video region, for encoding the current video block in the video region of the visual media data into a bitstream representation of the visual media data, wherein, in the LMCS processing step, an adaptive piecewise linear model is used to map luminosity samples in the video region and / or a luminosity-dependent chroma residual scaling operation is performed on chroma samples in the video region; and

[0482] The bitstream representation may selectively include syntax elements that indicate the application of LMCS processing steps to video regions.

[0483] 8. A method for decoding visual media, comprising:

[0484] The codec tree structure associated with the current video block is determined from the bitstream representation of the visual media data, which includes the video region containing the current video block.

[0485] At least based on the codec tree structure, determine whether a syntax element is included in the bitstream representation, wherein the syntax element indicates the application of a Luminosity Mapping and Chroma Scaling (LMCS) processing step to a video region, wherein in the LMCS processing step, an adaptive piecewise linear model is used to map luminosity samples, and luminosity-dependent chroma residual scaling is performed on the chroma samples; and

[0486] Generate the current video block from the bitstream representation.

[0487] 9. The method according to any one or more of Clauses 7-8, wherein, when syntactic elements are excluded in the bitstream representation, the syntactic elements indicating the application of the LMCS processing steps are inferred.

[0488] 10. The method described in any one or more of Clauses 7-9, wherein, in the case that the codec tree structure is a dual codec tree structure type, the syntax element is inferred as a Boolean false value.

[0489] 11. The method according to any one or more of clauses 7-10, wherein, in the case that the codec tree structure is a dual codec tree structure type, syntax elements are excluded from the bitstream representation and inferred based on one or more of the following:

[0490] (1) Messages notified by signaling in the Sequence Parameter Set (SPS), Video Parameter Set (VPS), Picture Parameter Set (PPS), Picture Header, Strip Header, Slice Header, Maximum Codec Unit (LCU), LCU Line, LCU Group, or Codec Unit (CU).

[0491] (2) Hash hit rate of IBC mode and / or inter-frame mode in codec images / slices / strips / reconstruction regions associated with other video blocks.

[0492] (3) The block dimension of the current video block or at least one of its neighboring blocks.

[0493] (4) The block shape of the current video block or at least one neighboring block.

[0494] (5) The current quantization parameter (QP) of the current video block or at least one neighboring block.

[0495] (6) Indication of the color format of the current video block.

[0496] (7) A single or dual codec tree structure associated with the current video block.

[0497] (8) The current video block's strip type, segment type, or image type, or

[0498] (9) Time-domain layer ID.

[0499] 12. The method described in any one or more of Clauses 6 or 11, wherein the color format of the current video block is 4:2:0 or 4:4:4.

[0500] 13. A video encoder apparatus comprising a processor configured to implement the methods described in any one or more of clauses 1-12.

[0501] 14. A video decoder apparatus comprising a processor configured to implement the methods described in any one or more of clauses 1-12.

[0502] 15. A computer-readable medium having code stored thereon, the code embodying processor-executable instructions for implementing the methods described in any one or more of clauses 1-12.

[0503] In this document, the terms "video processing" or "visual media processing" can refer to video encoding, video decoding, video compression, or video decompression. For example, a video compression algorithm may be applied during the conversion from the pixel representation of the video to the corresponding bitstream representation, or vice versa. As defined by the syntax, the bitstream representation of the current video block may, for example, correspond to bits juxtaposed within the bitstream or bits distributed at different positions within the bitstream. For example, a macroblock may be encoded based on the error residual values ​​of the transformation and encoding / decoding, and also using bits in the header and other fields in the bitstream. Furthermore, during the conversion, the decoder may, based on this determination, parse the bitstream knowing that some fields may or may not be present, as described in the above technical solutions. Similarly, the encoder may determine whether certain syntax fields are included and generate the codec representation accordingly by including or excluding syntax fields from the encoded representation.

[0504] From the foregoing, it will be understood that specific embodiments of the currently disclosed technology have been described for illustrative purposes, but various modifications can be made without departing from the scope of the invention. Therefore, the technology disclosed herein is not limited except by the appended claims.

[0505] The embodiments of the subject matter and functional operation described in this patent document can be implemented in various systems, digital electronic circuits, or in computer software, firmware, or hardware including the structures disclosed in this specification and their equivalents, or in combinations thereof. Embodiments of the subject matter described in this specification can be implemented as one or more computer program products, i.e., one or more modules of computer program instructions encoded on a tangible, non-transitory, computer-readable storage medium for execution by a data processing apparatus or for controlling the operation of a data processing apparatus. The computer-readable medium can be a machine-readable storage device, a machine-readable storage substrate, a memory device, a combination of materials that implement machine-readable propagation signals, or a combination thereof. The terms "data processing unit" or "data processing apparatus" encompass all means, devices, and machines for processing data, including, for example, a programmable processor, a computer, or multiple processors or computers. In addition to hardware, the apparatus may also include code that creates an operating environment for the computer program in question, for example, code constituting processor firmware, a protocol stack, a database management system, an operating system, or a combination thereof.

[0506] Computer programs (also known as programs, software, software applications, scripts, or code) can be written in any programming language, including compiled or interpreted languages, and can be deployed in any form, including as standalone programs or as modules, components, subroutines, or other units suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a document system. A program may be stored as a portion of a file containing other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple coordinating files (e.g., a file storing one or more modules, subroutines, or portions of code). Computer programs can be deployed to execute on one or more computers located at a single site or distributed across multiple sites interconnected by a communication network.

[0507] The processes and logic flows described in this specification can be executed by one or more programmable processors, which run one or more computer programs to perform functions by manipulating input data and generating output. The processes and logic flows can also be executed by dedicated logic circuitry, and the device can be implemented as dedicated logic circuitry, such as an FPGA (field programmable gate array) or an ASIC (application specific integrated circuit).

[0508] For example, processors suitable for executing computer programs include general-purpose and special-purpose microprocessors, as well as any one or more processors in any type of digital computer. Typically, a processor receives instructions and data from read-only memory or random access memory, or both. The basic components of a computer are a processor for executing instructions and one or more memory devices for storing instructions and data. Typically, a computer will also include one or more mass storage devices for storing data, such as magnetic disks, magneto-optical disks, or optical disks, or operatively coupled to receive data from or transfer data to one or more mass storage devices, or both. However, a computer does not necessarily have to have such devices. Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and storage devices, including, for example, semiconductor storage devices, such as programmable read-only memory (EPROM), programmable read-only memory (EEPROM), and flash memory devices. The processor and memory may be supplemented or incorporated into special-purpose logic circuitry.

[0509] The description and accompanying drawings are to be considered exemplary only, where exemplary means example. As used herein, the singular forms “a,” “an,” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. Furthermore, the use of “or” is intended to include “and / or”, unless the context clearly indicates otherwise.

[0510] While this patent document contains numerous details, these should not be construed as limiting any invention or the scope of the claims, but rather as descriptions of features characteristic of specific embodiments of a particular invention. Certain features described in the context of individual embodiments in this patent document may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments. Furthermore, although the foregoing features may be described as functioning in certain combinations, or even initially claimed in this manner, in some cases one or more features may be removed from the claimed combination, and the claimed combination may refer to a sub-combination or a variation of a sub-combination.

[0511] Similarly, although operations are described in a specific order in the accompanying drawings, this should not be construed as requiring these operations to be performed in the specific order or sequence shown, or requiring all of the shown operations to obtain the desired result. Furthermore, the separation of various system components in the embodiments described in this patent document should not be construed as requiring such separation in all embodiments.

[0512] Only a few implementation methods and examples are described, and other implementation methods, enhancements and variations can be made based on the content described and illustrated in this patent document.

Claims

1. A method for visual media encoding, comprising: To encode a current video block within a video region of visual media data into a bitstream representation of the visual media data, a decision is made regarding the application of a cross-component linear model (CCLM) prediction mode, based at least on or inferred from the codec tree structure associated with the video region, wherein, under the CCLM prediction mode, a linear model is used to predict the chroma samples of the current video block on the reconstructed luminance samples; and The bitstream representation selectively includes syntax elements, wherein the syntax elements indicate the application of a CCLM prediction mode based at least on or at least inferred from the codec tree structure; Wherein, if the syntax element is excluded from the bitstream representation, the syntax element indicating the application of the CCLM prediction mode is inferred; In the case where the codec tree structure is a dual codec tree structure type, the syntax elements are excluded from the bitstream representation.

2. The method according to claim 1, wherein, When the codec tree structure is a dual codec tree structure type, the syntax element is inferred as a Boolean false value.

3. The method according to claim 1, wherein, When the codec tree structure is a dual codec tree structure type, syntax elements are excluded from the bitstream representation and inferred based on the block dimension of the current video block or at least one of its neighboring blocks.

4. The method according to claim 1, wherein, When the codec tree structure is a dual codec tree structure type, syntax elements are excluded from the bitstream representation and inferred based on one or more of the following: (1) Messages signaled in the Sequence Parameter Set (SPS), Video Parameter Set (VPS), Picture Parameter Set (PPS), Picture Header, Strip Header, Slice Header, Maximum Codec Unit (LCU), LCU Line, LCU Group, or Codec Unit (CU). (2) Hash hit rate of IBC mode and / or inter-frame mode in codec images / slices / strips / reconstruction regions associated with other video blocks, (3) The block shape of the current video block or at least one neighboring block, (4) The current quantization parameter (QP) of the current video block or at least one neighboring block. (5) Indication of the color format of the current video block, (6) A single or dual codec tree structure associated with the current video block, (7) The current video block's strip type, segment type, or image type, or (8) Time-domain layer ID.

5. A method for decoding visual media, comprising: The codec tree structure associated with the current video block is determined from the bitstream representation of the visual media data, which includes the video region containing the current video block. At least based on the codec tree structure, determine whether a syntax element is included in the bitstream representation, wherein the syntax element indicates the application of a cross-component linear model (CCLM) prediction mode to the current video region, wherein, under CCLM prediction mode, a linear model is used to predict the chroma samples of the current video block on the reconstructed luma samples of the current video block; and Generate the current video block from the bitstream representation; In this case, when syntax elements are excluded from the bitstream representation, the syntax elements indicating the application of the CCLM prediction mode are inferred; In the case where the codec tree structure is a dual codec tree structure type, the syntax elements are excluded from the bitstream representation.

6. The method according to claim 5, wherein, When the codec tree structure is a dual codec tree structure type, the syntax element is inferred as a Boolean false value.

7. The method according to claim 5, wherein, When the codec tree structure is a dual codec tree structure type, syntax elements are excluded from the bitstream representation and inferred based on the block dimension of the current video block or at least one of its neighboring blocks.

8. The method according to claim 5, wherein, When the codec tree structure is a dual codec tree structure type, syntax elements are excluded from the bitstream representation and inferred based on one or more of the following: (1) Messages signaled in the Sequence Parameter Set (SPS), Video Parameter Set (VPS), Picture Parameter Set (PPS), Picture Header, Strip Header, Slice Header, Maximum Codec Unit (LCU), LCU Line, LCU Group, or Codec Unit (CU). (2) Hash hit rate of IBC mode and / or inter-frame mode in codec images / slices / strips / reconstruction regions associated with other video blocks, (3) The block shape of the current video block or at least one neighboring block, (4) The current quantization parameter (QP) of the current video block or at least one neighboring block. (5) Indication of the color format of the current video block, (6) A single or dual codec tree structure associated with the current video block, (7) The current video block's strip type, segment type, or image type, or (8) Time-domain layer ID.

9. A video encoder device comprising a processor configured to implement the method according to any one or more of claims 1-8.

10. A video decoder apparatus comprising a processor configured to implement the method according to any one or more of claims 1-8.

11. A computer-readable medium having code stored thereon, the code embodying processor-executable instructions for carrying out the methods according to any one or more of claims 1-8.

Images