Base size constraint for chroma intra mode
By adjusting chroma block constraints and prediction modes based on chroma formats, the solution addresses inefficiencies in handling small chroma blocks, enhancing coding efficiency and hardware suitability in video coding standards.
Patent Information
- Application Number
- JP2023130792
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Priority Date
- 2019-09-27
- Filing Date
- 2023-08-10
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2040-09-21
AI Technical Summary
Current video coding standards face inefficiencies in handling small chroma blocks, particularly in screen content coding, due to constraints that limit coding efficiency and hardware implementation suitability, and do not account for different chroma formats effectively.
The proposed solution involves modifying the constraints on chroma block sizes and prediction modes based on chroma formats, allowing flexible partitioning and prediction modes for small chroma blocks, and optimizing encoding processes to improve coding efficiency and hardware compatibility.
This approach enhances coding efficiency by allowing more flexible handling of small chroma blocks, improving compression performance, and making the encoding process more suitable for hardware implementation across various chroma formats.
Smart Images

Figure 0007581447000057 
Figure 0007581447000058 
Figure 0007581447000059
Abstract
Description
[Technical field]
[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application is a divisional application of International Patent Application No. PCT / CN2020 / 116472, filed on September 21, 2020, which is a divisional application of International Patent Application No. PCT / CN2019 / 107139, filed on September 21, 2019, and International Patent Application No. PCT / CN2019 / 108760, filed on September 27, 2019. The entire disclosures of the aforementioned applications are incorporated by reference as part of the disclosure of this application.
[0002] Technical Field This document concerns video and image encoding and decoding technologies. [Background technology]
[0003] Digital video accounts for the largest bandwidth usage on the Internet and other digital communications networks. Bandwidth demands for digital video usage are expected to continue to increase as the number of connected user devices capable of receiving and displaying video increases. Summary of the Invention [Means for solving the problem]
[0004] The disclosed techniques may be used by embodiments of video or image decoders or encoders in which reference images are used in video encoding or decoding.
[0005] In one example aspect, a method of video processing is disclosed that includes performing a conversion between a chroma block of a video region of a video picture of a video and a coded representation of the video according to rules, the rules specifying that the chroma block is not allowed to be represented in the coded representation using an intra-coding mode because the chroma block has a size M×N, where M and N are integers indicating a width and a height of the chroma block, respectively, and the intra-coding mode includes coding the chroma block based on a previously coded video region of the video picture.
[0006] In another exemplary aspect, another method of video processing is disclosed that includes performing a conversion between chroma blocks of a video region of a video picture of a video and a coded representation of the video according to rules, the rules specifying that the chroma blocks having a size M×N are not allowed to be represented in the coded representation using an intra coding mode when a dual tree partitioning is used, where M and N are integers indicating a width and a height of the chroma blocks, respectively, and the intra coding mode includes coding the chroma blocks based on a previously coded video region of the video picture.
[0007] In yet another exemplary aspect, another method of video processing is disclosed that includes performing a conversion between chroma blocks of a video region of a video picture of a video and a coded representation of the video according to rules, the rules specifying that the chroma blocks having a size M×N are not allowed to be represented in the coded representation using an intra coding mode when single-tree partitioning is used, where M and N are integers indicating a width and a height of the chroma blocks, respectively, and the intra coding mode includes coding the chroma blocks based on a previously coded video region of the video picture.
[0008] In yet another exemplary aspect, another method of video processing is disclosed, the method including performing a conversion between chroma blocks of a video region of a video and a coded representation of the video according to rules, the rules specifying that the chroma blocks having a size M×N are not allowed to be represented in the coded representation using a combined inter- and intra-prediction (CIIP) mode that combines inter-prediction and intra-prediction, where M and N are integers indicating a width and a height of the chroma blocks, respectively.
[0009] In yet another exemplary aspect, another method of video processing is disclosed, the method including performing a transformation between a video region of a video and a coded representation of the video, the rules specifying whether and how a maximum transform size applicable to the video region is signaled depends on a coding mode type used to represent the video region in the coded representation, the coding mode type being (i) a near-lossless coding mode in which a quantization parameter (QP) applied to the video region is less than a threshold, (ii) a lossless coding mode in which transformation of residual data for the video region is bypassed, or (iii) a lossy coding mode in which transformation of residual data for the video region is skipped.
[0010] In yet another exemplary aspect, the above-described method may be implemented by a video encoder device comprising a processor.
[0011] In yet another exemplary aspect, the above-described methods may be implemented by a video decoder device comprising a processor.
[0012] In yet another exemplary aspect, the methods may be embodied in processor-executable instructions and stored on a computer-readable program medium.
[0013] These and other aspects are described further in this document. [Brief description of the drawings]
[0014] [Figure 1] 1 shows an example of an intra block copying encoding tool.
[0015] [Diagram 2] An example of a block coded in palette mode is shown below.
[0016] [Diagram 3] 13 illustrates an example of using a palette predictor to signal palette items.
[0017] [Figure 4] 4 shows an example of horizontal and vertical traverse scans.
[0018] [Diagram 5] An example of palette index encoding is shown below.
[0019] [Figure 6] 6 shows an example of an intra prediction mode of 67.
[0020] [Figure 7] Examples of the left and top neighbours of the current block are shown.
[0021] [Figure 8] Examples of ALF filter shapes (chroma: 5x5 diamond, luma: 7x7 diamond) are shown.
[0022] [Figure 9] 13 shows an example of a subsampled Laplacian computation.
[0023] [Figure 10] 13 shows an example of modified block classification at a virtual boundary.
[0024] [Figure 11] 13 is an example illustration of modified ALF filtering for the luma component at the virtual boundary.
[0025] [Figure 12] Four examples of 1-D 3-pixel patterns for pixel classification in EO are shown.
[0026] [Figure 13] Shows that the four bands are grouped together and represented by their starting band positions.
[0027] [Figure 14] The top and left neighboring blocks used in the CIIP weight derivation are shown.
[0028] [Figure 15] 1 illustrates luma mapping with chroma scaling architecture.
[0029] [Figure 16] Here is an example of SCIPU.
[0030] [Figure 17A] FIG. 1 is a block diagram of an example hardware platform that may be used to implement the techniques described herein. [Figure 17B] FIG. 1 is a block diagram of an example hardware platform that may be used to implement the techniques described herein.
[0031] [Figure 18] 1 is a flow chart for an exemplary method of video processing.
[0032] [Figure 19] 13 shows examples of spatial merge candidate locations.
[0033] [Figure 20] 13 shows examples of candidate pairs considered for redundancy check of spatial merge candidates.
[0034] [Figure 21] 1 shows a flow chart for an exemplary method of video processing.
[0035] [Figure 22] 1 is a block diagram illustrating an example video encoding system.
[0036] [Figure 23] 1 is a block diagram illustrating an encoder according to some embodiments of the present disclosure.
[0037] [Figure 24] 2 is a block diagram illustrating a decoder according to some embodiments of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0038] This document provides various techniques that may be used by a decoder of an image or video bitstream to improve the quality of the decompressed or decoded digital video or image. For brevity, the term "video" is used herein to include both sequences of pictures (traditionally called videos) and individual images. Additionally, video encoders may also implement these techniques during the encoding process to reconstruct the decoded frames that are used for further encoding.
[0039] Section headings are used herein for ease of understanding and do not limit the embodiments and techniques to the corresponding section, such that embodiments from one section can be combined with embodiments from other sections.
[0040] 1. Overview
[0041] This paper relates to video coding technology, specifically palette coding using base colors based representation in video coding. It can be applied to existing video coding standards such as HEVC or to a yet to be finalized standard (Versatile Video Coding). It can also be applied to future video coding standards or video codecs.
[0042] 2. Early discussions
[0043] Video coding standards have evolved primarily through the development of well-known ITU-T and ISO / IEC standards. ITU-T produced H.261 and H.263, ISO / IEC produced MPEG-1 and MPEG-4 Visual, and both organizations jointly produced H.262 / MPEG-2 Video, H.264 / MPEG-4 Advanced Video Coding (AVC), and H.265 / HEVC standards. Since H.262, video coding standards have been based on a hybrid video coding structure in which temporal prediction and transform coding are utilized. In 2015, VCEG and MPEG jointly established the Joint Video Exploration Team (JVET) to explore future video coding technologies beyond HEVC. Since then, many new methods have been adopted by JVET and put into reference software named the Joint Exploration Model (JEM). In April 2018, the Joint Video Experts Team (JVET) between VCEG (Q6 / 16) and ISO / IEC JTC1 SC29 / WG11 (MPEG) was launched to work on the VVC standard, which aims to achieve a 50% bitrate reduction compared to HEVC.
[0044] 2.1 Intra-block copy
[0045] HEVC Screen Content Coding extensions (HEVC-SCC) and the current VVC Test Model (VTM-4.0) now employ Intra Block Copy (IBC), also known as picture referencing. IBC extends the concept of motion compensation from inter-frame coding to intra-frame coding. As shown in Figure 1, when IBC is applied, the current block is predicted by a reference block in the same picture. Samples in the reference block must already be reconstructed before the current block is coded or decoded. IBC is not very efficient for most camera-captured sequences, but shows significant coding gain for screen content. The reason is that there are many repeating patterns, such as icons and text characters, in screen content pictures. IBC can effectively remove redundancy between these repeating patterns. In HEVC-SCC, an inter-coded coding unit (CU) can apply IBC if it selects the current picture as its reference picture. In this case, MV is renamed to block vector (BV), and BV always has integer pixel precision. To be compatible with the Main Profile HEVC, the current picture is marked as a "long-term" reference picture in the Decoded Picture Buffer (DPB). It should be noted that in the Multiview / 3D Video Coding standard, inter-view reference pictures are also marked as "long-term" reference pictures.
[0046] After traversing the BV to find the reference block, a prediction can be generated by copying the reference block. The residual is obtained by subtracting the reference pixel from the original signal. Then, transformation and quantization can be applied as in other coding modes.
[0047] FIG. 1 is an illustration of intra-block copying.
[0048] However, if the reference block is outside the picture, or overlaps with the current block, or is outside the reconstructed area, or is outside the valid area constrained by some constraints, some or all pixel values are undefined. To deal with such problems, there are basically two solutions. One is to not allow such situations, for example in bitstream conformance. The other is to apply padding for those undefined pixel values. The following sub-sections will explain these solutions in detail.
[0049] 2.2 IBC in HEVC Screen Content Coding Extension In the screen content coding extension of HEVC, when a block uses the current picture as a reference, it should be ensured that the entire reference block is within the available reconstruction area, as shown in the following specification text. [Table 1]
[0050] In this way, cases where the reference block overlaps with the current block or is outside the picture do not occur: there is no need to pad the reference block or the predicted block.
[0051] 2.3 IBC in the VVC test model In the current VVC test model, i.e., VTM-4.0 design, the entire reference block should be with the current coding tree unit (CTU) and does not overlap with the current block. Therefore, there is no need to pad the reference block or the prediction block. The IBC flag is coded as the prediction mode of the current CU. Thus, for each CU, there are a total of three prediction modes: MODE_INTRA, MODE_INTER, and MODE_IBC.
[0052] 2.3.1 IBC Merge Mode In IBC merge mode, indices that point to entries in the IBC merge candidate list are parsed from the bitstream. The construction of the IBC merge list can be summarized as the following sequence of steps: Step 1: Derive spatial candidates Step 2: Inserting HMVP candidates · Step 3: Insertion of pairwise average candidates.
[0053] In the derivation of spatial merge candidates, up to four merge candidates are selected from among the candidates located at the positions shown in FIG. 19. The order of derivation is A1, B1, B0, A0, B2. Position B2 is considered only if any of the PUs at positions A1, B1, B0, A0 are not available (e.g., because they belong to another slice or tile) or are not coded in IBC mode. After the candidate at position A1 is added, the insertion of the remaining candidates is subjected to a redundancy check that ensures that candidates with the same motion information are removed from the list, improving the coding efficiency. In order to reduce the computational complexity, the aforementioned redundancy check does not consider all possible candidate pairs. Instead, only the pairs linked by the arrows in FIG. 20 are considered, and a candidate is added to the list only if the corresponding candidates used for the redundancy check do not have the same motion information.
[0054] If after the insertion of the spatial candidate, the size of the IBC merge list is still smaller than the maximum IBC merge list size, an IBC candidate from the HMVP table may be inserted. When inserting an HMVP candidate, a redundancy check is performed.
[0055] Finally, the pairwise average candidates are inserted into the IBC merge list.
[0056] If the reference block identified by the merge candidate is outside the picture, or overlaps with the current block, or is outside the reconstructed region, or is outside the valid region constrained by some constraint, the merge candidate is called an invalid merge candidate.
[0057] Note that it is possible for invalid merge candidates to be inserted into the IBC merge list.
[0058] 2.3.2 IBC AMVP mode In IBC AMVP mode, an AMVP index that points to an entry in an IBC AMVP list is parsed from the bitstream. The construction of an IBC AMVP list can be summarized as following a series of steps: Step 1: Derive spatial candidates ○Check A0, A1 until a usable candidate is found. ○Check B0, B1, B2 until a usable candidate is found. Step 2: Inserting HMVP candidates · Step 3: Insertion of zero candidates.
[0059] If, after the insertion of the spatial candidates, the IBC AMVP list size is still smaller than the maximum IBC AMVP list size, then IBC candidates from the HMVP table may be inserted.
[0060] Finally, the zero candidate is inserted into the IBC AMVP list.
[0061] 2.4 Palette Mode The basic idea behind the palette mode is that pixels in a CU are represented by a small set of representative color values. This set is called the palette. It is also possible to indicate samples that are outside the palette by signaling an escape symbol followed by a (possibly quantized) component value. This kind of sample is called an escape sample. The palette mode is shown in Figure 2.
[0062] Figure 2 shows an example of a block coded in palette mode.
[0063] 2.5 Palette Modes in HEVC Screen Content Coding Extension (HEVC-SCC) In palette mode in HEVC-SCC, a predictive method is used to code the palette and index map.
[0064] 2.5.1 Palette Item Encoding For coding of palette entries, a palette predictor is maintained. The maximum size of the palette and the palette predictor are signaled in the SPS. In HEVC-SCC, palette_predictor_initializer_present_flag is introduced in the PPS. If this flag is 1, an item to initialize the palette predictor is signaled in the bitstream. The palette predictor is initialized at the beginning of each CTU row, each slice, and each tile. Depending on the value of palette_predictor_initializer_present_flag, the palette predictor is either reset to 0 or initialized using the palette predictor initializer entries signaled in the PPS. In HEVC-SCC, a palette predictor initializer of size 0 allows explicit disabling of the palette predictor initialization at the PPS level.
[0065] For each item in the palette predictor, a reuse flag is signaled to indicate whether it is part of the current palette. This is shown in Figure 3. The reuse flag is transmitted using a run-length coding of zeros. Then the number of the new palette item is signaled using an exponential Golomb code of degree 0. Finally, the component value for the new palette item is signaled.
[0066] FIG. 3 shows an example of the use of a palette predictor to signal palette items.
[0067] 2.5.2 Palette Index Encoding Palette indices are coded using a horizontal and vertical transverse scan as shown in Figure 4. The scan order is explicitly signaled in the bitstream using palette_transpose_flag. For the remainder of the subsection, the scan is assumed to be horizontal.
[0068] FIG. 4 shows an example of horizontal and vertical transverse scanning.
[0069] The palette index is coded using two main palette sample modes: 'INDEX' and 'COPY_ABOVE'. As explained above, the escape symbol is also signaled as 'INDEX' mode and is assigned an index equal to the maximum palette size. The mode is signaled using a flag, except when the top row or the previous mode was 'COPY_ABOVE'. In 'COPY_ABOVE' mode, the palette index of the sample in the row above is copied. In 'INDEX' mode, the palette index is signaled explicitly. For both 'INDEX' and 'COPY_ABOVE' modes, a run value is signaled that specifies the number of subsequent samples that are also coded in the same mode. An escape component value is signaled for each escape symbol if the escape symbol is part of a run of 'INDEX' or 'COPY_ABOVE' modes. The coding of the palette index is shown in Figure 5.
[0070] This syntax order is achieved as follows: First, the number of index values for the CU is signaled. Following this, the actual index value for the entire CU is signaled using a truncated binary encoding. Both the number of indices and the index value are coded in bypass mode. This groups the bypass bins associated with the index. Then, the palette sample mode (if required) and runs are signaled in an interleaved manner. Finally, the component escape values corresponding to the escape samples for the entire CU are grouped together and coded in bypass mode.
[0071] An additional syntax element, last_run_type_flag, is signaled after signaling the index value, which eliminates the need to signal a run value corresponding to the last run in a block along with the number of indices.
[0072] In HEVC-SCC, palette mode is also enabled for 4:2:2, 4:2:0, and monochrome chroma formats. The signaling of palette items and palette indices is nearly identical for all chroma formats. For non-monochrome formats, each palette item consists of three components. For monochrome formats, each palette item consists of a single component. For subsampled chroma directions, chroma samples are associated with luma sample indices that are divisible by two. After reconstructing the palette index for a CU, if a sample only has a single component associated with it, only the first component of the palette item is used. The only difference in signaling is for the escape component values. For each escape sample, the number of escape component values signaled can vary depending on the number of components associated with that sample. For each escape sample, the number of escape component values signaled can vary depending on the number of components associated with that sample.
[0073] In VVC, because a dual-tree coding structure is used to code intra slices, the luma component and the two chroma components may have different palettes and palette indices, and the two chroma components share the same palette and palette indices.
[0074] Figure 5 shows an example of palette index encoding.
[0075] 2.6 Intra-mode coding in VVC To capture any edge direction presented in natural video, the number of directional intra modes in VTM5 is expanded from the 33 used in HEVC to 65. The new directional modes not in HEVC are indicated as dotted red arrows in Figure 6, while the planar and DC modes remain the same. These denser directional intra prediction modes apply to all block sizes and to both luma and chroma intra prediction.
[0076] In VTM5, some conventional angular intra-prediction modes are adaptively replaced with wide-angle intra-prediction modes for non-square blocks.
[0077] In HEVC, each intra-coded block has a square shape, with the length of each side being a power of 2. Thus, no division operations are required to generate an intra predictor using DC mode. In VTM5, blocks can have rectangular shapes, which in the general case requires the use of block-wise division operations. To avoid division operations for DC prediction, only the long side is used to calculate the average for non-square blocks.
[0078] FIG. 6 shows examples of 67 intra prediction modes.
[0079] In order to keep the complexity of the most probable mode (MPM) list generation low, an intra-mode coding method with six MPMs is used by considering two available neighboring intra-modes. To build the MPM list, three aspects are considered: · Default · Intra Mode Nearby intra mode -Derived intra mode.
[0080] The unified 6-MPM list is used for intra blocks, whether MRL and ISP coding tools are applied or not. The MPM list is constructed based on the intra modes of the left and above neighboring blocks. Assuming that the mode of the left block is denoted as Left and the mode of the above block is denoted as Above, the unified MPM list is as follows (the left and above blocks are shown in Figure 7):
[0081] FIG. 7 is an example of the left and top neighbors of the current block. If no neighboring blocks are available, the intra mode defaults to planar. If modes Left and Above are both non-angular modes: ○MPM list → {plane, DC, V, H, V-4, V+4} If one of the modes Left and Above is an angular mode and the other is non-angular: Set Mode Max to the larger of the Left and Above modes. ○MPM list → {Plane, Max, DC, Max-1, Max+1, Max-2} If Left and Above are both angular and different: Set Mode Max to the larger of the Left and Above modes. If the difference between the modes Left and Above is between 2 and 62 (inclusive): · MPM List → {Plane, Left, Above, DC, Max-1, Max+1} Otherwise: · MPM List → {Plane, Left, Above, DC, Max-2, Max+2} If Left and Above are both angular and the same: ○MPM list → {Plane, Left, Left-1, Left+1, DC, Left-2}
[0082] In addition, the first bin of the mpm index codeword is CABAC context coded. A total of three contexts are used, corresponding to whether the current intra block is MRL-enabled, ISP-enabled, or a normal intra block.
[0083] During the 6MPM list generation process, pruning is used to remove duplicate modes, so that only unique modes can be included in the MPM list. A truncated binary code (TBC) is used for entropy coding of the 61 non-MPM modes.
[0084] A total of eight intra modes are allowed for chroma intra mode coding, including five traditional intra modes and three cross-component linear model modes (CCLM, LM_A, LM_L). The chroma mode signaling and derivation process is shown in Table 2. The chroma mode coding directly depends on the intra prediction mode of the corresponding luma block. Since separate block partition structures for luma and chroma components are enabled separately in I slice, one chroma block may correspond to multiple luma blocks. Therefore, for chroma DM mode, the intra prediction mode of the corresponding luma block that covers the center position of the current chroma block is directly inherited. [Table 2]
[0085] 2.7 Quantized Residual Block Differential Pulse Code Modulation (QR-BDPCM) In JVET-N0413, quantized residual block differential pulse-code modulation (QR-BDPCM) is proposed for efficiently encoding screen content.
[0086] The prediction directions used in QR-BDPCM can be vertical and horizontal prediction modes. Intra prediction is done for the whole block by sample copying in the prediction direction (horizontal or vertical prediction) similar to intra prediction. The residual is quantized and the delta between the quantized residual and its predictor (horizontal or vertical) quantized value is coded. This is derived by: For a block of size M(rows)×N(columns), r i,j Let Q(r) be the prediction residual after performing intra prediction horizontally (copying the left neighbor pixel value line by line through the predicted block) or vertically (copying the upper neighbor line for each line in the predicted block) using unfiltered samples from the upper or left block boundary samples. Let Q(r) be the prediction residual after performing intra prediction horizontally (copying the left neighbor pixel value for each line through the predicted block) or vertically (copying the upper neighbor line for each line in the predicted block) using unfiltered samples from the upper or left block boundary samples. i,j ) is the residual r i,j Let r denote the quantized version of r. The residual is the difference between the values of the original block and the predicted block. Then, block DPCM is applied to the quantized residual samples, resulting in the element
number
number
number
[0087] For horizontal prediction, a similar rule applies and the residual quantized samples are given by:
number
[0088] Residual Quantization Samples
number
[0089] At the decoder side, the above calculation is reversed to give Q(r i,j ), 0≦i≦M-1, 0≦j≦N-1. For vertical prediction,
number
number
[0090] The dequantized residual Q -1 (Q(r i,j ) is added to the intrablock prediction to generate a reconstructed sample value.
[0091] The main advantage of this scheme is that the inverse DPCM can be performed on-the-fly during coefficient parsing by simply adding a predictor as the coefficients are parsed, or it can be performed after parsing.
[0092] 2.8 Adaptive Loop Filter In VTM5, an Adaptive Loop Filter (ALF) with block-based filter adaptation is applied: for the luma component, one of 25 filters is selected for each 4x4 block based on local gradient direction and activity.
[0093] 2.8.1.1 Filter Shape In VTM5, two diamond filter shapes are used (as shown in Figure 8): a 7x7 diamond shape is applied for the luma component and a 5x5 diamond shape is applied for the chroma components.
[0094] FIG. 8 shows examples of ALF filter shapes (chroma: 5×5 diamond, luma 7×7 diamond).
[0095] 2.8.1.2 Block Classification For the luma component, each block is classified into one of 25 classes. The classification index C is determined by its directionality D and the quantized activity value
number
number
[0096] To calculate D and ^A, first the horizontal, vertical, and two diagonal gradients are calculated using the 1-D Laplacian:
number
[0097] The indices i and j refer to the coordinates of the top-left sample in a 4x4 block, and R(i,j) denotes the reconstructed sample at coordinate (i,j).
[0098] To reduce the complexity of block classification, a subsampled 1-D Laplacian computation is applied. The same subsampled positions are used for gradient computation in all directions, as shown in Figure 9.
[0099] Figure 9 shows an example of subsampled Laplacian computation: (a) Subsampled positions for vertical gradients, (b) Subsampled positions for horizontal gradients, (c) Subsampled positions for diagonal gradients, and (d) Subsampled positions for diagonal gradients.
[0100] Then, the maximum and minimum values of the horizontal and vertical gradients are
number
[0101] The maximum and minimum of the two diagonal gradients are
number
[0102] To derive the directional value D, these values are compared with each other and with two thresholds t1 and t2:
number
[0103] The activation value A is calculated as follows:
number
[0104] A is further quantized to the range 0 to 4 (inclusive), and the quantized value is
number
[0105] For the chroma components within a picture, no classification method is applied, i.e. a single set of ALF coefficients is applied to each chroma component.
[0106] 2.8.1.3 Geometric Transformation of Filter Coefficients and Clipping Values Before filtering each 4x4 block, a geometric transformation, such as a rotation or a diagonal and vertical flip, is applied to the filter coefficients f(k,l) and the corresponding filter clipping values c(k,l) depending on the gradient values calculated for that block. This is equivalent to applying these transformations to samples within the filter support region. The idea is to make different blocks to which ALF is applied more similar by aligning their orientations.
[0107] Three geometric transformations are introduced, including diagonal, vertical flip, and rotation:
number
[0108] where K is the size of the filter and 0≦k,l≦K-1 are the coefficient coordinates such that position (0,0) is the top-left corner and position (K-1,K-1) is the bottom-right corner. A transformation is applied to the filter coefficients f(k,l) and clipping values c(k,l) depending on the gradient value calculated for that block. The relationship between the transformation and the four gradients in the four directions is summarized in the following table. [Table 3]
[0109] 2.8.1.4 Filter Parameter Signaling In VTM5, ALF filter parameters are signaled in adaptive parameter sets (APS). In one APS, up to 25 sets of luma filter coefficients and clipping value indexes and up to one set of chroma filter coefficients and clipping value indexes can be signaled. To reduce bit overhead, filter coefficients of different classifications can be merged. In the slice header, the index of the APS used for the current slice is signaled.
[0110] The clipping value index decoded from the APS allows to determine the clipping values using a luma table of clipping values and a chroma table of clipping values. These clipping values depend on the internal bit depth. More precisely, the luma table of clipping values and the chroma table of clipping values are obtained by the following formula:
number
[0111] where B is equal to the internal bit depth and N is equal to 4, which is the number of clipping values allowed in VTM5.0.
[0112] The filtering process can be controlled at the CTB level. A flag is always signaled to indicate whether the ALF is applied to the luma CTB. The luma CTB can select a filter set among 16 fixed filter sets and a filter set from the APS. A filter set index is signaled on the luma CTB to indicate which filter set is applied. The 16 fixed filter sets are predefined and hard-coded in both the encoder and the decoder.
[0113] The filter coefficients are quantized with a norm equal to 128. To limit the multiplication complexity, coefficient values at non-central positions are quantized with a norm equal to 128. 7 From 2 7 Bitstream conformance is applied such that the coefficients are in the range -1, inclusive. The central position coefficients are not signaled in the bitstream and are assumed to be equal to 128.
[0114] 2.8.1.5 The filtering process On the decoder side, when ALF is enabled for the CTB, each sample in the CU is filtered, giving sample values such as:
number
[0115] where f(k,l) represents the decoded filter coefficients, K(x,y) is the clipping function, and c(k,l) represents the decoded clipping parameters. The variables k and l vary between -L / 2 and L / 2, and L represents the filter length. The clipping function K(x,y)=min(y,max(-y,x)), which is equal to the function Clip3(-y,y,x).
[0116] 2.8.1.6 Virtual Boundary Filtering Process for Line Buffer Reduction In VTM5, to reduce the line buffer requirements of ALF, a modified block classification and filtering is adopted for samples near horizontal CTU boundaries. For this purpose, a virtual boundary is defined as a line by shifting the horizontal CTU boundary by “N” samples, where N is equal to 4 for luma components and 2 for chroma components, as shown in Figure 10.
[0117] FIG. 10 shows an example of modified block classification at the virtual boundary.
[0118] As shown in FIG. 11, a modified block classification is applied to the luma component, and thus the activity value A is scaled by taking into account the reduction in the number of samples used in the 1D Laplacian gradient calculation.
[0119] For filtering, a symmetric padding operation on the virtual boundary is used for both luma and chroma components. As shown in Fig. 11, if the sample to be filtered is located below the virtual boundary, the neighboring samples located above the virtual boundary are padded, while the corresponding samples on the other side are also padded symmetrically.
[0120] FIG. 11 shows an example of modified ALF filtering for the luma component at the virtual boundary.
[0121] 2.9 Sample Adaptive Offset (SAO) Sample Adaptive Offset (SAO) is applied to the reconstructed signal after the deblocking filter by using the offset specified for each CTB by the encoder. The HM encoder first makes a decision whether the SAO process is applied to the current slice. If SAO is applied to the slice, each CTB is classified into one of five SAO types, as shown in Table 2-. The concept of SAO is to reduce distortion by classifying pixels into categories and adding an offset to the pixels in each category. SAO operations include Edge Offset (EO), which uses edge attributes for pixel classifications of SAO types 1-4, and Band Offset (BO), which uses pixel intensity for pixel classifications of SAO type 5. Each applicable CTB has SAO parameters, including sao_merge_left_flag, sao_merge_up_flag, SAO type, and four offsets. If sao_merge_left_flag is equal to 1, the current CTB reuses the SAO type and offset of the left CTB. If sao_merge_up_flag is equal to 1, the current CTB reuses the SAO type and offset of the CTB above. [Table 4]
[0122] 2.9.1 Operation of each SAO type Edge offset uses four 1-D 3-pixel patterns to classify the current pixel p by considering edge direction information, as shown in Figure 12. From left to right, these are 0 degrees, 90 degrees, 135 degrees, and 45 degrees.
[0123] Figure 12 shows four examples of 1-D 3-pixel patterns for pixel classification in EO.
[0124] Each CTB is classified into one of five categories according to Table 2-7. [Table 5]
[0125] Band Offset (BO) classifies all pixels in one CTB region into 32 uniform bands by using the most significant 5 bits of the pixel value as the band index. In other words, the pixel intensity range is divided into 32 equal segments ranging from zero to the maximum intensity value (e.g., 255 for 8-bit pixels). As shown in Figure 13, four adjacent bands are grouped, and each group is indicated by its leftmost position. The encoder searches all positions to obtain the group with the maximum distortion reduction by compensating for the offset of each band.
[0126] FIG. 13 is an example where four bands are grouped and represented by their starting band positions.
[0127] 2.10 Combined Inter and Intra Prediction (CIIP) In VTM5, if a CU is coded in merge mode, if the CU contains at least 64 luma samples (i.e., CU width × CU height is 64 or more) and both CU width and CU height are less than 128 luma samples, an additional flag is signaled to indicate whether the Combined Inter / Intra Prediction (CIIP) mode applies to the current CU. As the name suggests, CIIP prediction combines the inter prediction signal with the intra prediction signal. Inter prediction signal in CIIP mode P inter is derived using the same inter prediction process as applied in the normal merge mode, resulting in an intra prediction signal P intra is derived according to the normal intra prediction process in planar mode. The intra and inter prediction signals are then combined using a weighted average, where the weight values are calculated depending on the coding modes of the upper and left neighboring blocks (shown in FIG. 14) as follows: · If the top neighbor is available and intra-coded, set isIntraTop to 1, otherwise set isIntraTop to 0; · If the left neighbor is available and intra-coded, set isIntraLeft to 1, otherwise set isIntraLeft to 0; · If (isIntraLeft+isIntraLeft) is equal to 2, then wt is set to 3; · Else if (isIntraLeft+isIntraLeft) is equal to 1 then wt is set to 2; Otherwise, set wt to 1.
[0128] The CIIP forecast is formed as follows: P CIIP =((4-wt)*P inter +wt*P intra +2)>>2 (3-2)
[0129] FIG. 14 shows an example of the top and left neighboring blocks used in the CIIP weight derivation.
[0130] 2.11 Luma Mapping with Chroma Scaling (LMCS) In VTM5, a coding tool called luma mapping with chroma scaling (LMCS) is added as a new processing block before the loop filter. LMCS has two main components: 1) in-loop mapping of the luma component based on an adaptive piecewise linear model, and 2) for the chroma component, luma-dependent chroma residual scaling is applied. Figure 15 shows the LMCS architecture from the decoder's perspective. The dotted blocks in Figure 15 indicate where processing is applied in the mapped domain, including inverse quantization, inverse transform, luma intra prediction, and summation of luma residual with luma prediction. The unpatterned blocks in Figure 15 indicate where processing is applied in the original (i.e., unmapped) domain, including loop filters such as deblocking, ALF, and SAO, motion compensated prediction, chroma intra prediction, summation of chroma prediction with chroma residual, and storage of the decoded picture as a reference picture. The checkered block in Figure 15 is the new LMCS functional block, which contains the forward and inverse mapping of the luma signal, and the luma-dependent chroma scaling process. Like most other tools in VVC, LMCS can be enabled / disabled at the sequence level using the SPS flag.
[0131] Figure 15 shows an example of luma mapping with a chroma scaling architecture.
[0132] 2.12 Dual Tree Splitting In the current VVC design, for an I-slice, using an implicit quadtree partitioning, each CTU can be partitioned into coding units with 64x64 luma samples, and these coding units are the root of two separate coding tree syntax structures for luma and chroma.
[0133] In the dual tree in intra picture, different partitioning can be applied in chroma coding tree compared to luma coding tree, so dual tree introduces longer coding pipeline, and small chroma blocks such as 2x2, 4x2, 2x4 are allowed in QTBT MinQTSizeC value range as well as MinBtSizeY and MinTTSizeY in chroma tree. It brings difficulties to practical decoder design. In addition, some prediction modes such as CCLM, planar and angular modes require multiplication. To mitigate the above issues, small chroma block size (2x2 / 2x4 / 4x2) is constrained in dual tree as partitioning restriction.
[0134] 2.13 Minimum Chroma Intra Prediction Unit (SCIPU) in JVET-O0050 Small chroma sizes are not suitable for hardware implementation. In the case of dual trees, chroma blocks with too small size are not allowed. However, in the case of single trees, VVC draft 5 still allows 2x2, 2x4, 4x2 chroma blocks. To limit the size of chroma blocks, in a single coding tree, a SCIPU is defined in JVET-O0050 as a coding tree node whose chroma block size is equal to or larger than TH chroma samples and has at least one child luma block smaller than 4TH luma samples, where TH is set to 16 in this contribution. In each SCIPU, it is required that either all CBs are inter or all CBs are non-inter, i.e., either intra or IBC. In the case of non-inter SCIPUs, it is further required that the chroma of the non-inter SCIPU is not further split and the luma of the SCIPU is allowed to be further split. This way the minimum chroma intra CB size is 16 chroma samples, and 2x2, 2x4 and 4x2 chroma CBs are eliminated. Furthermore, chroma scaling is not applied in the case of non-inter SCIPU.
[0135] An example of two SCIPUs is shown in Fig. 16. In Fig. 6(a), one chroma CB and three luma CBs (4x8, 8x8, 4x8 luma CBs) of an 8x4 chroma sample form one SCIPU, because the ternary tree (TT) division from an 8x4 chroma sample results in a chroma CB smaller than 16 chroma samples. In Fig. 16(b), one chroma CB (left side of the 8x4 chroma sample) and three luma CBs (8x4, 4x4, 4x4 luma CBs) of a 4x4 chroma sample form one SCIPU, and the other chroma CB (right side of the 8x4 chroma sample) and two luma CBs (8x4, 8x4 luma CBs) of a 4x4 sample form one SCIPU. This is because a binary tree (BT) division from 4x4 chroma samples results in a chroma CB that is smaller than 16 chroma samples.
[0136] FIG. 16 shows an example of SCIPU.
[0137] If the current slice is an I slice or the current SCIPU has a 4x4 luma partition in it after one further division, the type of the SCIPU is presumed to be non-inter (because inter 4x4 is not allowed in VVC); otherwise, the type of the SCIPU (inter or non-inter) is indicated by a single signaled flag before parsing the CU in the SCIPU.
[0138] 2.14 Small Chroma Block Constraints in VVC Draft 6 In VVC draft 6 (JVET-O2001-vE.docx), the constraints for small chroma blocks are implemented as follows (relevant parts are marked in bold italics): [Table 6] TIFF0007581447000025.tif231170TIFF0007581447000026.tif112170
[0139] 2.14.1.1 Coding Unit Syntax [Table 7] [Table 8]
[0140] Acceptable quadrant process The inputs to this process are: The coding block size in luma samples, cbSize; · Multi-type tree depth mttDepth, Variable treeType that specifies whether a single tree (SINGLE_TREE) or a dual tree is used to split the CTU, and if a dual tree is used, whether the luma component (DUAL_TREE_LUMA) or chroma component (DUAL_TREE_CHROMA) is currently being processed. [Table 9] [The contents of the bitmap are also listed in text: The variable modeType specifies whether intra (MODE_INTRA), IBC (MODE_IBC), palette (MODE_PLT), and inter coding modes can be used (MODE_TYPE_ALL), or whether only intra, IBC, and palette coding modes can be used (MODE_TYPE_INTRA), or whether only inter coding modes can be used (MODE_TYPE_INTER) for the coding unit in the coding tree node.]
[0141] The output of this process is the variable allowSplitQt.
[0142] The variable allowSplitQt is derived as follows: allowSplitQt is set equal to FALSE if one or more of the following conditions are true: treeType is equal to SINGLE_TREE or DUAL_TREE_LUMA and cbSize is less than or equal to MinQtSizeY -treeType is equal to DUAL_TREE_CHROMA and cbSize / SubWidthC is equal to or less than MinQtSizeC mttDepth is not equal to 0 -treeType is equal to DUAL_TREE_CHROMA and (cbSize / SubWidthC) is 4 or less [Table 10] Otherwise, allowSplitQt is set equal to TRUE.
[0143] Acceptable Dichotomy Process The inputs to this process are: - Bisection split mode btSplit, · cbWidth, the coding block width in luma samples; · cbHeight, the coding block height in luma samples; The position (x0,y0) of the top-left luma sample of the considered coding block relative to the top-left luma sample of the picture · Multi-type tree depth mttDepth, · The maximum multitype tree depth with offset maxMttDepth, · Maximum binary tree size maxBtSize, · minimum quadtree size minQtSize, Partition index partIdx, Variable treeType that specifies whether a single tree (SINGLE_TREE) or a dual tree is used to split the CTU, and if a dual tree is used, whether the luma component (DUAL_TREE_LUMA) or chroma component (DUAL_TREE_CHROMA) is currently being processed. [Table 11] [The contents of the bitmap are also listed in text: The variable modeType specifies whether intra (MODE_INTRA), IBC (MODE_IBC), palette (MODE_PLT), and inter coding modes can be used (MODE_TYPE_ALL), or whether only intra, IBC, and palette coding modes can be used (MODE_TYPE_INTRA), or whether only inter coding modes can be used (MODE_TYPE_INTER) for the coding unit in the coding tree node.]
[0144] The output of this process is the variable allowBtSplit. [Table 12] The variables parallelTtSplit and cbSize are derived as specified in Table 6-2.
[0145] The variable allowBtSplit is derived as follows: allowBtSplit is set equal to FALSE if one or more of the following conditions are true: -cbSize is less than or equal to MinBtSizeY cbWidth is greater than maxBtSize cbHeight is greater than maxBtSize mttDepth is greater than or equal to maxMttDepth -treeType is equal to DUAL_TREE_CHROMA and (cbWidth / SubWidthC)*(cbHeight / SubHeightC) is less than or equal to 16 [Table 13] Otherwise, if all of the following conditions are true, allowBtSplit is set equal to FALSE: btSplit is equal to SPLIT_BT_VER y0+cbHeight is greater than pic_height_in_luma_samples Otherwise, if all of the following conditions are true, allowBtSplit is set equal to FALSE: btSplit is equal to SPLIT_BT_VER · cbHeight is greater than MaxTbSizeY x0+cbWidth is greater than pic_width_in_luma_samples Otherwise, allowBtSplit is set equal to FALSE if all of the following conditions are true: btSplit is equal to SPLIT_BT_HOR · cbWidth is greater than MaxTbSizeY y0+cbHeight is greater than pic_height_in_luma_samples Otherwise, if all of the following conditions are true, allowBtSplit is set equal to FALSE: x0+cbWidth is greater than pic_width_in_luma_samples y0+cbHeight is greater than pic_height_in_luma_samples cbWidth is larger than minQtSize Otherwise, if all of the following conditions are true, allowBtSplit is set equal to FALSE: btSplit is equal to SPLIT_BT_HOR x0+cbWidth is greater than pic_width_in_luma_samples y0+cbHeight is less than or equal to pic_height_in_luma_samples Otherwise, if all of the following conditions are true, allowBtSplit is set equal to FALSE: mttDepth is greater than 0 partIdx is equal to 1 · MttSplitMode[x0][y0][mttDepth-1] is equal to parallelTtSplit Otherwise, if all of the following conditions are true, parallelBtSplit is set equal to FALSE: btSplit is equal to SPLIT_BT_VER -cbWidth is less than or equal to MaxTbSizeY · cbHeight is greater than MaxTbSizeY Otherwise, if all of the following conditions are true, allowBtSplit is set equal to FALSE: btSplit is equal to SPLIT_BT_HOR · cbWidth is greater than MaxTbSizeY -cbHeight is less than or equal to MaxTbSizeY Otherwise, allowBtSplit is set equal to TRUE.
[0146] Acceptable Trichotomy Process The inputs to this process are: -Thirds split mode ttSplit, · cbWidth, the coding block width in luma samples; · cbHeight, the height of the coding block in luma samples; The position (x0,y0) of the top-left luma sample of the considered coding block relative to the top-left luma sample of the picture · Multi-type tree depth mttDepth · The maximum multitype tree depth with offset maxMttDepth, - Maximum ternary tree size maxTtSize, Variable treeType that specifies whether a single tree (SINGLE_TREE) or a dual tree is used to split the CTU, and if a dual tree is used, whether the luma component (DUAL_TREE_LUMA) or chroma component (DUAL_TREE_CHROMA) is currently being processed. [Table 14] [The contents of the bitmap are also listed in text: The variable modeType specifies whether intra (MODE_INTRA), IBC (MODE_IBC), palette (MODE_PLT), and inter coding modes can be used (MODE_TYPE_ALL), or whether only intra, IBC, and palette coding modes can be used (MODE_TYPE_INTRA), or whether only inter coding modes can be used (MODE_TYPE_INTER) for the coding unit in the coding tree node.]
[0147] The output of this process is the variable allowTtSplit. [Table 15] The variable cbSize is derived as specified in Table 6-3.
[0148] The variable allowTtSplit is derived as follows: allowTtSplit is set equal to FALSE if one or more of the following conditions are true: -cbSize is 2×MinTtSizeY or less cbWidth is greater than Min(MaxTbSizeY,maxTtSize) cbHeight is greater than Min(MaxTbSizeY,maxTtSize) mttDepth is greater than or equal to maxMttDepth x0+cbWidth is greater than pic_width_in_luma_samples y0+cbHeight is greater than pic_height_in_luma_samples -treeType is equal to DUAL_TREE_CHROMA and (cbWidth / SubWidthC)*(cbHeight / SubHeightC) is less than or equal to 32 [Table 16] Otherwise, allowTtSplit is set equal to TRUE.
[0149] pred_mode_flag equal to 0 specifies that the current coding unit is coded in inter prediction mode. pred_mode_flag equal to 1 specifies that the current coding unit is coded in intra prediction mode. If pred_mode_flag is not present, it is inferred as follows: [Table 17]
[0150] The variables CuPredMode[chType][x][y], for x=x0..x0+cbWidth-1 and y=y0..y0+cbHeight-1, are derived as follows: · If pred_mode_flag is equal to 0, CuPredMode[chType][x][y] is set equal to MODE_INTER. Otherwise (pred_mode_flag is equal to 1), CuPredMode[chType][x][y] is set equal to MODE_INTRA.
[0151] pred_mode_ibc_flag equal to 1 specifies that the current coding unit is coded in IBC prediction mode. pred_mode_ibc_flag equal to 0 specifies that the current coding unit is not coded in IBC prediction mode. If pred_mode_ibc_flag is not present, it is inferred as follows: [Table 18] If pred_mode_ibc_flag is equal to 1, the variable CuPredMode[chType][x][y] is set equal to MODE_IBC for x=x0..x0+cbWidth-1 and y=y0..y0+cbHeight-1.
[0152] 3. Issues 1. Currently, IBC is considered as MODE_TYPE_INTRA, so small chroma blocks are not tolerated, which leads to unnecessary coding efficiency loss. 2. Currently, the palette is considered as MODE_TYPE_INTRA, so small chroma blocks are not tolerated, which leads to unnecessary coding efficiency loss. 3. Currently, the small chroma block constraint does not take into account color subsampling formats. 4. Currently, the same partition and prediction mode constraints on small blocks are applied to all chroma formats. However, it may be desirable to design different constraint mechanisms for small blocks in 4:2:0 and 4:2:2 chroma formats. 5. Currently, palette mode flag signaling depends on modeType, which is undesirable as palettes may not enforce small block constraints. 6. Currently, IBC mode flags are inferred to be 0 for P / B slices where cu_skip_flag is equal to 1 but MODE_TYPE is equal to MODE_TYPE_INTRA, which is illegal in parsing. 7. Currently, non-4x4 luma IBC modes are not allowed for SCIPU luma blocks, which may be undesirable and may cause loss of coding efficiency. 8.2xH chroma blocks are still acceptable, but are not suitable for hardware implementation. 9. CIIP is considered as MODE_INTER, but uses intra prediction, which in some cases violates constraints. 10.When SCIPU is applied, delta QP for chroma may be signaled depending on luma split. For example, if the current block dimension is 16x8 in luma samples and split by vertical TT, local dual tree may be applied. It is specified that qgOnC=qgOnC && (cbSubdiv+2<=cu_chroma_qp_offset_subdiv). So, if cbSubdiv+2<=cu_chroma_qp_offset_subdiv, qgOnC is set to zero. This conditional setting assumes that chroma components are also split by TT. In local dual tree, chroma components may not be split, so cbSubdiv may be larger than cu_chroma_qp_offset_subdiv. To allow signaling of delta QP for chroma, IsCuChromaQpOffsetCoded should be set to 0. However, since qgOnC is set to 0, IsCuChromaQpOffsetCoded is not set to 0. 11. The maximum transform size for lossless encoding may be set differently than the maximum transform size for lossy encoding.
[0153] 4. Examples of technical solutions and implementations The following list should be considered as examples. These techniques should not be interpreted in a narrow sense. Moreover, these techniques can be combined in any way.
[0154] In this paper, an "M×N coding tree node" refers to an M×N block, where M is the block width and N is the block height in luma samples, which may be further divided by QT / BT / TT, etc. For example, a block may be a QT node, a BT node, or a TT node. A coding tree node may be a coding unit (e.g., containing three color components in single-tree, two chroma color components in dual-tree chroma coding, or only a luma color component in dual-tree luma coding), or a luma coding block, or a chroma coding block. A "small coding tree node unit" may refer to a coding tree node with a block size M×N equal to 32 / 64 / 128 in luma samples.
[0155] Unless otherwise specified, the width W and height H for a coding block are measured in luma samples. For example, an M×N coding block means an M×N luma block and / or two (M / SubWidthC)×(N / SubHeightC) chroma blocks, where SubWidthC and SubHeightC are derived by the chroma format as follows: [Table 19]
[0156] 1. Whether and / or how to divide into small blocks may depend on the color format. In one example, for a 4:4:4 color format, constraints on the size of chroma blocks may follow constraints on the size of luma blocks. b. In one example, for a 4:2:2 color format, the constraints on the size of the chroma blocks may follow the constraints for a 4:2:0 color format. c. In one example, for 4:0:0 and / or 4:4:4 chroma formats, constraints on small block partitions and / or prediction modes may not be applied. d. In one example, constraints on small block partitions and / or prediction modes may be applied differently for different chroma formats. i. In one example, for an M×N (e.g., 8×8) coding tree node with horizontal BT splitting, in a 4:2:2 chroma format, horizontal BT splitting may be allowed for both chroma and luma blocks, while in a 4:2:0 chroma format, horizontal BT splitting may be allowed for luma blocks but disabled for chroma blocks. ii. In one example, for an M×N (e.g., 16×4) coding tree node with vertical BT split, in a 4:2:2 chroma format, vertical BT split may be allowed for both chroma and luma blocks, while in a 4:2:0 chroma format, vertical BT split may be allowed for luma blocks but disabled for chroma blocks. iii. In one example, for an M×N (e.g., 8×16) coding tree node with horizontal TT split, in a 4:2:2 chroma format, horizontal TT split may be allowed for both chroma and luma blocks, while in a 4:2:0 chroma format, horizontal TT split may be allowed for luma blocks but disabled for chroma blocks. iv. In one example, for an M×N (e.g., 32×4) coding tree node with vertical TT split, in a 4:2:2 chroma format, vertical TT split may be allowed for both chroma and luma blocks, while in a 4:2:0 chroma format, vertical TT split may be allowed for luma blocks but disabled for chroma blocks. v. In one example, for 4:0:0 and / or 4:4:4 color formats, the small block constraint may not apply. e. In one example, whether to enable SCIPU depends on the color format. i. In one example, SCIPU is enabled for 4:2:0 and 4:2:2 color formats. ii. In one example, SCIPU is disabled for 4:0:0 and / or 4:4:4 color formats. 1. In one example, modeType may always be equal to MODE_TYPE_ALL for 4:0:0 and / or 4:4:4 color formats. 2. In one example, modeTypeCondition may always be equal to 0 for 4:0:0 and / or 4:4:4 color formats.
[0157] 2. How to determine the prediction mode (and / or modeType) for a (sub)block of a coding tree node may depend on the chroma format. a. In one example, if one of the following conditions is true, the modeType of the (sub)block split by this coding tree node may be equal to MODE_TYPE_ALL for 4:2:2 chroma format, and for 4:2:0 chroma format, the modeType may be equal to either MODE_TYPE_INTRA or MODE_TYPE_INTER: i. M×N (e.g. 8×8) coding tree nodes with horizontal BT partitioning ii. M×N (e.g. 16×4) coding tree nodes with vertical BT partitioning iii. M×N (e.g. 8×16) coding tree nodes with horizontal TT partitioning iv. An M×N (say 32×4) coding tree node with a vertical TT partition.
[0158] 3. It is proposed to rename MODE_TYPE_INTRA to MODE_TYPE_NO_INTER and to restrict the use of MODE_INTER. In one example, if a coding unit's modeType is equal to MODE_TYPE_NO_INTER, then MODE_INTER is not allowed.
[0159] 4. It is proposed to rename MODE_TYPE_INTER to MODE_TYPE_NO_INTRA and to restrict the use of MODE_INTRA. In one example, if a coding unit's modeType is equal to MODE_TYPE_NO_INTRA, then MODE_INTRA is not allowed.
[0160] 5. The mode constraint flag may never be signaled in 4:2:2 and / or 4:0:0 and / or 4:4:4 chroma formats. In one example, if the mode constraint flag is not present, it may be inferred to be equal to 1. Alternatively, if the mode constraint flag is not present, it may be inferred to be equal to 0.
[0161] 6. Whether and / or how to apply SCIPU to an M×N coded block, where M is the block width and N is the block height, may depend on whether the color format is 4:2:0 or 4:2:2. a. In one example, in a 4:2:2 color format, for an M×N coded block, where M is the block width and N is the block height, SCIPU may be enabled only if M times N (denoted as M×N) is equal to 64 or 32. b. In one example, the M*N=128 coding tree node may never be treated as a SCIPU block in a 4:2:2 color format. c. In one example, a coding tree node with BT partitioning and M*N=64 may never be treated as a SCIPU block in a 4:2:2 color format. d. In one example, the coding tree node where split_qt_flag is equal to 1 and M*N=64 may be a SCIPU block in a 4:2:2 color format. e. In one example, a coding tree node with TT partition and M*N=64 may be treated as a SCIPU block in 4:2:2 color format. f. In one example, a coding tree node having a BT partition and M*N=32 may be treated as a SCIPU block in a 4:2:2 color format. g. In the above description, for SCIPU blocks of 4:2:2 color format, modeTypeCondition may always be equal to 1. h. In the above description, for SCIPU blocks in 4:2:2 color format, only MODE_TYPE_INTRA may be allowed for both the current block of the parent node and all sub-blocks under the child leaf nodes.
[0162] For 7.4:2:2 color format, the modeTypeCondition of the SCIPU block may always be equal to 1. In one example, modeTypeCondition may be equal to 0 or 1 for 4:2:2 color format. b. In one example, for a SCIPU block in 4:2:2 color format, modeTypeCondition may never be equal to 2.
[0163] For 8.4:2:2 color format, the modeType of the SCIPU block may always be equal to MODE_TYPE_INTRA. In one example, modeType may be equal to MODE_TYPE_ALL or MODE_TYPE_INTRA for 4:2:2 color format. b. In one example, for SCIPU blocks in 4:2:2 color format, MODE_TYPE_INTER may be disabled.
[0164] 9. Whether block partitioning is allowed may depend on modeType and / or block size. In one example, whether BT and / or TT splitting is allowed for a block may depend on the modeType. i. In one example, if modeType is equal to MODE_TYPE_INTER, then BT splitting may not be allowed for the current coding block (eg, allowBtSplit is set equal to false). ii. In one example, if modeType is equal to MODE_TYPE_INTER, then TT splitting may not be allowed for the current coding block (eg, allowTtSplit is set equal to false). b. In one example, whether BT and / or TT splitting is allowed for a block may depend on the modeType and block size. i. In one example, for an M×N coding block, where M is the block width and N is the block height, if M*N is less than or equal to 32 and modeType is equal to MODE_TYPE_INTER, then BT splitting may not be allowed (e.g., allowBtSplit is set to false). ii. In one example, for an M×N coding block, where M is the block width and N is the block height, if M*N is less than or equal to 64, and modeType is equal to MODE_TYPE_INTER, then TT splitting may not be allowed (e.g., allowTtSplit is set to false).
[0165] 10. If the coding tree's modeTypeCurr is equal to MODE_TYPE_INTER, the splitting of the coding tree may be constrained. In one example, if the coding tree's modeTypeCurr is equal to MODE_TYPE_INTER, then BT splitting may not be allowed. b. In one example, if the coding tree's modeTypeCurr is equal to MODE_TYPE_INTER, then TT splitting may not be allowed. c. In one example, if the coding tree's modeTypeCurr is equal to MODE_TYPE_INTER, then QT splitting may not be allowed. d. In one example, if the coding tree's modeTypeCurr is equal to MODE_TYPE_INTER and the luma block size is less than or equal to 32, then BT splitting may not be allowed. e. In one example, if the coding tree's modeTypeCurr is equal to MODE_TYPE_INTER and the luma block size is less than or equal to 64, then TT splitting may not be allowed.
[0166] 11. A coding unit whose treeType is DUAL_TREE_LUMA may be coded in inter mode. In one example, a coding unit coded in inter coding mode, ie, MODE_INTER, may contain only a luma component, even in the case of a color format with multiple color components. b. In one example, pred_mode_flag may need to be parsed for DUAL_TREE_LUMA blocks. c. In one example, for a DUAL_TREE_LUMA block coded in inter mode, the same constraints of inter mode for SINGLE_TREE may be applied. i. In one example, 4x4 DUAL_TREE_LUMA inter-blocks may not be allowed.
[0167] 12. Chroma intra blocks (and / or IBC blocks) with block width equal to M (e.g., M=2) chroma samples may not be allowed. In one example, 2×N (eg, N≦64) chroma intra blocks may not be allowed in the dual tree. i. In one example, if treeType is equal to DUAL_TREE_CHROMA and block width is equal to 4 chroma samples, vertical BT splitting may be disabled. ii. In one example, if treeType is equal to DUAL_TREE_CHROMA and block width is equal to 8 chroma samples, vertical TT splitting may be disabled. b. In one example, 2×N (e.g., N≦64) chroma-intra (and / or IBC) blocks may not be allowed in a single tree. i. In one example, for an M×N (eg, M=8 and N≦64) coding tree node with vertical BT partitioning, one of the following processes may be applied: Vertical BT splitting is not allowed for 1.4×N or 4×(N / 2) chroma blocks, but may be allowed for 8×N luma blocks. 2. 4xN or 4x(N / 2) chroma blocks may be coded by MODE_INTRA or MODE_IBC instead of vertical BT splitting. 3. Vertical BT splitting is allowed for both 8xN luma blocks and 4xN or 4x(N / 2) chroma blocks, but both luma and chroma blocks may not be coded in MODE_INTRA (e.g., they may be coded in MODE_INTER or MODE_IBC). ii. In one example, for an M×N (eg, M=16 and N≦64) coding tree node with vertical TT split, one of the following processes may be applied: Vertical TT splitting is not allowed for 1.8×N or 8×(N / 2) chroma blocks, but may be allowed for 16×N luma blocks. 2. 8xN or 8x(N / 2) chroma blocks may be coded by MODE_INTRA or MODE_IBC instead of vertical TT splitting. 3. Vertical TT splitting is allowed for both 16xN luma blocks and 8xN or 8x(N / 2) chroma blocks, but both luma and chroma blocks may not be coded in MODE_INTRA (e.g., they may be coded in MODE_INTER or MODE_IBC).
[0168] 13.IBC mode may be allowed for luma and / or chroma blocks, regardless of whether they are small block sizes. In one example, IBC mode may be allowed for luma blocks, including 8x4 / 8x8 / 16x4 and 4xN (such as N<=64) luma blocks, even when modeType is equal to MODE_TYPE_INTRA. b. In one example, IBC modes may be allowed for chroma blocks even if modeType is equal to MODE_TYPE_INTRA.
[0169] 14. Signaling of IBC prediction mode flags may depend on the prediction mode type (eg, MODE_TYPE_INTRA). a. In one example, the IBC prediction mode flag for a non-SKIP block (e.g., a coded block that is not coded in skip mode) may be explicitly signaled in the bitstream if treeType is not equal to DUAL_TREE_CHROMA and modeType is equal to MODE_TYPE_INTRA.
[0170] 15. The IBC prediction mode flag may be inferred depending on the CU SKIP flag and the mode type (eg, modeType). In one example, if the current block is coded in SKIP mode (eg, cu_skip_flag is equal to 1) and modeType is equal to MODE_TYPE_INTRA, then an IBC prediction mode flag (eg, pred_mode_ibc_flag) may be inferred to be equal to 1.
[0171] 16. Explicit signaling of palette mode flags may be independent of modeType. In one example, palette mode flag (eg, pred_mode_plt_flag) signaling may depend on slice type, block size, prediction mode, etc., but modeType can be anything. b. In one example, if modeType is equal to MODE_TYPE_INTER or MODE_TYPE_INTRA, the palette mode flags (e.g., pred_mode_plt_flag) are inferred to be 0.
[0172] 17. If modeType is equal to MODE_TYPE_INTER, then IBC mode may be allowed to be used. In one example, when modeType is equal to MODE_TYPE_INTRA, chroma IBC may not be allowed. b. In one example, when modeType is equal to MODE_TYPE_INTRA or MODE_TYPE_INTER, the IBC mode may be allowed to be used. c. In one example, the IBC mode may be allowed to be used regardless of what the modeType is. d. In one example, within one SCIPU, both IBC and Inter modes may be allowed. e. In one example, the size of an IBC chroma block may always correspond to the size of the corresponding luma block. f. In one example, if modeType is equal to MODE_TYPE_INTER and the coding unit size is 4x4 in luma, signaling of pred_mode_ibc_flag may be skipped and pred_mode_ibc_flag may be inferred to be equal to 1.
[0173] 18.If modeType is MODE_TYPE_INTER, palette mode is allowed to be used. In one example, if modeType is MODE_TYPE_INTRA, then chroma palettes may not be allowed. b. In one example, when modeType is equal to MODE_TYPE_INTRA or MODE_TYPE_INTER, the IBC mode may not be allowed to be used. c. In one example, the IBC mode may be allowed to be used regardless of what the modeType is. d. In one example, when modeType is equal to MODE_TYPE_INTRA or MODE_TYPE_INTER, palette mode may be allowed to be used. e. In one example, palette mode may be allowed to be used regardless of what the modeType is. f. In one example, within one SCIPU, both palette and inter modes may be allowed. g. In one example, Palette, IBC and Inter modes may all be allowed within one SCIPU. h. In one example, the size of a palette chroma block may always correspond to the size of the corresponding luma block. i. In one example, if modeType is equal to MODE_TYPE_INTER and the coding unit size is 4x4 in luma, signaling of pred_mode_plt_flag is skipped and pred_mode_plt_flag is inferred to be equal to 1. j. In one example, if modeType is equal to MODE_TYPE_INTER and the coding unit size is 4x4 in luma, one message may be sent to indicate whether the current prediction mode is IBC or palette. k. In one example, whether to enable / disable palette mode may depend on the slice type and modeType. i. In one example, for I slices with MODE_TYPE_INTRA, palette mode may be enabled. ii. In one example, for P / B slices with MODE_TYPE_INTER, palette mode may be enabled.
[0174] 19. When palette mode is enabled, local dual trees may not be allowed. a. In one example, if palette mode is enabled, modeTypeCondition may always be set to 0.
[0175] 20. For small chroma blocks whose width is equal to M (e.g., M=2) or whose height is equal to N (e.g., N=2), the allowed intra prediction modes may be constrained to be different from those allowed for large chroma blocks. In one example, only a subset of the intra prediction modes of the available chroma intra prediction modes may be used. b. In one example, only the INTRA_DC mode may be used. c. In one example, only the INTRA_PLANAR mode may be used. d. In one example, only the INTRA_ANGULAR18 mode may be used. e. In one example, only the INTRA_ANGULAR50 mode may be used. f. In one example, CCLM mode may not be allowed.
[0176] 21. For small chroma blocks with width equal to M (e.g., M=2) or height equal to N (e.g., N=2), the transformation type may be constrained to be different from that allowed for large chroma blocks. In one example, only transform skip may be used. b. In one example, only a one-dimensional transform may be used. c. In one example, coding tools that support multiple types of transforms are not allowed. i. Alternatively, signaling of coding tools that support multiple types of transforms is omitted.
[0177] 22. CIIP may be considered as MODE_TYPE_INTRA. In one example, CIIP mode may be allowed if dual tree splitting is used. i. In one example, CIIP mode may be allowed if the CU type is DUAL_TREEE_CHROMA. b. Alternatively, CIIP may be considered as MODE_TYPE_INTER. i. In one example, if the chroma block width is equal to M (e.g., M=2), then CIIP mode may not be allowed. ii. In one example, if the chroma block width is equal to M (eg, M=2), the intra prediction modes for chroma in CIIP may be restricted to simple intra prediction modes. 1. In one example, if the chroma block width is equal to M (eg, M=2), then INTRA_DC may be used for chroma intra prediction. 2. In one example, if the chroma block width is equal to M (eg, M=2), then INTRA_ANGULAR18 may be used for chroma intra prediction. 3. In one example, if the chroma block width is equal to M (eg, M=2), then INTRA_ANGULAR50 may be used for chroma intra prediction. iii. In one example, intra prediction modes for chroma in the CIIP may be restricted to simple intra prediction modes. 1. In one example, INTRA_DC may be used for chroma intra prediction. 2. In one example, INTRA_ANGULAR18 mode may be used for chroma intra prediction. 3. In one example, the INTRA_ANGULAR50 mode may be used for chroma intra prediction.
[0178] 23. For the above items, the variables M and / or N may be predefined or signaled. In one example, M and / or N may further depend on the color format (e.g., 4:2:0, 4:2:2, 4:4:4).
[0179] 24.modeType may be extended to cover more types. In one example, modeType may be MODE_TYPE_IBC. When modeType is equal to MODE_TYPE_IBC, the prediction mode is inferred to be IBC. i. In one example, pred_mode_flag is not signaled in this case. ii. In one example, pred_mode_ibc_flag is not signaled in this case. iii. In one example, pred_mode_plt_flag is not signaled in this case. In one example, modeType may be MODE_TYPE_PALETTE. When modeType is equal to MODE_TYPE_PALETTE, the prediction mode is inferred to be palette mode. i. In one example, pred_mode_flag is not signaled in this case. ii. In one example, pred_mode_ibc_flag is not signaled in this case. iii. In one example, pred_mode_plt_flag is not signaled in this case. c. In one example, mode_constraint_flag may be replaced with an index to indicate which of the allowed modeTypes is to be used.
[0180] 25. In one example, whether a QT division is allowed for a block of dimensions W×H may depend on the modeType combined with the dimensions. For example, if modeType is equal to MODE_TYPE_INTER, W is equal to 8, and H is equal to 8, then QT spit is not allowed.
[0181] 26. In one example, whether a vertical TT partition is allowed for a block of dimensions W×H may depend on the modeType combined with the dimensions. For example, if modeType is equal to MODE_TYPE_INTER, W is equal to 16, and H is equal to 4, then vertical TT spits are not allowed.
[0182] 27. In one example, whether horizontal TT division is allowed for a block of dimensions W×H may depend on the modeType combined with the dimensions. For example, if modeType is equal to MODE_TYPE_INTER, W is equal to 4, and H is equal to 16, then horizontal TT split is not allowed.
[0183] 28. In one example, whether vertical BT partitioning is allowed for a block of dimensions W×H may depend on the modeType combined with the dimensions. For example, if modeType is equal to MODE_TYPE_INTER, W is equal to 8, and H is equal to 4, then vertical BT split is not allowed.
[0184] 29. In one example, whether horizontal TT partitioning is allowed for a block of dimensions W×H may depend on the modeType combined with the dimensions. For example, if modeType is equal to MODE_TYPE_INTER, W is equal to 4, and H is equal to 8, then horizontal BT split is not allowed.
[0185] 30. In one example, whether the prediction mode of a CU is inferred by modeType may depend on the color components and / or the block dimensions W×H. For example, the prediction mode of a chroma CU is inferred by modeType, but the prediction mode of a luma CU is signaled rather than inferred by modeType. i. For example, the prediction mode of a luma CU is signaled instead of inferred by modeType if W>4 or H>4.
[0186] 31. When SCIPU is applied, whether and / or how to signal information related to the delta QP of the first component may depend on how the first component is partitioned. a. In one example, when SCIPU is applied, whether and / or how to signal information related to the delta QP of the first component may depend on how the first component is partitioned and may be decoupled from how the second component is partitioned. b. In one example, the first component is luma and the second component is chroma. c. In one example, the first component is chroma and the second component is luma.
[0187] 32. When a dual tree and / or local dual tree coding structure is applied, any variable related to the delta QP of the first component cannot be modified during the decoding or parsing process of the second component. In one example, a local dual tree coding structure may be used according to SCIPU. b. In one example, the first component is luma and the second component is chroma. i. The variable may be IsCuQpDeltaCoded. c. In one example, the first component is chroma and the second component is luma. i. The variable may be IsCuChromaQpOffsetCoded.
[0188] 33. When SCIPU applies, information related to the delta QP of a component (e.g., luma or chroma) is In a particular region where the luma and chroma components are required to share the same mode type (such as MODE_TYPE_INTER or MODE_TYPE_INTRA), it may be signaled at most once. In one example, a particular region is considered a quantization group.
[0189] 34. MxN chroma intra blocks with block width M and block height N may not be allowed. In one example, DC prediction may be used for M×N chroma intra prediction, e.g., M=2 chroma samples, N=8 and / or 16 and / or 32 chroma samples. i. In one example, the reconstructed neighboring pixels may not be used for DC prediction. ii. In one example, the neighboring reference pixels may be set equal to 1<<(bitDepth-1) for DC prediction. iii. In one example, PDPC may not be applied for DC prediction. iv. In one example, DC mode information may be uncoded and derived for MxN chroma intra blocks. In one example, CCLM prediction may be used for M×N chroma intra prediction, where M=2 chroma samples and N=8 and / or 16 and / or 32 chroma samples. i. In one example, the reconstructed neighboring pixels may not be used for CCLM prediction. ii. In one example, CCLM prediction pred C =a*rec L +b, and the chroma prediction value pred C is the luma reconstruction value rec L When derived from, the parameters a and b may be initialized to default fixed values (e.g. a=0, b=512) at the beginning of a video unit such as SPS / VPS / PPS / PPS / picture / subpicture / slice / tile / brick / CTU row / CTU / VPDU / CU / PU / TU. 1. In one example, a first-in-first-out table may be maintained on the fly to update the CCLM parameters a and b. iii. In one example, PDPC may not be applied to CCLM predictions. iv. In one example, CCLM mode information may be derived rather than coded for MxN chroma intra blocks. v. Alternatively, the proposed method may be applied to other types of CCLM coded chroma blocks whose width is not equal to M or whose height is not equal to N. 1. In one example, the CCLM parameters of a previously coded block may be utilized for the current block instead of being derived based on the reconstructed information of neighboring blocks of the current block. 2. In one example, CCLM parameters may be stored in a table and updated on the fly, such as by using a FIFO. c. In one example, M×N chroma intra blocks may not be allowed in the dual tree, eg, M=2 chroma samples, N=8 and / or 16 and / or 32 chroma samples. i. In one example, if treeType is equal to DUAL_TREE_CHROMA, X-splitting may be disabled for coding tree nodes with dimensions equal to (M*a) × (N*b), where M*a is the width of the coding tree node in units of chroma samples and N*b is the height of the coding tree node in units of chroma samples. 1. In one example, X is a vertical BT partition, with a=2, b=1. 2. In one example, X is a horizontal BT division, with a=1, b=2. 3. In one example, X is a vertical TT partition, with a=4, b=1. 4. In one example, X is a horizontal TT division, a=1, b=4. d. In one example, M×N chroma intra blocks may not be allowed in a single tree, eg, M=2 chroma samples, N=8 and / or 16 and / or 32 chroma samples. i. In one example, in a single tree, X-splitting may be disabled for chroma components of coding tree nodes with dimensions equal to (M*a*subWidthC)×(N*b*subHeightC), where M*a is the width of the coding tree node in luma samples, N*b is the height of the coding tree node in luma samples, and subWidthC and subHeightC are chroma subsampling ratios in the width and height dimensions, e.g., subWidthC and subHeightC are equal to 2 for a 4:2:0 chroma format. 1. In one example, X = vertical BT division, a = 2, b = 1 2. In one example, X = vertical TT division, a = 4, b = 1 3. In one example, X = horizontal BT division, a = 1, b = 2 4. In one example, X = horizontal TT division, a = 1, b = 2 5. In one example, X = horizontal TT division, a = 1, b = 4 6. In one example, an unsplit chroma block may have dimensions equal to (M*a)×(N*d). 7. In one example, a non-split chroma block may be determined as MODE_TYPE_INTRA, and only intra prediction (and / or IBC and / or PLT modes) may be allowed for this chroma block. 8. In one example, X-splitting may be allowed for the luma component of this coding tree node, e.g., the luma coding tree node may be further split into multiple luma child nodes, each of which may be determined to be MODE_TYPE_INTER, during which only inter prediction (and / or IBC mode) may be allowed. 9. Alternatively, the chroma components may be further split along with the co-located luma component, but the mode type may be equal to MODE_TYPE_INTER and only inter prediction (and / or IBC modes) may be allowed. ii. In one example, MxN chroma intra blocks are not allowed in CIIP. 1. In one example, CIIP may not be allowed for coding blocks with dimensions equal to (M*subWidthC)×(N*subHeightC) in luma samples. a) If CIIP is not allowed, the associated syntax elements, such as the CIIP flag, may not be coded in the bitstream. 2. In one example, the chroma blending process in the CIIP may only be filled with predictions from the inter part. a) In one example, only luma intra prediction may be applied to CIIP coded blocks, while chroma intra prediction may not be applied to CIIP coded blocks with dimensions equal to (M*subWidthC)×(N*subHeightC) in luma sample units. 3. In one example, DC prediction may be used for M×N chroma intra prediction for a CIIP block, as described in item 34(a). 4. In one example, CCLM prediction may be used for M×N chroma intra prediction for CIIP blocks, as described in item 34(b).
[0190] Relationship between losslessSizeY and maxCTB / maxTBsize / maxTS sizes 35. The maximum transform size may be signaled and / or estimated differently for lossless (and / or near lossless) and lossy encoding. In one example, the maximum transform size for lossless and / or near-lossless encoding may be referred to by other terms such as "maximum residual size" or "maximum representation size" since it does not involve a transform. In this disclosure, it may be referred to as "maxLosslessTbSizeY." b. In one example, lossless encoding for a block may refer to cu_transquant_bypass_flag equal to 1 for that block. c. In one example, a nearly lossless encoding for a block may refer to an encoding with a QP smaller than a given threshold. i. In one example, the threshold may be indicated in the DPS / VPS / SPS / PPS / tile / slice / brick header. d. In one example, a nearly lossless encoding for a block may refer to an encoding with a QP less than a given threshold, and transform_skip_flag is equal to 1 for that block. i. In one example, the threshold may be indicated in the DPS / VPS / SPS / PPS / tile / slice / brick header. e. A first maximum transform size (e.g., maxLosslessTbSizeY) for lossless and / or near-lossless encoding may be set conditional on a second maximum transform size that may be used for lossy encoding, and / or a maximum coding tree unit size, and / or a maximum transform skip size. f. The maximum transform size for lossless and / or near-lossless encoding may be signaled at the video unit level, such as syntax element (SE) in SPS / VPS / PPS / sub-picture / slice / tile / brick / CTU / VPDU. i. In one example, the SE may be signaled depending on the maximum CTB size (eg, log2_ctu_size_minus5, CtbSizeY, etc.). 1. For example, a flag may be signaled if log2_ctu_size_minus5 is greater than N (eg, N=0). ii. In one example, the SE may be signaled depending on the maximum transform size (eg, sps_max_luma_transform_size_64_flag, MaxTbSize, etc.). 1. For example, SE may be signaled when sps_max_luma_transform_size_64_flag is equal to M (eg, M=1). iii. In one example, the SE may be signaled depending on the maximum transform skip size (eg, log2_transform_skip_max_size_minus2, MaxTsSize, etc.). 1. For example, SE may be signaled if log2_transform_skip_max_size_minus2 is greater than K (eg, K=3). iv. In one example, a delta value between log(maxLosslessTbSizeY) and X may be signaled. 1. For example, X may be equal to log2_ctu_size_minus5. 2. For example, X may be equal to sps_max_luma_transform_size_64_flag. 3. For example, X may be equal to log2_transform_skip_max_size_minus2. 4. For example, X may be equal to a fixed integer. 5. In one example, a delta-K may be signaled, where K is an integer such as 1. v. In one example, a delta value between maxLosslessTbSizeY and X may be signaled. 1. For example, X may be equal to CtbSizeY 2. For example, X may be equal to MaxTbSize 3. For example, X may be equal to MaxTsSize 4. For example, X may be equal to a fixed integer. 5. In one example, a delta-K may be signaled, where K is an integer such as 1. vi. In one example, maxLosslessTbSizeY of a video unit (sequence / picture / subpicture / slice / tile / CTU / VPDU / CU / PU / TU, etc.) may be derived from the SE. vii.SE may be a flag. viii. If the flag is not present, it may be presumed to be equal to a positive integer N. g. In one example, if lossless and / or near lossless encoding is applied to the video unit (eg, cu_transquant_bypass_flag is equal to 1), the maximum transform size for the video unit may be set equal to maxLosslessTbSizeY other than maxTbSizeY. h. Alternatively, the maximum transform size for lossless and / or near-lossless encoding for a video unit (e.g., sequence / picture / sub-picture / slice / tile / CTU / VPDU / CU / PU / TU) may be derived instead of signaled. i. For example, it may be set equal to a fixed integer value M (such as M=32 in luma samples). ii. For example, it may be set equal to the second largest transform size that may be used for lossy encoding. iii. For example, it may be set equal to the maximum coding tree unit size. iv. For example, it may be set equal to the VPDU size. v. For example, it may be set equal to the maximum transform skip block size. vi. For example, it may be set equal to the maximum residual coding block size.
[0191] 36. For an MxN residual block with lossless (and / or near-lossless) coding, it may be split into multiple smaller residual sub-blocks depending on maxLosslessTbSizeY. After division, the width and height of every sub-block is no larger than maxLosslessTbSizeY. b. In one example, the sub-blocks may be divided in a recursive manner until the width and height of all sub-blocks are less than or equal to maxLosslessTbSizeY. 5. Embodiments Newly added parts are highlighted in bold and italics (alternatively underlined), and parts deleted from the VVC Working Draft are enclosed in double brackets ([[a]] represents the deletion of the letter "a"). Modifications are based on the latest VVC Working Draft (JVET-O2001-v11).
[0192] 5.1 Example embodiment #1 The following embodiment is for constraints on small block partitions and prediction modes only apply to 4:2:0 and 4:4:4 chroma formats (not to 4:0:0 and 4:4:4 chroma formats).
[0193] 7.4.9.4 Coding Tree Semantics The variable modeTypeCondition is derived as follows: · modeTypeCondition is set to 0 if any of the following conditions are true: slice_type==I and qtbtt_dual_tree_intra_flag is equal to 1 modeTypeCurr is not equal to MODE_TYPE_ALL · chroma_format_idc equals 0 · chroma_format_idc equals 3 Otherwise, if any of the following conditions are true, modeTypeCondition is set to 1: cbWidth*cbHeight is equal to 64 and split_qt_flag is equal to 1 cbWidth*cbHeight is equal to 64 and MttSplitMode[x0][y0][mttDepth] is equal to SPLIT_TT_HOR or SPLIT_TT_VER cbWidth*cbHeight is equal to 32 and MttSplitMode[x0][y0][mttDepth] is equal to SPLIT_BT_HOR or SPLIT_BT_VER Otherwise, if any of the following conditions are true, modeTypeCondition is set to 1+(slice_type!=I? 1 : 0): cbWidth*cbHeight is equal to 64 and MttSplitMode[x0][y0][mttDepth] is equal to SPLIT_BT_HOR or SPLIT_BT_VER cbWidth*cbHeight is equal to 128 and MttSplitMode[x0][y0][mttDepth] is equal to SPLIT_TT_HOR or SPLIT_TT_VER Otherwise, modeTypeCondition is set to 0.
[0194] 5.2 Example embodiment #2 The following embodiment concerns signaling of palette mode flags independent of modeType. 7.3.8. Coding Unit Syntax [Table 20]
[0195] 5.3 Exemplary embodiment #3 In the following embodiment, the IBC prediction mode flag is estimated depending on the CU SKIP flag and modeType.
[0196] pred_mode_ibc_flag equal to 1 specifies that the current coding unit is coded in IBC prediction mode. pred_mode_ibc_flag equal to 0 specifies that the current coding unit is not coded in IBC prediction mode.
[0197] If pred_mode_ibc_flag is not present, it is inferred as follows: · If cu_skip_flag[x0][y0] is equal to 1, cbWidth is equal to 4, and cbHeight is equal to 4, pred_mode_ibc_flag is inferred to be equal to 1. Otherwise, if both cbWidth and cbHeight are equal to 128, pred_mode_ibc_flag is inferred to be equal to 0. Otherwise, if cu_skip_flag[x0][y0] is equal to 1 and modeType is equal to MODE_TYPE_INTRA, pred_mode_ibc_flag is inferred to be equal to 1. Otherwise, if modeType is equal to MODE_TYPE_INTER, pred_mode_ibc_flag is inferred to be equal to 0. Otherwise, if treeType is equal to DUAL_TREE_CHROMA, pred_mode_ibc_flag is inferred to be equal to 0. Otherwise, pred_mode_ibc_flag is inferred to be equal to the value of sps_ibc_enabled_flag when decoding an I slice, or equal to 0 when decoding a P or B slice.
[0198] If pred_mode_ibc_flag is 1, then the variable CuPredMode[chType][x][y] is set equal to MODE_IBC, for x=x0..x0+cbWidth-1 and y=y0..y0+cbHeight-1.
[0199] 5.4 Exemplary embodiment #4 The following embodiments relate to the signaling of the IBC prediction mode flag depending on MODE_TYPE_INTRA and / or that IBC mode is allowed for a luma block regardless of whether it is a small block size or not.
[0200] 7.3.8. Coding Unit Syntax [Table 21]
[0201] 5.5 Exemplary embodiment #5 The following embodiment relates to applying different intrablock constraints for 4:2:0 and 4:2:2 color formats.
[0202] 7.4.9.4 Coding Tree Semantics The variable modeTypeCondition is derived as follows: · modeTypeCondition is set to 0 if any of the following conditions are true: slice_type==I and qtbtt_dual_tree_intra_flag is equal to 1 modeTypeCurr is not equal to MODE_TYPE_ALL Otherwise, if any of the following conditions are true, modeTypeCondition is set equal to 1: cbWidth*cbHeight is equal to 64 and split_qt_flag is equal to 1 cbWidth*cbHeight is equal to 64 and MttSplitMode[x0][y0][mttDepth] is equal to SPLIT_TT_HOR or SPLIT_TT_VER cbWidth*cbHeight is equal to 32 and MttSplitMode[x0][y0][mttDepth] is equal to SPLIT_BT_HOR or SPLIT_BT_VER Otherwise, if any of the following conditions are true, modeTypeCondition is set equal to 1 + (slice_type != I ? 1:0): cbWidth*cbHeight is equal to 64, MttSplitMode[x0][y0][mttDepth] is equal to SPLIT_BT_HOR or SPLIT_BT_VER, and chroma_format_idc is equal to 1 cbWidth*cbHeight is equal to 128, MttSplitMode[x0][y0][mttDepth] is equal to SPLIT_TT_HOR or SPLIT_TT_VER, and chroma_format_idc is equal to 1 cbWidth is equal to 8, cbHeight is equal to 8, MttSplitMode[x0][y0][mttDepth] is equal to SPLIT_BT_VER, and chroma_format_idc is equal to 2 cbWidth is equal to 4, cbHeight is equal to 16, MttSplitMode[x0][y0][mttDepth] is equal to SPLIT_BT_HOR, and chroma_format_idc is equal to 2 cbWidth is equal to 16, cbHeight is equal to 8, MttSplitMode[x0][y0][mttDepth] is equal to SPLIT_TT_VER, and chroma_format_idc is equal to 2 cbWidth is equal to 4, cbHeight is equal to 32, MttSplitMode[x0][y0][mttDepth] is equal to SPLIT_TT_HOR, and chroma_format_idc is equal to 2 Otherwise, modeTypeCondition is set to 0.
[0203] 5.6 Exemplary embodiment #6 The following embodiment relates to disallowing 2xN chroma intra blocks in a single tree.
[0204] 7.4.9.4 Semantics of the Coding Tree The variable modeTypeCondition is derived as follows: · modeTypeCondition is set to 0 if any of the following conditions are true: slice_type==I and qtbtt_dual_tree_intra_flag is equal to 1 modeTypeCurr is not equal to MODE_TYPE_ALL Otherwise, if any of the following conditions are true, modeTypeCondition is set to 1: cbWidth*cbHeight is equal to 64 and split_qt_flag is equal to 1 cbWidth*cbHeight is equal to 64 and MttSplitMode[x0][y0][mttDepth] is equal to SPLIT_TT_HOR or SPLIT_TT_VER cbWidth*cbHeight is equal to 32 and MttSplitMode[x0][y0][mttDepth] is equal to SPLIT_BT_HOR or SPLIT_BT_VER Otherwise, if any of the following conditions are true, modeTypeCondition is set to 1 + (slice_type != I ? 1:0): cbWidth*cbHeight is equal to 64 and MttSplitMode[x0][y0][mttDepth] is equal to SPLIT_BT_HOR or SPLIT_BT_VER cbWidth*cbHeight is equal to 128 and MttSplitMode[x0][y0][mttDepth] is equal to SPLIT_TT_HOR or SPLIT_TT_VER cbWidth is equal to 8 and MttSplitMode[x0][y0][mttDepth] is equal to SPLIT_BT_VER cbWidth is equal to 16 and MttSplitMode[x0][y0][mttDepth] is equal to SPLIT_TT_VER Otherwise, modeTypeCondition is set to 0.
[0205] 5.7 Exemplary embodiment #7 The following embodiment relates to disallowing 2xN chroma intra blocks in the dual tree.
[0206] 6.4.2 Acceptable Dichotomy Processes The variable allowBtSplit is derived as follows: allowBtSplit is set equal to FALSE if one or more of the following conditions are true: -cbSize is less than or equal to MinBtSizeY cbWidth is greater than maxBtSize cbHeight is greater than maxBtSize mttDepth is greater than or equal to maxMttDepth -treeType is equal to DUAL_TREE_CHROMA and (cbWidth / SubWidthC)*(cbHeight / SubHeightC) is less than or equal to 16 btSplit is equal to SPLIT_BT_VER, treeType is equal to DUAL_TREE_CHROMA, and (cbWidth / SubWidthC) is 4 or less treeType is equal to DUAL_TREE_CHROMA and modeType is equal to MODE_TYPE_INTRA ……
[0207] 6.4.3 Acceptable Trichotomy Processes The variable allowTtSplit is derived as follows: allowTtSplit is set equal to FALSE if one or more of the following conditions are true: -cbSize is 2×MinTtSizeY or less cbWidth is greater than Min(MaxTbSizeY,maxTtSize) cbHeight is greater than Min(MaxTbSizeY,maxTtSize) mttDepth is greater than or equal to maxMttDepth x0+cbWidth is greater than pic_width_in_luma_samples y0+cbHeight is greater than pic_height_in_luma_samples -treeType is equal to DUAL_TREE_CHROMA and (cbWidth / SubWidthC)*(cbHeight / SubHeightC) is less than or equal to 32 btSplit is equal to SPLIT_TT_VER, treeType is equal to DUAL_TREE_CHROMA, and (cbWidth / SubWidthC) is 8 or less treeType is equal to DUAL_TREE_CHROMA and modeType is equal to MODE_TYPE_INTRA Otherwise, allowTtSplit is set equal to TRUE.
[0208] 5.8 Exemplary embodiment #8 The following embodiment is related to enabling MODE_IBC for the SCIPU chroma block.
[0209] 7.3.8.5 Coding Unit Syntax [Table 22]
[0210] 5.9 Exemplary embodiment #9 regarding prohibition of block division when modeType is MODE_TYPE_INTER (Solution 1)
[0211] 6.4.2 Acceptable Dichotomy Processes The variable allowBtSplit is derived as follows: allowBtSplit is set equal to FALSE if one or more of the following conditions are true: -cbSize is less than or equal to MinBtSizeY cbWidth is greater than maxBtSize cbHeight is greater than maxBtSize mttDepth is greater than or equal to maxMttDepth treeType is equal to DUAL_TREE_CHROMA and (cbWidth / SubWidthC)*(cbHeight / SubHeightC) is less than or equal to 16 · modeType is equal to MODE_TYPE_INTER and cbWidth*cbHeight is less than or equal to 32 · treeType is equal to DUAL_TREE_CHROMA and modeType is equal to MODE_TYPE_INTRA ……
[0212] 6.4.3 Acceptable Trichotomy Processes The variable allowTtSplit is derived as follows: allowTtSplit is set equal to FALSE if one or more of the following conditions are true: -cbSize is 2×MinTtSizeY or less cbWidth is greater than Min(MaxTbSizeY,maxTtSize) cbHeight is greater than Min(MaxTbSizeY,maxTtSize) mttDepth is greater than or equal to maxMttDepth x0+cbWidth is greater than pic_width_in_luma_samples y0+cbHeight is greater than pic_height_in_luma_samples -treeType is equal to DUAL_TREE_CHROMA and (cbWidth / SubWidthC)*(cbHeight / SubHeightC) is less than or equal to 32 modeType is equal to MODE_TYPE_INTER and cbWidth*cbHeight is less than or equal to 64 treeType is equal to DUAL_TREE_CHROMA and modeType is equal to MODE_TYPE_INTRA Otherwise, allowTtSplit is set equal to TRUE.
[0213] 5.10 Exemplary embodiment #10 regarding prohibition of block division when modeType is MODE_TYPE_INTER (Solution 2)
[0214] 6.4.2 Acceptable Dichotomy Processes The variable allowBtSplit is derived as follows: allowBtSplit is set equal to FALSE if one or more of the following conditions are true: -cbSize is less than or equal to MinBtSizeY cbWidth is greater than maxBtSize cbHeight is greater than maxBtSize mttDepth is greater than or equal to maxMttDepth treeType is equal to DUAL_TREE_CHROMA and (cbWidth / SubWidthC)*(cbHeight / SubHeightC) is less than or equal to 16 modeType is equal to MODE_TYPE_INTER · treeType is equal to DUAL_TREE_CHROMA and modeType is equal to MODE_TYPE_INTRA ……
[0215] 6.4.3 Acceptable Trichotomy Processes The variable allowTtSplit is derived as follows: allowTtSplit is set equal to FALSE if one or more of the following conditions are true: -cbSize is 2×MinTtSizeY or less cbWidth is greater than Min(MaxTbSizeY,maxTtSize) cbHeight is greater than Min(MaxTbSizeY,maxTtSize) mttDepth is greater than or equal to maxMttDepth x0+cbWidth is greater than pic_width_in_luma_samples y0+cbHeight is greater than pic_height_in_luma_samples · treeType is equal to DUAL_TREE_CHROMA and (cbWidth / SubWidthC)*(cbHeight / SubHeightC) is less than or equal to 32 modeType is equal to MODE_TYPE_INTER treeType is equal to DUAL_TREE_CHROMA and modeType is equal to MODE_TYPE_INTRA Otherwise, allowTtSplit is set equal to TRUE.
[0216] 5.11 Exemplary embodiment #11 The following embodiment relates to constraints on further splitting of the coding tree when MODE_TYPE_INTER is derived.
[0217] 7.3.8.4 Coding Tree Syntax [Table 23]
[0218] 5.12 Exemplary embodiment #12 The following embodiment is for constraints on small block partitions and prediction modes do not apply when palette mode is enabled.
[0219] 7.4.9.4 Coding Tree Semantics The variable modeTypeCondition is derived as follows: · modeTypeCondition is set to 0 if any of the following conditions are true: slice_type==I and qtbtt_dual_tree_intr_flag is equal to 1 modeTypeCurr is not equal to MODE_TYPE_ALL sps_palette_enabled_flag is equal to 1 Otherwise, if any of the following conditions are true, modeTypeCondition is set to 1: cbWidth*cbHeight is equal to 64 and split_qt_flag is equal to 1 cbWidth*cbHeight is equal to 64 and MttSplitMode[x0][y0][mttDepth] is equal to SPLIT_TT_HOR or SPLIT_TT_VER cbWidth*cbHeight is equal to 32 and MttSplitMode[x0][y0][mttDepth] is equal to SPLIT_BT_HOR or SPLIT_BT_VER Otherwise, if any of the following conditions are true, modeTypeCondition is set to 1 + (slice_type != I ? 1:0): cbWidth*cbHeight is equal to 64 and MttSplitMode[x0][y0][mttDepth] is equal to SPLIT_BT_HOR or SPLIT_BT_VER cbWidth*cbHeight is equal to 128 and MttSplitMode[x0][y0][mttDepth] is equal to SPLIT_TT_HOR or SPLIT_TT_VER Otherwise, modeTypeCondition is set to 0.
[0220] 5.13 Exemplary embodiment #13 The following embodiment is for a small chroma intra block constraint for a 4:2:2 color format.
[0221] 7.4.9.4 Coding Tree Semantics The variable modeTypeCondition is derived as follows: · modeTypeCondition is set equal to 0 if any of the following conditions are true: slice_type==I and qtbtt_dual_tree_intr_flag is equal to 1 modeTypeCurr is not equal to MODE_TYPE_ALL Otherwise, if any of the following conditions are true, modeTypeCondition is set to 1: cbWidth*cbHeight is equal to 64 and split_qt_flag is equal to 1 cbWidth*cbHeight is equal to 64 and MttSplitMode[x0][y0][mttDepth] is equal to SPLIT_TT_HOR or SPLIT_TT_VER cbWidth*cbHeight is equal to 32 and MttSplitMode[x0][y0][mttDepth] is equal to SPLIT_BT_HOR or SPLIT_BT_VER In all other cases, chroma_format_idc equals 1, If any of the following conditions are true, modeTypeCondition is set to 1+(slice_type != I ? 1:0). cbWidth*cbHeight is equal to 64 and MttSplitMode[x0][y0][mttDepth] is equal to SPLIT_BT_HOR or SPLIT_BT_VER cbWidth*cbHeight is equal to 28 and MttSplitMode[x0][y0][mttDepth] is equal to SPLIT_TT_HOR or SPLIT_TT_VER Otherwise, modeTypeCondition is set to 0.
[0222] 5.14 Delta QP Signaling in SCIPU Example #1 [Table 24] TIFF0007581447000045.tif225170TIFF0007581447000046.tif81170
[0223] 5.15 Delta QP Signaling in SCIPU Example #2 [Table 25] TIFF0007581447000048.tif225170TIFF0007581447000049.tif81170
[0224] 5.16 Delta QP Signaling in SCIPU Example #3 [Table 26] TIFF0007581447000051.tif225170TIFF0007581447000052.tif81170
[0225] 5.17 Delta QP Signaling in SCIPU Example #4 [Table 27] TIFF0007581447000054.tif225170TIFF0007581447000055.tif81170
[0226] 5.18 Disable 2x8 Chroma Intra Block Example #14 6.4.2. Acceptable Dichotomy Processes The variable allowBtSplit is derived as follows: allowBtSplit is set equal to FALSE if one or more of the following conditions are true: -cbSize is less than or equal to MinBtSizeY cbWidth is greater than maxBtSize cbHeight is greater than maxBtSize mttDepth is greater than or equal to maxMttDepth -treeType is equal to DUAL_TREE_CHROMA and (cbWidth / SubWidthC)*(cbHeight / SubHeightC) is less than or equal to 16 treeType is equal to DUAL_TREE_CHROMA, btSplit is equal to SPLIT_BT_VER, (cbWidth / SubWidthC) is equal to 4, and (cbHeight / SubHeightC) is equal to 8 treeType is equal to DUAL_TREE_CHROMA, btSplit is equal to SPLIT_BT_HOR, (cbWidth / SubWidthC) is equal to 2, and (cbHeight / SubHeightC) is equal to 16 treeType is equal to DUAL_TREE_CHROMA and modeType is equal to MODE_TYPE_INTRA ……
[0227] 6.4.3. Acceptable Trichotomy Processes The variable allowTtSplit is derived as follows: · allowTtSplit is set equal to FALSE if one or more of the following conditions are true; -cbSize is 2×MinTtSizeY or less cbWidth is greater than Min(MaxTbSizeY,maxTtSize) cbHeight is greater than Min(MaxTbSizeY,maxTtSize) mttDepth is greater than or equal to maxMttDepth x0+cbWidth is greater than pic_width_in_luma_samples y0+cbHeight is greater than pic_height_in_luma_samples · treeType is equal to DUAL_TREE_CHROMA and (cbWidth / SubWidthC)*(cbHeight / SubHeightC) is less than or equal to 32 treeType is equal to DUAL_TREE_CHROMA, btSplit is equal to SPLIT_TT_VER, (cbWidth / SubWidthC) is equal to 8, (cbHeight / SubHeightC) is equal to 8 treeType is equal to DUAL_TREE_CHROMA, btSplit is equal to SPLIT_TT_HOR, (cbWidth / SubWidthC) is equal to 2, and (cbHeight / SubHeightC) is equal to 32 treeType is equal to DUAL_TREE_CHROMA and modeType is equal to MODE_TYPE_INTRA Otherwise, allowTtSplit is set equal to TRUE.
[0228] 7.4.9.4. Coding Tree Semantics The variable modeTypeCondition is derived as follows: · modeTypeCondition is set to 0 if any of the following conditions are true: slice_type==I and qtbtt_dual_tree_intr_flag is equal to 1 modeTypeCurr is not equal to MODE_TYPE_ALL Otherwise, if any of the following conditions are true, modeTypeCondition is set to 1: cbWidth*cbHeight is equal to 64 and split_qt_flag is equal to 1 cbWidth*cbHeight is equal to 64 and MttSplitMode[x0][y0][mttDepth] is equal to SPLIT_TT_HOR or SPLIT_TT_VER cbWidth*cbHeight is equal to 32 and MttSplitMode[x0][y0][mttDepth] is equal to SPLIT_BT_HOR or SPLIT_BT_VER Otherwise, if any of the following conditions are true, modeTypeCondition is set to 1 + (slice_type != I ? 1:0): cbWidth*cbHeight is equal to 64 and MttSplitMode[x0][y0][mttDepth] is equal to SPLIT_BT_HOR or SPLIT_BT_VER cbWidth*cbHeight is equal to 128 and MttSplitMode[x0][y0][mttDepth] is equal to SPLIT_TT_HOR or SPLIT_TT_VER cbWidth is equal to 8, (cbHeight / SubHeightC) is equal to 8, and MttSplitMode[x0][y0][mttDepth] is equal to SPLIT_BT_VER cbWidth is equal to 16, (cbHeight / SubHeightC) is equal to 8, and MttSplitMode[x0][y0][mttDepth] is equal to SPLIT_TT_VER. cbWidth is equal to 4, (cbHeight / SubHeightC) is equal to 16, and MttSplitMode[x0][y0][mttDepth] is equal to SPLIT_BT_HOR or SPLIT_TT_HOR. cbWidth is equal to 4, (cbHeight / SubHeightC) is equal to 32, and MttSplitMode[x0][y0][mttDepth] is equal to SPLIT_TT_HOR. Otherwise, modeTypeCondition is set to 0.
[0229] 7.4.9.7. Merge data semantics ciip_flag[x0][y0] specifies whether a combination of inter-picture merging and intra-picture prediction is applied to the current coding unit. The array index x0,y0 specifies the position (x0,y0) of the top-left luma sample of the considered coding block relative to the top-left luma sample of the picture. If ciip_flag[x0][y0] is not present, it is inferred as follows: ciip_flag[x0][y0] is inferred to be equal to 1 if all of the following conditions are true: sps_ciip_enabled_flag is equal to 1 general_merge_flag[x0][y0] is equal to 1 merge_subblock_flag[x0][y0] is equal to 0 regular_merge_flag[x0][y0] is equal to 0 cbWidth is less than 128 -cbHeight is less than 128 cbWidth is greater than 4 and cbHeight is not equal to 16 cbWidth*cbHeight is 64 or more Otherwise, ciip_flag[x0][y0] is inferred to be equal to 0.
[0230] 7.3.8.7. Merge data syntax [Table 28]
[0231] FIG. 17A is a block diagram of a video processing device 1700. The device 1700 may be used to implement one or more of the methods described herein. The device 1700 may be embodied in a smartphone, a tablet, a computer, an Internet of Things (IoT) receiver, etc. The device 1700 may include one or more processors 1702, one or more memories 1704, and video processing hardware 1706. The processor 1702 may be configured to implement one or more methods described herein. The memory(s) 1704 may be used to store data and codes used to implement the methods and techniques described herein. The video processing hardware 1706 may be used to implement some techniques described herein in a hardware circuit.
[0232] FIG. 17B is another example of a block diagram of a video processing system in which the disclosed techniques may be implemented. FIG. 17B is a block diagram illustrating an example video processing system 1710 in which various techniques disclosed herein may be implemented. Various implementations may include some or all of the components of the system 1710. The system 1710 may include an input 1712 for receiving video content. The video content may be received in a raw or uncompressed format, e.g., in the form of 8 or 10-bit multi-component pixel values, or may be received in a compressed or encoded format. The input 1712 may represent a network interface, a peripheral bus interface, or a storage interface. Examples of network interfaces include wired interfaces, such as Ethernet, passive optical network (PON), and wireless interfaces, such as Wi-Fi or cellular interfaces.
[0233] The system 1710 may include an encoding component 1714 that may implement various encoding or encoding methods described herein. The encoding component 1714 may reduce the average bit rate of the video from the input 1712 to the output of the encoding component 1714 to generate an encoded representation of the video. Thus, encoding techniques are sometimes referred to as video compression or video transcoding techniques. The output of the encoding component 1714 may be stored or transmitted via a communication connection as represented by component 1716. The stored or communicated bitstream (or encoded) representation of the video received at the input 1712 may be used by component 1718 to generate pixel values or displayable video that are sent to the display interface 1720. The process of generating a user-viewable video from the bitstream representation is sometimes referred to as video decompression. Additionally, although certain video processing operations are referred to as “encoding” operations or tools, it will be understood that the encoding tools or operations are used in an encoder and corresponding decoding tools or operations that reverse the results of the encoding are performed in a decoder.
[0234] Examples of peripheral bus interfaces or display interfaces include Universal Serial Bus (USB) or High Definition Multimedia Interface (MDMI) or DisplayPort, etc. Examples of storage interfaces include SATA (Serial Advanced Technology Attachment), PCI, IDE interfaces, etc. The techniques described herein may be embodied in a variety of electronic devices, such as mobile phones, laptops, smartphones, or other devices capable of performing digital data processing and / or video display.
[0235] FIG. 22 is a block diagram illustrating an example video encoding system 100 that can utilize the technique table of this disclosure.
[0236] 22, video encoding system 100 may include a source device 110 and a destination device 120. Source device 110 may be referred to as a video encoding device and may generate encoded video data. Destination device 120 may be referred to as a video decoding device and may decode the encoded video data generated by source device 110.
[0237] The source device 110 may include a video source 112 , a video encoder 114 , and an input / output (I / O) interface 116 .
[0238] The video source 112 may include a source such as a video capture device, an interface for receiving video data from a video content provider, and / or a computer graphics system for generating video data, or a combination of such sources. The video data may include one or more pictures. The video encoder 114 encodes the video data from the video source 112 to generate a bitstream. The bitstream may include a sequence of bits that form a coded representation of the video data. The bitstream may include coded pictures and associated data. A coded picture is a coded representation of a picture. The associated data may include sequence parameter sets, picture parameter sets, and other syntax structures. The I / O interface 116 may include a modulator / demodulator (modem) and / or a transmitter. The encoded video data may be transmitted directly to the destination device 120 via the I / O interface 116 over the network 130a. The encoded video data may also be stored on a storage medium / server 130b for access by the destination device 120.
[0239] The destination device 120 may include an I / O interface 126 , a video decoder 124 , and a display device 122 .
[0240] The I / O interface 126 may include a receiver and / or a modem. The I / O interface 126 may obtain encoded video data from the source device 110 or the storage medium / server 130b. The video decoder 124 may decode the encoded video data. The display device 122 may display the decoded video data to a user. The display device 122 may be integrated with the destination device 120 or may be external to the destination device 120 configured to interface with an external display device.
[0241] The video encoder 114 and the video decoder 124 may operate according to a video compression standard, such as the High Efficiency Video Coding (HEVC) standard, the Versatile Video Coding (VVC) standard, and other current and / or future standards.
[0242] FIG. 23 is a block diagram illustrating an example of a video encoder 200, which may be the video encoder 114 in the system 100 shown in FIG.
[0243] Video encoder 200 may be configured to perform any or all of the techniques of this disclosure. In the example of FIG. 9, video encoder 200 includes multiple functional components. The techniques described in this disclosure may be shared among various components of video encoder 200. In some examples, a processor may be configured to perform any or all of the techniques described in this disclosure.
[0244] Functional components of the video encoder 200 may include a splitting unit 201, a prediction unit 202 which may include a mode selection unit 203, a motion estimation unit 204, a motion compensation unit 205, and an intra prediction unit 206, a residual generation unit 207, a transform unit 208, a quantization unit 209, an inverse quantization unit 210, an inverse transform unit 211, a reconstruction unit 212, a buffer 213, and an entropy coding unit 214.
[0245] In other examples, the video encoder 200 may include more, fewer, or different functional components. In one example, the prediction unit 202 may include an intra block copy (IBC) unit. The IBC unit may perform prediction in an IBC mode, where at least one reference picture is the picture in which the current video block is located.
[0246] Furthermore, some components, such as the motion estimator 204 and the motion compensator 205, may be highly integrated, but are depicted separately in the example of FIG. 11 for purposes of illustration.
[0247] The partition unit 201 can partition a picture into one or more video blocks. The video encoder 200 and the video decoder 300 can support a variety of video block sizes.
[0248] The mode selection unit 203 may select one of the coding modes, intra or inter, for example based on the error result, and provide the resulting intra- or inter-coded block to a residual generation unit 207 that generates residual block data, and to a reconstruction unit 212 that reconstructs the encoded block for use as a reference picture. In some examples, the mode selection unit 203 may select a combined intra- and inter-prediction (CIIP) mode, where prediction is based on an inter-prediction signal and an intra-prediction signal. The mode selection unit 203 may also select the resolution of the motion vector for the block (e.g., sub-pixel or integer pixel precision) in the case of inter-prediction.
[0249] To perform inter prediction on the current video block, motion estimation unit 204 may generate motion information for the current video block by comparing one or more reference frames from buffer 213 to the current video block. Motion compensation unit 205 may determine a predicted video block for the current video block based on the motion information and decoded samples of pictures from buffer 213 other than the picture associated with the current video block.
[0250] Motion estimation unit 204 and motion compensation unit 205 may perform different operations on the current video block depending on whether the current video block is in an I slice, a P slice, or a B slice, for example.
[0251] In some examples, the motion estimation unit 204 may perform unidirectional prediction for the current video block, and the motion estimation unit 204 may search reference pictures in list 0 or list 1 for a reference video block for the current video block. The motion estimation unit 204 may then generate a reference index indicating a reference picture in list 0 or list 1 that contains the reference video block, and a motion vector indicating a spatial displacement between the current video block and the reference video block. The motion estimation unit 204 may output the reference index, the prediction direction indicator, and the motion vector as motion information for the current video block. The motion compensation unit 205 may generate a predicted video block for the current block based on the reference video block indicated by the motion information for the current video block.
[0252] In another example, the motion estimation unit 204 may perform bidirectional prediction for the current video block, and the motion estimation unit 204 may search the reference pictures in list 0 for a reference video block for the current video block and may also search the reference pictures in list 1 for another reference video block for the current video block. The motion estimation unit 204 may then generate a reference index indicating the reference pictures in list 0 and list 1 that contain the reference video blocks and a motion vector indicating a spatial displacement between the reference video block and the current video block. The motion estimation unit 204 may output the reference index and the motion vector for the current video block as motion information for the current video block. The motion compensation unit 205 may generate a prediction video block for the current video block based on the reference video block indicated by the motion information for the current video block.
[0253] In some examples, the motion estimation unit 204 may output a full set of motion information for the decoding process of the decoder.
[0254] In some examples, the motion estimator 204 may not output a full set of motion information for the current video. Rather, the motion estimator 204 may signal the motion information of the current video block by reference to motion information of another video block. For example, the motion estimator 204 may determine that the motion information of the current video block is sufficiently similar to the motion information of a neighboring video block.
[0255] In one example, motion estimation unit 204 may indicate a value in a syntax structure associated with a current video block that indicates to video decoder 300 that the current video block has the same motion information as another video block.
[0256] In another example, motion estimation unit 204 may identify another video block and a motion vector difference (MVD) in a syntax structure associated with the current video block. The motion vector difference indicates a difference between the motion vector of the current video block and the motion vector of the indicated video block. Video decoder 300 may determine the motion vector of the current video block using the motion vector of the indicated video block and the motion vector difference.
[0257] As mentioned above, video encoder 200 may predictively signal motion vectors. Two examples of predictive signaling techniques that may be implemented by video encoder 200 include advanced motion vector prediction (AMVP) and merge mode signaling.
[0258] The intra predictor 206 may perform intra prediction on the current video block. When the intra predictor 206 performs intra prediction on the current video block, the intra predictor 206 may generate predictive data for the current video block based on decoded samples of other video blocks in the same picture. The predictive data for the current video block may include the video block to be predicted and various syntax elements.
[0259] Residual generator 207 may generate residual data for the current video block by subtracting (e.g., as indicated by a minus sign) the prediction video block for the current video block from the current video block. The residual data for the current video block may include residual video blocks that correspond to different sample components of the samples in the current video block.
[0260] In other examples, such as in skip mode, residual data for the current video block may not exist and residual generator 207 may not perform a subtraction operation.
[0261] The transform processor 208 may generate one or more transform coefficient video blocks for the current video block by applying one or more transforms to the residual video block associated with the current video block.
[0262] After the transform unit 208 generates a transform coefficient video block associated with the current video block, the quantization unit 209 may quantize the transform coefficient video block associated with the current video block based on one or more quantization parameter (QP) values associated with the current video block.
[0263] An inverse quantization unit 210 and an inverse transform unit 211 may apply inverse quantization and inverse transform, respectively, to the transform coefficient video block to reconstruct a residual video block from the transform coefficient video block. A reconstruction unit 212 adds the reconstructed residual video block to corresponding samples from one or more prediction video blocks generated by the prediction unit 202 to generate a reconstructed video block related to the current block for storage in a buffer 213.
[0264] After the reconstructor 212 reconstructs the video blocks, a loop filtering operation may be performed to reduce video blocking artifacts in the video blocks.
[0265] The entropy encoder 214 may receive data from other functional components of the video encoder 200. Once the entropy encoder 214 receives the data, the entropy encoder 214 may perform one or more entropy encoding operations to generate entropy coded data and output a bitstream including the entropy coded data.
[0266] FIG. 24 is a block diagram illustrating an example of a video decoder 300, which may be the video decoder 114 in the system 100 shown in FIG.
[0267] The video decoder 300 may be configured to perform any or all of the techniques of this disclosure. In the example of FIG. 24, the video decoder 300 includes multiple functional components. The techniques described in this disclosure may be shared among various components of the video decoder 300. In some examples, a processor may be configured to perform any or all of the techniques described in this disclosure.
[0268] In the example of Figure 24, the video decoder 300 includes an entropy decoding unit 301, a motion compensation unit 302, an intra prediction unit 303, an inverse quantization unit 304, an inverse transform unit 305, a reconstruction unit 306, and a buffer 307. The video decoder 300 may, in some examples, perform a decoding pass that is generally the reverse of the encoding pass described with respect to the video encoder 200 (e.g., Figure 23).
[0269] The entropy decoding unit 301 may retrieve an encoded bitstream. The encoded bitstream may include entropy coded video data (e.g., encoded blocks of video data). The entropy decoding unit 301 may decode the entropy coded video data, and from the entropy decoded video data, the motion compensation unit 302 may determine motion information including motion vectors, motion vector precision, reference picture list index, and other motion information. The motion compensation unit 302 may determine such information, for example, by implementing AMVP and merge mode.
[0270] The motion compensation unit 302 may perform interpolation, possibly based on an interpolation filter, to generate a motion compensated block. An identifier for the interpolation filter used with sub-pixel accuracy may be included in the syntax element.
[0271] The motion compensation unit 302 may use an interpolation filter used by video encoder 200 during encoding of the video block to calculate interpolated values for sub-integer pixels of the reference block. The motion compensation unit 302 may determine the interpolation filter used by video encoder 200 according to the received syntax information and generate the prediction block using the interpolation filter.
[0272] The motion compensation unit 302 may use a portion of the syntax information to determine the size of the blocks used to encode the frames and / or slices of the encoded video sequence, partitioning information describing how each macroblock of a picture of the encoded video sequence is partitioned, a mode indicating how each partition is encoded, one or more reference frames (and reference frame lists) for each inter-coded block, and other information for decoding the encoded video sequence.
[0273] The intra prediction unit 303 may form a prediction block from spatially neighboring blocks, for example using an intra prediction mode received in the bitstream. The inverse quantization unit 303 inverse quantizes, i.e., dequantizes, the quantized video block coefficients provided in the bitstream and decoded by the entropy decoding unit 301. The inverse transform unit 303 applies an inverse transform.
[0274] The reconstruction unit 306 may add the residual blocks with corresponding prediction blocks generated by the motion compensation unit 202 or intra prediction unit 303 to form decoded blocks. If desired, a deblocking filter may also be applied to filter the decoded blocks to remove blockiness artifacts. The decoded video blocks are then stored in a buffer 307, which provides reference blocks for subsequent motion compensation / intra prediction.
[0275] 18 is a flow chart for a method 1800 of processing video. The method 1800 includes, for conversion between a video domain of a video and a coded representation of the video domain, parsing 1802 the coded representation according to syntax rules that define a relationship between a chroma block size and a color format of the video domain;
[0276] Some embodiments of the disclosed techniques include making a judgment or decision to enable a video processing tool or mode. In one example, if a video processing tool or mode is enabled, an encoder may use or implement the tool or mode in processing blocks of video, but may not necessarily modify the resulting bitstream based on the use of the tool or mode. That is, conversion of blocks of video to a bitstream representation of video uses the video processing tool or mode when enabled based on the judgment or decision. In another example, if the video processing tool or mode is enabled, a decoder processes the bitstream with knowledge that the bitstream has been modified based on the video processing tool or mode. That is, conversion of a bitstream representation of video to blocks of video is performed using the video processing tool or mode that is enabled based on the judgment or decision.
[0277] Some embodiments of the disclosed techniques include making a judgment or decision to disable a video processing tool or mode. In one example, when a video processing tool or mode is disabled, an encoder does not use the tool or mode in converting blocks of video into a bitstream representation of the video. In another example, when a video processing tool or mode is disabled, a decoder processes the bitstream with the knowledge that the bitstream has not been modified using the video processing tool or mode that was disabled based on the judgment or decision.
[0278] In this document, the term "video processing" can refer to video encoding, video decoding, video compression, or video decompression. For example, a video compression algorithm may be applied during conversion from a pixel representation of a video to a corresponding bitstream representation, or vice versa. The bitstream representation of a current video block may correspond to bits that are co-located or spread to different locations in the bitstream, for example, as defined by the syntax. For example, a macroblock may be encoded with a transformed and coded error residual value and using bits in the header and other fields in the bitstream.
[0279] The following provisions may be implemented as exemplary implementations of the disclosed technology.
[0280] The first set of clauses may include the following clauses relating to the techniques described in item 1 of the previous section: 1. A method for processing video, comprising the steps of: for conversion between a video domain of a video and an encoded representation of said video domain, parsing said encoded representation according to syntax rules defining a relationship between chroma block sizes and color formats of said video domain; and performing said conversion by performing said parsing according to said syntax rules. 2. The method of clause 1, wherein the color format is 4:4:4 and the syntax rules specify that the chroma blocks are subject to the same size constraints as luma blocks. 3. The method of claim 1, wherein the color format is 4:2:2 and the syntax rules specify that the chroma blocks are subject to the same size constraints as in a 4:2:0 color format. 4. The method of any one of clauses 1 to 3, wherein the syntax specifies that prediction modes and small block partitions are used in a chroma format dependent manner. 5. The method of claim 1, wherein the syntax rules define that a feature of minimum allowed size is enabled for conversion of the video region based on the color format of the video region.
[0281] The following provisions may be implemented in conjunction with the additional techniques described in item 2 of the previous section. 6. A method of video processing comprising the steps of: determining an encoding mode for a coding tree node of the video based on attributes of the video and a chroma format of the video; and performing a conversion between an encoded representation of the video and a video block of the coding tree node using the determined encoding mode. 7. The attribute is: i. The coding node is an M × N coding tree node with a horizontal binary tree partition; ii. The coding node is an M×N coding tree node with a vertical binary tree partition; iii. The coding node is an M × N coding tree node with a horizontal ternary tree partition; or iv. If the coding node is an M × N coding tree node with a vertical ternary tree partition, 7. The method according to clause 6, wherein the encoding mode is determined to be MODE_TYPE_ALL for the chroma format being 4:2:2, and MODE_TYPE_INTRA or MODE_TYPE_INTER for the chroma format being 4:2:0. 8. The method of claim 7, wherein M=8 or 16 or 32 and N=4 or 8 or 16.
[0282] The following provisions may be implemented in conjunction with the additional techniques described in item 12 of the previous section. 9. A video processing method comprising the steps of: determining, based on a rule, whether a certain size of chroma blocks is allowed in a video domain of a video; and performing a conversion between the video domain and an encoded representation of the video domain based on the determination. 10. The method of claim 9, wherein the rules specify that 2xN chroma blocks are prohibited because the video region includes a dual tree partition. 11. The method of claim 9, wherein the rules specify that 2N chroma blocks are not allowed because the video region includes a single tree partition. 12. The method according to clause 10 or 11, wherein N≦64.
[0283] The following provisions may be implemented in conjunction with the additional techniques described in items 13, 14, and 15 of the previous section. 13. A video processing method comprising the steps of: determining that a coding mode is allowed for a video domain based on rules that allow use of coding modes for certain video conditions; and performing a conversion between a coded representation of pixels in the video domain and pixels of the video domain based on the determination. 14. The method according to clause 13, wherein the video condition is a block size, and the rules allow the use of intra block copy mode for small block sizes of luma blocks. 15. The method of claim 14, wherein the smaller block size comprises a luma block size of 8x4, 8x8, 16x4 or 4xN. 16. The method according to clause 13, wherein said rules allow the use of intra block copy mode for transformation of a video domain using a MODE_TYPE_INTER mode encoding. 17. The method according to clause 13, wherein the rules allow the use of palette coding mode for transformation of video domains using MODE_TYPE_INTER mode coding.
[0284] The following provisions may be implemented in conjunction with the additional techniques described in items 16, 17, and 18 of the previous section. 18. A method of video processing comprising: performing a conversion between a video block of a video and an encoded representation of the video block using a video encoding mode, wherein syntax elements signaling the encoding mode are selectively included in the encoded representation based on a rule. 19. The method of claim 18, wherein the video coding mode is an intrablock coding mode and the rules specify using the type of the video coding mode to control the inclusion of the syntax element in the coded representation. 20. The method according to clause 19, wherein the rules provide for explicitly signalling non-SKIP blocks. 21. The method of claim 18, wherein the rules provide for implicitly signaling an intra block copy flag based on a skip flag and a mode type of the video block. 22. The method of clause 18, wherein the coding mode is a palette coding mode, and the rules specify selectively including a palette coding indicator based on a mode type of the video block.
[0285] The following provisions may be implemented in conjunction with the additional techniques described in item 21 of the previous section. 23. A method of video processing comprising: for a chroma block having a size smaller than a threshold size, determining that a transform type used during conversion between the chroma block and a coded representation of the chroma block is different from a transform type used for a corresponding luma block transform; and performing the conversion based on the determination. 24. The method of claim 23, wherein the threshold size is M×N, where M is 2.
[0286] The following provisions may be implemented in conjunction with the additional techniques described in item 22 of the previous section. 25. The method of any one of clauses 1 to 24, wherein the conversion uses a combined inter and intra prediction mode as MODE_TYPE_INTRA mode. 26. The method according to any one of clauses 18 to 22, wherein the conversion uses a combined inter and intra prediction mode as a MODE_TYPE_INTER mode. For example, when CIIP is considered as MODE_TYPE_INTER, the methods described in items 14 to 17 of the previous section may be applied. Alternatively, when the methods described in clauses 14 to 16 are applied, CIIP may be considered as MODE_TYPE_INTER.
[0287] The following provisions may be implemented in conjunction with the additional techniques described in items 3-6 of the previous section. 27. A method of video processing, comprising: determining, based on certain encoding conditions of a video region, whether a minimum chroma block rule is implemented during a conversion between an encoded representation of a video region and pixel values of the video region; and performing the conversion based on the determination. 28. The method of claim 17, wherein the encoding conditions include a color format of the video region. 29. The method of claim 18, wherein the video region has a width of M pixels and a height of N pixels, and the encoding condition is further dependent on the values of M and / or N. 30. The method of claim 29, wherein for video areas having 4:2:2 color format and M*N=32 or M*N=64, the minimum chroma block rule is enabled.
[0288] The following provisions may be implemented in conjunction with the additional techniques described in items 7 through 11 of the previous section. 31. A video processing method comprising the steps of: for converting between an encoded representation of a video region in 4:2:2 format and pixel values of the video region, determining a mode type to be used for the conversion based on whether a minimum chroma block rule is enabled for the video region; and performing the conversion based on the determination. 32. The method of clause 31, wherein the mode type of the video region is set to 1 because the video region has a 4:2:2 format and the minimum chroma block rule is enabled. 33. The method of clause 31, wherein determining the mode type includes determining the mode type to be an INTRA type because the minimum chroma block rule is enabled for the video region. 34. The method of clause 31, wherein determining the mode type includes determining that the mode type INTER is disabled because the minimum chroma block rule is enabled for the video region.
[0289] The following provisions may be implemented in conjunction with the additional techniques described in items 7 through 11 of the previous section. 35. A method of video processing, comprising: for converting between a coded representation of a video block and a video block of a video, determining whether block splitting is allowed during the conversion based on a mode type used during the conversion or dimensions of the video block; and performing the conversion using the determination. 36. The method of claim 35, wherein the block partitioning comprises a binary tree partitioning or a ternary tree partitioning. 37. The method of any one of clauses 35-36, wherein if the mode type is INTER mode, block partitioning is based on constraint rules that allow or prohibit partition types.
[0290] The following provisions may be implemented in conjunction with the additional techniques described in item 34 of the previous section. 38. A video processing method comprising the steps of: determining to apply a special processing mode for chroma blocks of size M×N for conversion between an encoded representation of a video segment of a video and the video segment, where M×N is an integer; and performing the conversion based on the determination. 39. The method of claim 38, wherein the special processing mode disables use of chroma blocks during the transform. 40. The method of claim 38, wherein the special processing mode uses DC prediction for intra prediction of the chroma blocks. 41. The method of clause 38, wherein the special processing mode includes using a component transverse linear model to predict intra coefficients of chroma blocks from corresponding downsampled luma coefficients. 42. The method of any of clauses 38-41, wherein the special processing mode disables use of the chroma blocks because the video segment uses dual tree partitioning. 43. The method of any of clauses 38 to 41, wherein the special processing mode disables use of the chroma blocks for the video segment using single tree partitioning. 44. The method of any of clauses 1 to 43, wherein the converting comprises encoding the video into the coded representation. 45. A method according to any preceding clause, wherein the converting comprises decoding the encoded representation to generate pixel values of the video. 46. A video decoding device comprising a processor configured to implement the methods according to one or more of clauses 1 to 45. 47. A video encoding apparatus comprising a processor configured to implement the methods according to one or more of clauses 1 to 45. 48. A computer program product having computer code stored thereon which, when executed by a processor, causes the processor to perform a method according to any of clauses 1 to 45. 49. Any method, apparatus or system described herein.
[0291] The second set of provisions may be implemented in conjunction with the additional techniques described in items 34-36 of the previous section. 1. A video processing method (e.g., method 2100 shown in FIG. 21 ), comprising: performing 2102 a conversion between a chroma block of a video region of a video picture of a video and an encoded representation of the video according to rules, the rules specifying that the chroma block is prohibited from being represented in the encoded representation using an intra-coding mode because the chroma block has a size M×N, where M and N are integers indicating a width and height of the chroma block, respectively, and the intra-coding mode comprises encoding the chroma block based on a previously encoded video region of the video picture. 2. The method of claim 1, wherein the rules further specify that a DC prediction mode is used for the chroma block, the DC prediction mode deriving a predicted value of a chroma component based on an average of neighboring blocks of the chroma block. 3. The method of claim 1, wherein the rules further specify that a component cross-linear model (CCLM) prediction mode is used for the chroma block, the CCLM prediction mode using a linear mode to derive predicted values of chroma components from other components. 4. The method of claim 2 or 3, wherein M=2 chroma samples and N=8, 16, or 32 chroma samples. 5. The method of any one of clauses 2 and 3, wherein the rules further specify that for DC or CCLM prediction modes, reconstructed neighboring pixels are not used. 6. The method of claim 2, wherein the rules further specify that neighboring reference pixels are set to 1<<(bitDepth-1) for DC prediction modes. 7. The method of claim 2 or 3, wherein the rules further specify that a position-dependent intra-prediction combining (PDPC) method is not applied for the DC prediction mode or the CCLM prediction mode, and the PDPC method combines neighboring samples with a prediction signal for the video region to generate a refined prediction signal for the video region. 8. The method of any one of clauses 2 to 3, wherein the rules further specify that information regarding the DC prediction mode or the CCLM prediction mode is derived without being encoded for the chroma block. 9. The method according to clause 8, wherein the rules further specify that for a CCLM prediction mode that satisfies predC=a*recL+b, the chroma prediction value predC is derived from the luma reconstruction value recL and the parameters a and b, which are initialized to a default fixed value at the start of the video domain. 10. The method of clause 9, wherein a first-in-first-out table is maintained for updating the parameters a and b. 11. The method according to any of clauses 1 to 10, wherein the rule is applied to another chroma block coded using the CCLM prediction and having a width not equal to M or a height not equal to N. 12. The method according to clause 11, wherein CCLM parameters of a previously coded video region are utilized for said video region instead of being derived based on reconstruction information of neighboring blocks of said video region. 13. The method of claim 11, wherein the CCLM parameters are stored in a table and updated using a first-in, first-out table. 14. A method of video processing comprising: performing a conversion between chroma blocks of a video region of a video picture of a video and a coded representation of the video according to rules, the rules specifying that the chroma blocks having size M×N are prohibited from being represented in the coded representation using an intra coding mode when dual tree partitioning is used, M and N being integers indicating a width and height of the chroma blocks, respectively, and the intra coding mode comprising coding the chroma blocks based on a previously coded video region of the video picture. 15. The method of claim 14, wherein M=2 chroma samples and N=8, 16, or 32 chroma samples. 16. The method of claim 14, wherein the rules further specify that for dual tree partitioning, a partitioning scheme is disabled for video regions that are coding tree nodes having dimensions equal to (M*a)×(N*b), where M*a is the number of chroma samples in the width of the coding tree node and N*b is the number of chroma samples in the height of the coding tree node. 17. The method of claim 16, wherein the certain partitioning scheme is a vertical binary tree (BT) partitioning in which the video region is partitioned into two parts along the vertical direction, where a=2 and b=1. 18. The method of claim 16, wherein the certain partitioning scheme is a horizontal binary tree (BT) partitioning in which the video region is partitioned into two parts along the horizontal direction, where a=1 and b=2. 19. The method of claim 16, wherein the certain partitioning scheme is a vertical ternary tree (TT) partitioning in which the video region is partitioned into three parts along the vertical direction, where a=4 and b=1. 20. The method of claim 16, wherein the certain partitioning scheme is a horizontal ternary tree (TT) partitioning in which the video region is partitioned into three parts along the horizontal direction, where a=1 and b=4. 21. A method of video processing comprising: performing a conversion between chroma blocks of a video region of a video picture of a video and a coded representation of the video according to rules, the rules specifying that the chroma blocks having size M×N are prohibited from being represented in the coded representation using an intra coding mode when single-tree partitioning is used, M and N being integers indicating a width and height of the chroma blocks, respectively, and the intra coding mode comprising coding the chroma blocks based on a previously coded video region of the video picture. 22. The method of claim 21, wherein M=2 chroma samples and N=8, 16, or 32 chroma samples. 23. The method of clause 21 or 22, wherein the rules further specify that for single tree partitioning, a partitioning scheme is disabled for chroma components of the video region that is a coding tree node having dimensions equal to (M*a*subWidthC)x(N*b*subHeightC), where M*a is the number of luma samples in the width of the coding tree node, N*b is the number of luma samples in the height of the coding tree node, and subWidthC and subHeightC indicate chroma subsampling ratios in width and height, respectively. 24. The method of claim 23, wherein the certain partitioning scheme is a vertical binary tree (BT) partitioning in which the video region is partitioned into two parts along the vertical direction, where a=2 and b=1. 25. The method of claim 23, wherein the certain partitioning scheme is a vertical ternary tree (TT) partitioning in which the video region is partitioned into three parts along the vertical direction, where a=4 and b=1. 26. The method of claim 23, wherein the certain partitioning scheme is a horizontal binary tree (BT) partitioning in which the video region is partitioned into two parts along the horizontal direction, where a=1 and b=2. 27. The method of claim 23, wherein the certain partitioning scheme is a horizontal ternary tree (TT) partitioning in which the video region is partitioned into three parts along the horizontal direction, where a=1 and b=2. 28. The method of claim 23, wherein the certain partitioning scheme is a horizontal ternary tree (TT) partitioning in which the video region is partitioned into three parts along the horizontal direction, where a=1 and b=4. 29. The method of claim 23, wherein the chroma block has dimensions equal to (M*a)×(N*d). 30. The method of claim 23, wherein the coding type of the chroma blocks is MODE_TYPE_INTRA, which allows the use of intra mode, palette mode, and intra block copy mode for the transform. 31. The method of clause 23, wherein the rules further specify that the certain partitioning scheme is allowed for a luma component of the coding tree node. 32. The method of claim 31, wherein the coding tree nodes are split into child nodes and inter and / or intra block copy modes are allowed for the child nodes. 33. The method of clause 22, wherein the rules further specify that chroma components are split along with a co-located luma component, and that inter and / or intra block copy modes are allowed for the chroma components. 34. A video processing method comprising the step of performing a conversion between chroma blocks of a video region of a video and a coded representation of said video according to rules, said rules specifying that said chroma blocks having a size M×N are prohibited from being represented in said coded representation using a combined inter- and intra-prediction (CIIP) mode that combines inter-prediction with intra-prediction, where M and N are integers indicating the width and height of said chroma blocks, respectively. 35. The method of claim 34, wherein the rules further specify that CIIP mode is not allowed for chroma components of the video region that is a coding tree node having dimensions equal to (M*subWidthC)×(N*subHeightC), whereby subWidthC and subHeightC indicate chroma subsampling ratios in the width and height dimensions, respectively. 36. The method of clause 34, wherein performing the transformation includes a chroma mixing process that determines a chroma prediction block for the chroma block by mixing predictions for the chroma block using the inter prediction when the video region is coded using the CIIP mode. 37. The method of clause 36, wherein the rules further specify that luma intra prediction is applied to the video region and that chroma intra prediction is not applied to the video region having dimensions equal to (M*subWidthC)×(N*subHeightC), whereby subWidthC and subHeightC indicate chroma subsampling ratios in the width and height dimensions, respectively. 38. The method of claim 34, wherein the rules further specify that DC prediction is used for the chroma blocks having the size M×N if the video region is coded using the CIIP mode. 39. The method according to clause 34, wherein the rules further specify that if the video region is coded using the CIIP mode, for chroma blocks having size M×N, CCLM prediction using a linear mode is used to derive a predicted value of a chroma component from another component. 40. A video processing method comprising: performing a transformation between a video region of a video and a coded representation of the video according to rules, the rules specifying whether and how a maximum transform size applicable to the video region is signaled depends on a coding mode type used to represent the video region in the coded representation, the coding mode type being (i) a near-lossless coding mode in which a quantization parameter (QP) applied to the video region is less than a threshold, (ii) a lossless coding mode in which transformation of residual data for the video region is bypassed, or (iii) a lossy coding mode in which transformation of residual data for the video region is skipped. 41. The method of claim 40, wherein the rules further specify that a maximum transform size for the lossless and / or near lossless encoding modes is equal to maxLosslessTbSizeY. 42. The method of clause 40, wherein the lossless coding mode is signaled at a coding unit level using syntax element cu_transquant_bypass_flag. 43. The method of claim 40, wherein the threshold is signaled in a decoder parameter set (DPS), a video parameter set (VPS), a sequence parameter set (SPS), a picture parameter set (PPS), a tile, slice, or brick header. 44. The method of clause 40, wherein transform_skip_flag is equal to 1 for a near lossless coding mode of the video region. 45. The method of clause 40, wherein the rules further specify that a first maximum transform size for the lossless and / or near lossless encoding modes of the video region is set based on i) another maximum transform size used for the lossy encoding mode, ii) a maximum coding tree unit size, and / or iii) a maximum transform skip size. 46. The method of clause 40, wherein the maximum transform size for the lossless or near-lossless encoding mode of the video region is signaled at a video unit level corresponding to a syntax element (SE) in a sequence parameter set (SPS), a video parameter set (VPS), a picture parameter set (PPS), a sub-picture, a slice, a tile, a brick, a coding tree unit (CTU), or a virtual pipeline data unit (VPDU). 47. The method of claim 46, wherein the SE is signaled depending on a maximum coding tree block size (CTB), a maximum transform size, and / or a maximum transform skip size. 48. The method of clause 47, wherein SE is signaled if log2_ctu_size_minus5 is greater than N, where N is an integer. 49. The method of claim 47, wherein SE is signaled if sps_max_luma_transform_size_64_flag is equal to M, where M is an integer. 50. The method of clause 47, wherein if log2_transform_skip_max_size_minus2 is greater than K, then SE is signaled. 51. The method of clause 47, wherein a difference between log(maxLosslessTbSizeY) and X is signaled, where maxLosslessTbSizeY indicates a maximum transform size for the near lossless or lossless encoding mode of the video region. 52. The method of clause 51, wherein X is equal to log2_ctu_size_minus5, sps_max_luma_transform_size_64_flag, log2_transform_sk_max_size_minus2, or a fixed integer. 53. The method of clause 47, wherein a difference between maxLosslessTbSizeY and X is signaled, where maxLosslessTbSizeY indicates a maximum transform size for the near lossless or lossless encoding mode of the video region. 54. The method of claim 47, wherein a value corresponding to a difference between maxLosslessTbSizeY and XK is signaled, where maxLosslessTbSizeY indicates a maximum transform size for the near lossless or lossless encoding mode of the video region, and K is an integer. 55. The method of clause 53 or 54, wherein X is equal to CtbSizeY, MaxTbSize, MaxTsSize, or a fixed integer. 56. The method of clause 46, wherein maxLosslessTbSizeY of the video region is derived from the SE, where maxLosslessTbSizeY indicates the maximum transform size for the near lossless or lossless encoding mode of the video region. 57. The method of claim 46, wherein the syntax element corresponds to a flag. 58. The method of clause 57, wherein if the flag is not present, a value that is a positive integer value is presumed. 59. The method of clause 40, wherein the rules further specify that the maximum transform size is derived for the lossless and / or near lossless encoding modes of the video domain. 60. The method of clause 59, wherein the maximum transform size is set equal to i) a fixed integer value, ii) another maximum transform size used for the lossy coding mode, iii) a maximum coding tree unit size, iv) a VPDU size, v) a maximum transform skip block size, or vi) a maximum residual coding block size. 61. A method of video processing, comprising the steps of: performing a transformation between a video domain of a video and a coded representation of said video according to a rule; 4. The method of claim 3, wherein the rules specify that a residual block of a video domain is divided into one or more residual sub-blocks for lossless and / or near-lossless encoding based on a maximum transform size of the video domain to which lossless and / or near-lossless encoding is applied, the lossless encoding including bypassing transform of residual blocks for the video domain, and the near-lossless encoding using a quantization parameter (QP) less than a threshold. 62. The method of clause 61, wherein the rules further specify that a width and height of the one or more residual sub-blocks resulting after the division is no greater than the maximum transform size. 63. The method of clause 62, wherein the rules further provide for repeating the division of the residual block until a width and height of the one or more residual sub-blocks is less than or equal to the maximum transform size. 64. A method according to any preceding clause, wherein the converting comprises encoding the video into the coded representation. 65. A method according to any preceding clause, wherein the converting comprises decoding the encoded representation to generate the video. 66. A video processing device comprising a processor configured to implement the methods according to one or more of clauses 1 to 65. 67. A computer readable medium having program code stored thereon, the program code, when executed, causing a processor to implement the method set out in any one or more of clauses 1 to 65. 68. A computer-readable medium storing a coded or bitstream representation generated according to any of the methods described above.
[0292] The disclosed and other solutions, examples, embodiments, modules, and functional operations described herein can be implemented in digital electronic circuitry, or in computer software, firmware, or hardware, including the structures disclosed herein and their structural equivalents, or in one or more combinations thereof. The disclosed and other embodiments can be implemented as one or more computer program products, i.e., one or more modules of computer program instructions encoded on a computer-readable medium for execution by or for controlling the operation of a data processing apparatus. The computer-readable medium can be a machine-readable storage device, a machine-readable storage substrate, a memory device, a composition of matter embodying a machine-readable propagated signal, or one or more combinations thereof. The term "data processing apparatus" encompasses all apparatus, devices, and machines for processing data, including, for example, a programmable processor, a computer, or multiple processors or computers. An apparatus can include, in addition to hardware, code that creates an execution environment for the computer program in question, e.g., code constituting a processor firmware, a protocol stack, a database management system, an operating system, or one or more combinations thereof. A propagated signal is an artificially generated signal, for example a machine-generated electrical, optical, or electromagnetic signal generated to encode information for transmission to a suitable receiver device.
[0293] A computer program (also known as a program, software, software application, script, or code) can be written in any form of programming language, including compiled or interpreted languages, and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. A program can be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in several coordinated files (e.g., a file storing one or more modules, subprograms, or code portions). A computer program can be deployed to be executed on one computer, or on several computers located at one site or distributed across several sites and interconnected by a communications network.
[0294] The processes and logic flows described herein may be performed by one or more programmable processors executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows may also be performed by, and apparatus may be implemented as, special purpose logic circuitry, such as an FPGA (field programmable gate array) or an ASIC (application specific integrated circuit).
[0295] Processors suitable for executing computer programs include, for example, both general purpose and special purpose microprocessors, and any one or more processors of any kind of digital computer. Generally, a processor receives instructions and data from a read-only memory or a random access memory, or both. The essential elements of a computer are a processor for executing instructions and one or more memory devices for storing instructions and data. Generally, a computer also includes, or is operatively coupled to receive data from, or transfer data to, one or more mass storage devices, e.g., magnetic disks, magneto-optical disks, or optical disks, for storing data. However, a computer need not have such devices. Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, including, for example, semiconductor memory devices, e.g., EPROM, EEPROM, and flash memory devices; magnetic disks, e.g., internal hard disks or removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks. The processor and memory may be supplemented by, or incorporated in, special purpose logic circuitry.
[0296] Although this patent document contains many specifics, these should not be construed as limitations on the scope of any subject matter or what may be claimed, but rather as descriptions of features that may be specific to certain embodiments of a particular technique. Certain features described in this patent document in the context of separate embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented in multiple embodiments separately or in any suitable subcombination. Furthermore, although features may be described above as acting in certain combinations, and even initially claimed as such, one or more features from a claimed combination may, in some cases, be deleted from the combination, and the claimed combination may be directed to a subcombination or a variation of the subcombination.
[0297] Similarly, although the figures depict acts in a particular order, this should not be construed as requiring that such acts be performed in the particular order depicted, or sequentially, or that all of the depicted acts be performed in order to achieve desired results. Further, the separation of various system components in the embodiments described in this patent document should not be understood as requiring such separation in all embodiments.
[0298] Only a few implementations and examples have been described; other implementations, improvements and modifications may be made based on what is described and explained in this patent document.
Claims
1. 13. A method of video processing comprising: determining whether to split the chroma blocks according to a rule related to a tree type of the chroma blocks and a dimension of the chroma blocks for conversion between a chroma block of a video and a bitstream of the video; performing said conversion based on said determination; The rule specifies that splitting the chroma block having a tree type of DUAL_TREE_CHROMA to generate a chroma sub-block with width 2 is not permitted; If the mode type of the chroma block is MODE_TYPE_INTRA, the rules further specify that splitting the chroma block to generate a chroma sub-block with width 4 and height 2 is also not allowed. method.
2. 2. The method of claim 1, wherein the rules specify that if the tree type of the chroma block is equal to DUAL_TREE_CHROMA and the width of the chroma block is equal to 4, then the chroma block is not allowed to be vertically bisected.
3. 3. The method of claim 1, wherein the rules specify that if the tree type of the chroma block is equal to DUAL_TREE_CHROMA and the width of the chroma block is equal to 8, then the chroma block is not allowed to be split into vertical thirds.
4. The method of claim 1 , wherein the rules further specify that a luma block corresponding to the chroma block is allowed to be split to generate luma sub-blocks.
5. 5. The method of claim 4, wherein the rules further specify that if the luma block corresponding to the chroma block has 64 luma samples, a slice type of the chroma block is an I slice, and a chroma format of the chroma block is 4:2:0, then splitting the chroma block having the tree type of DUAL_TREE_CHROMA using bisection is not allowed, and bisection of the luma block corresponding to the chroma block is allowed.
6. 6. The method of claim 4 or 5, wherein the rules further specify that if the luma block corresponding to the chroma block has 128 luma samples, the slice type of the chroma block is an I-slice, and the chroma format of the chroma block is 4:2:0, then splitting the chroma block having the tree type of DUAL_TREE_CHROMA using third splitting is not allowed, and third splitting of the luma block is allowed.
7. The method of claim 1 , wherein chroma blocks with a width less than 4 are not allowed to have a composite inter-intra prediction mode applied to them.
8. The method of claim 7 , wherein luma blocks with width equal to 4 and height equal to 16 are not allowed to have a composite inter-intra prediction mode applied to them.
9. The method of claim 1 , wherein the transforming comprises encoding the chroma blocks into the bitstream.
10. The method of claim 1 , wherein the transforming comprises decoding the chroma blocks from the bitstream.
11. 1. A video processing apparatus having a processor and a non-transitory memory having instructions that, when executed by the processor, cause the processor to: determining whether to split the chroma blocks according to a rule related to a tree type of the chroma blocks and a dimension of the chroma blocks for conversion between a chroma block of a video and a bitstream of the video; and performing the conversion based on the determination. the rules specify that splitting the chroma block having a tree type of DUAL_TREE_CHROMA to generate a chroma sub-block with width 2 is not permitted; If the mode type of the chroma block is MODE_TYPE_INTRA, the rules further specify that splitting the chroma block to generate a chroma sub-block with width 4 and height 2 is also not allowed. Device.
12. A non-transitory computer-readable storage medium storing instructions that cause a processor to: determining whether to split the chroma blocks according to a rule related to a tree type of the chroma blocks and a dimension of the chroma blocks for conversion between a chroma block of a video and a bitstream of the video; and performing the conversion based on the determination. the rules specify that splitting the chroma block having a tree type of DUAL_TREE_CHROMA to generate a chroma sub-block with width 2 is not permitted; If the mode type of the chroma block is MODE_TYPE_INTRA, the rules further specify that splitting the chroma block to generate a chroma sub-block with width 4 and height 2 is also not allowed. storage medium.
13. A method for storing a bitstream of video, comprising: determining, for a chroma block of the video, whether to split the chroma block according to a rule related to a tree type of the chroma block and a dimension of the chroma block; generating the bitstream based on the determination; storing the bitstream on a non-transitory computer readable recording medium; Including, the rules specify that splitting the chroma block having a tree type of DUAL_TREE_CHROMA to generate a chroma sub-block with width 2 is not permitted; If the mode type of the chroma block is MODE_TYPE_INTRA, the rules further specify that splitting the chroma block to generate a chroma sub-block with width 4 and height 2 is also not allowed. method.
Citation Information
Cited By
Size restriction based for chroma intra mode
US12294701B2
