Method, device and medium for processing video data

Abstract

Methods, apparatus, and systems for processing video are described. An example method includes performing conversion between a video including a video unit and a bitstream of the video, wherein the bitstream conforms to a format rule, and wherein the format rule provides that, responsive to whether or how the video unit is partitioned into slices, a first syntax element indicating whether a loop filtering operation is performed across a slice boundary is selectively included in the bitstream.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application is based on international patent application PCT / CN2021 / 076248 filed on February 9, 2021, and promptly claims priority to and benefits of international patent application No. PCT / CN2020 / 075216 filed on February 14, 2020. For all legal purposes, the entire disclosure of the foregoing application is incorporated herein by reference as part of the disclosure of this application. Technical Field

[0003] This patent document relates to image and video encoding and decoding. Background Art

[0004] Digital video accounts for the largest use of bandwidth on the Internet and other digital communications networks. As the number of connected user devices capable of receiving and displaying video increases, bandwidth requirements for digital video usage are expected to continue to grow. Summary of the invention

[0005] This document discloses techniques used by video encoder and decoder processes to process a codec representation of video using control information useful for decoding the codec representation.

[0006] In one example aspect, a video processing method is disclosed. The method includes: performing conversion between a video including a video unit and a bitstream of the video, wherein the bitstream complies with a format rule, and wherein the format rule provides that, in response to whether or how the video unit is partitioned into slices, a first syntax element indicating whether a loop filtering operation is performed across a slice boundary is selectively included in the bitstream.

[0007] In another example aspect, a video processing method is disclosed. The method includes: performing conversion between a video unit of a video and a bitstream of the video, wherein the bitstream conforms to a format rule, and wherein the format rule specifies that, in response to whether or how the video unit is partitioned into slices, a syntax element indicating whether to perform a loop filtering operation across a slice boundary is selectively included in the bitstream.

[0008] In another example aspect, a video processing method is disclosed. The method includes performing conversion between a video region of a video and a bitstream of the video, wherein the bitstream conforms to a format rule, and wherein the format rule specifies that information about the applicability of a tool for the conversion is indicated in the bitstream at a video slice level and / or at a video picture level, and wherein the tool maps luma samples to specific values ​​and selectively applies scaling operations to values ​​of chroma samples.

[0009] In another example aspect, a video processing method is disclosed. The method includes: performing conversion between a video region of a video and a bitstream of the video, wherein the conversion complies with a size rule, and wherein the size rule specifies a maximum size of a video region encoded using a transform skip (TS) codec scheme or a block-based delta pulse codec modulation (BDPCM) codec scheme, or a maximum size of a transform block for a video region according to a codec characteristic of the video region.

[0010] In another example aspect, a video processing method is disclosed. The method includes: performing conversion between a video region of a video and a bitstream of the video, wherein the bitstream conforms to a format rule, the format rule specifying that a minimum allowed codec block size used during conversion controls whether and / or how a maximum allowed transform block size is included in the bitstream.

[0011] In another example aspect, a video processing method is disclosed. The method includes: performing conversion between a video region of a video and a bitstream of the video, wherein the bitstream conforms to a format rule, the format rule specifying that a minimum allowed codec block size used during conversion controls whether and / or how an indication of a maximum size of a video region for encoding or decoding using a transform skip (TS) scheme or a block-based delta pulse codec modulation (BDPCM) scheme is included in the bitstream.

[0012] In another example aspect, a video processing method is disclosed. The method includes: performing conversion between a video unit of a video and a bitstream of the video, wherein the bitstream complies with a format rule, the format rule specifying that a minimum allowed codec block size used during conversion controls whether and / or how a field indicating enabling or disabling of a transform skip (TS) scheme or a block-based delta pulse codec modulation (BDPCM) scheme is included in the bitstream.

[0013] In another example aspect, a video processing method is disclosed. The method includes: performing conversion between a video unit of a video and a bitstream of the video according to a format rule, wherein the bitstream conforms to the format rule, and the format rule specifies that a minimum allowed codec block size used during conversion controls whether and / or how a field carrying information about applicability of a codec tool is included in a video region level of the bitstream.

[0014] In another example aspect, a video processing method is disclosed. The method includes: for conversion between a video region of a video and a bitstream of the video, due to a dual-tree coding scheme being enabled for the video region, determining that a partitioning scheme for a luma component and a chroma component of the video has different minimum allowed block sizes, and performing the conversion based on the determination.

[0015] In another example aspect, a video processing method is disclosed. The method includes: for a conversion between a video region of a video and a bitstream of the video, determining a maximum number of sub-block-based merge candidates allowed for the video region based on a rule; and performing the conversion based on the determination, wherein the rule stipulates that the maximum number of sub-block-based merge candidates used during the conversion is derived as a sum of a first variable and a second variable, wherein the first variable is equal to zero in response to affine prediction being disabled, and wherein the second variable is based on whether sub-block-based temporal motion vector prediction (sbTMVP) is enabled.

[0016] In another example aspect, a video processing method is disclosed. The method includes: performing conversion between a video region of a video and a bitstream of the video region by complying with a processing rule, wherein the processing rule is applicable to the conversion because the video is a 4:2:2 video or a 4:4:4 video, wherein the processing rule defines chrominance and luma alignment for one or more of the following: (a) the number of pixel rows between a virtual boundary of an adaptive loop filter (ALF) operation and a bottom boundary of a codec tree block (CTB); or (b) the filtering strength of a filter for a row between a virtual boundary of an ALF operation and a bottom boundary of a CTB; or (c) a padding method for padding luma and chroma samples in the same row.

[0017] In another example aspect, a video processing method is disclosed. The method includes: for conversion between a video unit of a video and a codec representation of the video, determining whether an indication of suitability of in-loop filtering across a video region of the video unit is included in the codec representation; and performing the conversion based on the determination.

[0018] In another example aspect, a video processing method is disclosed. The method includes: performing a conversion between a video region of a video and a codec representation of the video, wherein the codec representation conforms to a format rule, wherein the format rule specifies that information about the applicability of a luma mapping and chroma scaling (LMCS) tool for the conversion is indicated in the codec representation at a video slice level; wherein the LMCS tool includes constructing a current video block based on a first domain and a second domain during the conversion, and / or scaling a chroma residual in a luma-dependent manner.

[0019] In another example aspect, a video processing method is disclosed. The method includes: performing conversion between a video and a codec representation of the video, wherein the conversion complies with a size rule, the size rule specifying: during encoding, enforcing the size rule for a maximum size of a video region encoded using a transform skip decoding scheme or a delta pulse codec modulation decoding scheme, or during decoding, parsing and decoding the codec representation by enforcing the size rule for a maximum size of a video region decoded using a transform skip decoding scheme or a delta pulse codec modulation decoding scheme.

[0020] In another example aspect, a video processing method is disclosed. The method includes: performing a conversion between a video region of a video and a codec representation of the video, wherein the codec representation conforms to a format rule, the format rule providing that a minimum allowed transform block size used during the conversion controls whether or how an indication of a maximum allowed transform block size is included in the codec representation.

[0021] In another example aspect, a video processing method is disclosed. The method includes: performing conversion between a video unit of a video and a codec representation of the video according to a format rule, wherein the codec representation conforms to the format rule, and the format rule specifies that a minimum allowed codec block size used during conversion controls whether a field carrying information about applicability of a codec tool is included in a video region level.

[0022] In another example aspect, a video processing method is disclosed. The method includes: for a conversion between a video region of a video and a codec representation of the video, determining that a partitioning scheme for a luma component and a chroma component of the video has different minimum allowed block sizes for the luma component and the chroma component due to using a dual-tree codec for the conversion, and performing the conversion based on the determination.

[0023] In another example aspect, a video processing method is disclosed. The method includes: performing conversion between a video region of a video and a codec representation of the video region by conforming to a format rule for the codec representation; wherein the format rule specifies that a maximum number of subblock-based merge candidates used during the conversion is derivable as a sum of a first variable and a second variable, wherein a value of the first variable is controlled using affine prediction, and wherein a value of the second variable is controlled using subblock-based temporal motion vector prediction.

[0024] In another example aspect, a video processing method is disclosed. The method includes: performing conversion between a video region of the video and a codec representation of the video region by conforming to a processing rule applicable to the conversion, where the processing rule defines alignment of chrominance and luminance with one or more of: (a) the number of pixel rows between a virtual boundary of an adaptive loop filter operation and a bottom boundary of a codec tree block (CTB); or (b) a filter strength of a filter for a row between a virtual boundary of an adaptive loop filter operation and a bottom boundary of a codec tree block; or (c) a padding method for padding video samples in the same row.

[0025] In yet another exemplary aspect, a video encoder apparatus is disclosed. The video encoder includes a processor configured to implement the above method.

[0026] In yet another exemplary aspect, a video decoder apparatus is disclosed. The video decoder comprises a processor configured to implement the above method.

[0027] In yet another exemplary aspect, a computer readable medium is disclosed, on which code is stored. The code is in the form of processor executable code to implement one of the methods described herein.

[0028] These and other features are described herein. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 An example of a grid-scanning stripe partitioning of a picture is shown, where the picture is partitioned into 12 slices and 3 grid-scanning stripes.

[0030] Figure 2 An example of rectangular strip partitioning of a picture is shown, where the picture is partitioned into 24 slices (6 slice columns and 4 slice rows) and 9 rectangular strips.

[0031] Figure 3 An example of a picture partitioned into slices and rectangular strips is shown, where the picture is partitioned into 4 slices (2 slice columns and 2 slice rows) and 4 rectangular strips.

[0032] Figure 4 A picture partitioned into 15 slices, 24 slices and 24 sub-pictures is shown.

[0033] Figure 5 The nominal vertical and horizontal positions of 4:2:2 luma and chroma samples in the picture are shown.

[0034] Figure 6 Examples of ALF filter shapes are shown.

[0035] Figures 7A-7D A subsampled Laplace calculation is shown. Fig. 7A The subsampling locations of the vertical gradient are shown. Figure 7B The sub-sampling positions of the horizontal gradient are shown. Figure 7C The sub-sampling positions of the diagonal gradients are shown. Fig.7D The sub-sampling positions of the diagonal gradients are shown.

[0036] Figure 8 An example of loop filtering line buffer requirements for luma components in VTM-4.0 is shown.

[0037] Fig. 9 An example of loop filtering line buffer requirements for chroma components in VTM-4.0 is shown.

[0038] Fig.10 Improved block classification at virtual boundaries is shown.

[0039] Fig.11 An example of an improved ALF filter for brightness classification at a virtual boundary is shown.

[0040] Figures 12A-12C The improved luminance ALF filter at the virtual boundary is shown.

[0041] Fig.13 An example of repeated padding of the luma ALF filter at picture or sub-picture or slice or slice boundaries is shown.

[0042] Figures 14A-14D An example of ALF mirror population is shown.

[0043] Fig.15 A block diagram of an example video processing system is shown.

[0044] Fig.16 A block diagram of a video processing device is shown.

[0045] Fig.17 A flow chart illustrating an example method of video processing is shown.

[0046] Fig.18 is a block diagram illustrating a video encoding and decoding system according to some embodiments of the present disclosure.

[0047] Fig.19 is a block diagram illustrating an encoder according to some embodiments of the present disclosure.

[0048] Fig. 20 is a block diagram illustrating a decoder according to some embodiments of the present disclosure.

[0049] Figure 21-31 A flow chart illustrating an example method of video processing is shown. DETAILED DESCRIPTION

[0050] In this article, the section titles are used for ease of understanding and do not limit the applicability of the techniques and embodiments disclosed in each section to only that section. In addition, the use of H.266 terminology in some descriptions is only for ease of understanding and is not intended to limit the scope of the disclosed technology. Therefore, the technology described here is also applicable to other video codec protocols and designs.

[0051] 1. Overview

[0052] This article relates to video coding techniques. Specifically, the signaling of sub-pictures, slices and strips. These concepts can be applied alone or in various combinations to any video coding standard or non-standard video codec that supports multi-layer video coding, such as the Versatile Video Codec (VVC) under development.

[0053] 2. Abbreviations

[0054] APS Adaptive Parameter Set

[0055] AU Access Unit

[0056] AUD Access Unit Delimiter

[0057] AVC Advanced Video Codec

[0058] CLVS Codec Layer Video Sequence

[0059] CPB codec picture buffer

[0060] CRA Clear Random Access

[0061] CTU Codec Tree Unit

[0062] CVS codec video sequence

[0063] DPB decoded picture buffer

[0064] DPS Decoding Parameter Set

[0065] EOB End of bitstream

[0066] EOS sequence ends

[0067] GDR Progressive Decode Refresh

[0068] HEVC High Efficiency Video Codec

[0069] HRD Virtual Reference Decoder

[0070] IDR Instant Decode Refresh

[0071] JEM Joint Exploration Model

[0072] MCTS Motion Constraints Episode Set

[0073] NAL Network Abstraction Layer

[0074] OLS output layer set

[0075] PH Image Header

[0076] PPS Picture Parameter Set

[0077] PTL Profiles, Tiers, and Levels

[0078] PU Picture Unit

[0079] RBSP Raw Byte Sequence Payload

[0080] SEI Supplemental Enhancement Information

[0081] SPS Sequence Parameter Set

[0082] SVC Scalable Video Codec

[0083] VCL video codec layer

[0084] VPS Video Parameter Set

[0085] VTM VVC test model

[0086] VUI Video Availability Information

[0087] VVC Multi-functional Video Codec

[0088] 3. Preliminary Discussion

[0089] Video codec standards have evolved primarily through the development of the known ITU-T and ISO / IEC standards. ITU-T developed H.261 and H.263, ISO / IEC developed MPEG-1 and MPEG-4 video, and the two organizations jointly developed H.262 / MPEG-2 video and H.264 / MPEG-4 Advanced Video Codec (AVC) and H.265 / HEVC standards. Since H.262, video codec standards have been based on a hybrid video codec structure, which uses temporal prediction plus transform codec. In order to explore future video codec technologies beyond HEVC, VCEG and MPEG jointly established the Joint Video Exploration Team (JVET) in 2015. Since then, JVET has adopted many new methods and applied them to reference software called the Joint Exploration Model (JEM). JVET meetings are held simultaneously every quarter, and the goal of the new codec standard is to reduce the bit rate by 50% compared to HEVC. The new video codec standard was officially named Versatile Video Coding (VVC) at the JVET meeting in April 2018, when the first version of the VVC Test Model (VTM) was released. With the continuous efforts of VVC standardization, new codec technologies are adopted into the VVC standard at each JVET meeting. The VVC working draft and test model VTM are updated after each meeting. The VVC project now aims to achieve technical completion (FDIS) at the July 2020 meeting.

[0090] 3.1. Picture segmentation scheme in HEVC

[0091] HEVC includes four different picture segmentation schemes, namely normal slices, dependent slices, tiles, and wavefront parallel processing (WPP), which can be applied to maximum transmission unit (MTU) size matching, parallel processing, and reduction of end-to-end delay.

[0092] Regular slices are similar to those in H.264 / AVC. Each regular slice is encapsulated in its own NAL unit, and intra-picture prediction (intra-sample prediction, motion information prediction, codec mode prediction) and entropy coding dependencies across slice boundaries are disabled. Therefore, regular slices can be reconstructed independently of other regular slices in the same picture (although there may still be interdependencies due to loop filtering operations).

[0093] Regular slices are the only tool available for parallelization, which is also available in H.264 / AVC in almost the same form. Regular slice-based parallelization does not require much inter-processor or inter-core communication (except for inter-processor or inter-core data sharing for motion compensation when decoding predictive codec pictures, which is usually much heavier than inter-processor or inter-core data sharing due to intra-picture prediction). However, for the same reason, the use of regular slices may incur a large codec overhead due to the bit cost of the slice header and the lack of prediction across slice boundaries. In addition, regular slices (in contrast to the other tools mentioned below) also serve as a key mechanism for bitstream segmentation to match MTU size requirements due to their intra-picture independence and the fact that each regular slice is encapsulated in its own NAL. In many cases, the goals of parallelization and MTU size matching have conflicting requirements on the layout of slices in a picture. The realization of such situations led to the development of the parallelization tools mentioned below.

[0094] Dependent slices have short slice headers and allow the bitstream to be partitioned at treeblock boundaries without breaking any intra-picture prediction. Basically, dependent slices split a regular slice into multiple NAL units, reducing end-to-end latency by allowing part of a regular slice to be sent before encoding of the entire regular slice is complete.

[0095] In WPP, a picture is partitioned into a single row of codec tree blocks (CTBs). Entropy decoding and prediction are allowed to use data from CTBs in other partitions. Parallel processing is possible through parallel decoding of CTB rows, where the start of decoding of CTB rows is delayed by two CTBs to ensure that data related to CTBs above and to the right of the main CTB is available before the decoded main CTB. Using this staggered start (which looks like a wavefront when represented graphically), as many processors / cores as pictures contain CTB rows can be parallelized. Because intra-picture prediction between neighboring tree block rows within a picture is allowed, the inter-processor / inter-core communication required to implement intra-picture prediction can be substantial. WPP partitioning does not result in the generation of additional NAL units compared to when WPP partitioning is not applied, so WPP is not a tool for MTU size matching. However, if MTU size matching is required, regular slices can be used with WPP, but with certain codec overhead.

[0096] Slices define the horizontal and vertical boundaries that divide the image into slice columns and slice rows. Slice columns extend from the top of the image to the bottom of the image. Similarly, slice rows extend from the left side of the image to the right side of the image. The number of slices in a picture can be simply calculated by multiplying the number of slice columns by the number of slice rows.

[0097] Before decoding the top left CTB of the next slice in the order of the slice grid scan of one picture, the scanning order of the CTBs is changed to be local within the slice (in the order of the CTB grid scan of the slice). Similar to regular slices, slices break intra-picture prediction dependencies and entropy decoding dependencies. However, they do not need to be contained in independent NAL units (the same as WPP in this respect); therefore slices cannot be used for MTU size matching. Each slice can be processed by one processor / core, and in the case where a slice spans multiple slices, the inter-processor / inter-core communication required for intra-picture prediction between processing units decoding neighboring slices is limited to transmitting a shared slice header and loop filtering related to reconstruction sample points and metadata sharing. When more than one slice or WPP segment is contained in a slice, the entry point byte offset of each slice or WPP segment in the slice except the first is signaled in the slice header.

[0098] For simplicity, restrictions on the application of four different picture partitioning schemes are specified in HEVC. For most profiles specified in HEVC, a given codec video sequence cannot contain both slices and wavefronts. For each slice and slice, one or both of the following conditions must be met: 1) all codec tree blocks in a slice belong to the same slice; 2) all codec tree blocks in a slice belong to the same slice. Finally, a wavefront segment contains exactly one CTB row, and when WPP is used, if a slice starts within a CTB row, then the slice must end in the same CTB row.

[0099] The most recent revision of HEVC is specified in the JCT-VC output document JCTVC-AC1005, J. Boyce, A. Ramasubramonian, R. Skupin, G. J. Sullivan, A. Tourapis, Y.-K. Wang (eds.), "HEVC Additional Supplemental Enhancement Information (Draft 4)", published on October 24, 2017 at http: / / phenix.intevry.fr / jct / doc_end_user / documents- / 29_Macau / wg11 / JCTVC-AC1005-v2.zip. Contained within this revision, HEVC specifies three MCT-related SEI messages, namely the time-domain MCTS SEI message, the MCTS Extraction Information Set SEI message, and the MCTS Extraction Information Nesting SEI message.

[0100] The temporal MCTS SEI message indicates the presence of MCTS in the bitstream and signals the MCTS. For each MCTS, the motion vector is restricted to point to the full sample position within the MCTS and the fractional sample position that only needs to be interpolated from the full sample position within the MCTS, and the motion vector candidate predicted by the temporal motion vector derived from the block outside the MCTS is not allowed. In this way, each MCTS can be decoded independently in the absence of slices not included in the MCTS.

[0101] The MCTS extraction information set SEI message provides supplementary information that can be used for MCTS sub-bitstream extraction (specified as part of the semantics of the SEI message) to generate a bitstream that conforms to the MCTS set. The information consists of multiple extraction information sets, each of which defines multiple MCTS sets and contains RBSP bytes that replace the VPS, SPS, and PPS to be used in the MCTS sub-bitstream extraction process. When extracting a sub-bitstream according to the MCTS sub-bitstream extraction process, the parameter sets (VPS, SPS, and PPS) need to be rewritten or replaced, and the slice header needs to be slightly updated because one or all of the slice address related syntax elements (including first_slice_segment_in_pic_flag and slice_segment_address) usually need to have different values.

[0102] 3.2. Image Segmentation in VVC

[0103] In VVC, a picture is divided into one or more slice rows and one or more slice columns. A slice is a sequence of CTUs covering a rectangular area of ​​a picture. The CTUs in a slice are scanned in a grid scan order within the slice.

[0104] A slice consists of an integer number of complete slices or an integer number of consecutive complete CTU rows within a picture.

[0105] Two striping modes are supported, namely, grid scan striping mode and rectangular striping mode. In grid scan striping mode, a strip contains a complete sequence of slices in a slice grid scan of a picture. In rectangular striping mode, a strip contains multiple complete slices that together form a rectangular area of ​​a picture or multiple consecutive complete CTU rows that together form a slice of a rectangular area of ​​a picture. The slices within a rectangular stripe are scanned in a slice grid scan order within the rectangular area corresponding to the stripe.

[0106] A sub-picture consists of one or more strips that together cover a rectangular area of ​​the picture.

[0107] Figure 1 An example of grid-scanning stripe partitioning of a picture is shown, where the picture is divided into 12 slices and 3 grid-scanning stripes.

[0108] Figure 2 An example of rectangular strip partitioning of a picture is shown, where the picture is divided into 24 slices (6 slice columns and 4 slice rows) and 9 rectangular strips.

[0109] Figure 3 An example of a picture partitioned into slices and rectangular strips is shown, where the picture is divided into 4 slices (2 slice columns and 2 slice rows) and 4 rectangular strips.

[0110] Figure 4 An example of sub-picture partitioning of a picture is shown, where the picture is divided into 18 slices, 12 slices on the left, each covering one strip of 4x4 CTUs, and 6 slices on the right, each covering 2 vertically stacked strips of 2x2 CTUs, resulting in a total of 24 strips and 24 sub-pictures of different dimensions (each strip is a sub-picture).

[0111] 3.3.Signaling Notification of SPS / PPS / Picture Header / Slice Header in VVC (JVET-Q2001-vB)

[0112] 7.3.2.3 Sequence parameter set RBSP syntax

[0113]

[0114]

[0115]

[0116]

[0117]

[0118]

[0119]

[0120]

[0121]

[0122]

[0123]

[0124] 7.3.2.4 Picture parameter set RBSP syntax

[0125]

[0126]

[0127]

[0128]

[0129]

[0130]

[0131] 7.3.2.7 Picture header structure syntax

[0132]

[0133]

[0134]

[0135]

[0136]

[0137]

[0138]

[0139]

[0140] 7.3.7.1 General Strip Header Syntax

[0141]

[0142]

[0143]

[0144]

[0145]

[0146] 3.4. Color Space and Chroma Subsampling

[0147] A color space, also called a color model (or color system), is an abstract mathematical model that simply describes the range of colors as a tuple of numbers, usually 3 or 4 values ​​or color components (such as RGB). Basically, a color space is a refinement of a coordinate system and subspace.

[0148] For video compression, the most commonly used are YCbCr and RGB.

[0149] YCbCr, Y'CbCr or YPb / CbPr / Cr, also written as YCBCR or Y'CBCR, is a series of color spaces used as part of the color image pipeline in video and digital photography systems. Y' is the luminance component, CB and CR are the blue difference and red difference chrominance components. Y' (with superscript symbol) is different from Y (Y is luminance), which means that the light intensity is non-linearly encoded based on the gamma-corrected RGB primaries.

[0150] Chroma subsampling is the practice of encoding images at a lower resolution for chroma information than for luminance information, taking advantage of the fact that the human visual system is less sensitive to color differences than to luminance. 3.4.1 4:4:4

[0152] Each of the three Y'CbCr components has the same sample rate, so there is no chroma subsampling. This scheme is sometimes used in high-end film scanners and film post-production. 3.4.2 4:2:2

[0154] The two chroma components are sampled at half the luma sampling rate: the horizontal chroma resolution is halved, while the vertical chroma resolution remains unchanged. This reduces the bandwidth of the uncompressed video signal by one third with little visual difference. An example of nominal vertical and horizontal position for the 4:2:2 color format in the VVC working draft is shown below: Figure 5 shown.

[0155] Figure 5 Shows the nominal vertical and horizontal positions of the 4:2:2 luma and chroma samples in the picture. 3.4.3 4:2:0

[0157] In 4:2:0, the horizontal sampling is doubled compared to 4:1:1, but the vertical resolution is halved because the Cb and Cr channels are only sampled on every alternate line in this scheme. Therefore the data rate is the same. Cb and Cr are each subsampled by a factor of 2 horizontally and vertically. There are three variants of the 4:2:0 scheme with different horizontal and vertical positioning.

[0158] In MPEG-2, Cb and Cr are horizontally co-located. Cb and Cr are located between pixels in the vertical direction (alternately positioned).

[0159] In JPEG / JFIF, H.261, and MPEG-1, Cb and Cr are positioned alternately between alternate luma samples.

[0160] In 4:2:0DV, Cb and Cr are co-located in the horizontal direction. In the vertical direction, they are co-located on alternate lines.

[0161] Table 3: SubWidthC and SubHeightC values ​​derived from chroma_format_idc and separate_colour_plane_flag

[0162] chroma_format_idc separate_colour_plane_flag Chroma Format SubWidthC SubHeightC 0 0 Monochrome 1 1 1 0 4:2:0 2 2 2 0 4:2:2 2 1 3 0 4:4:4 1 1 3 1 4:4:4 1 1

[0163] 3.4. Adaptive Loop Filter (ALF)

[0164] In VVC, an adaptive loop filter (ALF) with block-based filter adaptation is applied. For the luminance component, one of 25 filters is selected for each 4×4 block based on the direction and activity of the local gradient.

[0165] 3.5.1 Filter Shape

[0166] Two diamond filter shapes were used (e.g. Figure 6 The luminance component uses a 7×7 diamond, and the chrominance component uses a 5×5 diamond.

[0167] Figure 6 Examples of ALF filter shapes are shown (chroma: 5×5 diamond, luma: 7×7 diamond).

[0168] 3.5.2 Block Classification

[0169] For the luminance component, each 4×4 block is classified into one of 25 classes. The class index C is based on its directionality D and activity The quantized value of is derived as follows:

[0170]

[0171] In order to calculate D and First, use the one-dimensional Laplace function to calculate the gradient in the horizontal, vertical and two diagonal directions:

[0172]

[0173]

[0174]

[0175]

[0176] Among them, the indexes i and j represent the coordinates of the upper left corner sample point in the 4×4 block, and R(i, j) represents the reconstructed sample point at the coordinate (i, j).

[0177] In order to reduce the complexity of block classification, a subsampled one-dimensional Laplace calculation is applied. Figures 7A-7D As shown, the same subsampling positions are used for gradient calculations in all directions.

[0178] Figures 7A-7D Shows subsampled Laplace calculations.

[0179] Then set the maximum and minimum values ​​of the horizontal and vertical gradients to:

[0180]

[0181] The maximum and minimum values ​​of the gradients in the two diagonal directions are set to:

[0182]

[0183] To derive the value of the directivity D, these values ​​are compared with each other and with two thresholds t1 and t2:

[0184] Step 1. If and are both true, D is set to 0.

[0185] Step 2. If Continue from step 3; otherwise continue from step 4.

[0186] Step 3. If D is set to 2; otherwise D is set to 1.

[0187] Step 4. If D is set to 4; otherwise, D is set to 3.

[0188] The activity value A is calculated as follows:

[0189]

[0190] A is further quantized to the range of 0 to 4, and the quantized value is expressed as

[0191] For the chrominance components in the picture, no classification method is applied, that is, a set of ALF coefficients is applied to each chrominance component.

[0192] 3.5.3 Geometric transformation of filter coefficients and clipping values

[0193] Before filtering each 4×4 luma block, geometric transformations such as rotation or diagonal and vertical flipping are applied to the filter coefficients f(k, l) and the corresponding filter clipping values ​​c(k, l) according to the gradient values ​​calculated for that block. This is equivalent to applying these transformations to the samples in the filter support region. The idea is to make different blocks to which the ALF is applied more similar by aligning their directionality.

[0194] Three geometric transformations are introduced, including diagonal, vertical flip, and rotation:

[0195] Diagonal:f D (k,l)=f(l,k),c D (k,l)=c(l,k), (3-9)

[0196] Flip vertically:f V (k,l)=f(k,Kl-1),c V (k,l)=c(k,Kl-1) (3-10)

[0197] Rotation:f R (k,l)=f(Kl-1,k),c R (k,l)=c(Kl-1,k) (3-11)

[0198] Where K is the size of the filter and 0≤k, l≤K-1 are the coefficient coordinates, so position (0,0) is at the top left corner and position (K-1, K-1) is at the bottom right corner. The transformation is applied to the filter coefficients f(k,l) and the clipping values ​​c(k,l) according to the gradient values ​​calculated for that block. The following table summarizes the relationship between the transformation and the four gradients in the four directions.

[0199] Table 3.2 Mapping between gradient calculation and transformation for a block

[0200] Gradient Value Transform <![CDATA[g d2 <g d1 and g h <g v ]]> No change <![CDATA[g d2 <g d1 and g v <g h ]]> diagonal <![CDATA[g d1 <g d2 and g h <g v ]]> Flip Vertically <![CDATA[g d1 <g d2 and g v <g h ]]> Rotation

[0201] 3.5.4 Filter parameter signal

[0202] The ALF filter parameters are signaled in an adaptive parameter set (APS). In one APS, up to 25 sets of luma filter coefficients and clipping value indices and up to 8 sets of chroma filter coefficients and clipping value indices can be signaled. To reduce bit overhead, filter coefficients of different classifications of luma components can be merged. In the slice header, the index of the APS for the current slice is signaled.

[0203] The clipping value index decoded from the APS allows the determination of the clipping values ​​using the clipping value tables for the luma and chroma components. These clipping values ​​depend on the internal bit depth. More precisely, the clipping values ​​are obtained by the following formula:

[0204] AlfClip = {round(2 B-α*n )for n∈[0..N-1]} (3-12)

[0205] B is equal to the internal bit depth, α is a predefined constant value equal to 2.35, and N is equal to 4, which is the number of cropping values ​​allowed in VVC.

[0206] In the slice header, up to 7 APS indices can be signaled to specify the chroma filter group used for the current slice. The filtering process can be further controlled at the CTB level. A flag is always signaled to indicate whether the ALF is applied to the chroma CTB. The chroma CTB can select a filter group among 16 fixed filter groups and the filter group from the APS. The filter group index is signaled for the chroma CTB to indicate which filter group is applied. The 16 fixed filter groups are predefined and hard-coded in both the encoder and the decoder.

[0207] For chroma components, the APS index is signaled in the slice header to indicate the chroma filter set used by the current slice. At the CTB level, if more than one set of chroma filters is set in the APS, the filter index is signaled for each chroma CTB.

[0208] The filter coefficients are quantized with a norm equal to 128. To limit the multiplication complexity, bitstream consistency is applied so that coefficient values ​​in non-central positions should be in the range: -2 7 to 2 7 -1, inclusive. Coefficients in center positions are not signaled in the bitstream and are considered equal to 128.

[0209] 3.5.5 Filtering process

[0210] At the decoder side, when ALF is enabled for a CTB, each sample R(i, j) within the CU is filtered to produce the sample value R'(i, j) as shown below,

[0211] R′(i,j)=R(i,j)+((∑ k≠0 ∑ l≠0 f(k,l)×K(R(i+k,j+l)-R(i,j),c(k,l))+64)>>7) (3-9)

[0212] Where f(k, l) represents the decoded filter coefficients, K(x, y) represents the clipping function, and c(k, l) represents the decoded clipping parameters. The variables k and l are and The clipping function K(x, y) = min(y, max(-y, x)) corresponds to the function Clip3(-y, y, x).

[0213] 3.5.6 Virtual Boundary Filtering Process for Reducing Line Buffer

[0214] In hardware and embedded software, picture-based processing is actually unacceptable because of the high requirements for picture buffers. Using an on-chip picture buffer is very expensive, and using an off-chip picture buffer significantly increases external memory access, power consumption, and data access latency. Therefore, in actual products, DF, SAO, and ALF are changed from picture-based decoding to LCU-based decoding. When LCU-based processing is used for DF, SAO, and ALF, the entire decoding process can be completed by the LCU in a grid scan in an LCU pipeline manner to process multiple LCUs in parallel. In this case, DF, SAO, and ALF require a row buffer because processing one LCU row requires pixels from the LCU row above. If an off-chip row buffer (such as DRAM) is used, the external memory bandwidth and power consumption will increase; if an on-chip row buffer (such as SRAM) is used, the chip area will increase. Therefore, although the row buffer is already much smaller than the picture buffer, it is still necessary to reduce the row buffer.

[0215] In VTM-4.0, if Figure 8 As shown, the total number of row buffers required for the chroma components is 11.25 rows. The explanation for the row buffer requirement is as follows: Deblocking of horizontal edges overlapping CTU edges cannot be performed because the decision and filtering require rows K, L, M, M from the first CTU and rows O, P from the bottom CTU. Therefore, deblocking of horizontal edges overlapping CTU boundaries is postponed until the lower CTU appears. Therefore, for rows K, L, M, N, the reconstructed chroma samples must be stored in the row buffer (4 rows). Then, SAO filtering can be performed on rows A to J. SAO filtering can be performed on row J because deblocking does not change the samples in row K. For SAO filtering of row K, the edge offset classification decision is only stored in the row buffer (i.e., 0.25 chroma rows). ALF filtering can only be performed on row AF. As shown Figure 8 As shown, ALF classification is performed on each 4×4 block. Each 4×4 block classification requires an active window of size 8×8, which in turn requires a 9×9 window to calculate a one-dimensional (1D) Laplacian function to determine the gradient.

[0216] Therefore, for the block classification requirement of the 4×4 blocks overlapping with rows G, H, I, J, SAO filters the samples below the virtual boundary. In addition, ALF classification requires SAO filtered samples of rows D, E, F. Furthermore, ALF filtering of row G requires the three SAO filtered rows D, E, F from the rows above. Therefore, the total row buffer requirement is as follows:

[0217] – Rows KN (horizontal DF pixels): 4 rows

[0218] – Row DJ (SAO filter pixels): 7 rows

[0219] – SAO edge offset classifier value between row J and row K: 0.25 rows

[0220] Therefore, the total number of luminance lines required is 7+4+0.25=11.25.

[0221] Similarly, the line buffer requirements for chrominance components are as follows Fig. 9 The line buffer requirement for evaluating the chrominance components is 6.25 lines.

[0222] Figure 8 An example of loop filter line buffer requirements for the luma component in VTM-4.0 is shown.

[0223] Fig. 9 An example of loop filter line buffer requirements for chroma components in VTM-4.0 is shown.

[0224] In order to eliminate the line buffer requirements of SAO and ALF, the concept of virtual boundary (VB) is introduced in the latest VVC to reduce the line buffer requirements of ALF. Modified block classification and filtering are used for samples near the horizontal CTU boundary. Figure 8 As shown, VB is the horizontal LCU boundary moved up by N pixels. For each LCU, SAO and ALF can process the pixels above VB before the lower LCU arrives, but cannot process the pixels below VB before the lower LCU arrives, which is caused by DF. Considering the hardware implementation cost, the space between the proposed VB and the horizontal LCU boundary is set to four pixels for the luminance component (e.g., Figure 8 or Fig.10 For example, N=4), for the chrominance component, it is set to two pixels (for example, N=2).

[0225] Fig.10 Modified block classification at virtual boundaries is shown.

[0226] Modified block classification applies to Fig.11 The luminance component is shown. For the 1D Laplacian gradient calculation of the 4×4 block located above the virtual boundary, only the samples above the virtual boundary are used. Similarly, for the 1D Laplacian gradient calculation of the 4×4 block below the virtual boundary, only the samples below the virtual boundary are used. By considering the reduced number of samples used in the 1D Laplacian gradient calculation, the quantization of the activity value A is scaled accordingly.

[0227] For filtering, both the luminance component and the chrominance component use a mirror (symmetric) padding operation at the virtual boundary. Fig.11 As shown in FIG. 1 , when the filtered sample point is below the virtual boundary, the neighboring sample points above the virtual boundary are filled in. At the same time, the corresponding sample points on the other side are also filled symmetrically.

[0228] For another example, if we fill a point at (i, j) (e.g. Fig. 12B If the dashed line P0A in the figure is used, then even if a sample point is available, fill the point at (m, n) (e.g., Fig. 12B The corresponding sample points with the same filter coefficients as the dotted line P3B in FIG. Figures 12A-12C shown.

[0229] Fig. 12A Shows 1 required row (on each side) of padding needed above / below the VB.

[0230] Fig. 12B Shows the 2 required rows (on each side) of padding needed above / below the VB.

[0231] Fig. 12C Shows the 3 required rows (on each side) of padding needed above / below the VB.

[0232] Unlike the mirrored (symmetric) padding method used at horizontal CTU boundaries, a repeated (one-sided) padding process is applied to slice, slice, and sub-picture boundaries when cross-boundary filtering is disabled. The repeated (one-sided) padding process also applies to picture boundaries. The padding samples are used for classification and filtering processes. Fig.13 An example of a repetition padding method for luma-ALF filtering at picture / sub-picture / slice / slice boundaries is shown.

[0233] 3.5.7 Adaptive Loop Filter Process in the Specification

[0234] 8.8.5.2 Codec Tree Block Filtering Process for Luma Samples

[0235] The input to this process is:

[0236] – Before the adaptive loop filtering process, reconstruct the luminance picture sample array recPicture,

[0237] –Filter and reconstruct the brightness picture sample array alfPicture L ,

[0238] – Luma position (xCtb, yCtb), specifies the top left sample point of the current luma codec treeblock relative to the top left sample point of the current picture.

[0239] The output of this process is the modified filtered reconstructed luminance picture sample array alfPicture L .

[0240] Call the filter index derivation process of Section 8.8.5.3, with the position (xCtb, yCtb) and the reconstructed luminance picture sample array recPicture as input, and filtIdx[x][y] and transposeIdx[x][y]-1 as output, where x, y=0.CtbSizeY.

[0241] In order to derive the filtered reconstructed luminance sample alfPicture L [x][y], with x, y = 0..CtbSizeY-1, filter each reconstructed luma sample in the current luma codec tree block recPicture[x][y] as follows:

[0242] – The array of luma filter coefficients f[j] and the array of luma clipping values ​​c[j] corresponding to the filter specified by filtIdx[x][y] are derived as follows, where j = 0..11:

[0243] – If AlfCtbFiltSetIdxY[xCtb>>CtbLog2SizeY][yCtb>>CtbLog2SizeY] is less than 16, then the following applies:

[0244] i=AlfCtbFiltSetIdxY[xCtb>>CtbLog2SizeY][yCtb>>CtbLog2SizeY] (1453)

[0245] f[j]=AlfFixFiltCoeff[AlfClassToFiltMap[i][filtIdx[x][y]]][j](1454)c[j]=2 BitDepth (1455)

[0246] – Otherwise (AlfCtbFiltSetIdxY[xCtb>>CtbLog2SizeY][yCtb>>CtbLog2SizeY] is greater than or equal to 16, then the following applies:

[0247] i=slice_alf_aps_id_luma[AlfCtbFiltSetIdxY[xCtb>>CtbLog2SizeY][yCtb>>CtbLog2SizeY]-16] (1456)

[0248] f[j]=AlfCoeff L [i][filtIdx[x][y]][j] (1457)

[0249] c[j]=AlfClip L[i][filtIdx[x][y]][j] (1458)

[0250] – The luma filter coefficients and the clipping value index idx are derived from transposeIdx[x][y] as follows:

[0251] – If transposeIndex[x][y] is equal to 1, then the following applies:

[0252] idx[]={9,4,10,8,1,5,11,7,3,0,2,6} (1459)

[0253] – Otherwise, if transposeIndex[x][y] is equal to 2, then the following applies:

[0254] idx[]={0,3,2,1,8,7,6,5,4,9,10,11} (1460)

[0255] – Otherwise, if transposeIndex[x][y] is equal to 3, then the following applies:

[0256] idx[]={9,8,10,4,3,7,11,5,1,0,2,6} (1461)

[0257] – Otherwise, the following applies:

[0258] idx[]={0,1,2,3,4,5,6,7,8,9,10,11} (1462)

[0259] – The position (h) of each corresponding luma sample (x, y) within a given array of chroma samples recPicture x+i , v y+j ) is derived as follows, where i, j = -3..3:

[0260] h x+i =Clip3(0,pic_width_in_luma_samples-1,xCtb+x+i) (1463)

[0261] v y+j =Clip3(0,pic_height_in_luma_samples-1,yCtb+y+j) (1464)

[0262] – The variables clipLeftPos, clipRightPos, clipTopPos, clipBottomPos, clipTopLeftFlag, and clipBotRightFlag are derived by calling the ALF boundary position derivation procedure specified in clause 8.8.5.5 with (xCtb, yCtb) and (x, y) as input.

[0263] – Modify the variable h by calling the ALF sample filling procedure specified in clause 8.8.5.6 x+i and v y+j , and (xCtb, yCtb), (h x+i , v y+j ), 0, clipLeftPos, clipRightPos, clipTopPos, clipBottomPos, clipTopLeftFlag and clipBotRightFlag as input.

[0264] –The derivation of the variable applyAlfLineBufBoundary is as follows:

[0265] – If the bottom boundary of the current codec tree block is the bottom boundary of the current picture and pic_height_in_luma_samples-yCtb<=CtbSizeY-4, applyAlfLineBufBoundary is set to 0:

[0266] – Otherwise, applyAlfLineBufBoundary is set to 1.

[0267] – In Table 45, the vertical sample position offsets y1, y2, y3 and the variable alfShiftY are specified according to the vertical luma sample position y and applyAlfLineBufBoundary.

[0268] –The variable curr is derived as follows:

[0269] curr=recPicture[h x ][v y ] (1465)

[0270] –The variable sum is derived as follows:

[0271] sum=f[idx[0]]*(Clip3(-c[idx[0]],c[idx[0]],recPicture[h x ][v y+y3 ]-curr)+

[0272] Clip3(-c[idx[0]],c[idx[0]],recPicture[h x ][v y-y3 ]-curr))+

[0273] f[idx[1]]*(Clip3(-c[idx[1]],c[idx[1]],recPicture[h x+1 ][v y+y2 ]-curr)+

[0274] Clip3(-c[idx[1]],c[idx[1]],recPicture[h x-1 ][v y-y2 ]-curr))+

[0275] f[idx[2]]*(Clip3(-c[idx[2]],c[idx[2]],recPicture[h x ][v y+y2 ]-curr)+

[0276] Clip3(-c[idx[2]],c[idx[2]],recPicture[h x ][v y-y2 ]-curr))+

[0277] f[idx[3]]*(Clip3(-c[idx[3]],c[idx[3]],recPicture[h x-1 ][v y+y2 ]-curr)+

[0278] Clip3(-c[idx[3]],c[idx[3]],recPicture[h x+1 ][v y-y2 ]-curr))+

[0279] f[idx[4]]*(Clip3(-c[idx[4]],c[idx[4]],recPicture[h x+2 ][v y+y1 ]-curr)+

[0280] Clip3(-c[idx[4]],c[idx[4]],recPicture[h x-2 ][v y-y1 ]-curr))+

[0281] f[idx[5]]*(Clip3(-c[idx[5]],c[idx[5]],recPicture[h x+1 ][v y+y1 ]-curr)+

[0282] Clip3(-c[idx[5]],c[idx[5]],recPicture[h x-1 ][v y-y1 ]-curr))+

[0283] f[idx[6]]*(Clip3(-c[idx[6]],c[idx[6]],recPicture[h x ][v y+y1 ]-curr)+

[0284] Clip3(-c[idx[6]],c[idx[6]],recPicture[h x ][v y-y1 ]-curr))+(1466)

[0285] f[idx[7]]*(Clip3(-c[idx[7]],c[idx[7]],recPicture[h x-1 ][v y+y1 ]-curr)+

[0286] Clip3(-c[idx[7]],c[idx[7]],recPicture[h x+1 ][v y-y1 ]-curr))+

[0287] f[idx[8]]*(Clip3(-c[idx[8]],c[idx[8]],recPicture[h x-2 ][v y+y1 ]-curr)+

[0288] Clip3(-c[idx[8]],c[idx[8]],recPicture[h x+2 ][v y-y1 ]-curr))+

[0289] f[idx[9]]*(Clip3(-c[idx[9]],c[idx[9]],recPicture[h x+3 ][v y ]-curr)+

[0290] Clip3(-c[idx[9]],c[idx[9]],recPicture[h x-3 ][v y ]-curr))+

[0291] f[idx

[10] ]*(Clip3(-c[idx

[10] ],c[idx

[10] ],recPicture[h x+2 ][v y ]-curr)+

[0292] Clip3(-c[idx

[10] ],c[idx

[10] ],recPicture[h x-2 ][v y ]-curr))+

[0293] f[idx

[11] ]*(Clip3(-c[idx

[11] ],c[idx

[11] ],recPicture[h x+1 ][v y ]-curr)+

[0294] Clip3(-c[idx

[11] ],c[idx

[11] ],recPicture[h x-1 ][v y ]-curr))sum=curr+((sum+64)>>alfShiftY) (1467)

[0295] – Modified filtering to reconstruct the brightness picture samples alfPicture L [xCtb+x][yCtb+y] is derived as follows:

[0296] alfPicture L [xCtb+x][yCtb+y]=Clip3(0,(1< <BitDepth)-1,sum)(1468)

[0297] Table 45 - Specification of y1, y2, y3 and alfShiftY according to vertical luminance sample position y and applyAlfLineBufBoundary

[0298]

[0299] 8.8.5.4 Codec Tree Block Filtering Process for Chroma Samples

[0300] The input to this process is:

[0301] – Before the adaptive loop filtering process, reconstruct the chrominance picture sample array recPicture,

[0302] –Filter and reconstruct the chrominance picture sample array alfPicture,

[0303] – Chroma position (xCtbC, yCtbC), specifies the top left sample point of the current chroma codec tree block relative to the top left sample point of the current picture,

[0304] – Alternative chroma filter index altIdx.

[0305] The output of this process is the modified filtered reconstructed chrominance picture sample array alfPicture.

[0306] The width and height ctbWidthC and ctbHeightC of the current chroma codec tree block are derived as follows:

[0307] ctbWidthC=CtbSizeY / SubWidthC (1500)

[0308] ctbHeightC=CtbSizeY / SubHeightC (1501)

[0309] To derive the filtered reconstructed chroma sample alfPicture[x][y], each reconstructed chroma sample in the current chroma codec tree block recPicture[x][y] is filtered as follows, where x = 0..ctbWidthC-1 and y = 0..ctbHeightC-1:

[0310] – The position (h) of each corresponding chrominance sample (x, y) in a given array x+i , v y+j ) is derived as follows, where i, j = -2..2:

[0311] h x+i =Clip3(0,pic_width_in_luma_samples / SubWidthC-1,xCtbC+x+i)(1502)

[0312] v y+j =Clip3(0,pic_height_in_luma_samples / SubHeightC-1,yCtbC+y+j)(1503)

[0313] – The variables clipLeftPos, clipRightPos, clipTopPos, clipBottomPos, clipTopLeftFlag, and clipBotRightFlag are derived by calling the ALF boundary position derivation process specified in Clause 8.8.5.5, where (xCtbC * SubWidthC, yCtbC * SubHeightC) and (x * SubWidthC, y * SubHeightC) are used as inputs.

[0314] – The variables h x+i and v y+j are modified by calling the ALF sample filling process specified in Clause 8.8.5.6, and (xCtb, yCtb), (h x+i , v y+j ), the set of variables isChroma equal to 1, clipLeftPos, clipRightPos, clipTopPos, clipBottomPos, clipTopLeftFlag, and clipBotRightFlag are used as inputs.

[0315] – The derivation of the variable applyAlfLineBufBoundary is as follows:

[0316] – If the bottom boundary of the current coded tree block is the bottom boundary of the picture and the bottom boundary of pic_height_in_luma_samples - (yCtbC * SubHeightC) < CtbSizeY – 4, applyAlfLineBufBoundary is set to 0.

[0317] – Otherwise, applyAlfLineBufBoundary is set to 1.

[0318] – In Table 45, the vertical sampling position offsets y1, y2, and the variable alfShiftC are specified according to the vertical chroma sampling position y and applyAlfLineBufBoundary.

[0319] – The derivation of the variable curr is as follows:

[0320] curr = recPicture[h x [v y (1504)

[0321] – The array f[j] of chroma filter coefficients and the array c[j] of chroma clip values are derived as follows, where j = 0.5:

[0322] f[j] = AlfCoeffC [slice_alf_aps_id_chroma][altIdx][j] (1505)

[0323] c[j]=AlfClip C [slice_alf_aps_id_chroma][altIdx][j] (1506)

[0324] –The variable sum is derived as follows:

[0325]

[0326] sum=curr+((sum+64)>>alfShiftC) (1508)

[0327] – The modified filtered reconstructed chrominance picture sample alfPicture[xCtbC+x][yCtbC+y] is derived as follows:

[0328] alfPicture[xCtbC+x][yCtbC+y]=Clip3(0,(1< <BitDepth)-1,sum) (1509)

[0329] Table 46 - Specification of y1, y2 and alfShiftC according to vertical chroma sample position y and applyAlfLineBufBoundary

[0330]

[0331] 4. Examples of technical problems solved by the public solution

[0332] The design of signaling notification of SPS / PPS / picture header / slice header in the existing VVC has the following problems:

[0333] 1) Even if there is only one tile, loop_filter_across_tiles_enabled_flag is signaled.

[0334] 2) Even if there is only one slice, loop_filter_across_slices_enabled_flag is signaled.

[0335] 3) LMCS information is signaled in the picture header instead of the slice header.

[0336] 4) The maximum allowed TS block size may be larger than the maximum CU size.

[0337] 5)MaxNumSubblockMergeCand is derived differently depending on whether affine is enabled.

[0338] 6) Apply mirror padding at the ALF virtual boundary to obtain unavailable samples and their corresponding ALF samples (e.g., luma ALF and chroma ALF), where the location of the ALF virtual boundary is used to determine which samples are unavailable and need to be filled. However, in the current design, for videos in 4:2:2 / 4:4:4 chroma format, the location of the ALF virtual boundary in the chroma ALF is inconsistent with the location in the luma ALF.

[0339] 7) In the current design, the ALF filtering of the virtual horizontal CTU boundary neighboring rows reduces the filter strength depending on the location of the virtual boundary. However, the wrong location of the ALF virtual boundary causes the filter strength to be reduced at unexpected sample rows.

[0340] 5. Technology and Example Embodiments

[0341] To address the above and other issues, the methods outlined below are disclosed. The listed items should be considered as examples to explain the general concept and should not be interpreted narrowly. In addition, these items can be applied alone or in combination in any way.

[0342] In the present disclosure, a neighboring (adjacent or non-adjacent) sample is “unavailable” if the neighboring (adjacent or non-adjacent) sample (or line, or row) is not allowed to be located in a different video processing unit (e.g., outside the current picture, or the current sub-picture, or the current slice, or the current strip, or the current brick, or the current CTU, or the current processing unit (e.g., the ALF processing unit or the narrow ALF processing unit), or any other current video unit), or unreconstructed or cross-filtered video processing units are not allowed.

[0343] The filling method for the ALF virtual boundary can be expressed as "mirror filling", where for the first unavailable sample located at (i, j) (or the first unavailable line j, or the first unavailable row i), it needs to be filled, and even if the second sample is available, it will be filled with the second sample defined by the "corresponding sample of the first sample" (or "corresponding line of the first line", or "corresponding row of the first row") in the filter support in the ALF (for example, the corresponding sample is located at (m, n) (or the corresponding line n, or the corresponding row m), (m, n) The distance of the current sample (or current line, or current row) is the same).

[0344] 1. Signaling of an indication of whether to perform a loop filtering operation across slice boundaries (eg, loop_filter_across_tiles_enabled_flag) may depend on whether and / or how a video unit (eg, a picture) is partitioned into slices.

[0345] a. In one example, loop_filter_across_tiles_enabled_flag is signaled only if the video unit is partitioned into more than one tile.

[0346] b. Alternatively, when the video unit has only one slice, signaling of loop_filter_across_tiles_enabled_flag is skipped.

[0347] c. In one example, loop_filter_across_tiles_enabled_flag is signaled only when the general constraint flag one_tile_per_pic_constraint_flag is equal to 0.

[0348] i. Alternatively, if the general constraint flag one_tile_per_pic_constraint_flag is equal to 1, then loop_filter_across_tiles_enabled_flag is inferred (or required) to be equal to 0.

[0349] 2. Signaling of an indication of whether to perform a loop filtering operation across slice boundaries (eg, loop_filter_across_slices_enabled_flag) may depend on whether and / or how a video unit (eg, a picture and / or a sub-picture) is partitioned into slices.

[0350] a. In one example, if the video unit is partitioned into only one slice, loop_filter_across_slices_enabled_flag is not signaled.

[0351] b. In one example, if each sub-picture is partitioned into only one slice (eg, single_slice_per_subpic_flag is equal to 1), loop_filter_across_slices_enabled_flag is not signaled.

[0352] c. In one example, if each sub-picture is partitioned into only one slice (eg, single_slice_per_subpic_flag is equal to 1), loop_filter_across_slices_enabled_flag is signaled.

[0353] d. In one example, if the picture is partitioned into slices in a non-rectangular manner (eg, rect_slice_flag is equal to 0), loop_filter_across_slices_enabled_flag is signaled.

[0354] e. In one example, if the picture is partitioned into slices in a rectangular manner (eg, rect_slice_flag is equal to 0) and the number of slices is equal to 1 (eg, num_slices_in_pic_minus1 is equal to 0), loop_filter_across_slices_enabled_flag is not signaled.

[0355] f. In one example, loop_filter_across_slices_enabled_flag is signaled only when the general constraint flag one_slice_per_pic_constraint_flag is equal to 0.

[0356] i. Alternatively, if the general constraint flag one_slice_per_pic_constraint_flag is equal to 1, then loop_filter_across_slices_enabled_flag is inferred (or required) to be equal to 0.

[0357] 3. LMCS information (eg, indication of LMCS usage, and / or usage of luma shaping, and / or adaptation_parameter_set_id of LMCS APS to be used and / or usage of chroma residual scaling) may be signaled in the slice header and / or picture header.

[0358] a. The LMCS information may include a first indication of whether LMCS is enabled, such as ph_lmcs_enabled_flag or slice_lmcs_enabled_flag.

[0359] b. The LMCS information may include a second indication of LMCS parameters, such as ph_lmcs_aps_id and / or ph_chroma_residual_scale_flag.

[0360] c.LMCS information can be signaled in the slice header and picture header.

[0361] d. LMCS information can be signaled in the picture header and slice header. When it is present in the slice header, the LMCS information of the picture header (if any) may be overwritten.

[0362] e.LMCS information can be signaled in either the slice header or the picture header, but not in both at the same time.

[0363] f. In one example, a syntax element may be signaled in the slice header to indicate which LMCS APS is used for the current slice.

[0364] g. In one example, a syntax element (eg, lmcs_info_in_ph_flag) is signaled in a higher level video unit such as SPS or PPS to indicate whether the LMCS information is signaled in a picture header or a slice header.

[0365] i. The syntax element may control only the first indication.

[0366] ii. The syntax element may control only the second indication.

[0367] iii. The syntax element may control the first indication and the second indication.

[0368] h. If the syntax element of the LMCS information in the picture header does not exist, it can be set to a default value.

[0369] i. If the syntax element of the LMCS information in the slice header does not exist, it can be set to a default value.

[0370] i. If the syntax element of the LMCS information in the slice header does not exist, it can be set to the value of the corresponding syntax element of the LMCS information in the picture header.

[0371] j. In one example, more than one adaptation_parameter_set_id (eg, a list of ph_lmcs_aps_id[]) of the LMCS APSs referred to by a slice associated with a PH may be signaled in the picture header.

[0372] i. In one example, the length of the ph_lmcs_aps_id[] list may depend on the number of slices in the picture.

[0373] ii. In one example, a syntax element may be signaled in the slice header to indicate which LMCS APS in the ph_lmcs_aps_id[] list is used for the current slice.

[0374] 4. The maximum allowed size of transform skip (TS) and / or BDPCM should be less than or equal to the size of the codec tree block (CTB).

[0375] a. For example, it is required that the maximum allowed width and height of TS and / or BDPCM of the luma block should be less than or equal to CtbSizeY.

[0376] b. For example, it is required that the maximum allowed width and height of TS and / or BDPCM of chroma blocks should be less than or equal to CtbSizeY / subWidthC and / or CtbSizeY / subHeightC.

[0377] i. Alternatively, it is required that the maximum allowed width of TS and / or BDPCM of chroma blocks should be less than or equal to CtbSizeY / subWidthC and / or CtbSizeY / subHeightC.

[0378] ii. Alternatively, it is required that the maximum allowed height of the TS and / or BDPCM of the chroma block should be less than or equal to CtbSizeY / subWidthC and / or CtbSizeY / subHeightC.

[0379] c. For example, it is required that log2_transform_skip_max_size_minus2 plus 2 should be less than or equal to CtbLog2SizeY.

[0380] d. For example, the maximum value of log2_transform_skip_max_size_minus2 is equal to CtbLog2SizeY–2.

[0381] e. For example, MaxTsSize is derived as

[0382] MaxTsSize=Min(CtbSizeY,1<<(log2_transform_skip_max_size_minus2+2)).

[0383] f. For example, MaxTsSize is derived as

[0384] MaxTsSize=1< <Min(log2_transform_skip_max_size_minus2+2,CtbLog2SizeY)。

[0385] g. The maximum allowed block size of TS and / or BDPCM for chroma blocks shall be less than or equal to the maximum transform block size of chroma blocks.

[0386] i. In one example, assuming that MaxTbSizeY represents the maximum transform size of the luma block, the maximum allowed width and height of the TS and / or BDPCM of the chroma block may be less than or equal to MaxTbSizeY / SubWidthC.

[0387] ii. In one example, assuming that MaxTbSizeY represents the maximum transform size of the luma block, the maximum allowed width of the TS and / or BDPCM of the chroma block may be less than or equal to MaxTbSizeY / SubWidthC.

[0388] iii. In one example, assuming that MaxTbSizeY represents the maximum transform size of the luma block, the maximum allowed height of the TS and / or BDPCM of the chroma block may be less than or equal to MaxTbSizeY / SubHeightC

[0389] h. The maximum allowed transform skip (TS) block size may be signaled using a binary syntax element (eg, "0" for 16 and "1" for 32).

[0390] 5. The maximum allowed transform skip (TS) block size and / or the maximum allowed transform block size shall not be less than the minimum decoding block size.

[0391] 6. Whether and / or how to signal or interpret or limit the maximum allowed transform block size (e.g., denoted as MaxTbSizeY in JVET-Q2001-vB) may depend on the minimum allowed codec block size (e.g., denoted as MinCbSizeY in JVET-Q2001-vB).

[0392] a. In one example, it is required that MaxTbSizeY must be greater than or equal to MinCbSizeY.

[0393] i. In one example, when MinCbSizeY is equal to 64, the value of sps_max_luma_transform_size_64_flag should be equal to 1.

[0394] ii. In one example, when MinCbSizeY is equal to 64, sps_max_luma_transform_size_64_flag is not signaled and is inferred to be 1.

[0395] 7. Whether and / or how to signal or interpret or limit the maximum allowed size of TS and / or BDPCM codecs (e.g., denoted as MaxTsSize in JVET-Q2001-vB) may depend on the minimum allowed codec block size (e.g., denoted as MinCbSizeY in JVET-Q2001-vB).

[0396] a. In one example, it is required that MaxTsSize must be greater than or equal to MinCbSizeY.

[0397] b. In one example, it is required that MaxTsSize must be less than or equal to W, where W is an integer, such as 32.

[0398] i. For example, it is required that MaxTsSize must satisfy MinCbSizeY<=MaxTsSize<=W.

[0399] ii. For example, when TS and / or BDPCM codecs are enabled, MinCbSizeY should be less than or equal to X.

[0400] c. In one example, when TS and / or BDPCM are enabled (eg, sps_transform_skip_enabled_flag is equal to 1), it is required that MaxTsSize must be greater than or equal to MinCbSizeY.

[0401] i. For example, when sps_transform_skip_enabled_flag is equal to 1, log2_transform_skip_max_size_minus2 should be greater than or equal to log2_min_luma_coding_block_size_minus2.

[0402] d. In one example, MaxTsSize=max(MaxTsSize,MinCbSizeY).

[0403] i. In another example, MaxTsSize=min(W, max(MaxTsSize, MinCbSizeY)), where W is an integer, such as 32.

[0404] e. In one example, when TS and / or BDPCM are enabled (eg, sps_transform_skip_enabled_flag is equal to 1), MaxTsSize = max(MaxTsSize, MinCbSizeY).

[0405] i. In one example, when TS and / or BDPCM are enabled (eg, sps_transform_skip_enabled_flag is equal to 1), MaxTsSize=min(W, max(MaxTsSize, MinCbSizeY)), where W is an integer, eg, 32. i.

[0406] f. In one example, the signaling of MaxTsSize (eg, log2_transform_skip_max_size_minus2 in JVET-Q2001-vB) may depend on MinCbSizeY.

[0407] i. In one example, the difference between log2_transform_skip_max_size_minus2 and log2_min_luma_coding_block_size_minus2 (denoted as log2_diff_max_transform_skip_min_coding_block) may be signaled to indicate MaxTsSize.

[0408] 1) For example, MaxTsSize=1<<(MinCbLog2SizeY+log2_diff_max_trasform_skip_min_coding_block).

[0409] 2) For example, MaxTsSize=min(W,1<<(MinCbLog2SizeY+log2_diff_max_trasform_skip_min_coding_block)), where W is an integer, such as 32.

[0410] 8. Whether and / or how to signal or interpret or limit the indication of TS and / or BDPCM codecs (e.g., denoted as sps_transform_skip_enabled_flag in JVET-Q2001-vB) may depend on the minimum allowed codec block size (e.g., denoted as MinCbSizeY in JVET-Q2001-vB).

[0411] a. In one example, when MinCbSizeY is equal to 64, sps_transform_skip_enabled_flag is not signaled and is inferred to be 0.

[0412] b. In one example, when MinCbSizeY is larger than the maximum allowed size of TS and / or BDPCM (eg, the maximum allowed size of TS and / or BDPCM is 32), sps_transform_skip_enabled_flag is not signaled and is inferred to be 0.

[0413] 9. Whether and / or how the indication of Codec Tool X is signaled or interpreted in the SPS / PPS / picture header / slice header may depend on the minimum allowed codec block size (eg, MinCbSizeY in JVET-Q2001-vB).

[0414] a. If the minimum allowed codec block size is greater than T, then the indication of codec tool X in the SPS / PPS / picture header / slice header may not be signaled and is inferred to be not used, where T is an integer such as 32.

[0415] b. If the minimum allowed codec block size is greater than T, then it must be indicated that codec tool X in the SPS / PPS / picture header / slice header is not used, where T is an integer such as 32.

[0416] c. Codec X may be Combined Inter-Intra Prediction (CIIP).

[0417] d. Codec X can be Multi-Transform Selection (MTS).

[0418] e. Codec X may be Segment Block Transform (SBT).

[0419] f. Codec X may be Symmetric Motion Vector Difference (SMVD).

[0420] g. Codec X can be BDOF.

[0421] h. Codec X can be affine prediction.

[0422] i. Codec tool X may be optical flow prediction refinement (PROF).

[0423] j. Codec Tool X may be decoder side motion vector refinement (DMVR).

[0424] k. Codec X can be bi-directional prediction with CU level weights (BCW).

[0425] 1. Codec X can be Merge with Motion Vector Difference (MMVD).

[0426] m. Codec X can be Geometric Partitioning Mode (GPM).

[0427] n.coidng tool X can be intra-block copying (IBC).

[0428] o. Codec X can be a palette codec.

[0429] p. Codec X may be Adaptive Color Transform (ACT).

[0430] q. Codec X may be Joint Cb-Cr Residual Codec (JCCR).

[0431] r. Codec X may be a Cross Component Linear Model prediction (CCLM).

[0432] s. The codec tool X can be a multi-reference line (MRL).

[0433] t. The codec tool X can be matrix-based intra prediction (MIP).

[0434] u. The codec tool X can be intra-subdivision prediction (ISP).

[0435] 10. When applying dual-tree coding, the minimum allowable block size for binary tree partitioning (e.g., MinBtSizeY in JVET-Q2001-vB) may be different for the luminance component and the chrominance component.

[0436] a. In one example, MinBtSizeY = 1 << MinBtLog2SizeY is the minimum allowable block size for binary tree partitioning of the luminance component, and MinBtSizeC = 1 << MinBtLog2SizeC is the minimum allowable block size for binary tree partitioning of the chrominance component, where MinBtLog2SizeY may not be equal to MinBtLog2SizeC.

[0437] i. MinBtLog2SizeY can be predicted by signaling MinCbLog2SizeY. For example, the difference between MinBtLog2SizeY and MinCbLog2SizeY can be signaled.

[0438] ii. MinBtLog2SizeC can be predicted by signaling MinCbLog2SizeY. For example, the difference between MinBtLog2SizeC and MinCbLog2SizeY can be signaled.

[0439] 11. When applying dual-tree coding, the minimum allowable block size for ternary tree partitioning (e.g., MinTtSizeY in JVET-Q2001-vB) may be different for the luminance component and the chrominance component. MinCbSizeY = 1 << MinCbLog2SizeY represents the minimum allowable coding block size.

[0440] a. In one example, MinTtSizeY = 1 << MinTtLog2SizeY is the minimum allowable block size for ternary tree partitioning of the luminance component, and MinTtSizeC = 1 << MinTtLog2SizeC is the minimum allowable block size for ternary tree partitioning of the chrominance component, where MinTtLog2SizeY may not be equal to MinTtLog2SizeC.

[0441] i.MinTtLog2SizeY can be predicted by MinCbLog2SizeY. For example, the difference between MinTtLog2SizeY and MinCbLog2SizeY can be signaled.

[0442] ii.MinTtLog2SizeC can be predicted by MinCbLog2SizeY. For example, the difference between MinTtLog2SizeC and MinCbLog2SizeY can be signaled.

[0443] 12. The maximum number of sub-block based merge candidates (e.g., MaxNumSubblockMergeCand) is derived as the sum of a first variable and a second variable, where the first variable is equal to zero if affine prediction is disabled (e.g., sps_affine_enabled_flag is equal to 0), and the second variable depends on whether sub-block based TMVP (sbTMVP) is enabled.

[0444] a. In one example, the first variable represents the number of allowed affine merge candidates.

[0445] b. In one example, the second variable may be set to (sps_sbtmvp_enabled_flag&&ph_temporal_mvp_enable_flag).

[0446] c. In one example, the first variable is derived as KS, where S is a value signaled by a syntax element (eg, five_minus_num_affine_merge_cand), and K is a fixed value such as 4 or 5.

[0447] d. In one example, MaxNumSubblockMergeCand = 5 - Five_minus_max_num_affine_merge_cand + (sps_sbtmvp_enabled_flag && ph_temporal_mvp_enable_flag).

[0448] e. In one example, MaxNumSubblockMergeCand = 4 - four_minus_max_num_affine_merge_cand + (sps_sbtmvp_enabled_flag && ph_temporal_mvp_enable_flag).

[0449] f. In one example, MaxNumSubblockMergeCand=Min(W,MaxNumSubblockMergeCand), where W is a fixed value, such as 5.

[0450] g. In one example, the indication of the first variable (eg, five_minus_max_num_affine_merge_cand or four_minus_max_num_affine_merge_cand) may be conditionally signaled.

[0451] i. In one example, signaling is only performed when sps_affine_enabled_flag is equal to 1.

[0452] ii. When not present, five_minus_max_num_affine_merge_cand is inferred to be K (eg, K=4 or 5).

[0453] 13. The number of rows (lines) between the ALF virtual boundary and the CTB bottom boundary of the luminance component and the chrominance component; and / or the filtering intensity for the rows between the ALF virtual boundary and the CTB bottom boundary of the luminance component and the corresponding rows of the chrominance component; and / or the filling method of the luminance and chrominance samples of the same row; alignment in 4:2:2 and 4:4:4 cases.

[0454] a. In one example, for 4:2:2 / 4:4:4 chroma format video, the vertical (and / or horizontal) position of the ALF virtual boundary (VB) in the chroma ALF should be aligned with the vertical (and / or horizontal) position of the ALF virtual boundary in the luma ALF. The vertical (and / or horizontal) position of ALF VB in the luma ALF is represented by vbPosY, and the vertical (and / or horizontal) position of ALF VB in the chroma ALF is represented by vbPosC.

[0455] i. In one example, when the vertical position (vbPosY) of ALF VB in the luma ALF is equal to CtbSizeY-S, the vertical position vbPosC of ALF VB of the chroma component may be set equal to (CtbSizeY-S) / SubHeightC.

[0456] ii. In one example, when the horizontal position vbPosY of ALF VB in luma ALF is equal to CtbSizeY-S, the horizontal position of ALF VB of chroma component, vbPosC=(CtbSizeY-S) / SubWidthC.

[0457] iii. In the above example, CtbSizeY specifies the luma codec treeblock size for each CTU and SubHeightC and SubWidthC as defined in Table 3-1. S is an integer, such as 4.

[0458] b. In one example, the padding method in the chroma ALF of K rows (and / or H columns) near the vertical (and / or horizontal) position of ALF VB should be aligned with the padding method in the luma ALF. Let Yc represent the vertical (or horizontal) chroma sample position.

[0459] i. In one example, when Yc is equal to vbPosC (e.g., Fig. 14B In the case of , the following can be applied:

[0460] 1) K (eg, K=2) unavailable rows above (or to the left) may be filled in. Alternatively, even if the corresponding K rows are available, the corresponding K rows of these unavailable rows below (or to the right) the current row may be filled in.

[0461] a. In one example, the current row can be used to fill the upper K unavailable rows.

[0462] i. In one example, Fig. 14B In the example, C1, C2, C3 above the current row can be set equal to C5, C6, C5. And C0 above the current row can be set equal to C6.

[0463] b. In one example, the current row may be used to fill the corresponding K rows.

[0464] i. In one example, Fig. 14B In the example, C3, C2, C1 below the current row can be set equal to C5, C6, C5. And C0 below the current row can be set equal to C6.

[0465] 2) The K unavailable rows below (or to the right) can be filled in. Alternatively, even if the corresponding K rows are available, the corresponding K rows of these unavailable rows above (or to the left) the current row can be filled in.

[0466] ii. In one example, when Yc is equal to vbPosC-M (eg, M=1, 2) (eg, Fig.14A and Fig.14D ), the following can be applied:

[0467] 1) K (eg, K=1, 2) unavailable rows below (or to the right) may be filled, and even if the corresponding K rows are available, the corresponding K rows of these unavailable rows above (or to the left) the current row may be filled.

[0468] a. In one example, when M is equal to 1, the current row can be used to fill the lower K unavailable rows.

[0469] i. In one example, Fig.14A In the example, C3, C2, C1 below the current row can be set equal to C5, C6, C5. And C0 below the current row can be set equal to C6.

[0470] b. In one example, when M is equal to 1, the current row may be used to fill the corresponding K rows.

[0471] i. In one example, Fig.14A In the example, C1, C2, C3 above the current row can be set equal to C5, C6, C5. And C0 above the current row can be set equal to C6.

[0472] c. In one example, when M is greater than or equal to 2, the bottommost row above the ALF virtual boundary (eg, vbPosC-1) may be used to fill the K unavailable rows below.

[0473] i. In one example, Fig.14D , C0 below the current row can be set equal to C6.

[0474] d. In one example, when M is greater than or equal to 2, the corresponding K rows (eg, vbPos - 2*M+1) may be filled with the corresponding row of the lowermost row above the ALF virtual boundary.

[0475] i. In one example, Fig.14D , C0 above the current row can be set equal to C6.

[0476] iii. In one example, when Yc is equal to vbPosC+N (eg, N=1) (eg, Fig. 14C In the case of , the following can be applied:

[0477] 1) K (eg, K=1) unavailable rows above (or to the left) may be filled in. Even if the corresponding K rows are available, the corresponding K rows of these unavailable rows below (or to the right) the current row may be filled in.

[0478] a. In one example, the current row can be used to fill the upper K unavailable rows.

[0479] i. In one example, Fig. 14C , C0 above the current row can be set equal to C6.

[0480] b. In one example, the current row may be used to fill the corresponding K rows.

[0481] i. In one example, Fig. 14C , C0 below the current row can be set equal to C6.

[0482] c. In one example, the ALF filter strength of the M rows (or N columns) around the vertical (or horizontal) position of ALF VB in the chroma ALF should be aligned with the filter strength in the luma ALF of the 4:2:2 / 4:4:4 chroma format video. The ALF filter strength of the luma ALF and chroma ALF is controlled by alfShiftY (e.g., in Table 45) and alfShiftC (e.g., in Table 46).

[0483] i. In one example, when Yc == vbPosC-M (eg, M=0, 1), alfShiftC=T1; when Yc!=vbPosC-M, alfShiftC=T2.

[0484] 1) In one example, T1=10, T2=7.

[0485] ii. In one example, when Yc == vbPosC+M (eg, M=0, 1), alfShiftC=T1; when Yc!=vbPosC+M, alfShiftC=T2.

[0486] 1) In one example, T1=10, T2=7.

[0487] 6. Examples

[0488] 6.1. Example of loop_filter_across_tiles_enabled_flag signaling notification

[0489] 7.3.2.4 Picture parameter set RBSP syntax

[0490]

[0491] 6.2. Example 1 of loop_filter_across_slices_enabled_flag signaling

[0492] 7.3.2.4 Picture parameter set RBSP syntax

[0493]

[0494] 6.3. Example 2 of loop_filter_across_slices_enabled_flag signaling

[0495] 7.3.2.4 Picture parameter set RBSP syntax

[0496]

[0497] 6.4. Example 3 of loop_filter_across_slices_enabled_flag signaling

[0498] 7.3.2.4 Picture parameter set RBSP syntax

[0499]

[0500]

[0501] 6.5. Example 4 of loop_filter_across_slices_enabled_flag signaling

[0502] 7.3.2.4 Picture parameter set RBSP syntax

[0503]

[0504] 6.6. LMCS Information Signaling Notification Example #1

[0505] 7.3.2.4 Picture parameter set RBSP syntax

[0506]

[0507] 6.7.LMCS Information Signaling Notification Example #2

[0508] 7.3.2.7 Picture parameter set RBSP syntax

[0509]

[0510]

[0511] 6.8.LMCS Information Signaling Example #3

[0512] 7.3.7.1 General Strip Header Syntax

[0513] 6.9. LMCS Information Signaling Example #4

[0514] 7.3.2.7 Picture parameter set RBSP syntax

[0515]

[0516] 6.10.LMCS Information Signaling Example #5

[0517] 7.3.7.1 Generic Strip Header Syntax

[0518]

[0519] 6.11.LMCS Information Signaling Example #6

[0520] Example 1: When slice_lmcs_enabled_flag is not present, it is inferred to be equal to ph_lmcs_enabled_flag.

[0521] Example 2: When slice_lmcs_aps_id does not exist, it is inferred to be equal to ph_lmcs_aps_id.

[0522] Example 3: When slice_chroma_residual_scale_flag is not present, it is inferred to be equal to ph_chroma_residual_scale_flag.

[0523] 6.12. Example 1 of Strip Information Signaling

[0524] 7.3.2.4 Picture parameter set RBSP syntax

[0525]

[0526]

[0527] 6.13. Embodiment of handling ALF virtual boundary

[0528] 8.8.5.4 Codec Tree Block Filtering Process for Chroma Samples

[0529] …

[0530] – In Table 46, the vertical sample position offsets y1, y2 and the variable alfShiftC are specified according to the vertical chroma sample position y and applyAlfLineBufBoundary.

[0531] –The variable curr is derived as follows:

[0532] curr=recPicture[h x ][v y ] (1504)

[0533] – The chroma filter coefficient array f[j] and the chroma clipping value array c[j] are derived as follows, where j = 0..5:

[0534] f[j]=AlfCoeff C[slice_alf_aps_id_chroma][altIdx][j] (1505)

[0535] c[j]=AlfClip C [slice_alf_aps_id_chroma][altIdx][j] (1506)

[0536] –The derivation of the variable sum is as follows:

[0537]

[0538] sum=curr+((sum+64)>>alfShiftC) (1508)

[0539] – The modified filtered reconstructed chrominance picture sample alfPicture[xCtbC+x][yCtbC+y] is derived as follows:

[0540] alfPicture[xCtbC+x][yCtbC+y]=Clip3(0,(1< <BitDepth)-1,sum) (1509)

[0541] Table 46 – Specifying y1, y2, and alfShiftC according to the vertical chroma sample position y and applyAlfLineBufBoundary

[0542]

[0543] Fig.15 A block diagram of an example video processing system 1900 in which various techniques of the present disclosure may be implemented is shown. Various implementations may include some or all of the components of system 1900. System 1900 may include an input 1902 for receiving video content. The video content may be received in a raw or uncompressed format (e.g., 8 or 10 bit multi-component pixel values), or may be received in a compressed or encoded format. Input 1902 may represent a network interface, a peripheral bus interface, or a storage interface. Examples of network interfaces include wired interfaces such as Ethernet, a passive optical network (PON), and wireless interfaces such as Wi-Fi or a cellular interface.

[0544] System 1900 may include a codec component 1904, which may implement various codec or encoding methods described in the present disclosure. Codec component 1904 may reduce the average bit rate of the video from input 1902 to the output of codec component 1904 to generate a codec representation of the video. Therefore, codec technology is sometimes referred to as video compression or video transcoding technology. The output of codec component 1904 may be stored or transmitted via a communication connection as represented by component 1906. The bitstream (or codec) representation of the storage or communication of the video received at input 1902 may be used by component 1908, which is used to generate pixel values ​​or displayable video sent to display interface 1910. The process of generating a user-viewable video from a bitstream representation is sometimes referred to as video decompression. In addition, although some video processing operations are referred to as "codec" operations or tools, it should be understood that the codec tool or operation is used at the encoder, and the corresponding decoding tool or operation of the inverse encoding result will be performed by the decoder.

[0545] Examples of peripheral bus interfaces or display interfaces may include Universal Serial Bus (USB) or High Definition Multimedia Interface (HDMI) or Display Port, etc. Examples of storage interfaces include SATA (Serial Advanced Technology Attachment), PCI, IDE interfaces, etc. The techniques described in the present disclosure may be implemented in various electronic devices, such as mobile phones, laptop computers, smart phones, or other devices capable of performing digital data processing and / or video display.

[0546] Fig.16 3600 is a block diagram of a video processing device 3600. Device 3600 may be used to implement one or more methods described in the present disclosure. Device 3600 may be located in a smartphone, a tablet, a computer, an Internet of Things (IoT) receiver, etc. Device 3600 may include one or more processors 3602, one or more memories 3604, and video processing hardware 3606. (Multiple) processors 3602 may be configured to implement one or more methods described in the present disclosure. Memory (multiple memories) 3604 may be used to store data and code for implementing the methods and techniques described in the present disclosure. Video processing hardware 3606 may be used in hardware circuits to implement some of the techniques described in the present disclosure.

[0547] Fig.18 is a block diagram illustrating an example video coding system 100 that may utilize the techniques of this disclosure.

[0548] like Fig.18As shown, the video coding system 100 may include a source device 110 and a target device 120. The source device 110 may generate encoded video data, and the source device 110 may be referred to as a video encoding device. The target device 120 may decode the encoded video data generated by the source device 110, and the target device 120 may be referred to as a video decoding device.

[0549] Source device 110 may include a video source 112 , a video encoder 114 , and an input / output (I / O) interface 116 .

[0550] The video source 112 may include, for example, a source of a video capture device, an interface for receiving video data from a video content provider, and / or a computer graphics system for generating video data, or a combination of these sources. The video data may include one or more pictures. The video encoder 114 encodes the video data from the video source 112 to generate a bit stream. The bit stream may include a bit sequence that forms a codec representation of the video data. The bit stream may include a codec picture and associated data. The codec picture is a codec representation of the picture. The associated data may include a sequence parameter set, a picture parameter set, and other grammatical structures. The I / O interface 116 may include a modulator / demodulator (modem) and / or a transmitter. The encoded video data may be directly sent to the target device 120 via the network 130a via the I / O interface 116. The encoded video data may also be stored on a storage medium / server 130b for access by the target device 120.

[0551] Target device 120 may include an I / O interface 126 , a video decoder 124 , and a display device 122 .

[0552] The I / O interface 126 may include a receiver and / or a modem. The I / O interface 126 may obtain encoded video data from the source device 110 or the storage medium / server 130b. The video decoder 124 may decode the encoded video data. The display device 122 may display the decoded video data to the user. The display device 122 may be integrated with the target device 120, or may be external to the target device 120, and the target device 120 may be configured to interface with an external display device.

[0553] The video encoder 114 and the video decoder 124 may operate according to a video compression standard, such as the High Efficiency Video Coding (HEVC) standard, the Versatile Video Coding (VVM) standard, and other current and / or other standards.

[0554] Fig.19 is a block diagram illustrating an example of a video encoder 200, which may be Fig.18 The video encoder 114 in the system 100 illustrated in FIG.

[0555] Video encoder 200 may be configured to perform any or all of the techniques of this disclosure. Fig.19 In the example shown, the video encoder 200 includes multiple functional components. The techniques described in this disclosure may be shared among the various components of the video encoder 200. In some examples, a processor may be configured to perform any or all of the techniques described in this disclosure.

[0556] The functional components of the video encoder 200 may include a segmentation unit 201, a prediction unit 202 which may include a mode selection unit 203, a motion estimation unit 204, a motion compensation unit 205 and an intra-frame prediction unit 206, a residual generation unit 207, a transform unit 208, a quantization unit 209, an inverse quantization unit 210, an inverse transform unit 211, a reconstruction unit 212, a buffer 213 and an entropy coding unit 214.

[0557] In other examples, the video encoder 200 may include more, fewer, or different functional components. In one example, the prediction unit 202 may include an intra-block copy (IBC) unit. The IBC unit may perform prediction in an IBC mode, where at least one reference picture is a picture where the current video block is located.

[0558] Furthermore, some components such as the motion estimation unit 204 and the motion compensation unit 205 may be highly integrated but are not shown in the figure for the purpose of description. Fig.19 are represented separately in the example.

[0559] The partitioning unit 201 may partition a picture into one or more video blocks. The video encoder 200 and the video decoder 300 may support various video block sizes.

[0560] The mode selection unit 203 may select one of the codec modes (intra or inter), for example, based on the error result, and provide the resulting intra or inter codec block to the residual generation unit 207 to generate residual block data and to the reconstruction unit 212 to reconstruct the codec block for use as a reference picture. In some examples, the mode selection unit 203 may select a combined intra and inter prediction (CIIP) mode, where the prediction is based on an inter prediction signal and an intra prediction signal. In the case of inter prediction, the mode selection unit 203 may also select a resolution of motion vectors for the block (e.g., sub-pixel or integer pixel resolution).

[0561] In order to perform inter-frame prediction on the current video block, the motion estimation unit 204 may generate motion information of the current video block by comparing the current video block with one or more reference frames from the buffer 213. The motion compensation unit 205 may determine a predicted video block of the current video block based on the motion information of pictures other than the picture associated with the current video block from the buffer 213 and decoded samples.

[0562] Motion estimation unit 204 and motion compensation unit 205 may perform different operations on the current video block, eg, depending on whether the current video block is in an I slice, a P slice, or a B slice.

[0563] In some examples, the motion estimation unit 204 may perform unidirectional prediction on the current video block, and the motion estimation unit 204 may search the reference picture of list 0 or list 1 for the reference video block of the current video block. Then, the motion estimation unit 204 may generate a reference index indicating the reference picture containing the reference video block in list 0 or list 1 and a motion vector indicating the spatial displacement between the current video block and the reference video block. The motion estimation unit 204 may output the reference index, the prediction direction indicator, and the motion vector as the motion information of the current video block. The motion compensation unit 205 may generate a predicted video block of the current block based on the reference video block indicated by the motion information of the current video block.

[0564] In other examples, the motion estimation unit 204 may perform bidirectional prediction on the current video block, the motion estimation unit 204 may search the reference picture in list 0 for the reference video block of the current video block, and may also search the reference picture in list 1 for another reference video block of the current video block. Then, the motion estimation unit 204 may generate a reference index indicating the reference picture containing the reference video block in list 0 and list 1, and a motion vector indicating the spatial displacement between the reference video block and the current video block. The motion estimation unit 204 may output the reference index and the motion vector of the current video block as the motion information of the current video block. The motion compensation unit 205 may generate a predicted video block of the current video block based on the reference video block indicated by the motion information of the current video block.

[0565] In some examples, motion estimation unit 204 may output a complete set of motion information for use in a decoding process by a decoder.

[0566] In some examples, motion estimation unit 204 may not output a complete set of motion information for the current video. Instead, motion estimation unit 204 may reference motion information of another video block to signal motion information for the current video block. For example, motion estimation unit 204 may determine that the motion information of the current video block is sufficiently similar to the motion information of a neighboring video block.

[0567] In one example, motion estimation unit 204 may indicate in a syntax structure associated with the current video block a value that indicates to video decoder 300 that the current video block has the same motion information as another video block.

[0568] In another example, motion estimation unit 204 may identify another video block and a motion vector difference (MVD) in a syntax structure associated with the current video block. The motion vector difference indicates the difference between the motion vector of the current video block and the motion vector of the indicated video block. Video decoder 300 may use the motion vector of the indicated video block and the motion vector difference to determine the motion vector of the current video block.

[0569] As described above, the video encoder 200 may predictively signal motion vectors. Two examples of predictive signaling techniques that may be implemented by the video encoder 200 include Advanced Motion Vector Prediction (AMVP) and merge mode signaling.

[0570] The intra prediction unit 206 may perform intra prediction on the current video block. When the intra prediction unit 206 performs intra prediction on the current video block, the intra prediction unit 206 may generate prediction data for the current video block based on decoded samples of other video blocks in the same picture. The prediction data for the current video block may include a prediction video block and various syntax elements.

[0571] The residual generation unit 207 may generate residual data for the current video block by subtracting (e.g., indicated by a negative sign) the prediction video block(s) of the current video block from the current video block. The residual data for the current video block may include residual video blocks corresponding to different sample components of the samples in the current video block.

[0572] In other examples, for the current video block, such as in skip mode, there may be no residual data for the current video block, and the residual generation unit 207 may not perform a subtraction operation.

[0573] Transform processing unit 208 may generate one or more transform coefficient video blocks for the current video block by applying one or more transforms to the residual video block associated with the current video block.

[0574] After transform processing unit 208 generates a transform coefficient video block associated with the current video block, quantization unit 209 may quantize the transform coefficient video block associated with the current video block based on one or more quantization parameter (QP) values ​​associated with the current video block.

[0575] The inverse quantization unit 210 and the inverse transform unit 211 may apply inverse quantization and inverse transform to the transform coefficient video block, respectively, to reconstruct a residual video block from the transform coefficient video block. The reconstruction unit 212 may add the reconstructed residual video block to corresponding samples of one or more predicted video blocks generated by the prediction unit 202 to generate a reconstructed video block associated with the current block stored in the buffer 213.

[0576] After the reconstruction unit 212 reconstructs the video block, a loop filtering operation may be performed to reduce video block artifacts in the video block.

[0577] The entropy coding unit 214 may receive data from other functional components of the video encoder 200. When the entropy coding unit 214 receives the data, the entropy coding unit 214 may perform one or more entropy coding operations to generate entropy-coded data and output a bitstream including the entropy-coded data.

[0578] Some embodiments of the disclosed technology include making a decision or determination to enable a video processing tool or mode. In one example, when a video processing tool or mode is enabled, the encoder will use the tool or implement the mode in the processing of a video block, but will not necessarily modify the generated bitstream based on the use of the tool or mode. That is, when a video processing tool or mode is enabled based on a decision or determination, the conversion from a video block to a bitstream representation of a video will use the video processing tool or mode. In another example, when a video processing tool or mode is enabled, a decoder will process the bitstream with the knowledge that the bitstream has been modified based on the video processing tool or mode. That is, the conversion from a bitstream representation of a video to a video block will be performed using the video processing tool or mode enabled based on a decision or determination.

[0579] Fig. 20 is a block diagram illustrating an example of a video decoder 300, which may be Fig.18 The video decoder 114 in the system 100 is shown.

[0580] Video decoder 300 may be configured to perform any or all of the techniques of this disclosure. Fig. 20 In the example shown, video decoder 300 includes multiple functional components. The techniques described in this disclosure may be shared among various components of video decoder 300. In some examples, a processor may be configured to perform any or all of the techniques described in this disclosure.

[0581] In such Fig. 20In the example shown, the video decoder 300 includes an entropy decoding unit 301, a motion compensation unit 302, an intra-frame prediction unit 303, an inverse quantization unit 304, an inverse transform unit 305, and a reconstruction unit 306 and a buffer 307. In some examples, the video decoder 300 can perform operations generally related to the video encoder 200 ( Fig.19 ) is the decoding channel that is the opposite of the encoding channel described by .

[0582] The entropy decoding unit 301 may obtain a coded bitstream. The coded bitstream may include entropy coded video data (e.g., coded video data blocks). The entropy decoding unit 301 may decode the entropy coded video data, and based on the entropy decoded video data, the motion compensation unit 302 may determine motion information, including motion vectors, motion vector precision, reference picture list index, and other motion information. For example, the motion compensation unit 302 may determine such information by performing AMVP and merge mode.

[0583] The motion compensation unit 302 may generate a motion compensated block, possibly performing interpolation based on an interpolation filter. An identifier for the interpolation filter at sub-pixel resolution may be included in the syntax element.

[0584] Motion compensation unit 302 may calculate interpolated values ​​for sub-integer pixels of a reference block using interpolation filters used by video encoder 20 during encoding of the video block. Motion compensation unit 302 may determine the interpolation filters used by video encoder 200 according to received syntax information and use the interpolation filters to produce a prediction block.

[0585] The motion compensation unit 302 may use some syntax information to determine the size of blocks used to encode (multiple) frames and / or (multiple) slices of the encoded video sequence, partitioning information describing how each macroblock of a picture of the encoded video sequence is partitioned, a mode indicating how each partition is encoded, one or more reference frames (and reference frame lists) for each inter-frame coded block, and other information for decoding the encoded video sequence.

[0586] The intra prediction unit 303 may form a prediction block from spatially neighboring blocks using, for example, an intra prediction mode received in the bitstream. The inverse quantization unit 303 inversely quantizes, i.e., dequantizes, the quantized video block coefficients provided in the bitstream and decoded by the entropy decoding unit 301. The inverse transform unit 303 applies an inverse transform.

[0587] The reconstruction unit 306 may add the residual block to the corresponding prediction block generated by the motion compensation unit 202 or the intra prediction unit 303 to form a decoded block. If necessary, a deblocking filter may also be applied to filter the decoded block to remove blocky artifacts. The decoded video block is then stored in a buffer 307, which provides reference blocks for subsequent motion compensation / intra prediction and also produces decoded video for presentation on a display device.

[0588] Next, a list of preferred solutions for some embodiments is provided.

[0589] The following solutions show example embodiments of the techniques discussed in the previous sections (eg, items 1 and 2).

[0590] 1. A video processing method (eg, Fig.17 The method 1700 shown in FIG. 1 includes: for a conversion between a video unit of a video and a codec representation of the video, determining ( 1702 ) whether an indication of suitability of in-loop filtering across video regions of the video unit is included in the codec representation; and performing ( 1704 ) the conversion based on the determination.

[0591] 2. A method as in solution 1, wherein the video unit includes a picture.

[0592] 3. The method of any of solutions 1-2, wherein the video region comprises a slice.

[0593] 4. A method as in any one of solutions 1-2, wherein the video area comprises strips.

[0594] The following solution shows an example embodiment of the technique discussed in the previous section (eg, item 3).

[0595] 5. A video processing method, comprising: performing a conversion between a video region of a video and a codec representation of the video, wherein the codec representation complies with a format rule, wherein the format rule stipulates that information about the applicability of a luminance mapping and chrominance scaling (LMCS) tool to the conversion is indicated in the codec representation at a video slice level; wherein the LMCS tool includes constructing a current video block based on a first domain and a second domain and / or scaling a chrominance residual in a luminance-dependent manner during the conversion.

[0596] The following solutions show example embodiments of the techniques discussed in the previous sections (eg, items 4, 5, 7, 8).

[0597] 6. A video processing method, comprising: performing conversion between a video and a codec representation of the video, wherein the conversion complies with a size rule, the size rule stipulating that: during encoding, the size rule is enforced for a maximum size of a video area encoded using a transform skip decoding scheme or an incremental pulse codec modulation decoding scheme, or during decoding, the codec representation is parsed and decoded by enforcing the size rule for a maximum size of a video area decoded using a transform skip decoding scheme or an incremental pulse codec modulation decoding scheme.

[0598] 7. The method of solution 6, wherein the size rule stipulates that the maximum size of the video region that is a transform skip block is less than or equal to the codec treeblock size.

[0599] 8. The method of solution 6, wherein the size rule specifies that the maximum size of a video region processed using the block-based delta-pulse mode codec is less than or equal to the codec treeblock size.

[0600] The following solution shows an example embodiment of the technique discussed in the previous section (eg, item 5).

[0601] 9. The method of solution 6, wherein the size rule stipulates that the maximum size of the video area used as a transform skip block is less than or equal to the minimum decoding block size.

[0602] 10. The method of solution 6, wherein the size rule stipulates that the maximum size of the video region processed using the block-based delta-pulse mode codec is less than or equal to the minimum codec block size.

[0603] The following solution shows an example embodiment of the technique discussed in the previous section (eg, item 7).

[0604] 11. A method as in any of solutions 6 to 10, wherein the size is indicated in a field in the codec representation, and wherein the minimum allowed codec block size for conversion controls where the field appears in the codec representation and / or how the size is interpreted from the field.

[0605] The following solution shows an example embodiment of the technique discussed in the previous section (eg, item 8).

[0606] 12. The method of any of solutions 6 to 10, wherein the minimum allowed codec block size for conversion controls whether a field indicating size rules is present in the codec representation and / or how the size is interpreted from the field.

[0607] The following solution shows an example embodiment of the technique discussed in the previous section (eg, item 6).

[0608] 13. A video processing method, comprising: performing conversion between a video region of a video and a codec representation of the video, wherein the codec representation conforms to a format rule, the format rule specifying that a minimum allowed transform block size used during conversion controls whether or how an indication of a maximum allowed transform block size is included in the codec representation.

[0609] 14. The method of solution 13, wherein the format rule specifies that the minimum allowed transform block size is greater than or equal to the minimum allowed coding block size.

[0610] The following solution shows an example embodiment of the technique discussed in the previous section (eg, item 9).

[0611] 15. A video processing method, comprising: performing conversion between video units of a video and a codec representation of the video according to format rules, wherein the codec representation complies with the format rules, and the format rules stipulate that a minimum allowed codec block size used during the conversion controls whether a field carrying information about the applicability of a codec tool in the conversion is included at the video region level.

[0612] 16. The method of solution 15, wherein the video region corresponds to a sequence parameter set or a picture parameter set or a picture header or a slice header.

[0613] 17. A method as in any of solutions 15-16, wherein the encoding and decoding tool comprises a combined inter-frame intra-frame prediction tool.

[0614] 18. A method as in any of solutions 15-16, wherein the codec tool comprises a multi-transform selection codec tool.

[0615] The following solution shows an example embodiment of the techniques discussed in the previous sections (eg, items 10, 11).

[0616] 19. A video processing method, comprising: for conversion between a video region of a video and a codec representation of the video, due to the use of a dual-tree codec for the conversion, determining that a partitioning scheme for a luminance component and a chrominance component of the video has different allowed minimum block sizes for the luminance component and the chrominance component, and performing the conversion based on the determination.

[0617] 20. A method as in solution 19, wherein the partitioning scheme comprises binary tree partitioning.

[0618] 21. A method as in Solution 19, wherein the partitioning scheme includes ternary tree partitioning.

[0619] The following solution shows an example embodiment of the technique discussed in the previous section (eg, item 12).

[0620] 22. A video processing method, comprising: performing conversion between a video region of a video and a codec representation of the video region by complying with format rules for the codec representation; wherein the format rules specify that a maximum number of sub-block based merge candidates used during the conversion can be derived as the sum of a first variable and a second variable, wherein the use of affine prediction controls the value of the first variable, and wherein the use of sub-block based temporal motion vector prediction controls the value of the second variable.

[0621] 23. The method of solution 22, wherein the first variable represents the number of allowed affine merge candidates.

[0622] The following solution illustrates an example embodiment of the technique discussed in the previous section (eg, item 13).

[0623] 24. A video processing method, comprising: because the video is 4:2:2 or 4:4:4, performing conversion between a video area of ​​the video and a codec representation area of ​​the video by complying with a processing rule applicable to the conversion, wherein the processing rule defines the alignment of chrominance and luminance to one or more of the following: (a) the number of pixel rows between a virtual boundary of an adaptive loop filter operation and a bottom boundary of a codec tree block; or (b) the filtering strength of a filter for a row between a virtual boundary of an adaptive loop filter operation and a bottom boundary of a codec tree block; or (c) a filling method for filling video samples in the same row.

[0624] 25. The method of solution 24, wherein the processing rule further defines a vertical alignment of a virtual boundary between chroma components and luma components.

[0625] 26. The method of solution 24, wherein the processing rule defines that the filling method for filling K rows and / or H columns of chrominance is aligned with the filling method for luma.

[0626] 27. The method of any of the above solutions, wherein the video area includes a video codec unit.

[0627] 28. A method according to any of the above solutions, wherein the video area comprises a video picture.

[0628] 29. A method as in any one of solutions 1 to 28, wherein the conversion comprises encoding the video into a codec representation.

[0629] 30. A method as in any one of solutions 1 to 28, wherein converting comprises decoding a codec representation to generate pixel values ​​of a video.

[0630] 31. A video decoding device, comprising a processor configured to implement the method in one or more of solutions 1 to 30.

[0631] 32. A video encoding apparatus, comprising a processor configured to implement the method in one or more of solutions 1 to 30.

[0632] 33. A computer program product having computer code stored thereon, which, when executed by a processor, causes the processor to implement the method of any one of solutions 1 to 30.

[0633] 34. A method, apparatus or system as herein described.

[0634] Fig.21 is a flow chart of an example method (2100) of video processing. Operation 2102 includes performing conversion between a video including a video unit and a bitstream of the video, wherein the bitstream complies with a format rule, and wherein the format rule provides that, in response to whether or how the video unit is partitioned into slices, a first syntax element indicating whether a loop filtering operation is performed across a slice boundary is selectively included in the bitstream.

[0635] In some embodiments of the method 2100, the video unit includes a picture. In some embodiments of the method 2100, the first syntax element is in a picture parameter set. In some embodiments of the method 2100, the format rule provides that in response to the video unit being partitioned into more than one slice, the first syntax element is included in the bitstream. In some embodiments of the method 2100, the format rule provides that in response to the video unit including only one slice, the first syntax element is not included in the bitstream. In some embodiments of the method 2100, the format rule provides that in response to the bitstream including a zero value for the second syntax element indicating whether there is one slice per picture as a constraint, the first syntax element is included in the bitstream.

[0636] In some embodiments of the method 2100, the format rule provides that, in response to a bitstream including a value of a second syntax element indicating whether there is one tile per picture as a constraint, the first syntax element is required to be equal to zero and included in the bitstream. In some embodiments of the method 2100, the second syntax element is one_tile_per_pic_constraint_flag. In some embodiments of the method 2100, the loop filtering operation includes at least one of a deblocking filtering operation, a sample adaptive offset operation, or an adaptive loop filtering operation. In some embodiments of the method 2100, the first syntax element is loop_filter_across_tiles_enabled_flag. In some embodiments of the method 2100, performing the conversion includes encoding the video into a bitstream.

[0637] In some embodiments of method 2100, performing the conversion includes generating a bitstream from the video, and the method further includes storing the bitstream in a non-transitory computer-readable recording medium. In some embodiments of method 2100, performing the conversion includes decoding the video from the bitstream. In some embodiments, the video decoding device includes a processor configured to implement the operations of method 2100 and related embodiments. In some embodiments, the video encoding device includes a processor configured to implement the operations of method 2100 and related embodiments.

[0638] In some embodiments, a computer program product having computer instructions thereon, the instructions, when executed by a processor, cause the processor to perform method 2100 and operations related to related embodiments. In some embodiments, a non-transitory computer-readable recording medium storing a bitstream of a video generated by a method performed by a video processing device, wherein the method includes: generating a bitstream from a video including a video unit; wherein the bitstream complies with a format rule, and wherein the format rule stipulates that, in response to whether or how the video unit is divided into slices, a first syntax element indicating whether a loop filtering operation is performed on a block slice boundary is selectively included in the bitstream. In some embodiments, a non-transitory computer-readable storage medium stores instructions that cause a processor to implement method 2100 and operations of related embodiments. In some embodiments, a bitstream generation method includes: generating a bitstream of a video according to the operations of method 2100 and related embodiments, and storing the bitstream in a computer-readable program medium.

[0639] Fig. 22 is a flow chart of an example method (2200) of video processing. Operation 2202 includes performing conversion between a video unit of a video and a bitstream of the video, wherein the bitstream conforms to a format rule, and wherein the format rule provides that, responsive to whether or how the video unit is partitioned into slices, a syntax element indicating whether to perform a loop filtering operation across a slice boundary is selectively included in the bitstream.

[0640] In some embodiments of the method 2200, the video unit comprises a picture. In some embodiments of the method 2200, the video unit comprises a sub-picture. In some embodiments of the method 2200, the syntax element is in a picture parameter set. In some embodiments of the method 2200, the rule provides that in response to the video unit being partitioned into only one slice, the syntax element is not included in the bitstream. In some embodiments of the method 2200, the rule provides that in response to each sub-picture of the video unit being partitioned into only one slice, the syntax element is not included in the bitstream. In some embodiments of the method 2200, the rule provides that in response to each sub-picture of the video unit being partitioned into only one slice, the syntax element is included in the bitstream.

[0641] In some embodiments of the method 2200, the rule specifies that, in response to a first flag equal to 1 specifying that each sub-picture consists of one and only one rectangular slice, the syntax element is included in the bitstream. In some embodiments of the method 2200, the first flag is single_slice_per_subpic_flag. In some embodiments of the method 2200, the first flag is in a picture parameter set.

[0642] In some embodiments of the method 2200, the rule provides that, in response to a video unit including a picture that is partitioned into a non-rectangular shape, the syntax element is included in the bitstream. In some embodiments of the method 2200, the rule provides that, in response to a second flag equal to 0 that specifies that each picture adopts a grid scan stripe mode, the syntax element is included in the bitstream. In some embodiments of the method 2200, the grid scan stripe mode is a non-rectangular stripe mode. In some embodiments of the method 2200, the second flag is rect_slice_flag. In some embodiments of the method 2200, the second flag is in a picture parameter set. In some embodiments of the method 2200, the rule provides that, in response to a video unit including a picture that is partitioned into a rectangular shape and the number of slices of the video unit is equal to 1, the syntax element is not included in the bitstream.

[0643] In some embodiments of the method 2200, the rule provides that, in response to a bitstream including a zero value of a syntax element indicating whether each picture has one slice as a constraint, the syntax element is included in the bitstream. In some embodiments of the method 2200, the rule provides that, in response to a bitstream including a one value of a syntax element indicating whether each picture has one slice as a constraint, the syntax element is set equal to 0 and included in the bitstream. In some embodiments of the method 2200, the loop filtering operation includes at least one of a deblocking filtering operation, a sample adaptive offset operation, or an adaptive loop filtering operation. In some embodiments of the method 2200, performing the conversion includes encoding the video into a bitstream. In some embodiments of the method 2200, performing the conversion includes generating a bitstream from the video, and the method also includes storing the bitstream in a non-transitory computer-readable recording medium.

[0644] In some embodiments of method 2200, performing the conversion includes decoding the video from a bitstream. In some embodiments, the video decoding device includes a processor configured to implement the operations of method 2200 and related embodiments. In some embodiments, the video encoding device includes a processor configured to implement the operations of method 2200 and related embodiments. In some embodiments, a computer program product having computer instructions thereon, the instructions, when executed by the processor, cause the processor to implement the operations of method 2200 and related embodiments. In some embodiments, a non-transitory computer-readable recording medium storing a bitstream of a video generated by a method performed by a video processing device, wherein the method includes: generating a bitstream from a video unit of the video; wherein the bitstream conforms to a format rule, and wherein the format rule specifies that, in response to whether or how the video unit is partitioned into slices, a syntax element indicating whether to perform a loop filtering operation across a slice boundary is selectively included in the bitstream.

[0645] In some embodiments, a non-transitory computer-readable storage medium stores instructions for causing a processor to implement the operations of method 2200 and related embodiments. In some embodiments, a bitstream generation method includes: generating a video bitstream according to the operations of method 2200 and related embodiments, and storing the bitstream on a computer-readable program medium.

[0646] Fig.23 is a flow chart of an example method (2300) of video processing. Operation 2302 includes performing conversion between a video region of a video and a bitstream of the video, wherein the bitstream conforms to a format rule, and wherein the format rule specifies that information regarding the applicability of a tool for the conversion is indicated in the bitstream at a video slice level and / or at a video picture level, and wherein the tool maps luma samples to specific values ​​and selectively applies a scaling operation to the values ​​of chroma samples.

[0647] In some embodiments of the method 2300, the format rules provide that the information about the applicability of the tool includes a first indication of whether the tool is enabled. In some embodiments of the method 2300, wherein the format rules provide that the information about the applicability of the tool includes a second indication of a parameter, and wherein the parameter includes an identifier of an adaptive parameter set (APS) for the tool at a video picture level, and / or a value indicating whether chroma residual scaling is enabled in a video picture level. In some embodiments of the method 2300, the format rules provide that the information about the applicability of the tool is indicated at both a video slice level and a video picture level.

[0648] In some embodiments of method 2300, the format rules provide that information about the applicability of the tool is indicated both at the video strip level and at the video picture level, and wherein previous information about the applicability of the tool included at the video picture level is overwritten by the information about the applicability of the tool included at the video strip level.

[0649] In some embodiments of the method 2300, the format rules provide that the information about the applicability of the tool is indicated in a video slice level or a video picture level. In some embodiments of the method 2300, the format rules provide that the information about the applicability of the tool is indicated in a video slice level to indicate an adaptation parameter set (APS) for a current slice of a video region. In some embodiments of the method 2300, the format rules provide that the bitstream includes a syntax element, the syntax element being in a sequence parameter set (SPS) or in a picture parameter set (PPS) to indicate whether the information about the applicability of the tool is indicated in a video slice level or a video picture level.

[0650] In some embodiments of the method 2300, the syntax element controls only the first indication. In some embodiments of the method 2300, the syntax element controls only the second indication. In some embodiments of the method 2300, the syntax element controls the first indication and the second indication. In some embodiments of the method 2300, the format rule provides that, in response to the information not being present in the picture header, the information about the applicability of the tool is indicated in a syntax element of the picture header with a default value. In some embodiments of the method 2300, the format rule provides that, in response to the information not being present in the slice header, the information about the applicability of the tool is indicated in a syntax element of the slice header with a default value. In some embodiments of the method 2300, wherein the format rule provides that the information about the applicability of the tool is indicated in a syntax element of the slice header, wherein the format rule provides that the syntax element has a value of a corresponding syntax element that indicates the information about the applicability of the tool in the picture header, and wherein the format rule provides that, in response to the information not being present in the slice header, the syntax element has a value of a corresponding syntax element.

[0651] In some embodiments of the method 2300, the format rules provide that, in response to a slice associated with a picture header that references identifiers of multiple APSs, identifiers of multiple APSs for tools that enable adaptive parameter sets (APSs) are indicated in the picture header. In some embodiments of the method 2300, the length of the list of identifiers of multiple APSs depends on the number of slices in the picture. In some embodiments of the method 2300, the format rules provide that a syntax element is included in the slice header, and wherein the syntax element indicates the APS enabled by the tool from the multiple APSs for the current slice. In some embodiments of the method 2300, when the tool is enabled, if the video area is from a luma component, then a switching of samples between a reshaped domain and an original domain of the video area is performed, or wherein, when enabled, if the video area is from a chroma component, a scaling of a chroma residual of the video area is performed.

[0652] Fig.24 2402 includes performing conversion between a video region of a video and a bitstream of the video, wherein the conversion complies with a size rule, and wherein the size rule specifies a maximum size of a video region encoded using a transform skip (TS) codec scheme or a block-based delta pulse codec modulation (BDPCM) codec scheme, or a maximum size of a transform block for a video region according to codec characteristics of the video region.

[0653] In some embodiments of the method 2400, the size rule provides that the maximum size of the video area is less than or equal to the size of the codec block (CTB). In some embodiments of the method 2400, the size rule provides that the maximum allowed width and height for the TS encoding or decoding scheme and / or for the BDPCM encoding or decoding scheme for the luma block is less than or equal to the CTB size. In some embodiments of the method 2400, the size rule provides that the maximum allowed width and height for the TS encoding or decoding scheme and / or for the BDPCM encoding or decoding scheme for the chroma block is less than or equal to the CTB size divided by subWidthC and / or the CTB size divided by subHeightC, where subWidthC and subHeightC depend on the chroma format of the video. In some embodiments of the method 2400, the size rule provides that the maximum allowed width for the TS encoding or decoding scheme and / or for the BDPCM encoding or decoding scheme for the chroma block is less than or equal to the CTB size divided by subWidthC and / or the CTB size divided by subHeightC.

[0654] In some embodiments of method 2400, the size rule stipulates that the maximum allowable height for a chroma block for a TS encoding or decoding scheme and / or for a BDPCM encoding or decoding scheme is less than or equal to the CTB size divided by subWidthC and / or the CTB size divided by subHeightC. In some embodiments of method 2400, the first value is equal to log2_transform_skip_max_size_minus2 plus 2, where the first value is less than or equal to a second value for CtbLog2SizeY, and where log2_transform_skip_max_size_minus2 plus 2 is equal to the log2 of the maximum block size for a TS encoding scheme or for a TS decoding scheme.

[0655] In some embodiments of method 2400, the first value describes that the maximum value for log2_transform_skip_max_size_minus2 is equal to the second value for CtbLog2SizeY minus 2, where log2_transform_skip_max_size_minus2 plus 2 is equal to the log2 of the maximum block size for a TS encoding scheme or for a TS decoding scheme. In some embodiments of method 2400, where the size rule stipulates that the maximum size of the video region as a transform skip square is the maximum of (CtbSizeY, 1<<(log2_transform_skip_max_size_minus2+2)), where << represents a left shift operation, where CtbSizeY is the CTB size, and where log2_transform_skip_max_size_minus2 plus 2 is equal to the log2 of the maximum block size for a TS encoding scheme or for a TS decoding scheme. In some embodiments of method 2400, the size rule stipulates that the maximum size of the video region as a transform skip square is 1<<Min(log2_transform_skip_max_size_minus2+2, CtbLog2SizeY), where << represents a left shift operation, where CtbSizeY is the CTB size, and where log2_transform_skip_max_size_minus2 plus 2 is equal to the log2 of the maximum block size for a TS encoding scheme or for a TS decoding scheme. In some embodiments of method 2400, the size rule stipulates that the maximum size for a chroma block for a TS encoding or decoding scheme and / or for a BDPCM encoding or decoding scheme is less than or equal to the size of the maximum transform block for a chroma block.

[0656] In some embodiments of the method 2400, the maximum allowed width and height for the TS encoding or decoding scheme and / or for the BDPCM encoding or decoding scheme for chroma blocks is less than or equal to the size of the maximum transform block for the luma block divided by SubWidthC. In some embodiments of the method 2400, the maximum allowed width for the TS encoding or decoding scheme and / or for the BDPCM encoding or decoding scheme for chroma blocks is less than or equal to the size of the maximum transform block for the luma block divided by SubWidthC. In some embodiments of the method 2400, the maximum allowed height for the TS encoding or decoding scheme and / or for the BDPCM encoding or decoding scheme for chroma blocks is less than or equal to the size of the maximum transform block for the luma block divided by subHeightC. In some embodiments of the method 2400, the size rule provides that the maximum size of the video area that is a transform skip block is indicated in the bitstream with a binary syntax element. In some embodiments of the method 2400, the size rule provides that the maximum size of the video area that is a transform skip block and / or the size of the maximum allowed transform block is greater than or equal to the size of the minimum encoding and decoding block.

[0657] Fig.25 2502 includes performing conversion between a video region of a video and a bitstream of the video, wherein the bitstream conforms to a format rule, the format rule specifying that a minimum allowed codec block size used during conversion controls whether and / or how a maximum allowed transform block size is included in the bitstream.

[0658] In some embodiments of the method 2500, the format rule specifies that the minimum allowed transform block size is greater than or equal to the minimum allowed codec block size. In some embodiments of the method 2500, in response to the minimum allowed codec block size being equal to 64, the value of sps_max_luma_transform_size_64_flag included in the bitstream is equal to 1. In some embodiments of the method 2500, in response to the minimum allowed codec block size being equal to 64, the value of sps_max_luma_transform_size_64_flag is not included in the bitstream and is inferred to be equal to 1.

[0659] Fig.26 2602 includes performing conversion between a video region of a video and a bitstream of the video, wherein the bitstream conforms to a format rule, the format rule specifying that a minimum allowed codec block size used during conversion controls whether and / or how an indication of a maximum size of a video region for encoding or decoding using a transform skip (TS) scheme or a block-based delta pulse codec modulation (BDPCM) scheme is included in the bitstream.

[0660] In some embodiments of the method 2600, the format rules specify that the maximum size of the video area is greater than or equal to the minimum allowed codec block size. In some embodiments of the method 2600, the format rules specify that the maximum size of the video area is less than or equal to W, where W is an integer. In some embodiments of the method 2600, W is 32. In some embodiments of the method 2600, the format rules specify that the maximum size of the video area is greater than or equal to the minimum allowed codec block size, wherein the format rules specify that the maximum size of the video area is less than or equal to W. In some embodiments of the method 2600, the format rules specify that when the TS scheme and / or the BDPCM scheme is enabled, the minimum allowed codec block size is less than or equal to X. In some embodiments of the method 2600, the format rules specify that when the TS scheme and / or the BDPCM scheme is enabled, the maximum size of the video area is greater than or equal to the minimum allowed codec block size.

[0661] In some embodiments of the method 2600, the format rule specifies that when the TS scheme is enabled, the log2 of the maximum size is greater than or equal to the log2 of the size of the minimum luma codec block. In some embodiments of the method 2600, the format rule specifies that the maximum size of the video area is the maximum of the maximum size or the minimum allowed codec block size. In some embodiments of the method 2600, wherein the format rule specifies that the maximum size of the video area is the minimum of a first value and a second value, wherein the first value is an integer W, and wherein the second value is the maximum of the maximum size or the minimum allowed codec block size. In some embodiments of the method 2600, the format rule specifies that when the TS scheme and / or the BDPCM scheme are enabled, the maximum size of the video area is the maximum of the maximum size or the minimum allowed codec block size. In some embodiments of the method 2600, wherein the format rule specifies that when the TS scheme and / or the BDPCM scheme are enabled, the maximum size of the video area is the minimum of a first value and a second value, wherein the first value is an integer W, and wherein the second value is the maximum of the maximum size or the minimum allowed codec block size.

[0662] In some embodiments of the method 2600, the format rule specifies that a maximum size of a video region is included in the bitstream based on a minimum allowed codec block size. In some embodiments of the method 2600, the format rule specifies that the maximum size of a video region is indicated by including in the bitstream the difference between the log 2 of the maximum size of the video region and the log 2 of the minimum allowed codec block size. In some embodiments of the method 2600, the format rule specifies: MaxTsSize = 1 << (MinCbLog2SizeY + log2_diff_max_transform_skip_min_coding_block), where << represents a left shift operation, where MaxTsSize is the maximum size of the video region, where MinCbLog2SizeY is the log 2 of the minimum coding unit size, and where log2_diff_max_transform_skip_min_coding_block is the log 2 of the difference between the maximum size of the video region and the minimum allowed codec block size in the bitstream.

[0663] In some embodiments of method 2600, the format rules specify:

[0664] MaxTsSize=min(W,1<<(MinCbLog2SizeY+log2_diff_max_trasform_skip_min_coding_block)), where << represents a left shift operation, where W is an integer, where MaxTsSize is the maximum size of the video area, where MinCbLog2SizeY is the log 2 of the minimum coding unit size, and where log2_diff_max_trasform_skip_min_coding_block is the log2 of the difference between the maximum size of the video area and the minimum allowed coding block size in the bitstream.

[0665] Fig. 27 is a flow chart of an example method (2700) of video processing. Operation 2702 includes performing conversion between a video unit of a video and a bitstream of the video according to a format rule, wherein the bitstream conforms to the format rule, and the format rule specifies that a minimum allowed codec block size used during conversion controls whether and / or how a field indicating enabling or disabling of a transform skip (TS) scheme or a block-based delta pulse codec modulation (BDPCM) scheme is included in the bitstream.

[0666] In some embodiments of the method 2700, the format rule provides that, in response to the minimum allowed codec block size being equal to 64, the field is inferred to be equal to 0 and is not included in the bitstream. In some embodiments of the method 2700, the format rule provides that, in response to the minimum allowed codec block size being greater than the maximum allowed size of the TS codec scheme and / or the BDPCM codec scheme, the field is inferred to be equal to 0 and is not included in the bitstream.

[0667] Fig.28 is a flow chart of an example method (2800) of video processing. Operation 2802 includes performing conversion between a video unit of a video and a bitstream of the video according to a format rule, wherein the bitstream conforms to the format rule, and the format rule specifies that a minimum allowed codec block size used during conversion controls whether and / or how a field carrying information about the applicability of a codec tool in the conversion is included at a video region level in the bitstream.

[0668] In some embodiments of the method 2800, the video area level corresponds to a sequence parameter set or a picture parameter set or a picture header or a slice header. In some embodiments of the method 2800, the format rule provides that in response to the minimum allowed codec block size being greater than an integer T, the field for the codec tool is inferred to be not used or not included in the bitstream. In some embodiments of the method 2800, the format rule provides that in response to the minimum allowed codec block size being greater than an integer T, the field for the codec tool indicates that the codec tool is not used or not included in the bitstream. In some embodiments of the method 2800, the codec tool includes a combined inter-frame intra-frame prediction (CIIP) tool. In some embodiments of the method 2800, the codec tool includes a multiple transform selection (MTS) codec tool. In some embodiments of the method 2800, the codec tool includes a segment block transform (SBT) codec tool. In some embodiments of the method 2800, the codec tool includes a symmetric motion vector difference (SMVD) codec tool. In some embodiments of the method 2800, the codec tool includes a bidirectional optical flow (BDOF) codec tool. In some embodiments of method 2800, the codec comprises an affine prediction codec.

[0669] In some embodiments of the method 2800, the codec tool includes an optical flow prediction refinement (PROF) codec tool. In some embodiments of the method 2800, the codec tool includes a decoder-side motion vector refinement (DMVR) codec tool. In some embodiments of the method 2800, the codec tool includes a CU-level weight bidirectional prediction (BCW) codec tool. In some embodiments of the method 2800, the codec tool includes a Merge codec with motion vector difference (MMVD) tool. In some embodiments of the method 2800, the codec tool includes a geometric partitioning mode (GPM) codec tool. In some embodiments of the method 2800, the codec tool includes an intra-block copy (IBC) codec tool. In some embodiments of the method 2800, the codec tool includes a palette codec tool. In some embodiments of the method 2800, the codec tool includes an adaptive color transform (ACT) codec tool. In some embodiments of the method 2800, the codec tool includes a joint Cb-Cr residual codec (JCCR) codec tool. In some embodiments of the method 2800, the codec comprises a cross-component linear prediction (CCLM) codec. In some embodiments of the method 2800, the codec comprises a multiple reference line (MRL) codec. In some embodiments of the method 2800, the codec comprises a matrix-based intra prediction (MIP) codec. In some embodiments of the method 2800, the codec comprises a sub-partition intra prediction (ISP) codec.

[0670] Fig.29 2900 is a flow chart of an example method (2900) for video processing. Operation 2902 includes, for conversion between a video region of a video and a bitstream of the video, determining that a partitioning scheme for a luma component and a chroma component of the video has different minimum allowed block sizes due to a dual-tree coding scheme being enabled for the video region. Operation 2904 includes performing the conversion based on the determination.

[0671] In some embodiments of method 2900, the partitioning scheme includes binary tree partitioning. In some embodiments of method 2900, the first minimum allowable block size for binary tree partitioning of the luminance component is MinBtSizeY = 1 << MinBtLog2SizeY, where the second minimum allowable block size for binary tree partitioning of the chrominance component is MinBtSizeC = 1 << MinBtLog2SizeC, where << represents a left shift operation, where MinBtSizeY is the first minimum allowable block size, where MinBtLog2SizeY is the log2 of MinBtSizeY, where MinBtSizeC is the first minimum allowable block size, where MinBtLog2SizeC is the log2 of MinBtSizeC, and where MinBtLog2SizeY is not equal to MinBtLog2SizeC. In some embodiments of method 2900, MinBtLog2SizeY is signaled based on the minimum decoding unit size. In some embodiments of method 2900, the difference between MinBtLog2SizeY and the log2 of the minimum decoding unit size is included in the bitstream. In some embodiments of method 2900, MinBtLog2SizeC is signaled based on the minimum decoding unit size.

[0672] In some embodiments of method 2900, the difference between MinBtLog2SizeC and the log2 of the minimum decoding unit size is included in the bitstream. In some embodiments of method 2900, the partitioning scheme includes ternary tree partitioning. In some embodiments of method 2900, the minimum allowable coding / decoding block size MinCbSizeY is equal to 1 << MinCbLog2SizeY, where << represents a left shift operation, and where MinCbLog2SizeY is the log2 of the minimum decoding unit size. In some embodiments of method 2900, the first minimum allowable block size for ternary tree partitioning of the luminance component is MinTtSizeY = 1 << MinTtLog2SizeY, where the second minimum allowable block size for ternary tree partitioning of the chrominance component is MinTtSizeC = 1 << MinTtLog2SizeC, where << represents a left shift operation, where MinTtSizeY is the first minimum allowable block size, where MinTtLog2SizeY is the log2 of MinTtSizeY, where MinTtSizeC is the first minimum allowable block size, where MinTtLog2SizeC is the log2 of MinTtSizeC, and where MinTtLog2SizeY is not equal to MinTtLog2SizeC.

[0673] In some embodiments of the method 2900, MinTtLog2SizeY is signaled based on a prediction of a minimum decoding unit size. In some embodiments of the method 2900, the difference between MinTtLog2SizeY and the log2 of the minimum decoding unit size is included in the bitstream. In some embodiments of the method 2900, MinTtLog2SizeC is signaled based on a prediction of a minimum decoding unit size. In some embodiments of the method 2900, the difference between MinTtLog2SizeC and the log2 of the minimum decoding unit size is included in the bitstream.

[0674] Fig.30 is a flow chart of an example method (3000) for video processing. Operation 3002 includes determining, for a conversion between a video region of a video and a bitstream of the video, a maximum number of sub-block based merge candidates allowed for the video region based on a rule. Operation 3004 includes performing the conversion based on the determination, wherein the rule provides that the maximum number of sub-block based merge candidates used during the conversion can be derived as a sum of a first variable and a second variable, wherein the first variable is equal to zero in response to affine prediction being disabled, and wherein the second variable is based on whether sub-block based temporal motion vector prediction (sbTMVP) is enabled.

[0675] In some embodiments of method 3000, the first variable represents the number of affine merge candidates allowed. In some embodiments of method 3000, the second variable is set to (sps_sbtmvp_enabled_flag&&ph_temporal_mvp_enable_flag). In some embodiments of method 3000, the first variable is derived as KS, where S is the value of syntax element signaling and K is a fixed value. In some embodiments of method 3000, the maximum number of merge candidates based on sub-blocks is MaxNumSubblockMergeCand, where the first variable is five_minus_max_num_affine_merge_cand, and MaxNumSubblockMergeCand=5-five_minus_max_num_affine_merge_cand+(sps_sbtmvp_enabled_flag&&ph_temporal_mvp_enable_flag). In some embodiments of the method 3000, wherein the maximum number of merge candidates based on subblocks is MaxNumSubblockMergeCand, wherein the first variable is four_minus_max_num_affine_merge_cand, and wherein MaxNumSubblockMergeCand=4-four_minus_max_num_affine_merge_cand+(sps_sbtmvp_enabled_flag&&ph_temporal_mvp_enable_flag). In some embodiments of the method 3000, wherein the maximum number of merge candidates based on subblocks is MaxNumSubblockMergeCand, and MaxNumSubblockMergeCand=Min(W,MaxNumSubblockMergeCand), wherein W is a fixed value. In some embodiments of the method 3000, the first variable is conditionally signaled in the bitstream. In some embodiments of the method 3000, in response to sps_affine_enabled_flag being equal to 1, the first variable is signaled in the bitstream. In some embodiments of the method 3000 , in response to the first variable not being present in the bitstream, five_minus_max_num_affine_merge_cand is inferred to be K.

[0676] Fig.31is a flow chart of an example method (3100) for video processing. Operation 3102 includes performing conversion between a video region of a video and a bitstream of the video region by conforming to a processing rule, wherein the processing rule is applicable to the conversion because the video is a 4:2:2 video or a 4:4:4 video, wherein the processing rule defines chroma and luma alignment for one or more of: (a) the number of pixel rows between a virtual boundary of an adaptive loop filter (ALF) operation and a bottom boundary of a codec tree block (CTB); or (b) the filtering strength of a filter for a row between a virtual boundary of an ALF operation and a bottom boundary of a CTB; or (c) a padding method for padding luma and chroma samples in the same row.

[0677] In some embodiments of the method 3100, the processing rules provide that the vertical position and / or the horizontal position aligns a virtual boundary between the chroma components and the luma components for the ALF operation. In some embodiments of the method 3100, wherein the processing rules provide that in response to the vertical position (vbPosY) of the virtual boundary for the luma component being equal to (CtbSizeY-S), the vertical position vbPosC of the virtual boundary for the chroma components is equal to (CtbSizeY-S) / SubHeightC. In some embodiments of the method 3100, the processing rules provide that in response to the horizontal position (vbPosY) of the virtual boundary for the luma component being equal to (CtbSizeY-S), the horizontal position (vbPosC) of the virtual boundary for the chroma components is equal to (CtbSizeY-S) / SubWidthC. In some embodiments of the method 3100, wherein CtbSizeY is the luma CTB size per codec tree unit (CTU), wherein S is an integer, wherein the values ​​of SubHeightC and / or SubWidthC are 1 or 2. In some embodiments of the method 3100, the processing rule provides that the padding method for the K rows and / or H columns of the chroma components near the vertical position and / or horizontal position of the virtual boundary is aligned with the padding method for the luma components, wherein Yc is the vertical or horizontal chroma sample position. In some embodiments of the method 3100, wherein Yc is equal to the vertical or horizontal position of the virtual boundary of the chroma components.

[0678] In some embodiments of method 3100, wherein the K unavailable rows above or to the left are filled, or wherein K rows of the corresponding K unavailable rows below or to the right of the virtual boundary are filled, or wherein K rows of the corresponding K available rows below or to the right of the virtual boundary are filled. In some embodiments of method 3100, wherein the K unavailable rows above are filled using a virtual boundary. In some embodiments of method 3100, wherein the samples of a first row are located immediately above the virtual boundary, wherein the samples of a second row immediately above the first row are set equal to the samples of the first row, and wherein the samples of a third row two rows above the first row are set equal to the samples of the first row. In some embodiments of method 3100, wherein the K rows are filled using a virtual boundary. In some embodiments of method 3100, wherein the sample points of a first row are located immediately below a virtual boundary, wherein the sample points of a second row immediately below the first row are set equal to the sample points of the first row, and wherein the sample points of a third row two rows below the first row are set equal to the sample points of the first row.

[0679] In some embodiments of method 3100, wherein the K unavailable rows below or to the right are filled, or wherein K rows of the corresponding K unavailable rows above or to the left of the virtual boundary are filled, or wherein K rows of the corresponding K available rows above or to the left of the virtual boundary are filled. In some embodiments of method 3100, wherein Yc is equal to the vertical or horizontal position of the virtual boundary for the chroma component minus M, where M is an integer. In some embodiments of method 3100, wherein the K unavailable rows below or to the right are filled, and wherein K rows of the corresponding K unavailable rows above or to the left of the virtual boundary are filled, or wherein K rows of the corresponding K available rows above or to the left of the virtual boundary are filled. In some embodiments of method 3100, wherein, in response to M being equal to 1, the K unavailable rows below are filled with the virtual boundary. In some embodiments of the method 3100, wherein the samples of the first row are located immediately below the virtual boundary, wherein the samples of the second row immediately below the first row are set equal to the samples of the first row, and wherein the samples of the third row two rows below the first row are set equal to the samples of the first row. In some embodiments of the method 3100, wherein, in response to M being equal to 1, the virtual boundary is used to fill the corresponding K unavailable rows or the corresponding K available rows. In some embodiments of the method 3100, wherein the samples of the first row are located immediately above the virtual boundary, wherein the samples of the second row immediately above the first row are set equal to the samples of the first row, and wherein the samples of the third row two rows above the first row are set equal to the samples of the first row.

[0680] In some embodiments of method 3100, wherein, in response to M being greater than or equal to 2, the lowermost row above the virtual boundary is used to fill the K unavailable rows below. In some embodiments of method 3100, wherein the first row of samples is located at a row two rows above the virtual boundary, wherein the samples of the second row located immediately below the virtual boundary are equal to the samples from the first row. In some embodiments of method 3100, wherein, in response to M being greater than or equal to 2, the corresponding K unavailable rows or the corresponding K available rows are filled with the corresponding rows of the lowermost row above the virtual boundary. In some embodiments of method 3100, wherein the first row of samples is located at a row two rows above the virtual boundary, wherein the samples of the second row located two rows above the first row are equal to the samples from the first row. In some embodiments of method 3100, Yc is equal to the vertical or horizontal position of the virtual boundary of the chroma component plus N, where N is an integer. In some embodiments of method 3100, wherein the K unavailable rows above or to the left are filled, and wherein K rows of the corresponding K unavailable rows below or to the right of the virtual border are filled, or wherein K rows of the corresponding K available rows below or to the right of the virtual border are filled. In some embodiments of method 3100, wherein the K unavailable rows above are filled using a virtual border.

[0681] In some embodiments of method 3100, wherein the samples of the first row are located two rows below the virtual boundary, and wherein the samples of the second row located immediately above the virtual boundary are equal to the samples from the first row. In some embodiments of method 3100, wherein the virtual boundary is used to fill the corresponding K unavailable rows or the corresponding K available rows. In some embodiments of method 3100, wherein the first row of samples is located two rows below the virtual boundary, and wherein the samples of the second row located two rows below the first row are equal to the samples from the first row. In some embodiments of method 3100, wherein the processing rule stipulates that the first filter strength for the M rows or N columns of the chroma components near the vertical position or horizontal position of the virtual boundary is aligned with the second filter strength for the luma component, wherein the second filter strength for the luma component and the first filter strength for the chroma component are controlled by alfShiftY and alfShiftC, respectively, and wherein Yc is the vertical or horizontal chroma sample position. In some embodiments of method 3100, wherein when Yc == vbPosC - M, alfShiftC = T1, when Yc! = vbPosC–M, alfShiftC=T2, where vbPosC is the horizontal position of the virtual boundary for the chroma components. In some embodiments of method 3100, T1=10, and T2=7. In some embodiments of method 3100, where Yc==vbPosC+M (eg, M=0,1), alfShiftC=T1, and when Yc!=vbPosC+M, alfShiftC=T2, where vbPosC is the horizontal position of the virtual boundary for the chroma components. In some embodiments of method 3100, T1=10, and T2=7.

[0682] In some embodiments of methods 2300-3100, performing the conversion includes encoding the video into a bitstream. In some embodiments of methods 2300-3100, performing the conversion includes generating a bitstream from the video, and the method also includes storing the bitstream in a non-transitory computer-readable recording medium. In some embodiments of methods 2300-3100, performing the conversion includes decoding the video from the bitstream. In some embodiments, the video decoding device includes a processor configured to implement the operations of methods 2300-3100 and related embodiments. In some embodiments, a computer program product having computer instructions thereon, when the instructions are executed by a processor, causes the processor to implement the operations of methods 2300-3100 and related embodiments. In some embodiments, a non-transitory computer-readable recording medium storing a bitstream of a video generated by a method performed by a video processing device, the method comprising: generating a bitstream from a video region of a video, wherein the bitstream complies with a format rule, the format rule providing that information about the applicability of a tool for conversion is indicated in the bitstream at a video slice level and / or at a video picture level, and wherein the tool maps luma samples to specific values ​​and selectively applies a scaling operation to the values ​​of chroma samples. In some embodiments, a non-transitory computer-readable storage medium storing instructions causes a processor to implement the operations of methods 2300-3100 and related embodiments. In some embodiments, a bitstream generation method comprising: generating a bitstream of a video according to the operations of methods 2300-3100 and related embodiments, and storing the bitstream on a computer-readable program medium. In some embodiments, a method, device, and bitstream generated according to the methods disclosed herein or the systems described.

[0683] As used herein, the term "video processing" may refer to video encoding, video decoding, video compression, or video decompression. For example, a video compression algorithm may be applied during conversion from a pixel representation of a video to a corresponding bitstream representation, or vice versa. For example, as defined by the syntax, the bitstream representation of the current video block may correspond to bits that are co-located or distributed at different locations within the bitstream. For example, a macroblock may be encoded based on error residual values ​​from a transform and codec, and bits may also be used in headers and other fields in the bitstream. In addition, as described in the above solution, during the conversion process, the decoder may parse the bitstream based on the determination knowing that certain fields may or may not be present. Similarly, the encoder may determine whether to include or not include certain syntax fields, and generate the codec representation accordingly by including or excluding syntax fields from the codec representation.

[0684] The disclosure and other solutions, examples, embodiments, modules and functional operations described in this patent document can be implemented in digital electronic circuits, or computer software, firmware or hardware, including the structures disclosed in this specification and their structural equivalents, or a combination of one or more thereof. The contents and other embodiments disclosed in this specification can be implemented as one or more computer program products, that is, one or more modules of computer program instructions encoded on a tangible and non-volatile computer-readable medium for data processing devices to execute or control the operation of the data processing device. The computer-readable medium can be a machine-readable storage device, a machine-readable storage substrate, a storage device, a material composition that affects a machine-readable propagation signal, or a combination of one or more thereof. The term "data processing unit" or "data processing device" includes all devices, equipment and machines for processing data, including, for example, a programmable processor, a computer or a multiprocessor or a computer group. In addition to hardware, the device may also include code that creates an execution environment for a computer program, for example, code constituting processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more thereof. A propagated signal is an artificially generated signal, for example, a machine-generated electrical, optical or electromagnetic signal, which is generated to encode information for transmission to a suitable receiver device.

[0685] A computer program (also referred to as a program, software, software application, script, or code) may be written in any form of programming language (including compiled or interpreted languages) and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. A program may be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program, or in multiple coordinated files (e.g., files that store one or more modules, subroutines, or portions of code). A computer program may be deployed for execution on one or more computers that are located at one site or distributed across multiple sites and interconnected by a communications network.

[0686] The processes and logic flows described in this specification may be performed by one or more programmable processors executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows may also be performed by special purpose logic circuits, and the apparatus may also be implemented as special purpose logic circuits, such as FPGAs (field programmable gate arrays) or ASICs (application specific integrated circuits).

[0687] For example, processors suitable for executing computer programs include general and special purpose microprocessors, and any one or more of any type of digital computer. Typically, the processor will receive instructions and data from a read-only memory or a random access memory or both. The basic components of a computer are a processor that executes instructions and one or more storage devices that store instructions and data. Typically, a computer will also include one or more mass storage devices for storing data, such as magnetic disks, magneto-optical disks, or optical disks, or be operatively coupled to one or more mass storage devices to receive data from them or transfer data to one or more mass storage devices, or both. However, a computer does not necessarily have such a device. Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, including, for example, semiconductor memory devices, such as EPROM, EEPROM, and flash memory devices; magnetic disks, such as internal hard disks or removable hard disks; magneto-optical disks; and CDROM and DVDROM disks. The processor and memory may be supplemented by, or incorporated into, dedicated logic circuits.

[0688] Although this patent document contains many details, they should not be construed as limitations on the scope of any invention or the claims, but rather as descriptions of features of particular embodiments of particular inventions. Certain features described in this patent document in the context of separate embodiments may also be implemented in combination in a single embodiment. Conversely, various functions described in the context of a single embodiment may also be implemented separately in multiple embodiments, or in any suitable subcombination. In addition, although the above-mentioned features may be described as working in certain combinations, or even initially claimed to be so, in some cases, one or more features in the claim combination may be removed from the combination, and the claim combination may be directed to subcombinations or variations of subcombinations.

[0689] Likewise, although operations are described in a particular order in the drawings, this should not be understood as requiring that such operations be performed in the particular order or order shown, or that all illustrated operations be performed, in order to achieve the desired results. In addition, the separation of various system components in the embodiments of this patent document should not be understood as requiring such separation in all embodiments.

[0690] Only a few implementations and examples are described, and other implementations, enhancements, and variations can be made based on what is described and illustrated in this patent document.

Claims

1. A method for processing video data, comprising: performing conversion between a video comprising a video unit and a bitstream of said video, wherein the bitstream complies with the format rules, and wherein the format rule specifies that, in response to whether the video unit is partitioned into more than one slice, a first syntax element indicating whether to allow a loop filtering operation to be performed across a slice boundary is selectively included in the bitstream, The format rule specifies that, in response to the bitstream containing a second constraint information syntax element with a value of zero, the first syntax element is allowed to be included in the bitstream, wherein the second constraint information syntax element indicates whether to constrain each picture to have only one slice, and the zero value of the second constraint information syntax element indicates that the constraint is not applied, and the second constraint information syntax element is a general constraint flag. The method of claim 1 , wherein the video unit is a picture.

3. The method of claim 1, wherein the first syntax element is in a picture parameter set.

4. The method of claim 1, wherein the format rule specifies that the first syntax element is included in the bitstream in response to the video unit being partitioned into more than one slice.

5. The method of claim 1, wherein the format rule provides that, in response to the video unit comprising only one slice, the first syntax element is excluded from the bitstream.

6. The method of claim 1 , wherein the format rule provides that, in response to the bitstream containing a second constraint information syntax element having a value of one, the first syntax element is inferred to be equal to zero and excluded from the bitstream, wherein the value of the second constraint information syntax element being one indicates that the constraint is applied, and the value of zero of the first syntax element indicates that the loop filtering operation across the slice boundary is disabled. The method of claim 1 , wherein the second constraint information syntax element is one_tile_per_pic_constraint_flag.

8. The method of claim 1, wherein the loop filtering operation comprises at least one of a deblocking filtering operation, a sample adaptive offset operation, or an adaptive loop filtering operation. 9 . The method of claim 1 , wherein the first syntax element is a loop_filter_across_tiles_enabled_flag.

10. The method of claim 1, wherein the converting comprises encoding the video into the bitstream.

11. The method of claim 1, wherein the converting comprises decoding the video from the bitstream.

12. An apparatus for processing video data, comprising a processor and a non-transitory memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to: performing conversion between a video comprising a video unit and a bitstream of said video, wherein the bitstream complies with the format rules, and wherein the format rule specifies that, in response to whether the video unit is partitioned into more than one slice, a first syntax element indicating whether to allow a loop filtering operation to be performed across a slice boundary is selectively included in the bitstream, The format rule specifies that, in response to the bitstream containing a second constraint information syntax element with a value of zero, the first syntax element is allowed to be included in the bitstream, wherein the second constraint information syntax element indicates whether to constrain each picture to have only one slice, and the zero value of the second constraint information syntax element indicates that the constraint is not applied, and the second constraint information syntax element is a general constraint flag.

13. The apparatus of claim 12, wherein the video unit is a picture; wherein the format rule specifies that, in response to the video unit being partitioned into more than one slice, the first syntax element is included in the bitstream; wherein the format rule provides that, in response to the video unit comprising only one slice, the first syntax element is excluded from the bitstream; wherein the loop filtering operation comprises at least one of a deblocking filtering operation, a sample adaptive offset operation or an adaptive loop filtering operation; and The first syntax element is loop_filter_across_tiles_enabled_flag.

14. The apparatus of claim 12, wherein the format rule provides that, in response to the bitstream containing a second constraint information syntax element having a value of one, the first syntax element is inferred to be equal to zero and excluded from the bitstream, wherein the value of the second constraint information syntax element is one indicating that the constraint is applied and the value of zero of the first syntax element indicates that the loop filtering operation across the slice boundary is disabled; and The second constraint information syntax element is one_tile_per_pic_constraint_flag.

15. A non-transitory computer-readable storage medium storing instructions that cause a processor to: performing conversion between a video comprising a video unit and a bitstream of said video, wherein the bitstream complies with the format rules, and wherein the format rule specifies that, in response to whether the video unit is partitioned into more than one slice, a first syntax element indicating whether to allow a loop filtering operation to be performed across a slice boundary is selectively included in the bitstream, The format rule specifies that, in response to the bitstream containing a second constraint information syntax element with a value of zero, the first syntax element is allowed to be included in the bitstream, wherein the second constraint information syntax element indicates whether to constrain each picture to have only one slice, and the zero value of the second constraint information syntax element indicates that the constraint is not applied, and the second constraint information syntax element is a general constraint flag.

16. The non-transitory computer-readable storage medium of claim 15, wherein the video unit is a picture; wherein the format rule specifies that, in response to the video unit being partitioned into more than one slice, the first syntax element is included in the bitstream; wherein the format rule provides that, in response to the video unit comprising only one slice, the first syntax element is excluded from the bitstream; wherein the loop filtering operation comprises at least one of a deblocking filtering operation, a sample adaptive offset operation or an adaptive loop filtering operation; and The first syntax element is loop_filter_across_tiles_enabled_flag.

17. The non-transitory computer-readable storage medium of claim 15, wherein the format rule provides that, in response to the bitstream containing a second constraint information syntax element having a value of one, the first syntax element is inferred to be equal to zero and excluded from the bitstream, wherein the value of the second constraint information syntax element is one indicating that the constraint is applied and the value of zero of the first syntax element indicates that the loop filtering operation across the slice boundary is disabled; and The second constraint information syntax element is one_tile_per_pic_constraint_flag.

18. A non-transitory computer-readable recording medium storing a bit stream of a video generated by a method performed by a video processing device, wherein the method comprises: generating a bitstream of a video comprising a video unit, wherein the bitstream complies with the format rules, and wherein the format rule specifies that, in response to whether the video unit is partitioned into more than one slice, a first syntax element indicating whether to allow a loop filtering operation to be performed across a slice boundary is selectively included in the bitstream, The format rule specifies that, in response to the bitstream containing a second constraint information syntax element with a value of zero, the first syntax element is allowed to be included in the bitstream, wherein the second constraint information syntax element indicates whether to constrain each picture to have only one slice, and the zero value of the second constraint information syntax element indicates that the constraint is not applied, and the second constraint information syntax element is a general constraint flag.

19. The non-transitory computer-readable recording medium of claim 18, wherein the video unit is a picture; wherein the format rule specifies that, in response to the video unit being partitioned into more than one slice, the first syntax element is included in the bitstream; wherein the format rule provides that, in response to the video unit comprising only one slice, the first syntax element is excluded from the bitstream; wherein the loop filtering operation comprises at least one of a deblocking filtering operation, a sample adaptive offset operation or an adaptive loop filtering operation; The first syntax element is loop_filter_across_tiles_enabled_flag; wherein the format rule provides that, in response to the bitstream comprising a second constraint information syntax element having a value of one, the first syntax element is inferred to be equal to zero and excluded from the bitstream, wherein the value of the second constraint information syntax element is one indicating that the constraint is applied and the value of the first syntax element is zero indicating that the loop filtering operation across the slice boundary is disabled; and The second constraint information syntax element is one_tile_per_pic_constraint_flag.

20. A method for storing a bit stream of a video, comprising: generating a bitstream of a video including the video unit; storing the bitstream in a non-transitory computer-readable storage medium, wherein the bitstream complies with the format rules, and wherein the format rule specifies that, in response to whether the video unit is partitioned into more than one slice, a first syntax element indicating whether to allow a loop filtering operation to be performed across a slice boundary is selectively included in the bitstream, The format rule specifies that, in response to the bitstream containing a second constraint information syntax element with a value of zero, the first syntax element is allowed to be included in the bitstream, wherein the second constraint information syntax element indicates whether to constrain each picture to have only one slice, and the zero value of the second constraint information syntax element indicates that the constraint is not applied, and the second constraint information syntax element is a general constraint flag.

21. A video decoding device comprising a processor configured to implement the method according to any one of claims 1-11.

22. A video encoding device comprising a processor configured to implement the method according to any one of claims 1-11.

23. A computer program product having computer instructions thereon, wherein: When the instructions are executed by a processor, the processor is enabled to implement the method according to any one of claims 1-11.

Images