Cross-component adaptive loop filter
By using CC-ALF to adjust the filter strength and two-stage signaling notification control LMCS mode in video encoding and decoding, the problem of insufficient bandwidth efficiency and visual quality in the prior art is solved, and more efficient video transmission and Simplified codec control.
Patent Information
- Application Number
- CN202180012659.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2020-02-03
- Filing Date
- 2021-02-02
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2041-02-02
AI Technical Summary
Existing video codec technology has shortcomings in bandwidth efficiency and visual quality, especially in high-resolution video transmission, filter intensity adjustment and visualization of cross-component adaptive loop filter (CC-ALF) at virtual boundaries The artifact problem has not been effectively solved, and the control complexity of LMCS codec mode is relatively high.
The filter intensity is adjusted at the virtual boundary by cross-component adaptive loop filter (CC-ALF), and the brightness mapping (LMCS) codec with chromatic scaling is controlled through a two-stage signaling notification mechanism, combined with remodeling processing and brightness-dependent chroma residual scaling, optimize the encoding and decoding conversion of video areas.
It improves the bandwidth efficiency and visual quality of video encoding and decoding, reduces visual artifacts at virtual boundaries, and simplifies the control of LMCS mode, improving the flexibility and efficiency of the encoding and decoding process.
Smart Images

Figure CN115176468B_ABST
Abstract
Description
[0001] CROSS-REFERENCE TO RELATED APPLICATIONS
[0002] This application is based on international patent application No. PCT / CN2021 / 074806 filed on February 2, 2021, which claims priority and benefits of international patent application No. PCT / CN2020 / 074222 filed on February 3, 2020. All of the above patent applications are hereby incorporated by reference in their entirety. Technical Field
[0003] This patent document relates to image and video encoding and decoding. Background Art
[0004] Digital video consumes the largest amount of bandwidth on the Internet and other digital communications networks. As the number of connected user devices capable of receiving and displaying video increases, bandwidth requirements for digital video usage are expected to continue to grow. Summary of the invention
[0005] This document discloses techniques that can be used by video encoders and decoders to use cross-component adaptive loop filtering and other chroma codec tools during video encoding or decoding.
[0006] In one example aspect, a video processing method is disclosed. The method includes, for a conversion between a video region of a chroma component of a video and a bitstream of the video, determining a manner to apply a cross-component adaptive loop filtering (CC-ALF) operation to a first sample point of the chroma component based on a position of a second sample point associated with the first sample point. The method also includes performing the conversion based on the determination.
[0007] In another example aspect, a video processing method is disclosed. The method includes performing a conversion between a video region of a video and a bitstream of the video. Based on syntax elements related to adaptive loop filtering (ALF) operations, one or more syntax elements related to use of cross-component adaptive loop filtering (CC-ALF) operations in the conversion are conditionally included in an adaptive parameter set.
[0008] In another example aspect, a video processing method is disclosed. The method includes performing conversion between a video region of a video and a bitstream of the video. The bitstream includes a first syntax element indicating whether an adaptive loop filter (ALF) or a cross-component adaptive loop filter (CC-ALF) is applied during the conversion of the video region.
[0009] In another example aspect, a video processing method is disclosed. The method includes performing conversion between a video region of a video and a codec representation of the video. Based on characteristics of the ALF operation, one or more syntax elements related to the ALF operation of a chrominance component are conditionally included in an adaptive parameter set.
[0010] In another example aspect, a video processing method is disclosed. The method includes performing conversion between a video and a video bitstream. The bitstream complies with a format rule that specifies that a two-level signaling mechanism is used to control the use of a luma mapping with chroma scaling (LMCS) codec mode, the two-level signaling mechanism including a first level in a first video region and a second level of signaling in a second video region associated with the first video region. The first level is higher than the second level. The LMCS codec mode includes performing conversion using a reshape process and a luma-dependent chroma residual scaling (CRS) process, in which a sample value represented in a first domain is mapped to a reshaped domain.
[0011] In another example aspect, a video processing method is disclosed. The method includes performing conversion between a video region of a video and a bitstream of the video. The bitstream complies with a format rule, which specifies that, in the case where a syntax flag indicates that a quadtree partitioning method is used to partition the video region, a value of a syntax element indicating a partitioning mode is not equal to any of the following items: (1) a horizontal ternary tree partitioning mode, (2) a horizontal binary tree partitioning mode, (3) a vertical ternary tree partitioning mode, or (4) a vertical binary tree partitioning mode.
[0012] In another example aspect, a video processing method is disclosed. The method includes, for a conversion between a current video block of a video and a codec representation of the video, determining a characteristic of a cross-component adaptive loop filter based on a position of a color component sample; and performing the conversion using the characteristic of the cross-component adaptive loop filter.
[0013] In another example aspect, another video processing method is disclosed. The method includes performing a conversion between a video region of a video and a codec representation of the video. One or more syntax elements related to use of cross-component adaptive loop filtering in the conversion are conditionally included in the codec representation based on whether another syntax element related to adaptive loop filtering is included in the codec representation.
[0014] In another example aspect, another video processing method is disclosed. The method includes performing a conversion between a video region of a video and a codec representation of the video. A first syntax element is included in the codec representation, which indicates whether an adaptive loop filter or a cross-component adaptive loop filter is applied during the conversion of the video region.
[0015] In another example aspect, another video processing method is disclosed. The method includes performing conversion between a video region of a video and a codec representation of the video. Depending on codec conditions, syntax elements related to adaptive loop filtering of chroma components are conditionally included in an adaptive parameter set.
[0016] In another example aspect, another video processing method is disclosed. The method includes performing conversion between a video region of a video and a codec representation of the video. The format rules of the codec representation define syntax elements that control the use of codec modes during conversion between two levels of the video. The codec modes include performing conversion of the video region using a reshaping process in which sample values are represented in a first domain and a second domain and a luma-dependent chroma residual scaling step.
[0017] In yet another example aspect, a video encoder apparatus is disclosed. The video encoder includes a processor configured to implement the above method.
[0018] In yet another exemplary aspect, a video decoder apparatus is disclosed. The video decoder includes a processor configured to implement the above method.
[0019] In yet another example aspect, a computer readable medium having code stored thereon is disclosed. The code embodies one of the methods described herein in the form of processor executable code.
[0020] These and other features are described throughout this document. BRIEF DESCRIPTION OF THE DRAWINGS
[0021] Figure 1 The nominal vertical and horizontal positions of 4:2:2 luma and chroma samples in the picture are shown.
[0022] Figure 2 An example of an encoder block diagram is shown.
[0023] Figure 3is an example picture of a CTU with 18×12 luma partitioned into 12 slices and 3 raster scan strips.
[0024] Figure 4 A picture with an 18x12 luma CTU partitioned into 24 slices and 9 rectangular strips is shown.
[0025] Figure 5 A picture (informative) is shown partitioned into 4 slices and 4 rectangular strips.
[0026] Figure 6 A picture (informative) that is partitioned into 15 slices, 24 stripes, and 24 sub-pictures.
[0027] Figure 7A-7C shows an example of a CTB that crosses a picture border, Fig. 7A It shows that K=M, L <N; Figure 7B Shows K <M,L=N; Figure 7C Shows K <M,L<N。
[0028] Figure 8 is a schematic diagram of picture samples on an 8×8 grid and horizontal and vertical block boundaries, as well as non-overlapping blocks of 8×8 samples that can be deblocked in parallel.
[0029] Fig. 9 Examples of pixels involved in filter on / off decisions and strong / weak filter selection are shown.
[0030] Fig.10 Examples of four one-dimensional directional patterns of EO point classification are shown: horizontal (EO class = 0), vertical (EO class = 1), 135° diagonal (EO class = 2), and 45° diagonal (EO class = 3).
[0031] Fig.11 Examples of ALF filter shapes are shown (chroma: 5×5 diamond, luma: 7×7 diamond).
[0032] Fig. 12A An example of a sub-sampled Laplacian calculation is shown.
[0033] Fig. 12B Another example of a sub-sampled Laplacian calculation is shown.
[0034] Fig. 12C Another example of a sub-sampled Laplacian calculation is shown.
[0035] Fig.12D Yet another example of a sub-sampled Laplacian calculation is shown.
[0036] Fig.13The filter line buffer requirements for the luma component in VTM-4.0 are shown.
[0037] Fig.14 The filter line buffer requirements for chroma components in VTM-4.0 are shown.
[0038] Fig.15 An example of modified block classification at a virtual boundary is shown.
[0039] Fig.16 An example of modified ALF filtering of the luma component at a virtual boundary is shown.
[0040] Fig.17A An example of modified luma ALF filtering at a virtual boundary is shown.
[0041] Fig. 17B Another example of modified luma ALF filtering at a virtual boundary is shown.
[0042] Fig. 17C Yet another example of modified luma ALF filtering at a virtual boundary is shown.
[0043] Fig.18 An example of repetitive padding for luma ALF filtering at picture / sub-picture / slice / slice boundaries is shown.
[0044] Fig.19A An example of placement of the CC-ALF relative to other loop filters is shown.
[0045] Fig.19B An example of a 4x6 diamond filter is shown.
[0046] Fig. 20 An example of a 3x4 diamond filter is shown.
[0047] Fig.21 An example of filling at the ALF virtual boundary for the CC-ALF of the chroma sample located at (x, y) is shown, and C2 represents the corresponding luma sample, and the white sample is the sample to be filled.
[0048] Fig. 22 is a block diagram illustrating an example video processing system in which the various techniques disclosed herein may be implemented.
[0049] Fig.23 is a block diagram of a video processing device according to the present technology.
[0050] Fig.24 is a flow chart of an example method of video processing.
[0051] Fig.25is a block diagram illustrating a video encoding and decoding system according to some embodiments of the present disclosure.
[0052] Fig.26 is a block diagram illustrating an encoder according to some embodiments of the present disclosure.
[0053] Fig. 27 is a block diagram illustrating a decoder according to some embodiments of the present disclosure.
[0054] Fig.28 is a flow chart representation of a video processing method according to the present technology.
[0055] Fig.29 is a flowchart representation of another video processing method according to the present technology.
[0056] Fig.30 is a flowchart representation of another video processing method according to the present technology.
[0057] Fig.31 is a flowchart representation of another video processing method according to the present technology.
[0058] Fig.32 is a flowchart representation of another video processing method according to the present technology.
[0059] Fig.33 is a flowchart representation of yet another video processing method according to the present technology. DETAILED DESCRIPTION
[0060] The section headings used in this document are intended to facilitate understanding, rather than to limit the applicability of the techniques and embodiments disclosed in each section to that section. In addition, the use of H.266 terminology in some descriptions is intended only to facilitate understanding, rather than to limit the scope of the disclosed techniques. Therefore, the techniques described herein are also applicable to other video codec protocols and designs.
[0061] 1. Overview
[0062] This document relates to video codec technology. Specifically, it relates to cross-component adaptive loop filter (CC-ALF) and other codec tools applied to chroma components. The technology can be applied to existing video codec standards (such as HEVC), or to standards that will be finalized (Universal Video Codec). The technology can also be applied to future video codec standards or video codecs.
[0063] 2. Preliminary Discussion
[0064] Video codec standards have evolved primarily through the well-known ITU-T and ISO / IEC standards. ITU-T developed H.261 and H.263, ISO / IEC developed MPEG-1 and MPEG-4 video, and the two organizations worked together to develop the H.262 / MPEG-2 video standard and the H.264 / MPEG-4 Advanced Video Coding (AVC) and H.265 / HEVC standards. Since H.262, video codec standards have been based on a hybrid video codec structure that uses temporal prediction plus transform codecs. To explore future video codec technologies beyond HEVC, VCEG and MPEG jointly established the Joint Video Exploration Team (JVET) in 2015. Since then, JVET has adopted many new methods and put them into a reference software called the Joint Exploration Model (JEM). In April 2018, the Joint Video Expert Team (JVET) between VCEG (Q6 / 16) and ISO / IEC JTC1 SC29 / WG11 (MPEG) was established to work on the VVC standard, with the goal of reducing the bit rate by 50% compared to HEVC.
[0065] 2.1. Color Space and Chroma Subsampling
[0066] A color space, also called a color model (or color system), is an abstract mathematical model that simply describes the range of colors as a tuple of numbers, usually 3 or 4 values or color components (such as RGB). Fundamentally, a color space is a refinement of a coordinate system and subspace.
[0067] For video compression, the most commonly used color spaces are YCbCr and RGB.
[0068] YCbCr, Y'CbCr or Y Pb / Cb Pr / Cr, also written as YCBCR or Y'CBCR, is a series of color spaces used as part of the color image pipeline in video and digital photography systems. Y' is the luminance component, CB and CR are the blue difference and red difference chrominance components. Y' (with a prime) is distinguished from Y (Y is luminance), which means that the light intensity is non-linearly encoded based on the gamma-corrected RGB primaries.
[0069] Chroma subsampling is the practice of encoding an image with lower precision for chroma information than for luminance information by taking advantage of the fact that the human visual system is less sensitive to color differences than to luminance. 2.1.1. 4:4:4
[0071] Each of the three Y'CbCr components has the same sample rate, so there is no chroma subsampling. This scheme is sometimes used in high-end film scanners and film post-production. 2.1.2. 4:2:2
[0073] The two chroma components are sampled at half the luma sampling rate: horizontal chroma accuracy is halved, while vertical chroma accuracy remains unchanged. This reduces the bandwidth of the uncompressed video signal by one third, but there is little visual difference. Figure 1 Examples of nominal vertical and horizontal positions for a 4:2:2 color format are depicted in . 2.1.3. 4:2:0
[0075] Compared to 4:1:1, the horizontal sampling in 4:2:0 is doubled, but the vertical accuracy is halved because the Cb and Cr channels are sampled only on every alternate lines in this scheme. Therefore the data rate is the same. Cb and Cr are subsampled by a factor of 2 in the horizontal and vertical directions respectively. There are three variants of the 4:2:0 scheme with different horizontal and vertical positioning.
[0076] In MPEG-2, Cb and Cr are co-located horizontally. Cb and Cr are located between pixels in the vertical direction (in the gap).
[0077] In JPEG / JFIF, H.261, and MPEG-1, Cb and Cr are located in the gaps between alternate luminance samples.
[0078] In 4:2:0, DV, Cb, and Cr are co-located horizontally. In the vertical direction, they are co-located on alternate lines.
[0079] Table 1: SubWidthC and SubHeightC values derived from chroma_format_idc and separate_colour_plane_flag
[0080]
[0081]
[0082] 2.2. Encoding and decoding process of typical video codecs
[0083] Figure 2An example of an encoder block diagram for VVC is shown, which contains three loop filter blocks: deblocking filter (DF), sample adaptive offset (SAO), and ALF. Unlike DF, which uses a predefined filter, SAO and ALF use the original samples of the current picture, the encoded and decoded side information of the signaling offset and filter coefficients, and reduce the mean square error between the original and reconstructed samples by adding an offset and by applying a finite impulse response (FIR) filter, respectively. ALF is located at the last processing stage of each picture and can be seen as a tool that tries to capture and repair artifacts created by previous stages.
[0084] 2.3. Definition of video unit
[0085] A picture is divided into one or more slice rows and one or more slice columns. A slice is a series of CTUs covering a rectangular area of a picture. The CTUs in a slice are scanned in raster scan order within the slice.
[0086] A slice consists of an integer number of complete slices or an integer number of consecutive complete CTU rows within a slice of a picture.
[0087] Two stripe modes are supported, namely raster scan stripe mode and rectangular stripe mode. In raster scan stripe mode, a stripe contains a complete sequence of slices in a slice raster scan of a picture. In rectangular stripe mode, a stripe contains multiple complete slices that together form a rectangular area of a picture, or multiple consecutive complete CTU rows that together form a slice of a rectangular area of a picture. The slices within a rectangular stripe are scanned in a slice raster scan order within the rectangular area corresponding to the stripe.
[0088] A sub-picture consists of one or more strips that together cover a rectangular area of the picture.
[0089] Figure 3 An example of raster scan striping partitioning of a picture is shown, where the picture is divided into 12 tiles and 3 raster scan strips.
[0090] Figure 4 An example from the VVC specification is shown, which shows an example of rectangular slice partitioning of a picture, where the picture is divided into 24 slices (6 slice columns and 4 slice rows) and 9 rectangular slices.
[0091] Figure 5 An example of a picture partitioned into slices and rectangular strips is shown, where the picture is divided into 4 slices (2 slice columns and 2 slice rows) and 4 rectangular strips.
[0092] Figure 6An example of sub-picture partitioning of a picture is shown, where the picture is partitioned into 15 slices, 24 slices, and 24 sub-pictures of different dimensions covering a 4×4 CTU.
[0093] 2.3.1.CTU / CTB size
[0094] In VVC, the CTU size signaled in SPS by the syntax element log2_ctu_size_minus2 can be as small as 4x4.
[0095] 7.3.2.3 Sequence parameter set RBSP syntax
[0096]
[0097]
[0098]
[0099] log2_ctu_size_minus2 plus 2 specifies the luma codec tree block size for each CTU.
[0100] log2_min_luma_coding_block_size_minus2 plus 2 specifies the minimum luma coding block size.
[0101] The variables CtbLog2SizeY, CtbSizeY, MinCbLog2SizeY, MinCbSizeY, MinTbLog2SizeY, MaxTbLog2SizeY, MinTbSizeY, MaxTbSizeY, PicWidthInCtbsY, PicHeightInCtbsY, PicSizeInCtbsY, PicWidthInMinCbsY, PicHeightInMinCbsY, PicSizeInMinCbsY, PicSizeInSamplesY, PicWidthInSamplesC, and PicHeightInSamplesC are derived as follows:
[0102] CtbLog2SizeY=log2_ctu_size_minus2+2 (7-9)
[0103] CtbSizeY=1< <CtbLog2SizeY (7-10)
[0104] MinCbLog2SizeY=log2_min_luma_coding_block_size_minus2+2 (7-11)
[0105] MinCbSizeY=1<<MinCbLog2SizeY (7-12)
[0106] MinTbLog2SizeY=2 (7-13)
[0107] MaxTbLog2SizeY=6 (7-14)
[0108] MinTbSizeY=1<<MinTbLog2SizeY (7-15)
[0109] MaxTbSizeY=1<<MaxTbLog2SizeY (7-16)
[0110] PicWidthInCtbsY=Ceil(pic_width_in_luma_samples÷CtbSizeY) (7-17)
[0111] PicHeightInCtbsY=Ceil(pic_height_in_luma_samples÷CtbSizeY) (7-18)
[0112] PicSizeInCtbsY=PicWidthInCtbsY*PicHeightInCtbsY (7-19)
[0113] PicWidthInMinCbsY=pic_width_in_luma_samples / MinCbSizeY (7-20)
[0114] PicHeightInMinCbsY=pic_height_in_luma_samples / MinCbSizeY (7-21)
[0115] PicSizeInMinCbsY=PicWidthInMinCbsY*PicHeightInMinCbsY (7-22)
[0116] PicSizeInSamplesY=pic_width_in_luma_samples*pic_height_in_luma_samples (7-23)
[0117] PicWidthInSamplesC=pic_width_in_luma_samples / SubWidthC (7-24)
[0118] PicHeightInSamplesC = pic_height_in_luma_samples / SubHeightC (7 - 25)
[0119] 2.3.2. CTUs in the Picture
[0120] Assume that the CTB / LCU size is represented by M×N (typically, M equals N as defined in HEVC / VVC), and for a CTB located at the border of a picture (or slice or strip or other type, taking the picture border as an example), K×L samples are within the picture border, where K < M or L < N. For Figure 7A-7C those CTBs depicted in
[0121] Fig. 7A Figure shows CTBs across the bottom picture border.
[0122] Figure 7B Figure shows CTBs across the right picture border.
[0123] Figure 7C Figure shows CTBs across the bottom - right picture border.
[0124] 2.4. Deblocking Filter (DB)
[0125] The input to the DB is the reconstructed samples before the loop filter.
[0126] First, filter the vertical edges in the picture. Then, use the samples modified by the vertical - edge filtering as the input to filter the horizontal edges in the picture. The vertical and horizontal edges in the CTB of each CTU are processed separately on the basis of the coding - decoding unit. The vertical edges of the coding - decoding blocks in the coding - decoding unit start filtering from the edge on the left side of the coding - decoding block and advance through the edges in their geometric order towards the right side of the coding - decoding block. The horizontal edges of the coding - decoding blocks in the coding - decoding unit start filtering from the edge on the top of the coding - decoding block and advance through the edges in their geometric order towards the bottom of the coding - decoding block.
[0127] Figure 8 Figure is a schematic diagram of picture samples on an 8×8 grid, as well as horizontal and vertical block boundaries, and non - overlapping blocks of 8×8 samples that can be de - blocked in parallel.
[0128] 2.4.1. Boundary Decision
[0129] Apply the filter to the 8×8 block boundaries. Additionally, it must be a transform - block boundary or a coding - decoding sub - block boundary (e.g., due to the use of affine motion prediction, ATMVP). For those boundaries that are not such, disable the filter.
[0130] 2.4.2. Boundary strength calculation
[0131] For transform block boundaries / codec subblock boundaries, if they are located in an 8×8 grid, they can be filtered and the bS[xD i ][yD j ](where [xD i ][yD j ] represents coordinates) are defined in Table 2 and Table 3 respectively.
[0132] Table 2: Boundary Strength (When SPS IBC is disabled)
[0133]
[0134] Table 3: Boundary Strength (when SPS IBC is enabled)
[0135]
[0136] 2.4.3. Deblocking decision for luminance component
[0137] The deblocking decision process is described in this subsection.
[0138] Fig. 9 Examples of pixels involved in filter on / off decisions and strong / weak filter selection are shown.
[0139] The wider-stronger brightness filter is a filter that is used only when all three conditions are met.
[0140] 2.4.4. Stronger Luma Deblocking Filter (Designed for Larger Blocks)
[0141] When samples on either side of the boundary belong to a large block, a bilinear filter is used. Samples belonging to a large block are defined as when the width of a vertical edge is >= 32 and the height of a horizontal edge is >= 32.
[0142] The bilinear filters are listed below.
[0143] In the above HEVC deblocking, the block boundary samples p from i=0 to Sp-1 are i and block boundary samples q from j = 0 to Sq-1 i (p i and q i is the i-th sample point in the row for filtering vertical edges, or the i-th sample point in the column for filtering horizontal edges) is replaced by linear interpolation as follows:
[0144] -p i ′=(f i*Middle s,t +(64-f i )*P s +32)>>6), crop to p i ±tcPD i
[0145] -q j ′=(g j *Middle s,t +(64-g j )*Q s +32)>>6), trim to q j ±tcPD j
[0146] Among them, the term tcPD i and tcPD j is the position-dependent clipping described in Section 0, and is given by j 、f i 、Middle s,t , P s and Q s .
[0147] 2.4.5. Chroma Deblocking Control
[0148] A chroma strong filter is used on both sides of the block boundary. Here, the chroma filter is selected when both sides of the chroma edge are greater than or equal to 8 (chroma position), and the decision meets the following three conditions: The first is the decision of boundary strength and large blocks. The proposed filter can be applied when the block width or height orthogonally across the block edge in the chroma sampling domain is equal to or greater than 8. The second and third are basically the same as the HEVC luma deblocking decision, which is an on / off decision and a strong filter decision, respectively.
[0149] In the first decision, the boundary strength (bS) is modified for chroma filtering and the conditions are checked sequentially. If one condition is met, the remaining conditions with lower priority are skipped.
[0150] Chroma deblocking is performed when bS is equal to 2, or bS is equal to 1 when a large block boundary is detected.
[0151] The second and third conditions are essentially the same as the HEVC luma strong filter decision, as shown below.
[0152] In the second condition:
[0153] Then d is derived as in HEVC luma deblocking.
[0154] When d is less than β, the second condition is true.
[0155] Under the third condition, the derivation of StrongFilterCondition is as follows:
[0156] dpq is derived as in HEVC.
[0157] sp3 = Abs(p3-p0), as derived in HEVC
[0158] sq3 = Abs(q0-q3), as derived in HEVC
[0159] As in the HEVC design, StrongFilterCondition = (dpq is less than (β>>2), sp3+sq3 is less than (β>>3), and Abs(p0-q0) is less than (5*t C +1)>>1)
[0160] 2.4.6. Strong Chroma Deblocking Filter
[0161] The following strong chroma deblocking filters are defined:
[0162] p2′=(3*p3+2*p2+p1+p0+q0+4)>>3
[0163] p1′=(2*p3+p2+2*p1+p0+q0+q1+4)>>3
[0164] p0′=(p3+p2+p1+2*p0+q0+q1+q2+4)>>3
[0165] The proposed chroma filter performs deblocking on a 4×4 chroma sample grid.
[0166] 2.4.7. Position-dependent clipping
[0167] Position dependent clipping tcPD is applied to the output samples of the luminance filtering process which involves modifying the strong and long filters of 7, 5 and 3 samples at the boundaries. Assuming a quantization error distribution, it is proposed to increase the clipping value for samples expected to have higher quantization noise, so that the reconstructed sample values are expected to deviate more from the true sample values.
[0168] For each P or Q boundary filtered using an asymmetric filter, a position-dependent threshold table is selected from the two tables provided to the decoder as side information (i.e., Tc7 and Tc3 in the table below), depending on the result of the decision processing in the section 0.
[0169] 2.4.8. Sub-block deblocking adjustment
[0170] To enable parallel friendly deblocking using long filters and subblock deblocking, the long filter is restricted to modifying at most 5 samples on one side using subblock deblocking (AFFINE or ATMVP or DMVR) (as shown in the luma control of the long filter). In addition, subblock deblocking is adjusted so that subblock boundaries on the 8×8 grid close to CU or implicit TU boundaries are restricted to modifying at most two samples on each side.
[0171] 2.5. Sample Adaptive Offset (SAO)
[0172] The input of SAO is the reconstructed samples after DB. The concept of SAO is to reduce the average sample distortion of the region by first classifying the region samples into multiple categories with the selected classifier, obtaining the offset of each category, and then adding the offset to each sample of the category, where the classifier index and the offset of the region are encoded in the bitstream. In HEVC and VVC, the region (the unit of SAO parameter signaling) is defined as CTU.
[0173] Two SAO types are used in HEVC to meet the low complexity requirements. These two types are edge offset (EO) and band offset (BO), which will be discussed in further detail below. The index of the SAO type is encoded and decoded (in the range [0,2]). For EO, the sample classification is based on the comparison between the current sample and the adjacent samples according to the one-dimensional direction mode: horizontal, vertical, 135° diagonal and 45° diagonal.
[0174] Fig.10 Examples of four one-dimensional directional patterns for EO point classification are shown: horizontal (EO class = 0), vertical (EO class = 1), 135° diagonal (EO class = 2), and 45° diagonal (EO class = 3).
[0175] For a given EO category, each sample point inside the CTB is classified into one of five categories. The current sample point value labeled "c" is compared with its two neighboring samples (labeled "a" and "b") along the selected 1D pattern. The classification rules for each sample point are summarized in Table 4: Sample point classification rules for edge offset. Category 1 and category 4 are associated with local valleys and local peaks along the selected 1D pattern, respectively. Category 2 and category 3 are associated with concave corners and convex corners along the selected 1D pattern, respectively. If the current sample point does not belong to EO category 1-4, it is category 0 and SAO is not applicable.
[0176] Table 4: Sample point classification rules for edge offset
[0177] category condition 1 c < a and c < b 2 (c<a&&c==b)||(c==a&&c<b) 3 (c>a&&c==b)||(c==a&&c>b) 4 c>a&&c>b 5 None of the above
[0178] 2.6. Adaptive Loop Filter (ALF)
[0179] In VVC, an adaptive loop filter (ALF) with block-based filter adaptation is applied. For the luminance component, one of 25 filters is selected for each 4×4 block based on the direction value and activity value of the local gradient.
[0180] Filter shape
[0181] Two diamond filter shapes are used (e.g. Fig.11 ). The 7×7 diamond is applied to the luma component and the 5×5 diamond is applied to the chroma components.
[0182] 2.6.2. Block classification
[0183] For the luma component, each 4×4 block is classified into one of 25 classes. The class index C is based on its direction value D and activity value The quantized value of is derived as follows:
[0184]
[0185] In order to calculate D and First, use the one-dimensional Laplacian operator to calculate the gradient in the horizontal, vertical and two diagonal directions:
[0186]
[0187]
[0188]
[0189]
[0190]
[0191] Where the indices i and j refer to the coordinates of the upper left sample point in the 4×4 block, and R(i,j) indicates the reconstructed sample point at the coordinate (i,j).
[0192] In order to reduce the complexity of block classification, a sub-sampled one-dimensional Laplacian operator is used for calculation. Figures 12A-12D As shown, the same subsampling positions are used for gradient calculations in all directions.
[0193] Figures 12A-12D An example of a sub-sampled Laplacian calculation is shown. Fig. 12A The sub-sampling positions of the vertical gradient are shown. Fig. 12B The sub-sampling positions of the horizontal gradient are shown. Fig. 12C The sub-sampling positions of the diagonal gradients are shown. Fig.12DThe sub-sampling positions of the diagonal gradients are shown.
[0194] Then the maximum and minimum values of the horizontal and vertical gradients D are set to:
[0195]
[0196] The maximum and minimum values of the gradients in the two diagonal directions are set to:
[0197]
[0198] To derive the value of the direction value D, these values are compared with each other and with two thresholds t1 and t2:
[0199] Step 1. If and If both are true, set D to 0.
[0200] Step 2. If Continue from step 3; otherwise continue from step 4.
[0201] Step 3. If Then set D to 2; otherwise set D to 1.
[0202] Step 4. If Then set D to 4; otherwise set D to 3.
[0203] The activity value A is calculated as follows:
[0204]
[0205] A is further quantized to the range of 0 to 4 (inclusive), and the quantized value is represented as
[0206] For the chroma components in a picture, no classification method is applied, ie, a single set of ALF coefficients is applied for each chroma component.
[0207] 2.6.3. Geometric transformation of filter coefficients and clipping values
[0208] Before filtering each 4×4 luma block, geometric transformations (such as rotations or diagonal and vertical flips) are applied to the filter coefficients f(k, l) and the corresponding filter clipping values c(k, l) according to the gradient values calculated for the block. This is equivalent to applying these transformations to the samples in the filter support region. The idea is to make different blocks to which the ALF is applied more similar by aligning the directionality of different blocks.
[0209] Three geometric transformations are introduced, including diagonal, vertical flip, and rotation:
[0210] Diagonal: f D (k, l) = f(l, k), c D (k, l) = c(l, k), (2-9)
[0211] Flip vertically: f V (k, l) = f(k, Kl-1), c V (k, l) = c (k, K = l-1) (2-10)
[0212] Rotation: f R (k, l) = f(Kl-1, k), c R (k, l) = c (K = l-1, k) (2-11)
[0213] Where K is the size of the filter, 0≤k, l≤K-1 are the coefficient coordinates, so that position (0, 0) is in the upper left corner and position (K-1, K-1) is in the lower right corner. The transformation is applied to the filter coefficients f(l, k) and the clipping value c(k, l) according to the gradient value calculated for the block. The following table summarizes the relationship between the transformation and the four gradients in the four directions.
[0214] Table 5: Mapping of gradients and transformations computed for a block
[0215] Gradient Value Transform <![CDATA[g d2 <g d1 And g h <g v ]]> No transformation <![CDATA[g d2 <g d1 And g v <g h ]]> diagonal <![CDATA[g d1 <g d2 And g h <g v ]]> Flip Vertically <![CDATA[g d1 <g d2 And g v <g h ]]> Rotation
[0216] 2.6.4. Filter parameter signaling
[0217] The ALF filter parameters are signaled in the Adaptation Parameter Set (APS). In one APS, up to 25 groups of luma filter coefficients and clipping value indices and up to 8 groups of chroma filter coefficients and clipping value indices can be signaled. To reduce bit overhead, filter coefficients of different classifications of luma components can be merged. In the slice header, the index of the APS for the current slice is signaled.
[0218] The clipping value index decoded from the APS allows the determination of the clipping value using the clipping value tables for both luma and chroma components. These clipping values depend on the internal bit depth. More precisely, the clipping value is obtained by the following formula:
[0219] AlfClip = {round(2 B-α*n ) for n∈[0..N-1]} (2-12)
[0220] Where B is equal to the internal bit depth, α is a predefined constant value equal to 2.35, and N is equal to 4, which is the number of cropping values allowed in VVC.
[0221] In the slice header, up to 7 APS indices can be signaled to specify the luma filter bank for the current slice. The filtering process can be further controlled at the CTB level. A flag is always signaled to indicate whether the ALF is applied to the luma CTB. The luma CTB can select a filter bank from 16 fixed filter banks and the filter bank from the APS. The filter bank index is signaled for the luma CTB to indicate which filter bank is applied. The 16 fixed filter banks are predefined and hard-coded in both the encoder and decoder.
[0222] For chroma components, the APS index is signaled in the slice header to indicate the chroma filter set being used for the current slice. At the CTB level, if there is more than one chroma filter set in the APS, the filter index is signaled for each chroma CTB.
[0223] The filter coefficients are quantized with a norm equal to 128. To limit the multiplication complexity, bitstream consistency is applied so that coefficient values in non-central positions should be within -2 7 to 2 7 The center position coefficient is not signaled in the bitstream and is considered to be equal to 128.
[0224] 2.6.5. Filtering
[0225] At the decoder side, when ALF is enabled for CTB, each sample R(i, j) within the CU is filtered to produce the sample value R′(i, j) as shown below,
[0226] R′(i, j)=R(i, j)+((∑ k≠0 ∑ l≠0 f(k,l)×K(R(i+k,j+l)-R(i,j),c(k,l))+64)>>7) (2-13)
[0227] Where f(k, l) represents the decoded filter coefficients, K(x, y) is the clipping function, and c(k, l) represents the decoded clipping parameters. The variables k and 1 are and where L represents the filter length. The clipping function K(x, y)=min(y, max(-y, x)) corresponding to the function Clip3(-y, y, x)
[0228] 2.6.6. Virtual Boundary Filtering for Line Buffer Reduction
[0229] In hardware and embedded software, picture-based processing is actually unacceptable due to its high requirements for picture buffers. Using on-chip picture buffers is very expensive, and using off-chip picture buffers will significantly increase external memory access, power consumption, and data access latency. Therefore, in actual products, DF, SAO, and ALF will change from picture-based decoding to LCU-based decoding. When LCU-based processing is used for DF, SAO, and ALF, for parallel processing of multiple LCUs, the entire decoding process can be completed in a LCU pipeline manner LCU by LCU in raster scanning. In this case, DF, SAO, and ALF require line buffers because processing one LCU row requires pixels from the above LCU rows. If an off-chip line buffer (such as DRAM) is used, the external memory bandwidth and power consumption will be increased; if an on-chip line buffer (such as SRAM) is used, the chip area will increase. Therefore, although the line buffer is already much smaller than the picture buffer, it is still desirable to reduce the line buffer.
[0230] In VTM-4.0, if Fig.13 As shown, the total number of line buffers required for the luma component is 11.25 lines. The explanation for the line buffer requirement is as follows: Since the decision and filtering require lines K, L, M, M from the first CTU and lines O, P from the bottom CTU, deblocking of horizontal edges overlapping the CTU edges cannot be performed. Therefore, deblocking of horizontal edges overlapping the CTU boundaries is postponed until the lower CTU arrives. Therefore, the reconstructed luma samples for lines K, L, M, N must be stored in the line buffer (4 lines). Then, SAO filtering can be performed on lines A to J. Since deblocking does not change the samples in line K, SAO filtering can be performed on line J. For SAO filtering of line K, the edge offset classification decision is only stored in the line buffer (0.25 luma lines). ALF filtering is performed only on line AF. As shown Fig.13 As shown, ALF classification is performed for each 4×4 block. Each 4×4 block classification requires an active window of size 8×8, which in turn requires a 9×9 window to compute a one-dimensional Laplacian to determine the gradient.
[0231] Therefore, for block classification of 4×4 blocks overlapping with rows G, H, I, J, the SAO filtered samples below the virtual boundary are required. In addition, the SAO filtered samples of rows D, E, F are necessary for ALF classification. Furthermore, ALF filtering of row G requires the three SAO filtered rows D, E, F from the above rows. Therefore, the total row buffer requirements are as follows:
[0232] – Row KN (horizontal DF pixels): 4 rows
[0233] – Row DJ (SAO filtered pixels): 7 rows
[0234] – SAO edge offset classifier value between row J and row K: 0.25 rows
[0235] Therefore, the total number of luminance lines required is 7+4+0.25=11.25.
[0236] Similarly, the line buffer requirements for the chrominance components are as follows Fig.14 The line buffer requirement for the chrominance components is estimated to be 6.25 lines.
[0237] Fig.13 The filter line buffer requirements for the luma component in VTM-4.0 are shown.
[0238] Fig.14 The filter line buffer requirements for chroma components in VTM-4.0 are shown.
[0239] In order to eliminate the line buffer requirements of SAO and ALF, the concept of virtual boundary (VB) is introduced in the latest VVC to reduce the line buffer requirements of ALF. Modified block classification and filtering are used for samples near the horizontal CTU boundary. Fig.13 As shown in FIG. 1 , VB is the horizontal LCU boundary shifted upward by N pixels. For each LCU, SAO and ALF can process pixels above VB before the lower LCU arrives, but cannot process pixels below VB until the lower LCU arrives, which is caused by DF. Considering the hardware implementation cost, the space between the proposed VB and the horizontal LCU boundary is set to four pixels of the luminance component (i.e., Fig.13 or Fig.15 , N=4) and two pixels of chrominance components (ie, N=2).
[0240] Fig.15 An example of modified block classification at a virtual boundary is shown.
[0241] like Fig.16 As shown, the modified block classification is applied to the luma component. For the one-dimensional Laplacian gradient calculation of the 4×4 block above the virtual boundary, only the samples above the virtual boundary are used. Similarly, for the one-dimensional Laplacian gradient calculation of the 4×4 block below the virtual boundary, only the samples below the virtual boundary are used. By considering the reduced number of samples used in the one-dimensional Laplacian gradient calculation, the quantization of the activity value A is scaled accordingly.
[0242] For the filtering process, a mirrored (symmetric) padding operation at the virtual boundary is used for both luma and chroma components. Fig.16 As shown in FIG. 1 , when the filtered sample point is located below the virtual boundary, the adjacent sample points located above the virtual boundary are filled in. At the same time, the corresponding sample points on the other side are filled in symmetrically.
[0243] For another example, if a sample point at (i, j) (e.g., Fig. 17B The dashed line P0A in the middle is filled, then the corresponding sample point at (m, n) (for example, Fig. 17B P3B in the dashed line, which share the same filter coefficients) is also filled even if the sample point is available (such as Figures 17A-17C shown).
[0244] Fig.17A -C shows an example of modified luma ALF filtering at a virtual boundary. Fig.17A shows a required line (on each side) of fill above / below VB, Fig. 17B shows the 2 required lines (on each side) that need to be filled above / below VB, Fig. 17C Shown are the 3 required lines (on each side) that need to be filled above / below VB.
[0245] Unlike the mirrored (symmetric) padding method used at horizontal CTU boundaries, repeated (one-sided) padding is applied to slice, slice, and sub-picture boundaries when filters across boundaries are disabled. Repeated (one-sided) padding is also applied to picture boundaries. Padding samples are used for classification and filtering. Fig.18 An example of a repetitive padding method for luma ALF filtering at picture / sub-picture / slice / slice boundaries is depicted.
[0246] Fig.18 An example of repetitive padding for luma ALF filtering at picture / sub-picture / slice / slice boundaries is shown.
[0247] 2.7. ALF VB with modified filter strength
[0248] The ALF virtual boundary processing avoids increasing the size of the adaptive loop filter's line buffer. The contribution claims that in some cases the processing can introduce visual artifacts due to the use of extreme padding for the nearest lines on each side of a virtual horizontal CTU boundary without compensating for this in the filter strength. To address this issue, the filter strength is reduced by increasing the right shift of the ALF filtering for lines adjacent to the virtual horizontal CTU boundary.
[0249] More specifically, when filtering rows adjacent to the ALF virtual boundary, the filter strength for both luma and chroma is reduced. Deletions are marked with [[ ]] and the corresponding normative additions are highlighted in bold italics and underlined as follows:
[0250] 8.8.5.2 Codec Tree Block Filtering for Luma Samples
[0251] The input to this process is:
[0252] – the reconstructed luminance picture sample array recPicture before adaptive loop filtering,
[0253] –Filtered reconstructed luminance picture sample array alfPicture L ,
[0254] – Luma position (xCtb, yCtb), specifies the top left sample point of the current luma codec treeblock relative to the top left sample point of the current picture.
[0255] The output of this process is the modified filtered reconstructed luminance picture sample array alfPicture L .
[0256] Call the derivation process of filter index clause 8.8.5.3, using the position (xCtb, yCtb) and the reconstructed luminance picture sample array recPicture as input, and using filtIdx[x][y] and transposeIdx[x][y] as output, where x, y = 0..CtbSizeY-1.
[0257] To derive the filtered reconstructed luminance sample alfPicture L [x][y], each reconstructed luma sample inside the current luma codec tree block recPicture[x][y] is filtered as follows, where x, y = 0..CtbSizeY-1:
[0258] – The array of luma filter coefficients f[j] and the array of luma clipping values c[j] corresponding to the filter specified by filtIdx[x][y], where j = 0..11, are derived as follows:
[0259] …
[0260] –The variable sum is derived as follows:
[0261]
[0262]
[0263] ....
[0265] Table 43 – y1, y2, y3 and y2 according to the vertical luma sample position y and applyAlfLineBufBoundary Specifications
[0266]
[0267]
[0268] 8.8.5.4 Codec Tree Block Filtering for Chroma Samples
[0269] The input to this process is:
[0270] – reconstructed chroma picture sample array recPicture before adaptive loop filtering,
[0271] – the filtered reconstructed chrominance picture sample array alfPicture,
[0272] – Chroma position (xCtbC, yCtbC), specifies the top left sample point of the current chroma codec tree block relative to the top left sample point of the current picture,
[0273] – Alternative chroma filter index altIdx.
[0274] The output of this process is the modified filtered reconstructed chrominance picture sample array alfPicture.
[0275] The width and height of the current chroma codec tree block ctbWidthC and ctbHeightC are derived as follows: ....
[0277] To derive the filtered reconstructed chroma sample alfPicture[x][y], each reconstructed chroma sample inside the current chroma codec treeblock recPicture[x][y] is filtered as follows, where x=0..ctbWidthC-1 and y=0..ctbHeightC-1:
[0278] – the position (h) of each of the corresponding chroma samples (x, y) within a given array of chroma samples recPicture x+i ,v y+j ) is derived as follows, where i, j = -2..2:
[0279] …
[0280] –The variable sum is derived as follows:
[0281]
[0282]
[0283]
[0284] Table 44 – y1, y2 and y3 according to the vertical chroma sample position y and applyAlfLineBufBoundary specification
[0285]
[0286] 2.8. Cross-component adaptive loop filter
[0287] Fig.19A Figure 2 shows the placement of the CC-ALF relative to the other loop filters. The CC-ALF operates by applying a linear diamond filter ( Fig.19B ) to operate, the chrominance component is expressed as
[0288]
[0289] in
[0290] (x,y) is the position of the chroma component i being refined
[0291] (x C ,y C ) is the brightness position based on (x,y)
[0292] S i is the filter support in luma for chroma component i
[0293] c i (x0,y0) represents the filter coefficient (2-14)
[0294] Fig.19A An example placement of the CC-ALF relative to other loop filters is shown. Fig.19B An example of a 4x6 diamond filter is shown.
[0295] The support region is centered around the brightness position (x C ,y C ) is calculated based on the spatial scaling factor between the luma and chroma planes. All filter coefficients are sent in the APS and have an 8-bit dynamic range. The APS can be referenced in the slice header. The CC-ALF coefficients for each chroma component of the slice are also stored in the buffer corresponding to the temporal sublayer. The use of slice level flags facilitates the reuse of these groups of temporal sublayer filter coefficients. The application of the CC-ALF filter is controlled on variable block sizes (i.e. 16×16, 32×32, 64×64, 128×128) and is signaled by a context codec flag received for each block of samples. For each chroma component, the block size is received at the slice level along with the CC-ALF enable flag. The boundary filling of the horizontal virtual boundaries utilizes repetition. For the remaining boundaries, the same type of padding as conventional ALF is used.
[0296] It is assumed that the sum of all CC-ALF filter coefficients is equal to 0. Therefore, in some embodiments, the filtering process is modified as follows:
[0297]
[0298] in is the corresponding luminance sample of the chrominance sample to be filtered; and I0(x C +x0,y C +y0) are the surrounding luma samples around the corresponding luma sample in the CC-ALF filter support. At the same time, the filter support is modified to a 3×4 diamond filter, where the center one in the second row is marked as the corresponding luma sample.
[0299] Fig. 20 An example of a 3×4 diamond filter is shown.
[0300] 2.8.1. Filling method at virtual boundaries
[0301] Similar to luma ALF / chroma ALF, in some embodiments, repetition padding is utilized at the ALF virtual boundary of CC-ALF. Fig.21 As shown, if the luma samples above or below the ALF virtual boundary are not available, the nearest sample line is used for filling. Similar to the ALF design, mirror filling is applied.
[0302] In the VVC specification, the detailed filtering process is defined as follows:
[0303] 8.8.5.7 Cross-component filtering
[0304] –The derivation of the variable sum is as follows:
[0305]
[0306] scaledSum=Clip3(-(1<<(BitDepth-1)),(1<<(BitDepth-1))-1,(sum+64)>>7)(1533)
[0307] sum = curr + scaledSum (1534)
[0308] – The modified filtered reconstructed chrominance picture samples ccAlfPicture[xCtbC+x][yCtbC+y] are derived as follows:
[0309] ccAlfPicture[xCtbC+x][yCtbC+y]=Clip3(0,(1< <BitDepth)-1,sum) (1535)
[0311] Table 47 – yP1 and yP2 specifications based on vertical luma sample position (y*subHeightC) and applyAlfLineBufBoundary
[0312]
[0313] (a)y*subHeightC==CtbSizeY–6; (b)y*subHeightC==CtbSizeY–5;
[0314] (c)y*subHeightC==CtbSizeY–4; (d)y*subHeightC==CtbSizeY–3
[0315] Fig.21 An example of filling of CC-ALF at the ALF virtual boundary for a chroma sample located at (x, y) is shown, and C2 represents the corresponding luma sample, the white sample being the one to be filled.
[0316] 2.8.2. Specification changes related to ALF VB processing
[0317] 8.8.5.7 Cross-component filtering for chroma sample codec tree blocks
[0318] The input to this process is:
[0319] – Reconstructed luminance picture sample array recPicture before luminance adaptive loop filtering L ,
[0320] –Filtered reconstructed chrominance picture sample array alfPicture C ,
[0321] – Chroma position (xCtbC, yCtbC), specifies the top left sample point of the current chroma codec tree block relative to the top left sample point of the current picture,
[0322] – Luma position (xCtb, yCtb), specifies the top left sample point of the current luma codec tree block relative to the top left sample point of the current picture
[0323] – The width of the chroma sample block ccAlfWidth
[0324] – The height of the chroma sample block ccAlfHeight
[0325] – Cross-component filter coefficients CcAlfCoeff[j], where j = 0..6
[0326] The output of this process is a modified filtered reconstructed chroma picture sample array ccAlfPicture.
[0327] In order to derive the filtered reconstructed chroma sample ccAlfPicture[xCtbC+x][yCtbC+y], each reconstructed chroma sample alfPicture inside the current chroma block C [xCtbC+x][yCtbC+y] is filtered as follows, where x = 0..ccAlfWidth–1, y = 0..ccAlfHeight–1:
[0328] – The luma position (xL, yL) corresponding to the current chroma sample at chroma position (xCtbC+x, yCtbC+y) is set equal to ((xCtbC+x)*SubWidthC, (yCtbC+y)*SubHeightC)
[0329] – In the array recPicture L The internal brightness position (h x+i ,v y+j ) is derived as follows, where i = -1..1, j = -1..2:
[0330] h x+i =Clip3(0,pic_width_in_luma_samples-1,xL+i) (8-1231)
[0331] v y+j =Clip3(0,pic_height_in_luma_samples-1,yL+j) (8-1234)
[0332] – The variables clipLeftPos, clipRightPos, clipTopPos, clipBottomPos, clipTopLeftFlag and clipBotRightFlag are derived by calling the ALF boundary position derivation procedure as specified in clause 8.8.5.5, with (xCtbC*SubWidthC, yCtbC*SubHeightC) and (x*SubWidthC, y*SubHeightC) as input.
[0333] – Variable h x+i and v y+j is modified by invoking the ALF sample filling process as specified in clause 8.8.5.6, where (xCtb, yCtb), (h x+i ,v y+j), with the variables isChroma, clipLeftPos, clipRightPos, clipTopPos, clipBottomPos, clipTopLeftFlag, and clipBotRightFlag set equal to 0 as input.
[0334] –The derivation of the variable applyAlfLineBufBoundary is as follows:
[0335] – If the bottom boundary of the current codec treeblock is the bottom boundary of the current picture and pic_height_in_luma_samples-yCtb<=CtbSizeY-4, then applyAlfLineBufBoundary is set equal to 0:
[0336] – Otherwise, applyAlfLineBufBoundary is set equal to 1.
[0337] – The vertical sample position offsets yP1 and yP2 are specified in Table 44 according to the vertical luma sample position y*subHeightC and applyAlfLineBufBoundary.
[0338] –The variable curr is derived as follows:
[0339] curr=alfPicture C [xCtbC+x][yCtbC+y] (8-1286)
[0340] – The array of cross-component filter coefficients f[j] is derived as follows, where j = 0..6:
[0341] f[j]=CcAlfCoeff[j] (8-1287)
[0342] –The derivation of the variable sum is as follows:
[0343]
[0344]
[0345] scaledSum=Clip3(-(1<<(BitDepth-1)),(1<<(BitDepth-1))-1,(sum+64)>>7)(8-1289)
[0346] sum = curr + scaledSum (8-1290)
[0347] – The modified filtered reconstructed chroma picture sample array ccAlfPicture[xCtbC+x][yCtbC+y] is derived as follows:
[0348] ccAlfPicture[xCtbC+x][yCtbC+y]=Clip3(0,(1< <BitDepth)-1,sum) (8-1291)
[0349] Table 44 – yP1 and yP2 specifications based on vertical luma sample position y*subHeightC and applyAlfLineBufBoundary
[0350]
[0351] 2.8.3. Signaling of CC-ALF related syntax elements
[0352] The CC-ALF filters are signaled in the APS together with the ALF related syntax elements.
[0353] 7.3.2.19 Adaptive Loop Filter Data Syntax
[0354]
[0355]
[0356]
[0357] 7.3.2.7 Picture header structure syntax
[0358]
[0359]
[0360]
[0361] 7.3.7 Strip Header Syntax
[0362] 7.3.7.1 Generic Strip Header Syntax
[0363]
[0364]
[0365] 7.3.10.2 Codec Tree Unit Syntax
[0366]
[0367]
[0368] 2.9. Luminance Mapping with Chroma Scaling (LMCS)
[0369] The LMCS mode operates based on two domains, including a first domain which is an original domain and a second domain which is a reshaped domain that maps luma samples to specific values according to a reshaped model.
[0370] The relevant syntax elements and semantics in the picture header and slice header are described as follows:
[0371] 7.3.2.7 Picture header structure syntax
[0372]
[0373]
[0374] 7.3.7 Strip Header Syntax
[0375] 7.3.7.1 Generic Strip Header Syntax
[0376]
[0377] ph_lmcs_enabled_flag equal to 1 specifies that luma mapping with chroma scaling is enabled for all slices associated with the PH. ph_lmcs_enabled_flag equal to 0 specifies that luma mapping with chroma scaling is disabled for one, more than one, or all slices associated with the PH. When not present, the value of ph_lmcs_enabled_flag is inferred to be equal to 0.
[0378] ph_lmcs_aps_id specifies the adaptation_parameter_set_id of the LMCS APS referenced by the slice associated with the PH. The TemporalId of the APS NAL unit with aps_params_type equal to LMCS_APS and adaptation_parameter_set_id equal to ph_lmcs_aps_id shall be less than or equal to the TemporalId of the picture associated with the PH.
[0379] ph_chroma_residual_scale_flag equal to 1 specifies that chroma residual scaling is enabled for all slices associated with the PH. ph_chroma_residual_scale_flag equal to 0 specifies that chroma residual scaling may be disabled for one, multiple, or all slices associated with the PH. When ph_chroma_residual_scale_flag is not present, it is inferred to be equal to 0.
[0380] slice_lmcs_enabled_flag equal to 1 specifies that luma mapping with chroma scaling is enabled for the current slice. slice_lmcs_enabled_flag equal to 0 specifies that luma mapping with chroma scaling is not enabled for the current slice. When slice_lmcs_enabled_flag is not present, it is inferred to be equal to 0.
[0381] 3. Examples of technical problems solved by the disclosed technical solution
[0382] The current design of CC-ALF has the following problems:
[0383] 1. In ALF processing, the shift representing the filter strength is adjusted based on the position of the sample to be filtered to resolve artifacts due to mirror padding. Similarly, CC-ALF may have visual quality issues at ALF virtual boundaries (VBs) because there is no adjustment for the filter strength / filter processing used to filter chroma samples across the ALF VB.
[0384] 2. CC-ALF is expected to be enabled only when luma ALF is applied (as designed in the picture header). However, two problems are found in the current VVC specification:
[0385] a. CC-ALF at the strip / picture / video level can be enabled only when ALF or Luma ALF is enabled. However, in CTB level, CC-ALF and ALF are controlled independently.
[0386] b. Even if the luma ALF filter is not signaled in the APS, the CC-ALF filter may be signaled.
[0387] 3. Chroma ALF is expected to be enabled only when luma ALF is applied (as designed in the picture header). However, two problems are found in the current VVC specification:
[0388] a.Strip / Picture / Video level Chroma ALF can be enabled only when ALF or Luma ALF is enabled. However, in CTB level, Chroma ALF and ALF are controlled independently.
[0389] b. Even if the luma filter is not signaled in the APS, the chroma ALF filter may be signaled.
[0390] 4. The codec tool for LMCS has two parts: luma mapping (reshaping process, represented by RP) and luma-dependent chroma residual scaling (CRS). In the picture header, CRS can be disabled (e.g., ph_chroma_residual_scale_flag is equal to 0) even if LMCS is enabled (i.e., in this case, only luma mapping is applied to the entire picture). However, in the slice header, there is only one flag (e.g., slice_lmcs_enabled_flag) to control both RP and CRS (which is not expected).
[0391] 5. When ph_lmcs_enabled_flag is true, the semantics of ph_lmcs_enabled_flag conflicts with the motivation of signaling slice-level LMCS flag. In current VVC, when ph_lmcs_enabled_flag is true, it means that all slices should enable LMCS. Therefore, there is no need to further signal the LMCS enabled flag in the slice header.
[0392] 4. Example Embodiments and Techniques
[0393] The following items should be considered as examples to explain general concepts. These items should not be interpreted in a narrow way. In addition, these items can be combined in any way.
[0394] In the present disclosure, the term "CC-ALF" refers to a codec that refines samples in a first color component (e.g., Cb) using sample values in a second color component (e.g., Y) or multiple color components (e.g., Y and Cr). It is not limited to existing CC-ALF technology. In addition, a "corresponding filter sample group" can be used to represent those samples included in the filter support. For example, for CC-ALF, a "corresponding filter sample group" can be used to represent co-located luma samples of chroma samples and adjacent luma samples of co-located luma samples, which are used to derive refinement / offset of chroma samples.
[0395] Handling ALF virtual boundaries for CC-ALF
[0396] 1. The filtering operation (eg, filter strength) in CC-ALF may depend on the location of the chroma samples to be filtered and / or the distance between the chroma samples and the ALF virtual boundary.
[0397] a. Optionally, it may depend on the distance between the corresponding luma sample and the ALF virtual boundary.
[0398] i. In one example, assume that the current chroma sample is located at (x, y) relative to the chroma CTB, and the corresponding luma sample is represented as (x*subWidthC, y*subHeightC), where (subWidthC, subHeightC) is derived from the color format.
[0399] 1) In one example, the filtering operation depends on (y*subHeightC) and / or the difference between (y*subHeightC) and ALFVB.
[0400] b. The CC-ALF filtering process is modified from equation (2-1) as follows:
[0401]
[0402] c.CC-ALF filtering is modified from equation (2-2) as follows:
[0403]
[0404] d. In the above sub-bullets b and c, the rounding offset OS and / or the filter strength represented by FS and / or the cropping boundaries T0 and / or T1 are not fixed.
[0405] i. Optionally, furthermore, the parameters OS and / or FS and / or T0 and / or T1 may be immediately signaled or derived.
[0406] ii. In one example, T0 and T1 may be set equal to those used in the non-linear luminance ALF.
[0407] Enable CC-ALF under the conditional check of Luma ALF
[0408] 2. CC-ALF related syntax elements in APS are signaled conditionally, such as depending on whether ALF / luminance ALF / chrominance ALF is signaled.
[0409] a. In one example, the signaling indicating that a CC-ALF filter is signaled (eg, alf_cc_cb_filter_signal_flag and / or alf_cc_cr_filter_signal_flag) may be under a conditional check of whether a luma ALF filter is signaled (eg, "if (alf_luma_filter_signal_flag)").
[0410] b. In one example, the signaling indicating that a CC-ALF filter is signaled (eg, alf_cc_cb_filter_signal_flag and / or alf_cc_cr_filter_signal_flag) may be under a conditional check of whether a chroma ALF filter is signaled (eg, “if (alf_chroma_filter_signal_flag)”).
[0411] c. In one example, if the indication(s) signaling the CC-ALF filter (eg, alf_cc_cb_filter_signal_flag and / or alf_cc_cr_filter_signal_flag) are not present in the bitstream, they are inferred to be 0.
[0412] d. It is required in a conforming bitstream that if the luma ALF filter is not signaled (eg, alf_luma_filter_signal_flag is zero), then the indication of the signaled CC-ALF filter (eg, alf_cc_cb_filter_signal_flag and / or alf_cc_cr_filter_signal_flag) must be zero.
[0413] e. It is required in a conforming bitstream that if the luma ALF filter is not signaled (eg, alf_chroma_filter_signal_flag is zero), then the indication of the signaled CC-ALF filter (eg, alf_cc_cb_filter_signal_flag and / or alf_cc_cr_filter_signal_flag) must be zero.
[0414] 3. The first syntax element (denoted as FSE) may control whether ALF and / or CCALF is applied to a given color component at the video region level.
[0415] a. In one example, a video region may be a sequence / video / slice / slice / sub-picture / picture / CTU / CTB / CU / PU / TU / CB / TB / PB.
[0416] b. In one example, the FSE may be signaled in the SPS / PPS / PH / slice header.
[0417] c. In one example, the first syntax element may control whether ALF and CCALF may be applied to any color component. If the first syntax element indicates "disabled" (eg, value equal to 0), neither ALF nor CCALF may be applied to any color component.
[0418] d. In one example, the first syntax element may control whether chroma ALF and CCALF may be applied to any chroma color component. If the first syntax element indicates "disabled" (eg, value equal to 0), neither chroma ALF nor CCALF may be applied to any chroma color component.
[0419] e. In one example, the first syntax element may control whether CCALF may be applied to Cb and Cr components. If the first syntax element indicates "disabled" (eg, value equal to 0), CCALF is disabled.
[0420] f. In one example, the first syntax element may control to which chroma components CCALF may be applied.
[0421] i. In one example, FSE is not a binary value.
[0422] ii. In one example, if the value is equal to 0, CCALF is disabled; if equal to 1 or 3, CCALF is enabled for the Cb color component; if equal to 2 or 3, CCALF is enabled for the Cr color component.
[0423] g. In one example, conditional on the first syntax element, one or more syntax elements of a lower level (smaller than the video area) may be signaled.
[0424] i. In one example, alf_cc_cb_filter_signal_flag and / or alf_cc_cr_filter_signal_flag and / or alf_luma_filter_signal_flag may be signaled under the conditional check of “if (FSE)”.
[0425] 4. CC-ALF related syntax elements (eg, in slice header or picture header, SPS / PPS / APS) may be conditionally signaled based on the enabling / signaling of chroma ALF.
[0426] a. In one example, the signaling of CC-ALF related syntax elements in the slice header may be under the conditional check of enabling / signaling chroma ALF for the same chroma component.
[0427] 5. Conditionally signal CC-ALF related syntax elements for color components in the codec tree block (CTB) level, such as depending on whether ALF / luma ALF / chroma ALF is applied to the current CTB and / or whether CC-ALF is enabled for the color component for the slice or picture containing the CTB (according to the associated picture header or PPS).
[0428] a. In one example, the signaling notification of alf_ctb_cc_cb_idc may be under the conditional check of “if (slice_cc_alf_cb_enabled_flag&&alf_ctb_flag[0][CtbAddrX][CtbAddrY])”.
[0429] b. In one example, the signaling notification of alf_ctb_cc_cr_idc may be under the conditional check of “if (slice_cc_alf_cr_enabled_flag&&alf_ctb_flag[0][CtbAddrX][CtbAddrY])”.
[0430] c. In one example, the signaling notification of alf_ctb_cc_cb_idc may be under the conditional check of “if (ph_cc_alf_cb_enabled_flag&&alf_ctb_flag[0][CtbAddrX][CtbAddrY])”.
[0431] d. In one example, the signaling notification of alf_ctb_cc_cr_idc can be under the conditional check of "if (ph_cc_alf_cr_enabled_flag&&alf_ctb_flag[0][CtbAddrX][CtbAddrY])".
[0432] e. In one example, the signaling notification of alf_ctb_cc_cb_idc may be under the conditional check of “if (slice_cc_alf_cb_enabled_flag&&alf_ctb_flag[1][CtbAddrX][CtbAddrY])”.
[0433] f. In one example, the signaling notification of alf_ctb_cc_cr_idc may be under the conditional check of “if (slice_cc_alf_cr_enabled_flag&&alf_ctb_flag[1][CtbAddrX][CtbAddrY])”.
[0434] g. In one example, the signaling notification of alf_ctb_cc_cb_idc may be under the conditional check of “if (ph_cc_alf_cb_enabled_flag&&alf_ctb_flag[2][CtbAddrX][CtbAddrY])”.
[0435] h. In one example, the signaling notification of alf_ctb_cc_cr_idc may be under the conditional check of “if (ph_cc_alf_cr_enabled_flag&&alf_ctb_flag[2][CtbAddrX][CtbAddrY])”.
[0436] Enable Chroma ALF under the conditional check of Luma ALF
[0437] 6. Conditionally signal chroma ALF related syntax elements in APS, e.g. depending on whether luma ALF is signaled.
[0438] a. In one example, the signaling of alf_chroma_filter_signal_flag may be under a conditional check of “if (alf_luma_filter_signal_flag)”.
[0439] 7. Conditionally signal chroma ALF related syntax elements for color components in the codec tree block (CTB) level, such as depending on whether luma ALF and / or chroma ALF are applied to the current CTB and / or whether chroma ALF for the color component is enabled for the slice containing the CTB.
[0440] a. In one example, the signaling notification of alf_ctb_flag[1][CtbAddrX][CtbAddrY] may be under the conditional check of “if ((slice_alf_chroma_idc==1||slice_alf_chroma_idc==3)&&alf_ctb_flag[0][CtbAddrX][CtbAddrY])”.
[0441] b. In one example, the signaling notification of alf_ctb_flag[2][CtbAddrX][CtbAddrY] may be under the conditional check of “if ((slice_alf_chroma_idc==2||slice_alf_chroma_idc==3)&&alf_ctb_flag[0][CtbAddrX][CtbAddrY])”.
[0442] In the discussion, “if (slice_alf_chroma_idc==1||slice_alf_chroma_idc==3)” is equivalent to “if (slice_alf_chroma_idc&1==1)”. “if (slice_alf_chroma_idc==2||slice_alf_chroma_idc==3)” is equivalent to “if (slice_alf_chroma_idc&2==1)”.
[0443] Related to LMCS
[0444] 8. Two-level control of LMCS is introduced (which includes two aspects: luma mapping (reshaping process, denoted by RP) and luma-dependent chroma residual scaling (CRS)), where higher-level (e.g., picture-level) and lower-level (e.g., strip-level) control are used. In addition, the following also applies:
[0445] a. In the first example, one or more of the following sub-bullets are applied:
[0446] i. A first indicator (e.g., ph_lmcs_enabled_flag) to enable / disable LMCS may be signaled at a higher level (e.g., in a picture header). More specifically, when the first indicator is equal to 1, it specifies that LMCS may be enabled for one or more or all slices associated with a picture header (PH); and when the first indicator is equal to 0, it specifies that LMCS is disabled for all slices associated with the PH. When the first indicator is not present, the value of the indicator is inferred to be equal to 0.
[0447] 1) Optionally, furthermore, the first indicator may be conditionally signaled according to the value of a LMCS enabled flag (eg, sps_lmcs_enabled_flag) in the sequence level.
[0448] ii. A second indicator (e.g., ph_chroma_residual_scale_flag) that enables / disables CRS may be signaled at a higher level (e.g., in a picture header). More specifically, when the second indicator is equal to 1, it specifies that CRS may be enabled for one, multiple, or all slices associated with the PH. And when the second indicator is equal to 0, it specifies that CRS is disabled for all slices associated with the PH. When the second indicator is not present, it is inferred to be equal to 0.
[0449] 1) Optionally, the second indicator can be conditionally signaled based on the value of the first indicator (e.g., whether it is equal to 1) and / or the color format, and / or separate plane codecs and / or "ChromaArrayType!=0" and / or whether the chroma tool is present or allowed.
[0450] iii. A third indicator (e.g., slice_lmcs_enabled_flag) to enable / disable LMCS for a lower level may be signaled at a lower level (e.g., in a slice header), and may be signaled under a conditional check of "the first indicator is true".
[0451] 1) It is required in a conformant bitstream that if the first indicator in a higher-level video unit (such as a picture header) associated with a lower-level video unit is 1, then the third indicator in at least one lower-level video unit (such as a slice) must be 1.
[0452] iv. A fourth indicator (eg, slice_chroma_residual_scale_enabled_flag) to enable / disable CRS for a lower level may be signaled at a lower level (eg, in a slice header).
[0453] 1) In one example, the signaling may be performed under a conditional check of "the second indicator is true" or "both the second and third indicators are true".
[0454] 2) It is required in the conforming bitstream that if the second indicator in the higher-level video unit (such as the picture header) associated with the lower-level video unit is 1, the fourth indicator in at least one lower-level video unit (such as the slice) must be 1.
[0455] b. In a second example, the first and / or third indicators mentioned in the first example may be used to control the use of RP instead of LMCS.
[0456] c. In the third example, the conditional check for signaling enabled / disabled LMCS in the slice level (eg, slice_lmcs_enabled_flag) may be replaced by "if (!ph_lmcs_enabled_flag)".
[0457] 9. The first syntax element may be signaled in the video region level, where the video region (e.g., such as a picture) is larger than a slice, and it indicates whether the LMCS / RP / CRS related syntax elements are present at the slice level (e.g., in the slice header) or at the video region level.
[0458] d. Optionally, in addition, under the conditional check whether the first syntax element indicates LMCS / RP / CRS related syntax elements in the video area level instead of the slice level, the LMCS / RP / CRS related syntax elements in the video area level can be signaled.
[0459] i. Optionally, in addition, when the first syntax element signals that LMCS / RP / CRS related syntax elements are present in the slice level, those LMCS / RP / CRS related syntax elements in the video area level (e.g., in the picture header) are not signaled.
[0460] e. Optionally, in addition, under the conditional check whether the first syntax element indicates that the LMCS / RP / CRS related syntax element exists in the slice level, the LMCS / RP / CRS related syntax element in the slice level can be signaled.
[0461] i. Optionally, in addition, when the first syntax element informs that LMCS / RP / CRS related syntax elements do not exist in the slice level, whether and / or how to enable LMCS / RP / CRS may depend on those syntax elements in the video area level (such as in the picture header).
[0462] 1) Optionally, in addition, all slices follow the same behavior indicated in the corresponding picture header.
[0463] f. Optionally, furthermore, the first syntax element may be signaled conditionally, such as only when the SPSLMCS enabled flag is true.
[0464] About MttSplitMode
[0465] 10. If split_qt_flag is equal to 1, then MttSplitMode[x0][y0][mttDepth] is set to a value not equal to any of SPLIT_TT_HOR, or SPLIT_BT_HOR, or SPLIT_TT_VER, or SPLIT_BT_VER. For example, if split_qt_flag is equal to 1, then MttSplitMode[x0][y0][mttDepth] is set to SPLIT_QT.
[0466] 11.allowBtSplit is derived based on a joint test of MttSplitMode[x0][y0][mttDepth] and split_qt_flag.
[0467] 12.modeTypeCondition is derived based on the joint test of MttSplitMode[x0][y0][mttDepth] and split_qt_flag.
[0468] 13. Perform cross-component chroma intra prediction mode check processing based on a joint test of MttSplitMode[x0][y0][mttDepth] and split_qt_flag.
[0469] General
[0470] 14. Whether and / or how to apply the above disclosed method can be signaled at the sequence level / picture level / slice level / slice group level, such as in the sequence header / picture header / SPS / VPS / DCI / PPS / APS / slice header / slice group header.
[0471] 15. Whether and / or how to apply the above disclosed method may depend on the codec information, such as color format, single / dual tree partitioning, and location of samples (eg, relative to CU / CTU).
[0472] 5. Examples
[0473] In the following examples, deletions are marked with [[ ]] and corresponding canonical additions are highlighted in bold, italics, and underlining.
[0474] 5.1. Example #1
[0475] In this embodiment, the semantics of LMCS-related syntax elements in the picture header are modified.
[0476] 7.3.2.7 Picture header structure syntax
[0477]
[0478]
[0479] Alternative #1
[0480] ph_lmcs_enabled_flag is equal to 1 to specify [[Yes]] [[for all slices associated with PH]] luma mapping with chroma scaling is enabled. ph_lmcs_enabled_flag equal to 0 specifies [[may be disabled for one or more or all slices associated with PH]] Luma mapping with chroma scaling. When not present, the value of ph_lmcs_enabled_flag is inferred to be equal to 0.
[0481] ph_lmcs_aps_id specifies the adaptation_parameter_set_id of the LMCS APS referenced by the slice associated with the PH. The TemporalId of the APS NAL unit with aps_params_type equal to LMCS_APS and adaptation_parameter_set_id equal to ph_lmcs_aps_id shall be less than or equal to the TemporalId of the picture associated with the PH.
[0482] ph_chroma_residual_scale_flag is equal to 1 to specify [[Yes]] Chroma residual scaling is enabled for all slices associated with PH. ph_chroma_residual_scale_flag equal to 0 specifies [[may be enabled for one or more or, disabled]] for all slices associated with PH. Chroma residual scale. When ph_chroma_residual_scale_flag is not present, it is inferred to be equal to 0.
[0483] Alternative #2
[0484] ph_lmcs_enabled_flag is equal to 1 to specify [[Yes]] for PH-related Luma mapping with chroma scaling is enabled for all slices. ph_lmcs_enabled_flag equal to 0 specifies that all slices associated with PH Luma mapping with chroma scaling.
[0485] ph_lmcs_aps_id specifies the adaptation_parameter_set_id of the LMCS APS referenced by the slice associated with the PH. The TemporalId of the APS NAL unit with aps_params_type equal to LMCS_APS and adaptation_parameter_set_id equal to ph_lmcs_aps_id shall be less than or equal to the TemporalId of the picture associated with the PH.
[0486] ph_chroma_residual_scale_flag is equal to 1 to specify [[Yes]] for PH-related Enable chroma residual scaling for all slices. ph_chroma_residual_scale_flag equal to 0 specifies [[may be applied to one or more or]] all slices associated with PH Chroma residual scale. When ph_chroma_residual_scale_flag is not present, it is inferred to be equal to 0.
[0487] 5.2. Example #2
[0488] In this embodiment, the semantics of the LMCS-related syntax elements in the picture header remain unchanged. However, in order to signal the LMCS enable flag of the lower level (e.g., slice), the conditional check of the LMCS enable flag of the higher level (e.g., picture) being true is replaced by the LMCS enable flag of the higher level being false.
[0489] In the following alternative, the condition "if (ph_lmcs_enabled_flag)" is replaced by "if (!ph_lmcs_enabled_flag)".
[0490] 7.3.7 Strip Header Syntax
[0491] 7.3.7.1 Generic Strip Header Syntax
[0492]
[0493] 5.3. Example #3
[0494] In this embodiment, a new syntax element is introduced in a higher level (eg, picture header) to indicate whether LMCS related syntax elements exist in a lower level (eg, slice header).
[0495] 7.3.2.7 Picture header structure syntax
[0496]
[0497]
[0498]
[0499] 7.3.7 Strip Header Syntax
[0500] 7.3.7.1 Generic Strip Header Syntax
[0501]
[0502]
[0503]
[0504] ph_lmcs_enabled_flag equal to 1 specifies that luma mapping with chroma scaling is enabled for all slices associated with the PH. ph_lmcs_enabled_flag equal to 0 specifies that luma mapping with chroma scaling may be disabled for one or more or all slices associated with the PH. When not present, the value of ph_lmcs_enabled_flag is inferred to be equal to 0.
[0505] ph_lmcs_aps_id specifies the adaptation_parameter_set_id of the LMCS APS referenced by the slice associated with the PH.The TemporalId of the APS NAL unit with aps_params_type equal to LMCS_APS and adaptation_parameter_set_id equal to ph_lmcs_aps_id shall be less than or equal to the TemporalId of the picture associated with the PH.
[0506] ph_chroma_residual_scale_flag equal to 1 specifies that chroma residual scaling is enabled for all slices associated with the PH. ph_chroma_residual_scale_flag equal to 0 specifies that chroma residual scaling may be disabled for one or more or all slices associated with the PH. When ph_chroma_residual_scale_flag is not present, it is inferred to be equal to 0.
[0507] 5.4. Example #4
[0508] This embodiment gives an example of controlling CC-ALF under chroma ALF.
[0509]
[0510]
[0511] 5.5. Example #5
[0512] This embodiment gives an example of modifying the filter strength of CC-ALF when filtering chroma samples around ALF VB.
[0513] 8.8.5.7 Cross-component filtering for chroma sample codec tree blocks
[0514] The input to this process is:
[0515] – Reconstructed luminance picture sample array recPicture before luminance adaptive loop filteringL ,
[0516] –Filtered reconstructed chrominance picture sample array alfPicture C ,
[0517] – Chroma position (xCtbC, yCtbC), specifies the top left sample point of the current chroma codec tree block relative to the top left sample point of the current picture,
[0518] – Luma position (xCtb, yCtb), specifies the top left sample point of the current luma codec tree block relative to the top left sample point of the current picture
[0519] – The width of the chroma sample block ccAlfWidth
[0520] – The height of the chroma sample block ccAlfHeight
[0521] – Cross-component filter coefficients CcAlfCoeff[j], where j = 0..6
[0522] The output of this process is a modified filtered reconstructed chroma picture sample array ccAlfPicture.
[0523] In order to derive the filtered reconstructed chroma sample ccAlfPicture[xCtbC+x][yCtbC+y], each reconstructed chroma sample alfPicture inside the current chroma block C [xCtbC+x][yCtbC+y] is filtered as follows, where x = 0..ccAlfWidth–1, y = 0..ccAlfHeight–1:
[0524] – The luma position (xL, yL) corresponding to the current chroma sample at chroma position (xCtbC+x, yCtbC+y) is set equal to ((xCtbC+x)*SubWidthC, (yCtbC+y)*SubHeightC)
[0525] – In the array recPicture L The internal brightness position (h x+i ,v y+j ) is derived as follows, where i = -1..1, j = -1..2:
[0526] h x+i =Clip3(0,pic_width_in_luma_samples-1,xL+i) (8-1231)
[0527] v y+j=Clip3(0,pic_height_in_luma_samples-1,yL+j) (8-1234)
[0528] – The variables clipLeftPos, clipRightPos, clipTopPos, clipBottomPos, clipTopLeftFlag and clipBotRightFlag are derived by calling the ALF boundary position derivation procedure as specified in clause 8.8.5.5, with (xCtbC*SubWidthC, yCtbC*SubHeightC) and (x*SubWidthC, y*SubHeightC) as input.
[0529] – Variable h x+i and v y+j is modified by invoking the ALF sample filling process as specified in clause 8.8.5.6, where (xCtb, yCtb), (h x+i ,v y+j ), with the variables isChroma, clipLeftPos, clipRightPos, clipTopPos, clipBottomPos, clipTopLeftFlag, and clipBotRightFlag set equal to 0 as input.
[0530] –The derivation of the variable applyAlfLineBufBoundary is as follows:
[0531] – If the bottom boundary of the current codec treeblock is the bottom boundary of the current picture and pic_height_in_luma_samples-yCtb<=CtbSizeY-4, then applyAlfLineBufBoundary is set equal to 0:
[0532] – Otherwise, applyAlfLineBufBoundary is set equal to 1.
[0533] – The vertical sample position offsets yP1 and yP2 are specified in Table 44 according to the vertical luma sample position y*subHeightC and applyAlfLineBufBoundary.
[0534] –The variable curr is derived as follows:
[0535] curr=alfPicture C [xCtbC+x][yCtbC+y] (8-1286)
[0536] – The array of cross-component filter coefficients f[j] is derived as follows, where j = 0..6:
[0537] f[j]=CcAlfCoeff[j] (8-1287)
[0538] –The derivation of the variable sum is as follows:
[0539]
[0540]
[0541]
[0542] sum = curr + scaledSum (8-1290)
[0543] – The modified filtered reconstructed chroma picture sample array ccAlfPicture[xCtbC+x][yCtbC+y] is derived as follows:
[0544] ccAlfPicture[xCtbC+x][yCtbC+y]=Clip3(0,(1< <BitDepth)-1,sum) (8-1291)
[0545] Table 44 – yP1 and yP2 specifications based on vertical luma sample position y*subHeightC and applyAlfLineBufBoundary
[0546]
[0547] In the newly added text related to ccalfShiftC above, the three variables S0, S1, and S2 are integer values, such as 10 or 7. In one example, depending on the position of the current chroma sample to be filled, RoundOffset can be set equal to (1<<(S0-1)), or (1<<(S1-1)), or (1<<(S2-1)). Alternatively, it can be set to a fixed value, such as 64; or the fixed value can depend on the precision of the CC-ALF coefficient.
[0548] Optionally, S0 is 4. In one example, S1 is 6. In one example, S1 is 7.
[0549] 5.6. Example #6
[0550] This embodiment shows an example of the MTT division mode.
[0551] The variable allowBtSplit is derived as follows:
[0552] – allowBtSplit is set equal to FALSE if one or more of the following conditions are true:
[0553] –cbSize is less than or equal to MinBtSizeY
[0554] –cbWidth is greater than maxBtSize
[0555] –cbHeight is greater than maxBtSize
[0556] –mttDepth is greater than or equal to maxMttDepth
[0557] –treeType is equal to DUAL_TREE_CHROMA, and (cbWidth / SubWidthC)*(cbHeight / SubHeightC) is less than or equal to 16
[0558] –treeType equals DUAL_TREE_CHROMA, and (cbWidth / SubWidthC) equals 4, and btSplit equals SPLIT_BT_VER
[0559] –treeType equals DUAL_TREE_CHROMA and modeType equals MODE_TYPE_INTRA
[0560] –cbWidth*cbHeight is equal to 32, and modeType is equal to MODE_TYPE_INTER
[0561] – Otherwise, if all of the following conditions are true, allowBtSplit is set equal to FALSE
[0562] –btSplit is equal to SPLIT_BT_VER
[0563] –y0+cbHeight is greater than pic_height_in_luma_samples
[0564] – Otherwise, if all of the following conditions are true, allowBtSplit is set equal to FALSE
[0565] –btSplit is equal to SPLIT_BT_VER
[0566] –cbHeight is greater than 64
[0567] –x0+cbWidth is greater than pic_width_in_luma_samples
[0568] – Otherwise, if all of the following conditions are true, allowBtSplit is set equal to FALSE
[0569] –btSplit is equal to SPLIT_BT_HOR
[0570] –cbWidth is greater than 64
[0571] –y0+cbHeight is greater than pic_height_in_luma_samples
[0572] – Otherwise, if all of the following conditions are true, allowBtSplit is set equal to FALSE
[0573] –x0+cbWidth is greater than pic_width_in_luma_samples
[0574] –y0+cbHeight is greater than pic_height_in_luma_samples
[0575] –cbWidth is greater than minQtSize
[0576] – Otherwise, if all of the following conditions are true, allowBtSplit is set equal to FALSE
[0577] –btSplit is equal to SPLIT_BT_HOR
[0578] –x0+cbWidth is greater than pic_width_in_luma_samples
[0579] –y0+cbHeight is less than or equal to pic_height_in_luma_samples
[0580] – Otherwise, allowBtSplit is set equal to FALSE if all of the following conditions are true:
[0581] –mttDepth is greater than 0
[0582] –partIdx is equal to 1
[0583]
[0584] –MttSplitMode[x0][y0][mttDepth-1] equals parallelTtSplit
[0585] – Otherwise, if all of the following conditions are true, allowBtSplit is set equal to FALSE
[0586] –btSplit is equal to SPLIT_BT_VER
[0587] –cbWidth is less than or equal to 64
[0588] –cbHeight is greater than 64
[0589] – Otherwise, if all of the following conditions are true, allowBtSplit is set equal to FALSE
[0590] –btSplit is equal to SPLIT_BT_HOR
[0591] –cbWidth is greater than 64
[0592] –cbHeight is less than or equal to 64
[0593] – Otherwise, allowBtSplit is set to TRUE.
[0594] 5.7. Example #7
[0595] The variable modeTypeCondition is derived as follows:
[0596] – modeTypeCondition is set equal to 0 if one or more of the following conditions are true:
[0597] –slice_type is equal to 1, and qtbtt_dual_tree_intra_flag is equal to 1.
[0598] –modeTypeCurr is not equal to MODE_TYPE_ALL.
[0599] –chroma_format_idc is equal to 0.
[0600] –chroma_format_idc is equal to 3.
[0601] – Otherwise, modeTypeCondition is set equal to 1 if one of the following conditions is true:
[0602] –cbWidth*cbHeight is equal to 64, and split_qt_flag is equal to 1.
[0603] –cbWidth*cbHeight is equal to 64, And MttSplitMode[x0][y0][mttDepth] is equal to SPLIT_TT_HOR or SPLIT_TT_VER.
[0604] –cbWidth*cbHeight is equal to 32, And MttSplitMode[x0][y0][mttDepth] is equal to SPLIT_BT_HOR or SPLIT_BT_VER.
[0605] – Otherwise, modeTypeCondition is set equal to 1+(slice_type != I?1:0) if one of the following conditions is true:
[0606] –cbWidth*cbHeight is equal to 64, And MttSplitMode[x0][y0][mttDepth] is equal to SPLIT_BT_HOR or SPLIT_BT_VER, and chroma_format_idc is equal to 1.
[0607] –cbWidth*cbHeight is equal to 128, And MttSplitMode[x0][y0][mttDepth] is equal to SPLIT_TT_HOR or SPLIT_TT_VER, and chroma_format_idc is equal to 1.
[0608] –cbWidth is equal to 8, And MttSplitMode[x0][y0][mttDepth] is equal to SPLIT_BT_VER.
[0609] –cbWidth is equal to 16, And MttSplitMode[x0][y0][mttDepth] is equal to SPLIT_TT_VER.
[0610] – Otherwise, modeTypeCondition is set equal to 0.
[0611] 5.8. Example #8
[0612] 8.4.4 Cross-component chroma intra prediction mode check processing
[0613] The input to this process is:
[0614] – Luma position (xCb, yCb), specifies the top left sample of the current chroma codec block relative to the top left luma sample of the current picture.
[0615] The output of this processing is:
[0616] – The variable CclmEnabled specifies whether the cross-component chroma intra prediction mode is enabled (TRUE) or not enabled (FALSE) for the current chroma codec block.
[0617] The variable CclmEnabled is derived as follows:
[0618] – If sps_cclm_enabled_flag is equal to 0, then CclmEnabled is set equal to 0.
[0619] – Otherwise, CclmEnabled is set equal to 1 if one or more of the following conditions are true:
[0620] –qtbtt_dual_tree_intra_flag is equal to 0.
[0621] –slice_type is not equal to I.
[0622] –CtbLog2SizeY is less than 6.
[0623] – Otherwise, the following applies:
[0624] – The variables xCb64, yCb64, yCb32 are derived as follows:
[0625] xCb64=(xCb>>6)<<6 (236)
[0626] yCb64=(yCb>>6)<<6 (237)
[0627] yCb32=(yCb>>5)<<5 (238)
[0628] – The variable CclmEnabled is set equal to 1 if one or more of the following conditions are true:
[0629] –CbWidth[1][xCb64][yCb64] is equal to 64, and CbHeight[1][xCb64][yCb64] is equal to 64.
[0630] – CqtDepth[1][xCb64][yCb64] is equal to CtbLog2SizeY-6, MttSplitMode[xCb64][yCb64][0] is equal to SPLIT_BT_HOR, CbWidth[1][xCb64][yCb32] is equal to 64, and CbHeight[1][xCb64][yCb32] is equal to 32.
[0631] –CqtDepth[1][xCb64][yCb64] is greater than CtbLog2SizeY-6.
[0632] – CqtDepth[1][xCb64][yCb64] is equal to CtbLog2SizeY-6, MttSplitMode[xCb64][yCb64][0] is equal to SPLIT_BT_HOR, and MttSplitMode[xCb64][yCb32][1] is equal to SPLIT_BT_VER.
[0633] – Otherwise, the variable CclmEnabled is set equal to 0.
[0634] When CclmEnabled is equal to 1 and one of the following conditions is true, CclmEnabled is set equal to 0:
[0635] – CbWidth[0][xCb64][yCb64] and CbHeight[0][xCb64][yCb64] are both equal to 64, and intra_subpartitions_mode_flag[xCb64][yCb64] is equal to 1.
[0636] – CbWidth[0][xCb64][yCb64] or CbHeight[0][xCb64][yCb64] is less than 64, and CqtDepth[0][xCb64][yCb64] is equal to CtbLog2SizeY-6.
[0637] Fig. 22 2200 is a block diagram illustrating an example video processing system 2200 in which various techniques disclosed herein may be implemented. Various implementations may include some or all of the components of system 2200. System 2200 may include an input 2202 for receiving video content. The video content may be received in a raw or uncompressed format (e.g., 8 or 10 bit multi-component pixel values), or may be in a compressed or encoded format. Input 2202 may represent a network interface, a peripheral bus interface, or a storage interface. Examples of network interfaces include wired interfaces such as Ethernet, a Passive Optical Network (PON), and wireless interfaces such as Wi-Fi or a cellular interface.
[0638] System 2200 may include a codec component 2204, which may implement various codecs or encoding methods described in this document. Codec component 2204 may reduce the average bit rate of the video from input 2202 to the output of codec component 2204 to generate a codec representation of the video. Therefore, codec technology is sometimes referred to as video compression or video transcoding technology. The output of codec component 2204 may be stored or transmitted via a communication connection represented by component 2206. Component 2208 may use a bitstream (or codec) representation of the storage or communication of the video received at input 2202 to generate pixel values or displayable video sent to display interface 2210. The process of generating user-visible video from a bitstream representation is sometimes referred to as video decompression. In addition, although some video processing operations are referred to as "codec" operations or tools, it should be understood that codec tools or operations are used at the encoder, and the corresponding decoding tools or operations of the inverted codec results will be performed by the decoder.
[0639] Examples of peripheral bus interfaces or display interfaces may include Universal Serial Bus (USB) or High Definition Multimedia Interface (HDMI) or DisplayPort, etc. Examples of storage interfaces include SATA (Serial Advanced Technology Attachment), PCI, IDE interfaces, etc. The techniques described in this document may be implemented in various electronic devices, such as mobile phones, laptops, smartphones, or other devices capable of performing digital data processing and / or video display.
[0640] Fig.23 is a block diagram of a video processing device 2300. The device 2300 may be used to implement one or more methods described herein. The device 2300 may be implemented in a smartphone, a tablet, a computer, an Internet of Things (IoT) receiver, etc. The device 2300 may include one or more processors 2302, one or more memories 2304, and video processing hardware 2306. The processor(s) 2302 may be configured to implement one or more methods described in this document. The memory(s) 2304 may be used to store data and code for implementing the methods and techniques described herein. The video processing hardware 2306 may be used to implement some of the techniques described in this document in hardware circuits.
[0641] Fig.25 is a block diagram illustrating an example video coding system 100 that may utilize the techniques of this disclosure.
[0642] like Fig.25As shown, the video codec system 100 may include a source device 110 and a destination device 120. The source device 110 generates encoded video data, which may be referred to as a video encoding device. The destination device 120 may decode the encoded video data generated by the source device 110, which may be referred to as a video decoding device.
[0643] Source device 110 may include a video source 112 , a video encoder 114 , and an input / output (I / O) interface 116 .
[0644] The video source 112 may include a source such as a video capture device, an interface for receiving video data from a video content provider, and / or a computer graphics system for generating video data, or a combination of these sources. The video data may include one or more pictures. The video encoder 114 encodes the video data from the video source 112 to generate a bit stream. The bit stream may include a bit sequence that forms a codec representation of the video data. The bit stream may include a codec picture and related data. The codec picture is a codec representation of the picture. Related data may include a sequence parameter set, a picture parameter set, and other grammatical structures. The I / O interface 116 may include a modulator / demodulator (modem) and / or a transmitter. The encoded video data may be directly sent to the destination device 120 via the network 130a via the I / O interface 116. The encoded video data may also be stored on a storage medium / server 130b for access by the destination device 120.
[0645] Destination device 120 may include an I / O interface 126 , a video decoder 124 , and a display device 122 .
[0646] The I / O interface 126 may include a receiver and / or a modem. The I / O interface 126 may obtain encoded video data from the source device 110 or the storage medium / server 130b. The video decoder 124 may decode the encoded video data. The display device 122 may display the decoded video data to the user. The display device 122 may be integrated with the destination device 120, or may be outside the destination device 120, and the destination device 120 may be configured to be connected to an external display device.
[0647] The video encoder 114 and the video decoder 124 may operate according to a video compression standard, such as the High Efficiency Video Codec (HEVC) standard, the Versatile Video Codec (VVM) standard, and other current and / or future standards.
[0648] Fig.26 is a block diagram showing an example of a video encoder 200, which may be Fig.25 The video encoder 114 in the system 100 is shown in FIG.
[0649] Video encoder 200 may be configured to perform any or all of the techniques of this disclosure. Fig.26 In the example of , video encoder 200 includes multiple functional components. The techniques described in this disclosure may be shared between various components of video encoder 200. In some examples, a processor may be configured to perform any or all of the techniques described in this disclosure.
[0650] The functional components of the video encoder 200 may include a segmentation unit 201, a prediction unit 202 (which may include a mode selection unit 203, a motion estimation unit 204, a motion compensation unit 205 and an intra-frame prediction unit 206), a residual generation unit 207, a transform unit 208, a quantization unit 209, an inverse quantization unit 210, an inverse transform unit 211, a reconstruction unit 212, a buffer 213 and an entropy coding unit 214.
[0651] In other examples, the video encoder 200 may include more, fewer, or different functional components. In one example, the prediction unit 202 may include an intra block copy (IBC) unit. The IBC unit may perform prediction in an IBC mode, where at least one reference picture is a picture where the current video block is located.
[0652] Furthermore, some components (eg, motion estimation unit 204 and motion compensation unit 205) may be highly integrated but are not shown for purposes of explanation. Fig.26 In the example, they are represented separately.
[0653] The partitioning unit 201 may partition a picture into one or more video blocks. The video encoder 200 and the video decoder 300 may support various video block sizes.
[0654] The mode selection unit 203 may select one of the coding modes (intra or inter) (e.g., based on the error result), and provide the resulting intra or inter coded block to the residual generation unit 207 to generate residual block data, and to the reconstruction unit 212 to reconstruct the coded block for use as a reference picture. In some examples, the mode selection unit 203 may select a combination of intra and inter prediction (CIIP) modes, where the prediction is based on inter prediction signaling and intra prediction signaling. In the case of inter prediction, the mode selection unit 203 may also select the precision of the motion vector for the block (e.g., sub-pixel or integer pixel precision).
[0655] In order to perform inter-frame prediction on the current video block, the motion estimation unit 204 may generate motion information of the current video block by comparing the current video block with one or more reference frames from the buffer 213. The motion compensation unit 205 may determine a predicted video block of the current video block based on the motion information of pictures other than the picture associated with the current video block from the buffer 213 and decoded samples.
[0656] Motion estimation unit 204 and motion compensation unit 205 may perform different operations on the current video block, eg, depending on whether the current video block is in an I slice, a P slice, or a B slice.
[0657] In some examples, motion estimation unit 204 may perform unidirectional prediction on the current video block, and motion estimation unit 204 may search for a reference video block of the current video block in a reference picture in list 0 or list 1. Motion estimation unit 204 may then generate a reference index indicating a reference picture in list 0 or list 1, the reference picture including the reference video block and a motion vector indicating a spatial displacement between the current video block and the reference video block. Motion estimation unit 204 may output the reference index, the prediction direction indicator, and the motion vector as motion information of the current video block. Motion compensation unit 205 may generate a predicted video block of the current block based on the reference video block indicated by the motion information of the current video block.
[0658] In other examples, the motion estimation unit 204 may perform bidirectional prediction on the current video block, and the motion estimation unit 204 may search for a reference video block of the current video block in the reference pictures in list 0, and may also search for another reference video block of the current video block in the reference pictures in list 1. The motion estimation unit 204 may then generate a reference index indicating the reference pictures in list 0 and list 1 that include the reference video block, and a motion vector indicating a spatial displacement between the reference video block and the current video block. The motion estimation unit 204 may output the reference index and the motion vector of the current video block as the motion information of the current video block. The motion compensation unit 205 may generate a predicted video block of the current video block based on the reference video block indicated by the motion information of the current video block.
[0659] In some examples, motion estimation unit 204 may output a complete set of motion information for use in the decoding process of a decoder.
[0660] In some examples, motion estimation unit 204 may not output a complete set of motion information for the current video. Instead, motion estimation unit 204 may reference motion information of another video block to signal motion information of the current video block. For example, motion estimation unit 204 may determine that the motion information of the current video block is sufficiently similar to the motion information of an adjacent video block.
[0661] In one example, motion estimation unit 204 may indicate a value in a syntax structure associated with the current video block that indicates to video decoder 300 that the current video block has the same motion information as another video block.
[0662] In another example, motion estimation unit 204 may identify another video block and a motion vector difference (MVD) in a syntax structure associated with the current video block. The motion vector difference indicates the difference between the motion vector of the current video block and the motion vector of the indicated video block. Video decoder 300 may use the motion vector of the indicated video block and the motion vector difference to determine the motion vector of the current video block.
[0663] As described above, the video encoder 200 may predictively signal motion vectors. Two examples of predictive signaling techniques that may be implemented by the video encoder 200 include advanced motion vector prediction (AMVP) and merge mode signaling.
[0664] The intra prediction unit 206 may perform intra prediction on the current video block. When the intra prediction unit 206 performs intra prediction on the current video block, the intra prediction unit 206 may generate prediction data for the current video block based on decoded samples of other video blocks in the same picture. The prediction data for the current video block may include the predicted video block and various syntax elements.
[0665] The residual generation unit 207 may generate residual data for the current video block by subtracting (e.g., indicated by a minus sign) the predicted video block of the current video block from the current video block. The residual data for the current video block may include residual video blocks corresponding to different sample components corresponding to samples in the current video block.
[0666] In other examples, the current video block may not have residual data for the current video block, such as in skip mode, and the residual generation unit 207 may not perform a subtraction operation.
[0667] Transform processing unit 208 may generate one or more transform coefficient video blocks for the current video block by applying one or more transforms to the residual video block associated with the current video block.
[0668] After transform processing unit 208 generates a transform coefficient video block associated with the current video block, quantization unit 209 may quantize the transform coefficient video block associated with the current video block based on one or more quantization parameter (QP) values associated with the current video block.
[0669] The inverse quantization unit 210 and the inverse transform unit 211 may apply inverse quantization and inverse transform to the transform coefficient video block, respectively, to reconstruct a residual video block from the transform coefficient video block. The reconstruction unit 212 may add the reconstructed residual video block to corresponding samples from one or more prediction video blocks generated by the prediction unit 202 to generate a reconstructed video block associated with the current block for storage in the buffer 213.
[0670] After reconstruction unit 212 reconstructs the video block, a loop filtering operation may be performed to reduce video block artifacts in the video block.
[0671] The entropy encoding unit 214 may receive data from other functional components of the video encoder 200. When the entropy encoding unit 214 receives the data, the entropy encoding unit 214 may perform one or more entropy encoding operations to generate entropy-encoded data and output a bitstream including the entropy-encoded data.
[0672] Fig. 27 is a block diagram showing an example of a video decoder 300, which may be Fig.25 The video decoder 114 in the system 100 is shown.
[0673] Video decoder 300 may be configured to perform any or all of the techniques of this disclosure. Fig. 27 In the example of , video decoder 300 includes multiple functional components. The techniques described in this disclosure can be shared between various components of video decoder 300. In some examples, a processor can be configured to perform any or all of the techniques described in this disclosure.
[0674] exist Fig. 27 In the example of FIG. 3 , the video decoder 300 includes an entropy decoding unit 301, a motion compensation unit 302, an intra-frame prediction unit 303, an inverse quantization unit 304, an inverse transform unit 305, a reconstruction unit 306, and a buffer 307. In some examples, the video decoder 300 may perform operations generally similar to those for the video encoder 200 ( Fig.26 ) is the decoding channel that is the opposite of the encoding process described.
[0675] The entropy decoding unit 301 may retrieve an encoded bitstream. The encoded bitstream may include entropy encoded video data (e.g., encoded video data blocks). The entropy decoding unit 301 may decode the entropy encoded video data, and from the entropy decoded video data, the motion compensation unit 302 may determine motion information, including motion vectors, motion vector precision, reference picture list indexes, and other motion information. The motion compensation unit 302 may determine such information, for example, by performing AMVP and merge modes.
[0676] The motion compensation unit 302 may generate a motion compensated block, and interpolation may be performed based on an interpolation filter. An identifier of an interpolation filter to be used with sub-pixel precision may be included in the syntax element.
[0677] Motion compensation unit 302 may calculate interpolation of sub-integer pixels of a reference block using interpolation filters as used by video encoder 20 during encoding of the video block. Motion compensation unit 302 may determine the interpolation filters used by video encoder 200 from received syntax information and use the interpolation filters to produce a prediction block.
[0678] The motion compensation unit 302 may use some syntax information to determine the size of blocks used to encode (multiple) frames and / or (multiple) slices of the encoded video sequence, partitioning information describing how each macroblock of a picture of the encoded video sequence is partitioned, a mode indicating how each partition is encoded, one or more reference frames (and reference frame lists) for each inter-coded block, and other information for decoding the encoded video sequence.
[0679] The intra prediction unit 303 may form a prediction block from spatially adjacent blocks using, for example, an intra prediction mode received in the bitstream. The inverse quantization unit 303 inversely quantizes (i.e., dequantizes) the quantized video block coefficients provided in the bitstream and decoded by the entropy decoding unit 301. The inverse transform unit 303 applies an inverse transform.
[0680] The reconstruction unit 306 may add the residual block to the corresponding prediction block generated by the motion compensation unit 202 or the intra prediction unit 303 to form a decoded block. If necessary, a deblocking filter may also be applied to filter the decoded block to remove block artifacts. The decoded video block is then stored in a buffer 307, which provides reference blocks for subsequent motion compensation / intra prediction and also generates decoded video for presentation on a display device.
[0681] Next, a list of technical solutions preferred by some embodiments is provided.
[0682] The following technical scheme illustrates an example embodiment of the technology discussed in the previous section (eg, item 1).
[0683] 1. A video processing method (eg, Fig.24 The method 2400 described in ), includes: for conversion between a current video block of a video and a coded representation of the video, determining a characteristic of a cross-component adaptive loop filter based on the position of color component samples (2402); and performing the conversion using the characteristic of the cross-component adaptive loop filter (2404).
[0684] 2. The method according to claim 1, wherein the characteristic comprises a filter strength of a cross-component adaptive loop filter.
[0685] 3. A method according to any one of technical solutions 1-2, wherein the color component is a chrominance component.
[0686] 4. A method according to any one of technical solutions 1-2, wherein the color component is a brightness component.
[0687] 5. The method according to any one of technical solutions 1-5, wherein the cross-component adaptive loop filtering is expressed as:
[0688]
[0689] in
[0690] (x,y) is the position of the chroma component i being refined
[0691] (x C ,y C ) is the brightness position based on (x,y)
[0692] S i is the filter support in luma of chroma component i
[0693] c i (x0,y0) represents the filter coefficients
[0694] The following technical scheme illustrates an example embodiment of the technology discussed in the previous section (eg, Item 2).
[0695] 6. A video processing method, comprising: performing a conversion between a video region of a video and a codec representation of the video, wherein one or more syntax elements related to the use of cross-component adaptive loop filtering in the conversion are conditionally included in the codec representation based on whether another syntax element related to adaptive loop filtering is included in the codec representation.
[0696] 7. A method according to technical solution 6, wherein the adaptive loop filtering includes brightness adaptive loop filtering.
[0697] 8. The method according to technical solution 6, wherein the adaptive loop filtering includes chrominance adaptive loop filtering.
[0698] The following technical scheme illustrates an example embodiment of the technology discussed in the previous section (eg, item 3).
[0699] 9. A video processing method, comprising: performing a conversion between a video region of a video and a codec representation of the video, wherein a first syntax element is included in the codec representation indicating whether an adaptive loop filter or a cross-component adaptive loop filter is applied during the conversion of the video region.
[0700] 10. A method according to technical solution 9, wherein the video area corresponds to a video sequence, or a video, or a video strip, or a video slice, or a video sub-picture, or a video picture, or a codec tree unit, or a codec tree block, or a codec unit, or a prediction unit, or a transform unit, or a codec block, or a transform block or a prediction block.
[0701] 11. A method according to any one of technical solutions 9-10, wherein the first syntax element is included at a sequence parameter set level or a picture parameter set level, or is included in a picture header or a slice header.
[0702] 12. A method according to any one of technical solutions 9-11, wherein the first syntax element indicates whether an adaptive loop filter or a cross-component adaptive loop filter is applied to the chrominance component.
[0703] 13. A method according to any one of technical solutions 9-12, wherein the first syntax element indicates whether an adaptive loop filter or a cross-component adaptive loop filter is applied to the luminance component.
[0704] The following technical scheme illustrates an example embodiment of the technology discussed in the previous section (eg, item 4).
[0705] 14. A method according to any one of technical solutions 1-13, wherein the codec representation conditionally includes syntax elements related to the cross-component adaptive loop filter based on whether the adaptive loop filter is applied to the chrominance component.
[0706] The following technical scheme illustrates an example embodiment of the techniques discussed in the previous sections (eg, items 6, 7).
[0707] 15. A video processing method, comprising: performing conversion between a video region of a video and a codec representation of the video, wherein syntax elements related to adaptive loop filtering of chroma components are conditionally included in an adaptive parameter set depending on codec conditions.
[0708] 16. A method according to technical solution 15, wherein the coding condition includes whether syntax elements related to adaptive filtering of luminance components in a video area are included in the coding representation.
[0709] 17. A method according to technical solution 16, wherein the video area includes a codec tree block, and wherein the codec condition includes (1) whether a luminance adaptive loop filter is applied to the codec tree block, or (2) whether a chrominance adaptive loop filter is applied to the codec tree block, or (3) whether a strip including the codec tree block is configured to use a chrominance adaptive loop filter.
[0710] The following technical scheme illustrates an example embodiment of the technology discussed in the previous section (eg, item 8).
[0711] 18. A video processing method, comprising: performing conversion between a video region of a video and a codec representation of the video, wherein format rules of the codec representation define syntax elements that control the use of a codec mode during conversion of two levels of the video, wherein the codec mode includes performing conversion of the video region using a reshaping process in which sample values are represented in a first domain and a second domain and a luminance-dependent chrominance residual scaling step.
[0712] 19. The method according to technical solution 18, wherein the syntax element includes a first indicator, which is at a higher level corresponding to a group of video regions including the video region.
[0713] 20. The method according to technical solution 19, wherein the first indicator is at a picture level and in a picture header, and the second indicator is at a slice level.
[0714] 21. A method according to any one of technical solutions 18-20, wherein the second indicator controls the use of the luminance-dependent chrominance residual scaling step independently of the first indicator.
[0715] 22. A method according to any one of technical solutions 18-20, wherein the format rule specifies a third indicator, which is located at a lower level than the first indicator and also indicates the applicability of the first indicator at the video area level.
[0716] 23. A method according to any one of technical solutions 18-22, wherein the video area includes a video strip.
[0717] 24. The method according to technical solution 18, wherein the video area includes multiple slices, and wherein the first indicator indicates whether the second indicator exists at the slice level.
[0718] 25. The method according to claim 18, wherein the video area comprises a plurality of slices, and wherein the absence of the first indicator indicates whether the second indicator is present at the slice level.
[0719] 26. A method according to any one of technical solutions 18-25, wherein the format rule stipulates that the syntax element exists conditionally depending on whether a flag for signaling the use of the syntax element is used in the encoded representation.
[0720] 27. A method according to any of the above technical solutions, wherein a syntax element in the codec representation indicates whether the method is used for conversion.
[0721] 28. A method according to any of the above technical solutions, wherein the conversion depends on whether the codec information meets conditions including color format or segmentation scheme or sample point positions used in the conversion.
[0722] 29. A method according to any one of technical solutions 1 to 28, wherein the conversion includes encoding the video into a codec representation.
[0723] 30. A method according to any one of technical solutions 1 to 28, wherein the conversion includes decoding the codec representation to generate pixel values of the video.
[0724] 31. A video decoding device, comprising a processor, which is configured to implement the method described in one or more of technical solutions 1 to 30.
[0725] 32. A video encoding device, comprising a processor, which is configured to implement the method described in one or more of technical solutions 1 to 30.
[0726] 33. A computer program product having computer code stored thereon, which, when executed by a processor, enables the processor to implement the method described in any one of Technical Solutions 1 to 30.
[0727] 34. The method, apparatus or system described in this document.
[0728] Fig.28 2 is a flowchart representation of a video processing method 2800 according to the present technology. The method 2800 includes, at operation 2810, for a conversion between a video region of a chroma component of a video and a bitstream of the video, determining a manner to apply a cross-component adaptive loop filtering (CC-ALF) operation to a first sample point of the chroma component based on a position of a second sample point associated with the first sample point. The method 2800 includes, at operation 2820, performing the conversion based on the determination.
[0729] In some embodiments, the method includes a specified filter strength for applying the CC-ALF operation. In some embodiments, the location of the second sample point includes a distance between the second sample point and a virtual boundary of the CC-ALF operation. In some embodiments, the first sample point is the same as the second sample point.
[0730] In some embodiments, the second sample corresponds to a luma sample associated with the first sample, and the location of the second sample includes a distance between the luma sample and a virtual boundary of the CC-ALF operation. In some embodiments, the first sample is located at (x, y) relative to the video area. The luma sample is located at (x×subWidthC, y×subHeightC), and (subWidthC, subHeightC) is determined based on the color format of the video. In some embodiments, the method includes applying the CC-ALF operation based on at least one of (y×subHeightC) or a difference between (y×subHeightC) and a virtual boundary of the CC-ALF operation.
[0731] In some embodiments, the manner in which the CC-ALF operation is applied to the first sample point is also based on the color format of the video. In some embodiments, the virtual boundary of the CC-ALF operation is the same as the virtual boundary of the adaptive loop filtering (ALF) operation. In some embodiments, the CC-ALF operation is represented as:
[0732] Among them, (x, y) represents the position of sample point i, (x C ,y C ) represents the position of the brightness sample point determined based on (x, y), c i (x0, y0) represents the filter coefficient, OS represents the rounding offset, and FS represents the filter strength. In some embodiments, the CC-ALF operation is represented as:
[0733] Among them, (x, y) represents the position of sample point i, (x C ,y C ) represents the position of the brightness sample point determined based on (x, y), c i (x0, y0) represents the filter coefficients, OS represents the rounding offset, FS represents the filter strength, and T0 and T1 represent the clipping boundaries. In some embodiments, T0 and T1 are variable values. In some embodiments, T0 and T1 are the same values used in nonlinear brightness adaptive linear filtering. In some embodiments, OS, FS, T0 and / or T1 are derived during conversion or included in the bitstream.
[0734] Fig.2929 is a flowchart representation of a video processing method 2900 according to the present technology. The method 2900 includes, at operation 2910, performing a conversion between a video region of a video and a bitstream of the video. Based on a syntax element related to an adaptive loop filtering (ALF) operation, one or more syntax elements related to the use of a cross-component adaptive loop filtering (CC-ALF) operation in the conversion are conditionally included in an adaptive parameter set. In some embodiments, the ALF operation includes a luma adaptive loop filtering. In some embodiments, the ALF operation includes a chroma adaptive loop filtering. In some embodiments, in the case where the syntax element related to the ALF operation is omitted in the bitstream, one or more syntax elements related to the use of the CC-ALF operation are inferred to be 0. In some embodiments, the bitstream conforms to a format rule that specifies that in the case where the syntax element related to the ALF operation is omitted in the bitstream, one or more syntax elements related to the use of the CC-ALF operation are set to 0. In some embodiments, one or more syntax elements related to the use of the CC-ALF operation are included in a slice header, a picture header, a sequence parameter set, a picture parameter set, or an adaptive parameter set. In some embodiments, one or more syntax elements related to the use of CC-ALF operations in a slice header are based on whether ALF operations are enabled for the same chroma components. In some embodiments, one or more syntax elements related to the use of CC-ALF operations for color components in a codec treeblock are conditionally included in the bitstream based on whether the ALF operations are applied to the codec treeblock and / or whether CC-ALF operations are enabled for a slice or picture that includes the codec treeblock.
[0735] Fig.30 3000 is a flowchart representation of a video processing method 3000 according to the present technology. The method 3000 includes, at operation 3010, performing conversion between a video region of a video and a bitstream of the video. A first syntax element is included in the bitstream indicating whether an adaptive loop filter (ALF) or a cross-component adaptive loop filter (CC-ALF) is applied during conversion of the video region.
[0736] In some embodiments, the video area includes a sequence, a slice, a slice, a sub-picture, a picture, a codec tree unit, a codec tree block, a codec unit, a picture unit, a transform unit, a codec block, a transform block, or a picture block. In some embodiments, a first syntax element is included in a sequence parameter set, a picture parameter set, a picture header, or a slice header. In some embodiments, the first syntax element indicates whether ALF or CC-ALF is applicable to any color component of the video. In some embodiments, the color component includes a chroma component. In some embodiments, the chroma component includes a Cr or Cb component. In some embodiments, the first syntax element indicates which color component ALF or CC-ALF is applicable to. In some embodiments, the first syntax element is not a binary value. In some embodiments, based on the first syntax element, one or more syntax elements are conditionally included in the bitstream of a second video area that is smaller than the video area.
[0737] Fig.31 is a flowchart representation of a video processing method 3100 according to the present technology. The method 3100 includes, at operation 3110, performing a conversion between a video region of a video and a codec representation of the video. Based on characteristics of an adaptive loop filtering (ALF) operation, one or more syntax elements related to an ALF operation of a chroma component are conditionally included in an adaptation parameter set.
[0738] In some embodiments, the characteristic includes whether ALF operations for luma components are included in the bitstream. In some embodiments, one or more syntax elements related to ALF operations are for color components in a codec treeblock, and the characteristic includes whether luma or chroma ALF operations are applied to the codec treeblock, and whether chroma ALF operations for color components are enabled for slices that include the codec treeblock.
[0739] Fig.32 is a flowchart representation of a video processing method 3200 according to the present technology. The method 3200 includes, at operation 3210, performing conversion between a video and a bitstream of the video. The bitstream conforms to a format rule that specifies that a two-level signaling mechanism is used to control the use of a luma mapping with chroma scaling (LMCS) codec mode, the two-level signaling mechanism including signaling at a first level for a first video region and at a second level for a second video region associated with the first video region, wherein the first level is higher than the second level. The LMCS codec mode includes performing conversion using a reshaping process and a luma-dependent chroma residual scaling (CRS) process, in which the reshaping process, sample values represented in the first domain are mapped to the reshaped domain.
[0740] In some embodiments, the two-level signaling mechanism includes signaling a first syntax element in a higher level, and signaling a second syntax element in a lower level based on the first syntax element. The first syntax element indicates whether the LMCS codec mode or processing associated with the LMCS codec mode is enabled for the first video region, and the format rule stipulates that the second syntax element is equal to 1 when the first syntax element is equal to 1. In some embodiments, the first syntax element indicates whether the reshaping process is enabled. In some embodiments, the first syntax element indicates whether the CRS process is enabled. In some embodiments, the first video region includes a picture header. In some embodiments, the second video region includes a slice.
[0741] In some embodiments, the first syntax element includes a first indicator that is conditionally included based on whether the LMCS codec mode is enabled at the sequence level. In some embodiments, the first syntax element includes a second indicator that is conditionally included based on whether the LMCS codec mode is enabled in a picture header, a color format of the video, a separate plane codec mode of the video, or whether a chroma tool is present or allowed. In some embodiments, the second syntax element includes a third indicator that is conditionally included in the bitstream based on the first indicator. In some embodiments, the second syntax element includes a fourth indicator that is conditionally included in the bitstream based on the second indicator and / or the third indicator.
[0742] In some embodiments, the two-level signaling mechanism includes signaling a first syntax element in a higher level, which indicates whether one or more syntax elements related to the LMCS codec mode exist in the first video area or the second video area. In some embodiments, when the first syntax element indicates that one or more syntax elements related to the LMCS codec mode exist in the second video area, the syntax elements related to the LMCS codec mode in the first video area are omitted in the bitstream. In some embodiments, when the first syntax element indicates that one or more syntax elements related to the LMCS codec mode exist in the first video area, the syntax elements related to the LMCS codec mode in the second video area are omitted in the bitstream.
[0743] Fig.33is a flowchart representation of a video processing method 3300 according to the present technology. The method 3300 includes, at operation 3310, performing conversion between a video region of a video and a bitstream of the video. The bitstream conforms to a format rule, which stipulates that in the case where a syntax flag indicates that the video region is partitioned using a quadtree partitioning method, the value of the syntax element indicating the partitioning mode is not equal to any of the following items: (1) horizontal ternary tree partitioning mode, (2) horizontal binary tree partitioning mode, (3) vertical ternary tree partitioning mode, or (4) vertical binary tree partitioning mode.
[0744] In some embodiments, whether the video region allows the binary tree partitioning method is determined based on syntax elements and syntax flags. In some embodiments, the conditions of the partitioning mode are determined based on syntax elements and syntax flags. In some embodiments, the cross-component chroma intra prediction mode is determined based on syntax elements and syntax flags.
[0745] In some embodiments, the method is applied at a sequence level, a picture level, a slice level or a slice group level, including a sequence header, a picture header, a sequence parameter set, a video parameter set, a picture parameter set, an adaptation parameter set, a slice header or a slice group header. In some embodiments, the method is applied based on the video codec information, the codec information including a color format, a segmentation method or a position of a sample relative to a codec unit or a codec tree unit.
[0746] In this document, the term "video processing" may refer to video encoding, video decoding, video compression, or video decompression. For example, a video compression algorithm may be applied during the conversion from a pixel representation of a video to a corresponding bitstream representation, or vice versa. For example, the bitstream representation of a current video block may correspond to bits spread across concatenated positions or different positions in the bitstream as defined by the syntax. For example, a macroblock may be encoded based on a transformed and encoded error residual value, and bits in a header and other fields in the bitstream may also be used.
[0747] The disclosed and other technical solutions, examples, embodiments, modules and functional operations described herein can be implemented in digital electronic circuits or computer software, firmware or hardware, including the structures disclosed herein and their structural equivalents, or a combination of one or more thereof. The disclosed embodiments and other embodiments can be implemented as one or more computer program products, i.e., one or more modules of computer program instructions encoded on a computer-readable medium for a data processing device to execute or control its operation. The computer-readable medium can be a combination of a machine-readable storage device, a machine-readable storage substrate, a storage device, a material that affects a machine-readable propagation signal, or one or more combinations thereof. The term "data processing device" includes all devices, equipment and machines for processing data, including, for example, a programmable processor, a computer or multiple processors or computers. In addition to hardware, the device may also include code that creates an execution environment for a computer program, for example, code that constitutes a processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more thereof. The propagated signal is an artificially generated signal, for example, a machine-generated electrical, optical or electromagnetic signal, which is generated to encode information for transmission to a suitable receiver device.
[0748] A computer program (also referred to as a program, software, software application, script, or code) may be written in any form of programming language (including compiled or interpreted languages) and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. A program may be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language file), in a single file dedicated to the program, or in multiple coordinated files (e.g., files that store one or more modules, subroutines, or portions of code). A computer program may be deployed for execution on one or more computers that are located at one site or distributed across multiple sites and interconnected by a communications network.
[0749] The processes and logic flows described in this document may be performed by one or more programmable processors executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows may also be performed by, and the apparatus may also be implemented as, special purpose logic circuits, such as an FPGA (field programmable gate array) or an ASIC (application specific integrated circuit).
[0750] For example, processors suitable for executing computer programs include general and special purpose microprocessors, and any one or more of any type of digital computer. Typically, the processor will receive instructions and data from a read-only memory or a random access memory or both. The basic elements of a computer are a processor that executes instructions and one or more storage devices that store instructions and data. Typically, a computer will also include one or more mass storage devices for storing data, such as magnetic disks, magneto-optical disks, or optical disks, or be operatively coupled to one or more mass storage devices to receive data from them or transfer data to one or more mass storage devices, or both. However, a computer does not necessarily have such a device. Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, including, for example, semiconductor memory devices, such as EPROM, EEPROM, and flash memory devices; magnetic disks, such as hard disks within a frame or removable hard disks; magneto-optical disks; and CD ROM and DVD ROM disks. The processor and memory may be supplemented by, or incorporated into, dedicated logic circuits.
[0751] Although this patent document contains many details, they should not be interpreted as limitations on any subject matter or the scope of the claims, but rather as descriptions of features of specific embodiments of specific technologies. Certain features described in this patent document in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various functions described in the context of a single embodiment can also be implemented separately in multiple embodiments, or in any suitable subcombination. In addition, although the above-mentioned features may be described as working in certain combinations, or even initially required to be so, in some cases, one or more features in the claim combination can be removed from the combination, and the combination of claims can be directed to subcombinations or variations of subcombinations.
[0752] Likewise, although operations are described in a particular order in the drawings, this should not be understood as requiring that such operations be performed in the particular order or order shown, or that all illustrated operations be performed, in order to achieve the desired results. In addition, the separation of various system components in the embodiments of this patent document should not be understood as requiring such separation in all embodiments.
[0753] Only a few implementations and examples are described, and other implementations, enhancements, and variations can be made based on what is described and illustrated in this patent document.
Claims
1. A method for processing video data, comprising: For a conversion between a video region of a chroma component of a video and a bitstream of the video, determining a filter strength in a cross-component loop filtering operation for a first sample of the video region based on a first distance between the first sample and a chroma virtual boundary and a second distance between a corresponding luma sample of the first sample and a luma virtual boundary; as well as performing said converting based on said filter strength, In the cross-component loop filtering operation, the cross-component filtering value of the first sample is derived using the reconstructed luma picture sample before the luma adaptive loop filtering process and the filtered reconstructed chroma picture sample after the chroma adaptive loop filtering process of the first sample, and the filter strength is applied to the reconstructed luma picture sample.
2. The method according to claim 1, wherein: The first sample point is located at (x, y) relative to the upper left sample point of the video area, and the corresponding luminance sample point of the first sample point is located at (x×subWidthC, y×subHeightC), and wherein subWidthC and subHeightC are chroma subsampling rates in horizontal and vertical directions, and the chroma subsampling rate is determined based on the color format of the video.
3. The method according to claim 1, wherein: The video area satisfies that a bottom boundary of the video area is not a bottom boundary of a current picture including the video area, or a difference between a height of the current picture in units of luma samples and a vertical position of a luma sample corresponding to an upper left sample of the video area is greater than a size of the codec tree block in units of the luma samples minus 4.
4. The method according to claim 2, wherein: The cross-component loop filtering operation has a 4×3 filter shape, -S, T, S, R are the y-coordinates of four sample rows in the 4×3 filter shape, wherein -S=-1, T=0, S=1, R=2, and the corresponding luma sample of the first sample is in the T=0 row.
5. The method according to claim 4, wherein: When the vertical position of the corresponding luma sample of the first sample is in CtbSizeY-3 or CtbSizeY-6 of the codec tree block, the luma sample (x, S) in the row at R is filled with the luma sample (x, R) in the row at S, where CtbSizeY is the size of the codec tree block in units of the luma samples.
6. The method according to claim 4, wherein: When the vertical position of the corresponding luma sample of the first sample point is CtbSizeY-4 or CtbSizeY-5, the luma sample (x, -S) in the row at -S is filled with the luma sample (x, T) in the row of the corresponding luma sample of the first sample point, the luma sample (x, S) in the row at S is filled with the luma sample (x, T) in the row of the corresponding luma sample of the first sample point, and the luma sample (x, R) in the row at R is filled with the luma sample (x, T) in the row of the corresponding luma sample of the first sample point, where CtbSizeY is the size of the codec tree block in units of the luma samples.
7. The method according to claim 1, wherein: The converting includes decoding the video from the bitstream.
8. The method according to claim 1, wherein: The converting includes encoding the video into the bitstream.
9. A device for processing video data, comprising a processor and a non-volatile memory with instructions thereon, wherein: The instructions, when executed by the processor, cause the processor to: For a conversion between a video region of a chroma component of a video and a bitstream of the video, determining a filter strength in a cross-component loop filtering operation for a first sample of the video region based on a first distance between the first sample and a chroma virtual boundary and a second distance between a corresponding luma sample of the first sample and a luma virtual boundary; as well as said performing conversion based on said filter strength, In the cross-component loop filtering operation, the cross-component filtering value of the first sample is derived using the reconstructed luma picture sample before the luma adaptive loop filtering process and the filtered reconstructed chroma picture sample after the chroma adaptive loop filtering process of the first sample, and the filter strength is applied to the reconstructed luma picture sample.
10. The device according to claim 9, wherein: The first sample point is located at (x, y) relative to the upper left sample point of the video area, and the corresponding luminance sample point of the first sample point is located at (x×subWidthC, y×subHeightC), and wherein subWidthC and subHeightC are chroma subsampling rates in horizontal and vertical directions, and the chroma subsampling rate is determined based on the color format of the video.
11. The device according to claim 9, wherein: The video area satisfies that a bottom boundary of the video area is not a bottom boundary of a current picture including the video area, or a difference between a height of the current picture in units of luma samples and a vertical position of a luma sample corresponding to an upper left sample of the video area is greater than a size of the codec tree block in units of luma samples minus 4.
12. The device according to claim 10, wherein: The cross-component loop filtering operation has a 4×3 filter shape, −S, T, S, R are y-coordinates of four sample rows in the 4×3 filter shape, wherein −S=−1, T=0, S=1, R=2, and the corresponding luma sample of the first sample is in the T=0 row; wherein, when the vertical position of the corresponding luma sample of the first sample is within CtbSizeY-3 or CtbSizeY-6 of the codec tree block, the luma sample (x, S) in the row at R is filled with the luma sample (x, R) in the row at S, wherein CtbSizeY is the size of the codec tree block in units of luma samples; or Wherein, when the vertical position of the corresponding luma sample of the first sample point is CtbSizeY-4 or CtbSizeY-5, the luma sample (x, -S) in the row at -S is filled with the luma sample (x, T) in the row at S, the luma sample (x, S) in the row at S is filled with the luma sample (x, T) in the row at S, and the luma sample (x, T) in the row at R is filled with the luma sample (x, R) in the row at R, where CtbSizeY is the size of the codec tree block in units of luma samples.
13. A non-transitory computer-readable storage medium storing instructions that cause a processor to: For conversion between a video region of a chroma component of a video and a bitstream of the video, determining a filter strength in a cross-component loop filtering operation for a first sample of the video region based on a first distance between the first sample and a chroma virtual boundary and a second distance between a corresponding luma sample of the first sample and a luma virtual boundary; and performing said converting based on said filter strength, in, In the cross-component loop filtering operation, a cross-component filtering value of the first sample is derived using a reconstructed luma picture sample before luma adaptive loop filtering and a filtered reconstructed chroma picture sample after chroma adaptive loop filtering of the first sample, and the filter strength is applied to the reconstructed luma picture sample.
14. The non-transitory computer-readable storage medium of claim 13, wherein: The first sample point is located at (x, y) relative to the upper left sample point of the video area, and the corresponding luminance sample point of the first sample point is located at (x×subWidthC, y×subHeightC), and wherein subWidthC and subHeightC are chroma subsampling rates in horizontal and vertical directions, and the chroma subsampling rate is determined based on the color format of the video.
15. The non-transitory computer-readable storage medium of claim 13, wherein: The video area satisfies that a bottom boundary of the video area is not a bottom boundary of a current picture including the video area, or a difference between a height of the current picture in units of luma samples and a vertical position of a luma sample corresponding to an upper left sample of the video area is greater than a size of the codec tree block in units of luma samples minus 4.
16. The non-transitory computer-readable storage medium of claim 14, wherein: The cross-component loop filtering operation has a 4×3 filter shape, −S, T, S, R are y-coordinates of four sample rows in the 4×3 filter shape, wherein −S=−1, T=0, S=1, R=2, and the corresponding luma sample of the first sample is in the T=0 row; wherein, when the vertical position of the corresponding luma sample of the first sample is within CtbSizeY-3 or CtbSizeY-6 of the codec tree block, the luma sample (x, S) in the row at R is filled with the luma sample (x, R) in the row at S, wherein CtbSizeY is the size of the codec tree block in units of luma samples; or Wherein, when the vertical position of the corresponding luma sample of the first sample point is CtbSizeY-4 or CtbSizeY-5, the luma sample (x, -S) in the row at -S is filled with the luma sample (x, T) in the row at S, the luma sample (x, S) in the row at S is filled with the luma sample (x, T) in the row at S, and the luma sample (x, T) in the row at R is filled with the luma sample (x, R) in the row at R, where CtbSizeY is the size of the codec tree block in units of luma samples.
17. A method for storing a bitstream of a video, comprising: For a video region of a chroma component of a video, determining a filter strength in a cross-component loop filtering operation for a first sample of the video region based on a first distance between the first sample and a chroma virtual boundary and a second distance between a corresponding luma sample of the first sample and a luma virtual boundary; generating the bitstream based on the filter strength; as well as storing the bitstream in a non-transitory computer-readable recording medium, In the cross-component loop filtering operation, the cross-component filtering value of the first sample is derived using the reconstructed luma picture sample before the luma adaptive loop filtering process and the filtered reconstructed chroma picture sample after the chroma adaptive loop filtering process of the first sample, and the filter strength is applied to the reconstructed luma picture sample.
18. The method for storing a bit stream of a video according to claim 17, wherein: The first sample point is located at (x, y) relative to the upper left sample point of the video area, and the corresponding luminance sample point of the first sample point is located at (x×subWidthC, y×subHeightC), and wherein subWidthC and subHeightC are chroma subsampling rates in horizontal and vertical directions, and the chroma subsampling rate is determined based on the color format of the video.
19. The method for storing a bit stream of a video according to claim 17, wherein: The video area satisfies that a bottom boundary of the video area is not a bottom boundary of a current picture including the video area, or a difference between a height of the current picture in units of luma samples and a vertical position of a luma sample corresponding to an upper left sample of the video area is greater than a size of the codec tree block in units of luma samples minus 4.
20. The method for storing a bit stream of a video according to claim 18, wherein: The cross-component loop filtering operation has a 4×3 filter shape, −S, T, S, R are y-coordinates of four sample rows in the 4×3 filter shape, wherein −S=−1, T=0, S=1, R=2, and the corresponding luma sample of the first sample is in the T=0 row; wherein, when the vertical position of the corresponding luma sample of the first sample is within CtbSizeY-3 or CtbSizeY-6 of the codec tree block, the luma sample (x, S) in the row at R is filled with the luma sample (x, R) in the row at S, wherein CtbSizeY is the size of the codec tree block in units of luma samples; or Wherein, when the vertical position of the corresponding luma sample of the first sample point is CtbSizeY-4 or CtbSizeY-5, the luma sample (x, -S) in the row at -S is filled with the luma sample (x, T) in the row at S, the luma sample (x, S) in the row at S is filled with the luma sample (x, T) in the row at S, and the luma sample (x, T) in the row at R is filled with the luma sample (x, R) in the row at R, where CtbSizeY is the size of the codec tree block in units of luma samples.
Citation Information
Patent Citations
Zero cell filtering
CN110662035A
Handling face discontinuities in 360-degree video coding
WO2019060443A1
