Filling of unavailable sample points in adaptive loop filtering

Abstract

A method of video processing includes: for a conversion between a current block of video and a bitstream representation of the video using a codec that accesses samples outside of a current block, determining that neighboring samples for the conversion are unavailable. Samples located at a boundary of a video unit that spans the video are considered unavailable samples. The method also includes performing the conversion by replacing the unavailable neighboring samples with padding samples based on the determination. The padding samples are determined using samples that are confined within a current processing unit associated with the current block.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to and the benefit of International Patent Application No. PCT / CN2019 / 110489, filed on October 10, 2019, International Patent Application No. PCT / CN2019 / 110681, filed on October 11, 2019, and International Patent Application No. PCT / CN2019 / 111114, filed on October 14, 2019, in a timely manner, pursuant to applicable patent law and / or the rules applicable to the Paris Convention. The entire disclosures of the above applications are incorporated herein by reference as a part of the disclosure of this application. Technical Field

[0003] This patent document generally relates to video encoding and decoding techniques. Background Art

[0004] Video codec standards have evolved primarily through the development of the well-known ITU-T and ISO / IEC standards. ITU-T produced H.261 and H.263, ISO / IEC produced MPEG-1 and MPEG-4 Visual, and the two organizations worked together to produce the H.262 / MPEG-2 Video, H.264 / MPEG-4 Enhanced Video Codec (AVC), and H.265 / High Efficiency Video Codec (HEVC) standards. Starting with H.262, video codec standards have been based on a hybrid video codec architecture that utilizes temporal prediction and transform codecs. To explore future video codec technologies beyond HEVC, VCEG and MPEG jointly established the Joint Video Exploration Team (JVET) in 2015. Since then, JVET has adopted many new methods and incorporated them into reference software called the Joint Exploration Model (JEM). In April 2018, the JVET between VCEG (Q6 / 16) and ISO / IEC JTC1 SC29 / WG11 (MPEG) was established to work on the development of the next-generation Versatile Video Codec (VVC) standard, with the goal of reducing the bit rate by 50% compared to HEVC. Summary of the Invention

[0005] Using the disclosed video encoding, transcoding, or decoding techniques, embodiments of a video encoder or decoder are able to handle virtual boundaries of codec treeblocks to provide better compression efficiency and simpler implementation of encoding or decoding tools.

[0006] In one example aspect, a method of video processing is disclosed. The method includes determining, for a conversion between a current block of video and a bitstream representation of the video using a codec that accesses samples outside of the current block, that neighboring samples for the conversion are unavailable. Samples located at a boundary across a video unit of the video are considered unavailable samples. The method also includes performing the conversion by replacing the unavailable neighboring samples with padding samples based on the determination. The padding samples are determined using samples confined to a current processing unit associated with the current block.

[0007] In another example aspect, a method of video processing is disclosed. The method includes, for converting between a current block of video and a bitstream representation of the video using a codec that accesses samples outside the current block, determining whether neighboring samples of the current block are in the same video unit as the current block. The method also includes performing the conversion based on the determination.

[0008] In another example aspect, a method of video processing is disclosed. The method includes performing a conversion between a current block of video and a bitstream representation of the video using a codec that accesses samples outside the current block. During the conversion, the availability of neighboring samples in an upper left region, an upper right region, a lower left region, or a lower right region of the current block is determined independently of samples in an upper neighboring region, a left neighboring region, a right neighboring region, or a lower neighboring region of the current block. Samples located at a boundary that spans video units of the video are considered unavailable samples.

[0009] In another example aspect, a method of video processing is disclosed. The method includes performing a conversion between a current processing unit of video and a bitstream representation of the video using a codec that accesses samples outside the current processing unit. During the conversion, unavailable neighboring samples of the current processing unit are filled in a predefined order, wherein samples located at a boundary across video units of the video are considered unavailable samples.

[0010] In another example aspect, a method of video processing is disclosed. The method includes performing conversion between video blocks of a video picture and a bitstream representation thereof. Here, the video blocks are processed using logical groupings of codec tree blocks, and the codec tree blocks are processed based on whether a bottom boundary of a bottom codec tree block is outside a bottom boundary of the video picture.

[0011] In another example aspect, another video processing method is disclosed, the method comprising determining a usage state of virtual samples during loop filtering based on a condition of a codec treeblock of a current video block, and performing a conversion between the video block and a bitstream representation of the video block consistent with the usage state of the virtual samples.

[0012] In yet another example aspect, another video processing method is disclosed that includes, during conversion between a video picture logically grouped into one or more video slices or video bricks and a bitstream representation of the video picture, determining to disable use of samples in another slice or brick in adaptive loop filtering processing, and performing the conversion consistent with the determination.

[0013] In yet another example aspect, another video processing method is disclosed. The method includes, during conversion between a current video block of a video picture and a bitstream representation of the current video block, determining that the current video block includes samples located at a boundary of a video unit of the video picture, and performing the conversion based on the determination, wherein performing the conversion includes generating virtual samples for loop filtering processing using a unified method that is the same for all boundary types in the video picture.

[0014] In yet another example aspect, another method of video processing is disclosed. The method includes, during a transition between a current video block of a video picture and a bitstream representation thereof, determining to apply one of a plurality of adaptive loop filter (ALF) sample selection methods available for the video picture during the transition, and performing the transition by applying the one of the plurality of ALF sample selection methods.

[0015] In yet another example aspect, another method of video processing is disclosed, the method comprising: performing a loop filtering operation on samples of the current video block of a video picture based on a boundary rule during conversion between a current video block and a bitstream representation of the current video block, wherein the boundary rule prohibits using samples of a virtual pipeline data unit (VPDU) that spans the video picture, and performing the conversion using a result of the loop filtering operation.

[0016] In yet another example aspect, another method of video processing is disclosed, the method comprising: performing a loop filtering operation on samples of the current video block of a video picture based on a boundary rule during conversion between a current video block and a bitstream representation of the current video block, wherein the boundary rule specifies, for a position of the current video block that crosses a video unit boundary, using samples generated without using padding, and performing the conversion using a result of the loop filtering operation.

[0017] In yet another example aspect, another method of video processing is disclosed, the method comprising: performing a loop filtering operation on samples of a current video block of a video picture during a conversion between the current video block and a bitstream representation of the current video block based on a boundary rule, wherein the boundary rule specifies selecting a filter for the loop filtering operation, the filter having dimensions such that samples of the current video block used during the loop filtering do not cross a boundary of a video unit of the video picture, and performing the conversion using a result of the loop filtering operation.

[0018] In another example aspect, another video processing method is disclosed. The method includes: performing a loop filtering operation on samples of a current video block of a video picture during a conversion between the current video block and a bitstream representation of the current video block based on a boundary rule; wherein the boundary rule specifies selecting a clipping parameter or a filter coefficient for the loop filtering operation based on whether sample padding is required for loop filtering, and performing the conversion using a result of the loop filtering operation.

[0019] In another example aspect, another video processing method is disclosed. The method includes: performing a loop filtering operation on samples of a current video block of a video picture during a conversion between the current video block and a bitstream representation of the current video block based on a boundary rule; wherein the boundary rule depends on a color component identifier of the current video block, and performing the conversion using a result of the loop filtering operation.

[0020] In another example aspect, a video encoding apparatus configured to perform the above method is disclosed.

[0021] In another example aspect, a video decoder configured to perform the above method is disclosed.

[0022] In another example aspect, a computer-readable medium is disclosed. The medium stores code that, when executed, causes a processor to implement one or more of the above methods.

[0023] The above and other aspects and features of the disclosed technology are described in more detail in the drawings, the specification, and the claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 An example of a picture having 18 by 12 luminance codec tree units (CTUs) is shown, which is partitioned into 12 slices and 3 raster scan stripes.

[0025] Figure 2 An example of a picture having 18 by 12 luminance CTUs is shown, which is partitioned into 24 slices and 9 rectangular stripes.

[0026] Figure 3 An example of a picture partitioned into 4 slices, 11 bricks, and 4 rectangular stripes is shown.

[0027] Figure 4A An example of a codec tree block (CTB) across a picture boundary when K = M and L < N is shown.

[0028] Figure 4B An example of a codec tree block (CTB) across a picture boundary when K < M and L = N is shown.

[0029] Figure 4CShows an example of a coding / decoding tree block (CTB) that spans a picture boundary when K < M and L < N.

[0030] Figure 5 Shows an example of an encoder block diagram.

[0031] Figure 6 Is an illustration of picture samples on an 8×8 grid where parallel deblocking is possible, as well as horizontal and vertical block boundaries and non - overlapping blocks of 8×8 samples.

[0032] Figure 7 Shows an example of pixels involved in filter on / off decision and strong / weak filter selection.

[0033] Figure 8 Shows four 1 - D directional patterns.

[0034] Figure 9 Shows examples of geometric adaptive loop filter (GALF) filter shapes (left: 5×5 rhombus, middle: 7×7 rhombus, right: 9×9 rhombus).

[0035] Figure 10 Shows the relative coordinates for 5×5 rhombus filter support.

[0036] Figure 11 Shows an example of the relative coordinates for 5×5 rhombus filter support.

[0037] Figure 12A Shows an example arrangement for Laplacian calculation for subsampling.

[0038] Figure 12B Shows another example arrangement for Laplacian calculation for subsampling.

[0039] Figure 12C Shows another example arrangement for Laplacian calculation for subsampling.

[0040] Figure 12D Shows another example arrangement for Laplacian calculation for subsampling.

[0041] Figure 13 Shows an example of the loop filter row buffer requirements for the luminance component in VTM - 4.0.

[0042] Figure 14 Shows an example of the loop filter row buffer requirements for the chrominance component in VTM - 4.0.

[0043] Figure 15A Shows an example of ALF block classification at the virtual boundary when N = 4.

[0044] Figure 15BAnother example of ALF block classification at a virtual boundary when N=4 is shown.

[0045] Figure 16A An example of modified luma ALF filtering at a virtual boundary is shown.

[0046] Figure 16B Another example of modified luma ALF filtering at a virtual boundary is shown.

[0047] Figure 16C Yet another example of modified luma ALF filtering at a virtual boundary is shown.

[0048] Figure 17A An example of modified chroma ALF filtering at a virtual boundary is shown.

[0049] Figure 17B Another example of modified chroma ALF filtering at a virtual boundary is shown.

[0050] Figure 18A An example of horizontal surround motion compensation is shown.

[0051] Figure 18B Another example of horizontal surround motion compensation is shown.

[0052] Figure 19 An example of a modified adaptive loop filter is shown.

[0053] Figure 20 An example of processing CTUs in a video picture is shown.

[0054] Figure 21 An example of a modified adaptive loop filter boundary is shown.

[0055] Figure 22 is a block diagram of an example of a video processing device.

[0056] Figure 23 is a flow chart of an example method for video processing.

[0057] Figure 24 An example of a HEC picture of a 3×2 layout is shown.

[0058] Figure 25 Examples of the number of fill lines for sample points of two boundaries are shown.

[0059] Figure 26 An example of processing of CTUs in a picture is shown.

[0060] Figure 27 Another example of processing of CTUs in a picture is shown.

[0061] Figure 28 Another example of the current sample point and the sample points that need to be accessed is shown.

[0062] Figure 29 Another example of filling in “unavailable” adjacent sample points is shown.

[0063] Figure 30 An example of samples that need to be utilized in the ALF classification process is shown.

[0064] Figure 31 An example of "unavailable" point filling is shown.

[0065] Figure 32A Shows the example of using the samples in the current processing unit to fill the "unavailable" samples.

[0066] Figure 32B Another example of filling “unavailable” samples using samples in the current processing unit is shown.

[0067] Figure 32C Another example of filling “unavailable” samples using samples in the current processing unit is shown.

[0068] Figure 32D Another example of using samples in the current processing unit to fill in “unavailable” samples is shown.

[0069] Figure 33A An example of using samples within the current processing unit to fill the "unavailable" samples of the current processing unit is shown.

[0070] Figure 33B Another example of using samples within the current processing unit to fill the "unavailable" samples of the current processing unit is shown.

[0071] Figure 33C Another example of using samples within the current processing unit to fill the "unavailable" samples of the current processing unit is shown.

[0072] Figure 33D Another example of using samples within the current processing unit to fill the “unavailable” samples of the current processing unit is shown.

[0073] Figure 34 An example of using samples within the current processing unit to fill the "unavailable" samples of the current processing unit is shown.

[0074] Figure 35 is a block diagram of an example video processing system in which the disclosed technology may be implemented.

[0075] Figure 36 is a block diagram illustrating an example video encoding and decoding system.

[0076] Figure 37 is a block diagram illustrating an encoder according to some embodiments of the present disclosure.

[0077] Figure 38 is a block diagram illustrating a decoder according to some embodiments of the present disclosure.

[0078] Figure 39 is a flow chart representing a method for video processing according to the present technology.

[0079] Figure 40 is a flowchart representing another method for video processing according to the present technology.

[0080] Figure 41 is a flowchart representing another method for video processing according to the present technology.

[0081] Figure 42 is a flowchart representing yet another method for video processing according to the present technology. DETAILED DESCRIPTION

[0082] The section headings used in this document are for ease of understanding and do not limit the embodiments disclosed in that section to only that section. In addition, although certain embodiments are described with reference to general video codecs or other specific video codecs, the disclosed techniques are also applicable to other video codec technologies. In addition, although some embodiments describe video encoding and decoding steps in detail, it will be understood that the corresponding steps of undoing the encoding will be implemented by the decoder. In addition, the term video processing covers video encoding or compression, video decoding or decompression, and video transcoding, wherein video pixels are represented from one compression format to another compression format or at a different compression bit rate.

[0083] 1. Brief Summary

[0084] This document relates to video coding techniques. Specifically, it relates to picture / slice / tile / brick boundary and virtual boundary coding, especially for nonlinear adaptive loop filters. It can be applied to existing video codec standards (such as HEVC) or to the finalized standard (Universal Video Codec). It may also be applicable to future video codec standards or video codecs.

[0085] 2. Preliminary Discussion

[0086] Video codec standards have evolved primarily through the development of the well-known ITU-T and ISO / IEC standards. ITU-T produced H.261 and H.263, ISO / IEC produced MPEG-1 and MPEG-4 Visual, and the two organizations worked together to produce the H.262 / MPEG-2 Video, H.264 / MPEG-4 Enhanced Video Codec (AVC), and H.265 / HEVC standards. Starting with H.262, video codec standards have been based on a hybrid video codec architecture that utilizes temporal prediction and transform codecs. To explore future video codec technologies beyond HEVC, VCEG and MPEG jointly established the Joint Video Exploration Team (JVET) in 2015. Since then, JVET has adopted many new methods and incorporated them into reference software called the Joint Exploration Model (JEM). In April 2018, the JVET (Joint Video Exploration Team) was established between VCEG (Q6 / 16) and ISO / IEC JTC1SC29 / WG11 (MPEG) to work on the Versatile Video Codec (VVC) standard with the goal of reducing bitrate by 50% compared to HEVC.

[0087] 2.1 Color Space and Chroma Subsampling

[0088] A color space, also called a color model (or color system), is an abstract mathematical model that describes the range of colors simply as a tuple of numbers, typically 3 or 4 values ​​or color components (e.g., RGB). Basically, a color space is a detailed description of a coordinate system and subspaces.

[0089] For video compression, the most commonly used color spaces are YCbCr and RGB.

[0090] YCbCr, Y'CbCr, or Y Pb / Cb Pr / Cr, also known as YCBCR or Y'CBCR, is a family of color spaces used as part of the color picture pipeline in video and digital photography systems. Y' is the luma component, while CB and CR are the blue-difference and red-difference chroma components. Y' (with the primaries) is distinguished from Y (luminance) to indicate that light intensity is nonlinearly encoded based on the gamma-corrected RGB primaries.

[0091] Chroma subsampling is the practice of encoding an image at a lower resolution for chroma information than for luminance information, taking advantage of the human visual system's lower sensitivity to color differences compared to luminance.

[0092] 2.1.1 Color Format 4:4:4

[0093] Each of the three Y'CbCr components has the same sampling rate, so there is no chroma subsampling. This scheme is sometimes used in high-end film scanners and film post-production.

[0094] 2.1.2 Color Format 4:2:2

[0095] The two chroma components are sampled at half the sampling rate of luma (half the horizontal chroma resolution). This reduces the bandwidth of uncompressed video signaling by one-third with almost no visual difference.

[0096] 2.1.3 Color Format 4:2:0

[0097] In 4:2:0, horizontal sampling is doubled compared to 4:1:1, but since the Cb and Cr channels are only sampled on alternate lines in this scheme, the vertical resolution is reduced by half. Therefore, the data rate remains the same. Cb and Cr are subsampled by a factor of 2 horizontally and vertically, respectively. There are three variants of the 4:2:0 scheme, with different horizontal and vertical positions.

[0098] In MPEG-2, Cb and Cr coexist horizontally. Cb and Cr are located between pixels vertically (with gaps).

[0099] In JPEG / JFIF, H.261, and MPEG-1, Cb and Cr are intermittently located between alternating luminance samples.

[0100] In 4:2:0 DV, Cb and Cr coexist horizontally and vertically on alternate lines.

[0101] 2.2 Various video units

[0102] A picture is divided into one or more slice rows and one or more slice columns. A slice is a sequence of CTUs covering a rectangular area of ​​a picture.

[0103] A slice is divided into one or more bricks, and each brick consists of multiple CTU rows within the slice.

[0104] A slice that is not split into multiple bricks is also called a brick. However, a brick that is a true subset of a slice is not called a slice.

[0105] A strip contains multiple slices of an image or multiple tiles of a slice.

[0106] Two striping modes are supported: raster scan striping and rectangular striping. In raster scan striping, a strip consists of a sequence of slices from a raster scan of the image. In rectangular striping, a strip consists of multiple tiles of the image, which together form a rectangular region of the image. The tiles in a rectangular strip are arranged in the strip's tile raster scan order.

[0107] Figure 1 An example of raster scan stripe partitioning of a picture is shown, where the picture is divided into 12 slices and 3 raster scan stripes.

[0108] Figure 2 An example of rectangular strip partitioning of a picture is shown, where the picture is divided into 24 slices (6 slice columns and 4 slice rows) and 9 rectangular strips.

[0109] Figure 3 An example of a picture divided into slices, tiles, and rectangular strips is shown, where the picture is divided into 4 slices (2 slice columns and 2 slice rows), 11 tiles (top left slice contains 1 tile, top right slice contains 5 tiles, bottom left slice contains 2 tiles, and bottom right slice contains 3 tiles), and 4 rectangular strips.

[0110] 2.2.1 CTU / CTB Size

[0111] In VVC, the CTU size signaled in SPS through the syntax element log2_ctu_size_minus2 can be as small as 4×4.

[0112] 7.3.2.3 Sequence Parameter Set RBSP Syntax

[0113]

[0114]

[0115] log2_ctu_size_minus2 plus 2 specifies the luma codec tree block size for each CTU.

[0116] log2_min_luma_coding_block_size_minus2 plus 2 specifies the minimum luma coding block size.

[0117] The variables CtbLog2SizeY, CtbSizeY, MinCbLog2SizeY, MinCbSizeY, MinTbLog2SizeY, MaxTbLog2SizeY, MinTbSizeY, MaxTbSizeY, PicWidthInCtbsY, PicHeightInCtbsY, PicSizeInCtbsY, PicWidthInMinCbsY, PicHeightInMinCbsY, PicSizeInMinCbsY, PicSizeInSamplesY, PicWidthInSamplesC, and PicHeightInSamplesC are derived as follows:

[0118] CtbLog2SizeY=log2_ctu_size_minus2+2 (7-9)

[0119] CtbSizeY=1<<CtbLog2SizeY (7-10)

[0120] MinCbLog2SizeY=log2_min_luma_coding_block_size_minus2+2(7-11)

[0121] MinCbSizeY=1<<MinCbLog2SizeY (7-12)

[0122] MinTbLog2SizeY=2 (7-13)

[0123] MaxTbLog2SizeY=6 (7-14)

[0124] MinTbSizeY=1<<MinTbLog2SizeY (7-15)

[0125] MaxTbSizeY=1<<MaxTbLog2SizeY (7-16)

[0126] PicWidthInCtbsY=Ceil(pic_width_in_luma_samples÷CtbSizeY) (7-17)

[0127] PicHeightInCtbsY=Ceil(pic_height_in_luma_samples÷CtbSizeY) (7-18)

[0128] PicSizeInCtbsY=PicWidthInCtbsY*PicHeightInCtbsY (7-19)

[0129] PicWidthInMinCbsY=pic_width_in_luma_samples / MinCbSizeY (7-20)

[0130] PicHeightInMinCbsY=pic_height_in_luma_samples / MinCbSizeY (7-21)

[0131] PicSizeInMinCbsY = PicWidthInMinCbsY * PicHeightInMinCbsY (7-22)

[0132] PicSizeInSamplesY = pic_width_in_luma_samples * pic_height_in_luma_samples (7-23)

[0133] PicWidthInSamplesC = pic_width_in_luma_samples / SubWidthC (7-24)

[0134] PicHeightInSamplesC = pic_height_in_luma_samples / SubHeightC (7-25)

[0135] 2.2.2 CTUs in the Picture

[0136] Assume the CTB / LCU size represented by M×N (typically, M equals N, as defined in HEVC / VVC), and for a CTB located on the boundary of a picture (or slice or strip or other type, taking the picture boundary as an example), K×L samples are within the picture boundary, where K < M or L < N. For those CTBs as Figures 4A-4C shown, the size of the CTB still equals M×N, however, the bottom boundary / right boundary of the CTB is outside the picture.

[0137] Figure 4A CTBs crossing the bottom picture boundary are shown. Figure 4B CTBs crossing the right picture boundary are shown. Figure 4C CTBs crossing the bottom-right picture boundary are shown.

[0138] Figures 4A-4C Examples of CTBs crossing the picture boundary are shown, (a) K = M, L < N; (b) K < M, L = N; (c) K < M, L < N

[0139] 2.3 Encoding and Decoding Processes of a Typical Video Codec

[0140] Figure 5An example of a VVC encoder block diagram is shown, which contains three loop filtering blocks: deblocking filter (DF), sample adaptive offset (SAO), and ALF (adaptive loop filter). Unlike DF, which uses a predefined filter, SAO and ALF use the original samples of the current picture to reduce the mean square error between the original samples and the reconstructed samples by adding an offset and applying a finite impulse response (FIR) filter, respectively. The offset and filter coefficients are signaled by coded side information. ALF is located at the last processing stage of each picture and can be seen as a tool that attempts to capture and repair artifacts created by previous stages.

[0141] 2.4 Deblocking Filter (DB)

[0142] The input to DB is the reconstructed samples before the loop filter.

[0143] First, vertical edges in the image are filtered. Then, horizontal edges in the image and the samples modified by the vertical edge filtering process are used as input for filtering. Vertical and horizontal edges in the CTBs of each CTU are processed separately on a per-codec unit basis. Vertical edges of codec blocks in a codec unit are filtered starting from the edge on the left-hand side of the codec block and moving through the edges in geometric order toward the right-hand side of the codec block. Horizontal edges of codec blocks in a codec unit are filtered starting from the edge at the top of the codec block and moving through the edges in geometric order toward the bottom of the codec block.

[0144] Figure 6 is an illustration of picture samples on an 8x8 grid that can be deblocked in parallel, along with horizontal and vertical block boundaries and non-overlapping blocks of 8x8 samples.

[0145] 2.4.1 Boundary Decision

[0146] The filter is applied on 8x8 block boundaries. Additionally, it must be a transform block boundary or a codec subblock boundary (e.g. due to the use of affine motion prediction (ATMVP)). The filter is disabled for those without such boundaries.

[0147] 2.4.1 Boundary strength calculation

[0148] For transform block boundaries / codec sub-block boundaries, if they are located in an 8×8 grid, they can be filtered and the bS[xD i ][yD j ](where [xD i ][yD j ] represents coordinates) are defined in Table 1 and Table 2 respectively.

[0149] Table 1. Boundary Strength (When SPS IBC is Disabled)

[0150]

[0151]

[0152] Table 2. Boundary Strength (When SPS IBC is Enabled)

[0153]

[0154] 2.4.3 Deblocking Decision for Luma Component

[0155] In this subsection the deblocking decision process is described.

[0156] Figure 7 Examples of pixels involved in filter on / off decisions and strong / weak filter selection are shown.

[0157] The wider, stronger luminance filter is a filter that is used only when conditions 1, 2, and 3 are all TRUE.

[0158] Condition 1 is the "large block condition." This condition checks whether the samples on the P and Q edges belong to large blocks, represented by the variables bSidePisLargeBlk and bSideQisLargeBlk, respectively. bSidePisLargeBlk and bSideQisLargeBlk are defined as follows.

[0159] bSidePisLargeBlk = ((edge ​​type is vertical and p0 belongs to a CU with width >= 32) || (edge ​​type is horizontal and p0 belongs to a CU with height >= 32))? TRUE:FALSE

[0160] bSideQisLargeBlk = ((edge ​​type is vertical and q0 belongs to a CU with width >= 32) || (edge ​​type is horizontal and q0 belongs to a CU with height >= 32))? TRUE:FALSE

[0161] Based on bSidePisLargeBlk and bSideQisLargeBlk, condition 1 is defined as follows.

[0162] Condition 1=(bSidePisLargeBlk||bSidePisLargeBlk)? TRUE:FALSE

[0163] Next, if condition 1 is true, condition 2 will be checked. First, the following variables are derived:

[0164] – First derive dp0, dp3, dq0, dq3 as in HEVC

[0165] – If p side is greater than or equal to 32

[0166] dp0=(dp0+Abs(p50-2*p40+p30)+1)>>1

[0167] dp3=(dp3+Abs(p53-2*p43+p33)+1)>>1

[0168] – If q side is greater than or equal to 32

[0169] dq0=(dq0+Abs(q50-2*q40+q30)+1)>>1

[0170] dq3=(dq3+Abs(q53-2*q43+q33)+1)>>1

[0171] Condition 2 = (d < β)? TRUE:FALSE

[0172] Where d=dp0+dq0+dp3+dq3.

[0173] If Condition1 and Condition2 are valid, it further checks whether any block uses the sub-block:

[0174]

[0175]

[0176] Finally, if both conditions 1 and 2 are valid, the proposed deblocking method will check condition 3 (large block strong filtering condition), which is defined as follows.

[0177] In condition 3 StrongFilterCondition, derive the following variables:

[0178] dpq is derived as in HEVC.

[0179] sp3 = Abs(p3 - p0), derived as in HEVC

[0180] If p sides is greater than or equal to 32

[0181]

[0182] sq3 = Abs(q0 - q3), derived as in HEVC

[0183] If q side is greater than or equal to 32

[0184]

[0185] As in HEVC, StrongFilterCondition = (dpq less than (β>>2), sp3+sq3 less than (3*β>>5), and Abs(p0-q0) less than (5*t C +1)>>1)? TRUE:FALSE.

[0186] 2.4.4 Stronger Deblocking Filter for Luma (Designed for Larger Blocks)

[0187] When the samples on either side of the boundary belong to a large block, a bilinear filter is used. Samples belonging to a large block are defined as: the width of the vertical edge is greater than or equal to 32, and the height of the horizontal edge is greater than or equal to 32.

[0188] Bilinear filters are listed below.

[0189] In the above HEVC deblocking, for the block boundary sample points p from i=0 to Sp-1 i and for block boundary samples q from j = 0 to Sq-1 j (pi and qi are the i-th sample in the row for filtering vertical edges, or the i-th sample in the column for filtering horizontal edges) are replaced by linear interpolation as follows:

[0190] —o i ′=(f i *Middle s,t +(64-f i )*P s +32)>>6), cut to p i ±tcPD i

[0191] —q j ′=(g j *Middle s,t +(64-g j )*Q s +32)>>6), cut to q j ±tcPD j

[0192] tcPD i and tcPD j The term is the position-dependent shear described in Section 2.4.7, and g j 、f i 、Middle s,t 、P s and Q s Given below.

[0193] 2.4.5 Deblocking Control for Chroma

[0194] A chroma strong filter is used on both sides of the block boundary. Here, the chroma filter is selected when both sides of the chroma edge are greater than or equal to 8 (chroma position), and the following decision with three conditions is met: The first condition is used for boundary strength and large block decisions. The proposed filter can be applied when the block width or height orthogonal to the block edge in the chroma sample domain is equal to or greater than 8. The second and third conditions are basically the same as the HEVC luma deblocking decision, which are the on / off decision and the strong filter decision, respectively.

[0195] In the first decision, for chroma filtering, the boundary strength (bS) is modified and the conditions are checked in sequence. If a condition is met, the remaining conditions with lower priority are skipped.

[0196] When a large block boundary is detected, chroma deblocking is performed when bS is equal to 2 or bS is equal to 1.

[0197] The second and third conditions are essentially the same as the HEVC luma strong filter decision as follows.

[0198] In the second condition:

[0199] d is derived as in HEVC luma deblocking.

[0200] The second condition is TRUE when d is less than β.

[0201] In the third condition, StrongFilterCondition is derived as follows:

[0202] dpq is derived as in HEVC.

[0203] sp3 = Abs(p3 - p0), derived as in HEVC

[0204] sq3 = Abs(q0 - q3), derived as in HEVC

[0205] As in the HEVC design, StrongFilterCondition = (dpq less than (β>>2), sp3+sq3 less than (β>>3), and Abs(p0-q0) less than (5*t C +1)>>1)

[0206] 2.4.6 Strong Deblocking Filter for Chroma

[0207] The following strong deblocking filter for chroma is defined:

[0208] p2′=(3*p3+2*p2+p1+p0+q0+4)>>3

[0209] p1′=(2*p3+p2+2*p1+p0+q0+q1+4)>>3

[0210] p0′=(p3+p2+p1+2*p0+q0+q1+q2+4)>>3

[0211] The proposed chroma filter performs deblocking on a 4×4 chroma sample grid.

[0212] 2.4.7 Position-dependent shearing

[0213] A position-dependent clipping tcPD is applied to the output samples of a luminance filtering process involving a strong filter and a long filter that modify 7, 5, and 3 samples at the boundaries. Assuming a quantization error distribution, it is recommended to increase the clipping value for samples that are expected to have higher quantization noise, and thus a larger deviation of the reconstructed sample value from the true sample value.

[0214] For each P or Q boundary filtered with an asymmetric filter, a position-dependent threshold table is selected from two tables (e.g., Tc7 and Tc3 listed below) provided as side information to the decoder, depending on the result of the decision process in Section 2.4.2:

[0215] Tc7={6,5,4,3,2,1,1}; Tc3={6,4,2};

[0216] tcPD=(Sp==3)? Tc3:Tc7;

[0217] For P or Q boundaries filtered with a short symmetric filter, a position-dependent threshold of lower magnitude is applied:

[0218] Tc3={3,2,1};

[0219] After defining the threshold, the filtered p' is clipped according to the tcP and tcQ values. i and q' i Sample value for clipping:

[0220] p” i =Clip3(p' i +tcP i ,p' i –tcP i ,p' i );

[0221] q” j =Clip3(q' j +tcQ j ,q' j –tcQ j,q' j );

[0222] Among them, p' i and q' i is the filtered sample value, p” i and q” j is the output sample value after clipping, tcPi tcPi is the clipping threshold derived from the VVC tc parameters and tcPD and tcQD. Function Clip3 is the clipping function as specified in VVC.

[0223] 2.4.8 Sub-block Deblocking Adjustment

[0224] To enable parallel friendly deblocking using long filters and subblock deblocking, as shown in the luminance control of long filters, the long filter is restricted to modifying at most 5 samples on the side where subblock deblocking (AFFINE or ATMVP or DMVR) is used. In addition, subblock deblocking is adjusted so that subblock boundaries close to CU or implicit TU boundaries on the 8×8 grid are restricted to modifying at most two samples on each side.

[0225] The following applies to sub-block boundaries that are not aligned with CU boundaries.

[0226]

[0227]

[0228] Where edges equal to 0 correspond to CU boundaries, edges equal to 2 or equal to orthogonalLength-2 correspond to sub-block boundaries 8 samples from a CU boundary, etc. ImplicitTU is true if implicit partitioning of TUs is used.

[0229] 2.5 SAO

[0230] The input of SAO is the reconstructed samples after DB. The concept of SAO is to reduce the average sample distortion of the region by first classifying the region samples into multiple categories using a selected classifier, obtaining an offset for each category, and then adding the offset to each sample of the category, where the classifier index and the offset of the region are encoded in the bitstream. In HEVC and VVC, the region (the unit used for SAO parameter signaling) is defined as a CTU.

[0231] HEVC uses two SAO types that meet low-complexity requirements. These types are edge offset (EO) and band offset (BO), which are discussed in detail below. The index of the SAO type is encoded (in the range [0, 2]). For EO, ​​sample classification is based on a comparison between the current sample and adjacent samples based on a one-dimensional orientation pattern (horizontal, vertical, 135° diagonal, and 45° diagonal).

[0232] Figure 8 Four one-dimensional orientation patterns for EO point classification are shown: horizontal (EO class = 0), vertical (EO class = 1), diagonal 135° (EO class = 2), and diagonal 45° (EO class = 3).

[0233] For a given EO category, each sample point within the CTB is classified into one of five categories. The current sample point value, labeled "c", is compared with its two neighbors along the selected 1-D pattern. Table I summarizes the classification rules for each sample point. Categories 1 and 4 are associated with local valleys and local peaks along the selected 1-D pattern, respectively. Categories 2 and 3 are associated with concave corners and convex corners along the selected 1-D pattern, respectively. If the current sample point does not belong to EO category 1-4, it is category 0 and SAO is not applied.

[0234] Table 3: Sample point classification rules for edge offset

[0235] category condition 1 c<a and c<b 2 (c<a&&c==b)||(c==a&&c<b) 3 (c>a&&c==b)||(c==a&&c>b) 4 c>a&&c>b 5 None of the above

[0236] 2.6 Geometric Transformation-Based Adaptive Loop Filter

[0237] The input of DB is the reconstructed samples after DB and SAO. Sample classification and filtering are based on the reconstructed samples after DB and SAO.

[0238] In some embodiments, a geometric transform based adaptive loop filter (GALF) with block-based filter adaptation is applied.For the luma component, one of 25 filters is selected for each 2x2 block according to the direction and activity of the local gradient.

[0239] 2.6.1 Filter Shape

[0240] In some embodiments, up to three diamond filter shapes (e.g., Figure 9 As shown in Figure 2.1 ...

[0241] 2.6.2 Block Classification

[0242] Each 2×2 block is classified into one of 25 classes. The classification index C is based on its directionality D and activity The quantitative value of is derived as follows:

[0243]

[0244] To calculate D and First, use the one-dimensional Laplace transform to calculate the gradient in the horizontal, vertical, and two diagonal directions:

[0245]

[0246]

[0247] The indices i and j refer to the coordinates of the upper left sample point in the 2×2 block, and R(i, j) indicates the reconstructed sample point at the coordinates (i, j).

[0248] Then set the D maximum and minimum values ​​of the gradient in the horizontal and vertical directions to:

[0249]

[0250] Set the maximum and minimum values ​​of the gradients in the two diagonal directions to:

[0251]

[0252] To derive the value of the directivity D, these values ​​are compared with each other using two thresholds t1 and t2:

[0253] Step 1. If and If both are true, set D to 0.

[0254] Step 2. If Then continue with step 3, otherwise continue with step 4.

[0255] Step 3. If Then set D to 2; otherwise, set D to 1.

[0256] Step 4. If Then set D to 4; otherwise, set D to 3.

[0257] The activity value A is calculated as:

[0258]

[0259] A is further quantized to the range of 0 to 4 (inclusive), and the quantized value is expressed as

[0260] For the chroma components in a picture, no classification method is applied, eg, a single ALF coefficient set is applied to each chroma component.

[0261] 2.6.1.2 Geometric Transformation of Filter Coefficients

[0262] Figure 10Relative coordinators supported by a 5×5 diamond filter are shown: left: diagonal, center: vertical flip, right: rotation.

[0263] Before filtering each 2×2 block, geometric transformations such as rotation or diagonal and vertical flipping are applied to the filter coefficients f(k,l) associated with the coordinates (k,l), depending on the gradient values ​​calculated for that block. This is equivalent to applying these transformations to the samples in the filter support region. The idea is to make different blocks with ALF applied more similar by aligning their directionality.

[0264] Three geometric transformations are introduced, including diagonal, vertical flip and rotation:

[0265]

[0266] Where K is the size of the filter, and 0≤k,l≤K-1 are the coefficient coordinates, such that position (0,0) is in the upper left corner and position (K-1,K-1) is in the lower right corner. Depending on the gradient value calculated for that block, a transformation is applied to the filter coefficients f(k,l). Table 4 summarizes the relationship between the transformation and the four gradients in the four directions. Figure 9 The transform coefficients for each position based on a 5x5 diamond are shown.

[0267] Table 4: Mapping between gradients and transformations computed for a block

[0268] Gradient value Transform <![CDATA[g d2 <g d1 and g h <g v ]]> No transformation <![CDATA[g d2 <g d1 and g v <g h ]]> diagonal <![CDATA[g d1 <g d2 and g h <g v ]]> Flip vertically <![CDATA[g d1 <g d2 and g v <g h ]]> Rotation

[0269] 2.6.1.3 Filter Parameter Signaling

[0270] In some embodiments, GALF filter parameters are signaled for the first CTU, for example, after the slice header and before the SAO parameters. Up to 25 sets of luma filter coefficients can be signaled. To reduce bit overhead, filter coefficients from different classes can be merged. Additionally, the GALF coefficients of a reference picture are stored and allowed to be reused as the GALF coefficients for the current picture. The current picture can choose to use the GALF coefficients stored for the reference picture and bypass GALF coefficient signaling. In this case, only the index of one of the reference pictures is signaled, and the stored GALF coefficients of the indicated reference picture are inherited for the current picture.

[0271] To support GALF temporal prediction, a candidate list of GALF filter sets is maintained. When decoding a new sequence begins, the candidate list is empty. After decoding a picture, the corresponding filter set can be added to the candidate list. Once the size of the candidate list reaches the maximum allowable value (e.g., 6), the new filter set overwrites the oldest set in decoding order, i.e., the first-in-first-out (FIFO) rule is applied to updating the candidate list. To avoid duplication, a set can only be added to the list if the corresponding picture does not use GALF temporal prediction. To support temporal scalability, there are multiple candidate lists of filter sets, and each candidate list is associated with a temporal layer. More specifically, each array assigned by a temporal layer index (TempIdx) can constitute the filter set of the previously decoded picture with a lower TempIdx. For example, the kth array is assigned to be associated with a TempIdx equal to k, and it only contains filter sets from pictures with a TempIdx less than or equal to k. After encoding or decoding a picture, the filter set associated with the picture will be used to update the arrays associated with equal or higher TempIdx.

[0272] Temporal prediction of the GALF coefficients is used for inter-coded frames to minimize signaling overhead. For intra frames, temporal prediction is not available, and a set of 16 fixed filters is assigned to each class. To indicate the use of fixed filters, a flag for each class is signaled, along with the index of the selected fixed filter if necessary. Even if a fixed filter is selected for a given class, the coefficients f(k, l) of the adaptive filter may still be sent for that class, in which case the filter coefficients applied to the reconstructed picture are the sum of the two sets of coefficients.

[0273] The filtering of the luma component can be controlled at the CU level. A signaling flag is used to indicate whether GALF is applied to the luma component of the CU. For chroma components, whether GALF is applied is indicated only at the picture level.

[0274] 2.6.1.4 Filtering

[0275] At the decoder side, when GALF is enabled for a block, each sample point R(i, j) in the block is filtered to obtain the sample value R'(i, j) as shown below, where L represents the filter length and f m,n represents the filter coefficient, and f(k,l) represents the decoded filter coefficient.

[0276]

[0277] Figure 11An example of relative coordinates for a 5×5 diamond filter support is shown, assuming that the coordinates (i, j) of the current sample point are (0, 0). Samples filled with the same color in different coordinates will be multiplied by the same filter coefficient.

[0278] 2.7 Geometric Transformation-Based Adaptive Loop Filter (GALF)

[0279] 2.7.1 GALF Example

[0280] In some embodiments, the filtering process of the adaptive loop filter is performed as follows:

[0281] (x,y)=∑ (i,j) w(i,j).I(x+i,y+j), (11)

[0282] Where the sample point I(x+i,y+j) is the input sample point, O(x,y) is the output sample point of the filter (for example, the filter result), and w(i,j) represents the filter coefficient. In fact, in VTM4.0, it uses integer arithmetic to implement fixed-point precision calculation:

[0283]

[0284] Where L represents the filter length and w(i,j) are the filter coefficients with fixed-point precision.

[0285] The current design of GALF in VVC has the following major changes:

[0286] (1) Eliminate adaptive filter shapes. The luminance component only allows a 7×7 filter shape, and the chrominance component only allows a 5×5 filter shape.

[0287] (2) The signaling of ALF parameters is removed from the slice / picture level to the CTU level.

[0288] (3) The class index calculation is performed at a 4×4 level instead of a 2×2 level. In addition, in some embodiments, a subsampled Laplacian calculation method for ALF classification is utilized. More specifically, the horizontal / vertical / 45° diagonal / 135° gradient does not need to be calculated for every sample point within a block. Instead, 1:2 subsampling is used.

[0289] Figures 12A-12D The sub-sampled Laplacian operation for CE2.6.2 is shown. Figure 12A shows the subsampling positions for the vertical gradient, Figure 12B shows the subsampling positions for the horizontal gradient, Figure 12C shows the subsampling positions for diagonal gradients, and Figure 12D The subsampling positions for diagonal gradients are shown.

[0290] 2.8 Nonlinear ALF

[0291] 2.8.1 Filter Reformulation

[0292] Formula (11) can be rewritten in the following expression without affecting the encoding and decoding efficiency:

[0293] O(x,y)=I(x,y)+∑ (i,j)≠(0,0) w(i,j).(I(x+i,y+j)-I(x,y)) (13)

[0294] Here, w(i,j) are the same filter coefficients as in formula (11) [except w(0,0), which is equal to 1 in formula (13) and equal to 1-∑ (i,j)≠(0,0) w(i,j)].

[0295] Using the filter formula (13) above, the efficiency of the ALF can be improved by introducing nonlinearity using a simple clipping function to reduce the influence of the neighboring sample value (I(x+i,y+j)) when the neighboring sample value is significantly different from the sample value currently being filtered (I(x,y)).

[0296] More specifically, the ALF filter is modified as follows:

[0297] O′(x,y)=I(x,y)+∑ (i,j)≠(0,0) w(i,j).K(I(x+i,y+j)-I(x,y),k(i,j)) (14)

[0298] where K(d,b)=min(b,max(-b,d)) is the shearing function and k(i,j) is the shearing parameter that depends on the (i,j) filter coefficients. The encoder performs an optimization to find the optimal k(i,j).

[0299] In some embodiments, a clipping parameter k(i,j) is specified for each ALF filter, and one clipping value is signaled for each filter coefficient. This means that up to 12 clipping values ​​can be signaled in the bitstream for each luma filter and up to 6 clipping values ​​can be signaled in the bitstream for each chroma filter.

[0300] In order to limit the signaling cost and encoder complexity, only the same 4 fixed values ​​as for INTER and INTRA slices can be used.

[0301] Since the variance of local differences in luma is typically higher than that in chroma, two different sets of filters are applied for luma and chroma. A maximum sample value in each set is also introduced (here 1024 for 10-bit depth) so that clipping can be disabled when not necessary.

[0302] A set of clipping values ​​used in some embodiments is provided in Table 5. The 4 values ​​were chosen by dividing the full range of sample values ​​for luma (encoded in 10 bits) and the range of chroma from 4 to 1024 approximately equally in the logarithmic domain.

[0303] More precisely, the brightness table of the clipped values ​​has been obtained by the following formula:

[0304] Where M = 2 10 And N = 4. (15)

[0305] Similarly, the chromaticity table of the clipping value is obtained according to the following formula:

[0306] Where M = 2 10 , N = 4 and A = 4. (16)

[0307] Table 5: Authorized shear values

[0308]

[0309] The selected clipping value is encoded in the "alf_data" syntax element by using the Golomb coding scheme corresponding to the index of the clipping value in the above Table 5. This coding scheme is the same as that of the filter index.

[0310] 2.9 Virtual Boundary

[0311] In both hardware and embedded software, picture-based processing is practically unacceptable due to its high picture buffer requirements. Using an on-chip picture buffer is very expensive, while using an off-chip picture buffer significantly increases external memory access, power consumption, and data access latency. Therefore, in actual products, DF, SAO, and ALF will switch from picture-based decoding to LCU-based decoding. When LCU-based processing is used for DF, SAO, and ALF, the entire decoding process can be completed by the LCUs in a raster scan in an LCU pipeline manner, processing multiple LCUs in parallel. In this case, DF, SAO, and ALF require a line buffer because processing a single LCU row requires pixels from that LCU row. Using an off-chip line buffer (e.g., DRAM) increases external memory bandwidth and power consumption; using an on-chip line buffer (e.g., SRAM) increases chip area. Therefore, even though pipeline buffers are already much smaller than picture buffers, reducing the line buffer size is still desirable.

[0312] In some embodiments, as Figure 13 As shown, the total number of line buffers required for the luma component is 11.25 lines. The line buffer requirement is explained as follows: Since the decision and filtering require lines K, L, M of the first CTU and lines O, P of the bottom CTU, deblocking of horizontal edges overlapping with CTU edges cannot be performed. Therefore, deblocking of horizontal edges overlapping with CTU boundaries is postponed until the lower layer CTU arrives. Therefore, for lines K, L, M, N, the reconstructed luma samples must be stored in the line buffer (4 lines). SAO filtering can then be performed on lines A to J. Since deblocking does not change the samples in line K, SAO filtering can be performed on line J. For SAO filtering of line K, the edge offset classification decision is only stored in the line buffer (i.e., 0.25 luma lines). ALF filtering can only be performed on lines AF. As shown Figure 13 As shown, ALF classification is performed on each 4×4 block. Each 4×4 block classification requires an active window of size 8×8, which in turn requires a 9×9 window to calculate the 1d Laplacian to determine the gradient.

[0313] Therefore, for block classification of 4×4 blocks overlapping with lines G, H, I, and J, the SAO filtered samples below the virtual boundary are required. In addition, ALF classification requires the SAO filtered samples of lines D, E, and F. Furthermore, ALF filtering of line G requires the SAO filtered samples of lines D, E, and F from the three above. Therefore, the total line buffer requirements are as follows:

[0314] -KN lines (horizontal DF pixels): 4 lines

[0315] -DJ lines (SAO filtered pixels): 7 lines

[0316] - SAO edge offset classifier value between row J and row K: 0.25

[0317] Therefore, the total number of luma rows required is 7+4+0.25=11.25.

[0318] Similarly, the line buffer for the chroma components is required to be Figure 14 The line buffer requirement for the chroma components is estimated to be 6.25 lines.

[0319] In order to eliminate the line buffer requirement of SAO and ALF, the concept of virtual boundary (VB) is introduced in the latest VVC. Figure 13 As shown in FIG, VB is moved upward by N pixels from the horizontal LCU boundary. For each LCU, SAO, and ALF, pixels above VB can be processed before the lower layer LCU appears, but pixels below VB cannot be processed until the lower layer LCU appears, which is caused by DF. Considering the hardware implementation cost, the interval between the proposed VB and the horizontal LCU boundary is set to four pixels of luma (e.g., Figure 13 N=4 in ) and two pixels of chrominance (e.g., Figure 9 N=2 in this example).

[0320] 2.9.1 Modified ALF Block Classification When VB Size N is 4

[0321] Figures 15A-15B The classification of modified blocks is shown for the case when the virtual boundary is 4 rows above the CTU boundary (N=4). Figure 15A As shown in FIG, for a 4×4 block starting from row G, block classification uses only rows E to J. However, the Laplace gradient calculation for the samples belonging to row J requires one or more rows below (row K). Therefore, row K is padded with row J.

[0322] Similarly, if Figure 15B As shown in FIG, for a 4×4 block starting from row K, block classification uses only rows K to P. However, the Laplace gradient calculation for the samples belonging to row K requires one or more rows above (row J). Therefore, row J is padded with row K.

[0323] 2.9.2 Filling on both sides of sample points across virtual boundaries

[0324] like Figures 16A-16C As shown, a truncated version of the filter is used to filter the luma samples belonging to rows close to the virtual boundary. Figure 16A For example, when filtering M rows, such as Figure 13As shown, for example, the central sample point of a 7×7 rhombus support is in row M. It needs to access the row above VB (represented by the thick line). In this case, the sample point above VB is copied from the lower-right of the sample point below VB. For example, the sample point P0 in the solid line is copied to the dotted position above. Symmetrically, even if the sample point at this position is available, the sample point P3 in the solid line is copied to the lower-right of the dotted position. The copied sample points are only used in the luminance filtering process.

[0325] The filling method for the ALF virtual boundary can be expressed as "filling on both sides", where if a sample point at (i, j) is filled (for example, Figure 16B P0A with a dotted line in Figure 16B ), then the corresponding sample point at (m, n) sharing the same filter coefficient is also filled (for example, Figures 16A-16C P3B with a dotted line in Figures 17A-17B ), even if the sample point is available, as Figures 16A-16C shown. In Figure 16A a 7×7 rhombus filter support, the center is the current sample point to be filtered. Figure 16B shows 1 required row above / below VB that needs to be filled. Figure 16C shows 2 required rows above / below VB that needs to be filled.

[0326] Similarly, as Figures 17A-17B shown, the filling-on-both-sides method is also used for chroma ALF filtering. Figures 17A-17B shows the modified chroma ALF filtering at the virtual boundary (5×5 rhombus filter support, the center is the current sample point to be filtered). Figure 17A shows 1 required row above / below VB that needs to be filled. Figure 17B shows 2 required rows above / below VB that needs to be filled.

[0327] 2.9.3 Alternative method for implementing bilateral filling when non-linear ALF is disabled

[0328] When non-linear ALF is disabled for CTB, for example, the shearing parameter k(i, j) in Equation (14) is equal to (1 << Bitdepth), and the filling process can be replaced by modifying the filter coefficients (also known as modified-coefficient-based ALF, MALF). For example, when filtering the sample points in the L / I row, the filter coefficient c5 is modified to c5'. In this case, there is no need to copy the luminance sample points from the solid line P0A to the dotted line P0A and from the solid line P3B to the dotted line P3B, as Figure 18AAs shown in this case, assuming that the current sample to be filtered is at (x, y), bilateral padding and MALF will produce the same result. Because K(d, b) = d, and due to padding, I(x-1, y-1) = I(x-1, y-2),

[0329] c5.K(I(x-1,y-1)-I(x,y),k(-1,-1))

[0330] +c1.K(I(x-1,y-2)-I(x,y),k(-1,-2))

[0331] =(c5+c1).K(I(x-1,y-1)-I(x,y),k(-1,-1)) (17)

[0332] However, when nonlinear ALF is enabled, MALF and two-sided padding may produce different filtering results because nonlinear parameters are associated with each coefficient, such as the shearing parameters are different for filter coefficients c5 and c1. Therefore, since K(d,b)!=d, even though I(x-1,y-1)=I(x-1,y-2) due to padding,

[0333] c5.K(I(x-1,y-1)-I(x,y),k(-1,-1))+c1.K(I(x-1,y-2)-I(x,y),k(-1,-2))! =(c5+c1).K(I(x-1,y-1)-I(x,y),k(-1,-1)) (18)

[0334] 2.10 ALF Filtering Specifications

[0335] Newly added parts are indicated by bold, italic and underline. Deleted parts are indicated by [[ ]].

[0336] 7.3.2.4 Picture Parameter Set RBSP Syntax

[0337]

[0338]

[0339]

[0340]

[0341] loop_filter_across_bricks_enabled_flag is equal to 1, which specifies that loop filtering operations can be performed across brick boundaries in pictures that reference PPS. loop_filter_across_bricks_enabled_flag is equal to 0, which specifies that loop filtering operations are not performed across brick boundaries in pictures that reference PPS. Loop filtering operations include deblocking filter, sample adaptive offset filter, and adaptive loop filter operations. When not present, the value of loop_filter_across_bricks_enabled_flag is inferred to be equal to 1.

[0342] loop_filter_across_slices_enabled_flag is equal to 1, indicating that loop filtering operations can be performed across slice boundaries in pictures that reference PPS. loop_filter_across_slice_enabled_flag is equal to 0, indicating that loop filtering operations are not performed across slice boundaries in pictures that reference PPS. Loop filtering operations include deblocking filter, sample adaptive offset filter, and adaptive loop filter operations. When not present, the value of loop_filter_across_slices_enabled_flag is inferred to be equal to 0.

[0343] When pps_loop_filter_across_virtual_boundaries_disabled_flag is equal to 1, it specifies that loop filtering operations can be disabled across virtual boundaries in pictures that reference the PPS. When pps_loop_filter_across_virtual_boundaries_disabled_flag is equal to 0, it specifies that such disabled loop filtering operations are not applied in pictures that reference the PPS. Loop filtering operations include deblocking filter, sample adaptive offset filter, and adaptive loop filter operations. When not present, the value of pps_loop_filter_across_virtual_boundaries_disabled_flag is inferred to be equal to 0.

[0344] pps_num_ver_virtual_boundaries specifies the number of pps_virtual_boundaries_pos_x[i] syntax elements present in the PPS. When pps_num_ver_virtual_boundaries is not present, it is inferred to be equal to 0.

[0345] 8.8.5.2 Codec Tree Block Filtering for Luma Samples

[0346] The inputs to this process are:

[0347] - Before adaptive loop filtering, the reconstructed luminance picture sample array recPicture L .

[0348] -Filtered and reconstructed brightness picture sample array alfPicture L .

[0349] -Specifies the luma position (xCtb, yCtb) of the top left sample of the current luma codec treeblock relative to the top left sample of the current picture.

[0350] The output of this process is the modified filtered reconstructed luminance picture sample array alfPicture L .

[0351] Use the position (xCtb, yCtb) and the reconstructed luminance picture sample array recPicture L The filter index derivation process of clause 8.8.5.3 is called with as input, and filtIdx[x][y] and transposeIdx[x][y] with x,y=0..CtbSizeY-1 as output.

[0352] For the filtered reconstructed luminance sample alfPicture L [x][y], each reconstructed luminance sample in the current luminance codec tree block recPictureL[x][y] is filtered as follows (x, y = 0..CtbSizeY-1).

[0353] - The luma filter coefficient array f[j] and luma clipping value array c[j] corresponding to the filter specified by filtIdx[x][y] are derived as follows, j = 0...11:

[0354] - If AlfCtbFiltSetIdxY[xCtb>>Log2CtbSize][yCtb>>Log2CtbSize] is less than 16, the following applies:

[0355] i=AlfCtbFiltSetIdxY[xCtb>>Log2CtbSize][yCtb>>Log2CtbSize] (8-1172)

[0356] f[j]=AlfFixFiltCoeff[AlfClassToFiltMap[i][filtidx]][j](8-1173)c[j]=2 BitdepthY (8-1174)

[0357] -else(AlfCtbFiltSetIdxY[xCtb>>Log2CtbSize][yCtb>>Log2CtbSize]

[0358] Greater than or equal to 16), the following applies:

[0359] i=slice_alf_aps_id_luma[AlfCtbFiltSetIdxY[xCtb>>Log2CtbSize][yCtb>>Log2CtbSize]-16] (8-1175)

[0360] f[j]=AlfCoeff L [i][filtIdx[x][y]][j] (8-1176)

[0361] c[j]=AlfClip L [i][filtIdx[x][y]][j] (8-1177)

[0362] - The luma filter coefficients and shear value index dx derived from transposeIdx[x][y] are as follows:

[0363] If transposeIndex[x][y] is equal to 1, then the following applies:

[0364] idx[ ]={9,4,10,8,1,5,11,7,3,0,2,6} (8-1178)

[0365] Otherwise, if transposeIndex[x][y] is equal to 2, the following applies:

[0366] idx[ ]={0,3,2,1,8,7,6,5,4,9,10,11} (8-1179)

[0367] Otherwise, if transposeIndex[x][y] is equal to 3, then the following applies:

[0368] idx[ ]={9,8,10,4,3,7,11,5,1,0,2,6} (8-1180)

[0369] - Otherwise, the following applies:

[0370] idx[ ]={0,1,2,3,4,5,6,7,8,9,10,11} (8-1181)

[0371] - The position (h) of each corresponding luma sample in a given luma sample array recPicture (where i, j = -3...3) x+i ,v y+j ) is derived as follows:

[0372] If pps_loop_filter_across_virtual_boundaries_disabled_flag is equal to 1 for any n=0...pps_num_ver_virtual_boundaries-1, and xCtb+x-PpsVirtualBoundariesPosX[n] is greater than or equal to 0 and less than 3, the following applies:

[0373] h x+i =Clip3(PpsVirtualBoundariesPosX[n],pic_width_in_luma_samples-1,xCtb+x+i) (8-1182)

[0374] Otherwise, if pps_loop_filter_across_virtual_boundaries_disabled_flag is equal to 1 for any n = 0..pps_num_ver_virtual_boundaries–1, and PpsVirtualBoundariesPosX[n]-xCtb-x is greater than 0 and less than 4, the following applies:

[0375] h x+i =Clip3(0,PpsVirtualBoundariesPosX[n]-1,xCtb+x+i) (8-1183)

[0376] - Otherwise, the following applies:

[0377] h x+i =Clip3(0,pic_width_in_luma_samples-1,xCtb+x+i) (8-1184)

[0378] If pps_loop_filter_across_virtual_boundaries_disabled_flag is equal to 1, and yCtb+y-PpsVirtualBoundariesPosY[n] is greater than or equal to 0 and less than 3, the following applies:

[0379] v y+j =Clip3(PpsVirtualBoundariesPosY[n],pic_height_in_luma_samples-1,yCtb+y+j) (8-1185)

[0380] Otherwise, if pps_loop_filter_across_virtual_boundaries_disabled_flag is equal to 1 for any n = 0..pps_num_hor_virtual_boundaries–1, and PpsVirtualBoundariesPosY[n]-yCtb-y is greater than 0 and less than 4, the following applies:

[0381] v y+j =Clip3(0,PpsVirtualBoundariesPosY[n]-1,yCtb+y+j) (8-1186)

[0382] - Otherwise, the following applies:

[0383] v y+j =Clip3(0,pic_height_in_luma_samples-1,yCtb+y+j) (8-1187)

[0384]

[0385]

[0386] - Depending on the horizontal luma sample position y and applyVirtualBoundary, the reconstructed sample offsets r1, r2 and r3 are specified in Table 8-22.

[0387] -The variable curr is derived as follows:

[0388] curr=recPicture L [h x ,v y ] (8-1188)

[0389] -The derivation of the variable sum is as follows:

[0390] sum=f[idx[0]]*(Clip3(-c[idx[0]],c[idx[0]],recPicture L [h x ,v y+r3]-curr)+Clip3(-c[idx[0]],c[idx[0]],recPicture L [h x ,v y-r3 ]-curr))+f[idx[1]]*(Clip3(-c[idx[1]],c[idx[1]],recPicture L [h x+1 ,v y+r2 ]-curr)+Clip3(-c[idx[1]],c[idx[1]],recPicture L [h x-1 ,v y-r2 ]-curr))+f[idx[2]]*(Clip3(-c[idx[2]],c[idx[2]],recPicture L [h x ,v y+r2 ]-curr)+Clip3(-c[idx[2]],c[idx[2]],recPicture L [h x ,v y-r2 ]-curr))+f[idx[3]]*(Clip3(-c[idx[3]],c[idx[3]],recPicture L [h x-1 ,v y+r2 ]-curr)+Clip3(-c[idx[3]],c[idx[3]],recPicture L [h x+1 ,v y-r2 ]-curr))+f[idx[4]]*(Clip3(-c[idx[4]],c[idx[4]],recPicture L [h x+2 ,v y+r1 ]-curr)+Clip3(-c[idx[4]],c[idx[4]],recPicture L [h x-2 ,v y-r1 ]-curr))+f[idx[5]]*(Clip3(-c[idx[5]],c[idx[5]],recPicture L [h x+1 ,v y+r1 ]-curr)+Clip3(-c[idx[5]],c[idx[5]],recPicture L [hx-1 ,v y-r1 ]-curr))+f[idx[6]]*(Clip3(-c[idx[6]],c[idx[6]],recPicture L [h x ,v y+r1 ]-curr)+Clip3(-c[idx[6]],c[idx[6]],recPicture L [h x ,v y-r1 ]-curr))+(8-1189)f[idx[7]]*(Clip3(-c[idx[7]],c[idx[7]],recPicture L [h x-1 ,v y+r1 ]-curr)+Clip3(-c[idx[7]],c[idx[7]],recPicture L [h x+1 ,v y-r1 ]-curr))+f[idx[8]]*(Clip3(-c[idx[8]],c[idx[8]],recPicture L [h x-2 ,v y+r1 ]-curr)+Clip3(-c[idx[8]],c[idx[8]],recPicture L [h x+2 ,v y-r1 ]-curr))+f[idx[9]]*(Clip3(-c[idx[9]],c[idx[9]],recPicture L [h x+3 ,v y ]-curr)+Clip3(-c[idx[9]],c[idx[9]],recPicture L [h x-3 ,v y ]-curr))+f[idx

[10] ]*(Clip3(-c[idx

[10] ],c[idx

[10] ],recPicture L [h x+2 ,v y ]-curr)+Clip3(-c[idx

[10] ],c[idx

[10] ],recPicture L [h x-2 ,v y]-curr))+f[idx

[11] ]*(Clip3(-c[idx

[11] ],c[idx

[11] ],recPicture L [h x+1 ,v y ]-curr)+Clip3(-c[idx

[11] ],c[idx

[11] ],recPicture L [h x-1 ,v y ]-curr))sum=curr+((sum+64)>>7)(8-1190)

[0391] - The derivation of the modified filtered reconstructed luminance picture sample alfPictureL[xCtb+x][yCtb+y] is as follows:

[0392] - If pcm_loop_filter_disabled_flag and pcm_flag[xCtb+x][yCtb+y] are both equal to 1, the following applies:

[0393] alfPicture L [xCtb+x][yCtb+y]=recPicture L [h x ,v y ] (8-1191)

[0394] Otherwise (pcm_loop_filter_disabled_flag is equal to 0 or pcm_flag[x][y] is equal to 0), the following applies:

[0395] alfPicture L [xCtb+x][yCtb+y]=Clip3(0,(1< <BitDepth Y )-1,sum) (8-1192)

[0396] Table 8-22: Regulations for r1, r2, and r3 based on the horizontal luminance sample position y and applyVirtualBoundary

[0397] condition r1 r2 r3 (y==CtbSizeY-5||y==CtbSizeY-4)&&(applyVirtualBoundary==1) 0 0 0 (y==CtbSizeY-6||y==CtbSizeY-3)&&(applyVirtualBoundary==1) 1 1 1 (y==CtbSizeY-7||y==CtbSizeY-2)&&(applyVirtualBoundary==1) 1 2 2 other 1 2 3

[0398] 8.8.5.4 Codec Tree Block Filtering for Chroma Samples

[0399] The inputs to this process are:

[0400] -reconstructed chroma picture sample array recPicture before adaptive loop filtering,

[0401] -Filtered and reconstructed chrominance picture sample array alfPicture,

[0402] -Specifies the chroma position (xCtbC, yCtbC) of the top left sample of the current chroma codec tree block relative to the top left sample of the current picture.

[0403] The output of this process is the modified filtered reconstructed chroma picture sample array alfPicture.

[0404] The width and height ctbWidthC and ctbHeightC of the current chroma codec tree block are derived as follows:

[0405] ctbWidthC=CtbSizeY / SubWidthC (8-1230)

[0406] ctbHeightC=CtbSizeY / SubHeightC (8-1231)

[0407] To derive the filtered reconstructed chroma samples alfPicture[x][y], each reconstructed chroma sample in the current chroma codec tree block recPicture[x][y] is filtered as follows, where x=0...ctbWidthC-1, y=0...ctbHeightC-1.

[0408] - In a given chroma sample array recPicture (where i, j = -2...2), the position (h) of each corresponding chroma sample (x, y) x+i ,v y+j ) is derived as follows:

[0409] If pps_loop_filter_across_virtual_boundaries_disabled_flag is equal to 1 for any n=0...pps_num_ver_virtual_boundaries-1, and xCtbC+x-PpsVirtualBoundariesPosX[n] / SubWidthC is greater than or equal to 0 and less than 2, the following applies:

[0410] h x+i =Clip3(PpsVirtualBoundariesPosX[n] / SubWidthC,

[0411] pic_width_in_luma_samples / SubWidthC-1,xCtbC+x+i)(8-1232)

[0412] Otherwise, if pps_loop_filter_across_virtual_boundaries_disabled_flag is equal to 1 for any n=0...pps_num_ver_virtual_boundaries-1, and PpsVirtualBoundariesPosX[n] / SubWidthC-xCtbC-x is greater than 0 and less than 3, the following applies:

[0413] h x+i =Clip3(0,PpsVirtualBoundariesPosX[n] / SubWidthC-1,xCtbC+x+i)(8-1233)

[0414] - Otherwise, the following applies:

[0415] h x+i =Clip3(0,pic_width_in_luma_samples / SubWidthC-1,xCtbC+x+i)(8-1234)

[0416] If pps_loop_filter_across_virtual_boundaries_disabled_flag is equal to 1, and yCtbC+y-PpsVirtualBoundariesPosY[n] / SubHeightC is greater than or equal to 0 and less than 2, the following applies:

[0417] v y+j =Clip3(PpsVirtualBoundariesPosY[n] / SubHeightC,pic_height_in_luma_samples / SubHeightC-1,yCtbC+y+j) (8-1235)

[0418] Otherwise, if pps_loop_filter_across_virtual_boundaries_disabled_flag is equal to 1 for any n = 0 ... pps_num_hor_virtual_boundaries - 1, and PpsVirtualBoundariesPosY[n] / SubHeightC-yCtbC-y is greater than 0 and less than 3, the following applies:

[0419] vy+j =Clip3(0,PpsVirtualBoundariesPosY[n] / SubHeightC-1,yCtbC+y+j)(8-1236)

[0420] - Otherwise, the following applies:

[0421] v y+j =Clip3(0,pic_height_in_luma_samples / SubHeightC-1,yCtbC+y+j) (8-1237)

[0422] -The variable applyVirtualBoundary is derived as follows:

[0423]

[0424] - Based on the horizontal luma sample position y and applyVirtualBoundary, the reconstructed sample offsets r1 and r2 are specified in Table 8-22.

[0425] -The variable curr is derived as follows:

[0426] curr=recPicture[h x ,v y ] (8-1238)

[0427] - The derivation of the chroma filter coefficient array f[j] and the chroma clipping value array c[j] is as follows, j=0...5.

[0428] f[j]=AlfCoeff C [slice_alf_aps_id_chroma][j] (8-1239)

[0429] c[j]=AlfClip C [slice_alf_aps_id_chroma][j] (8-1240)

[0430] -The derivation of the variable sum is as follows:

[0431]

[0432]

[0433] sum=curr+(sum+64)>>7) (8-1242)

[0434] - The derivation of the modified filtered reconstructed chrominance picture sample alfPicture[xCtbC+x][yCtbC+y] is as follows:

[0435] - If pcm_loop_filter_disabled_flag and pcm_flag[(xCtbC+x)*SubWidthC][(yCtbC+y)*SubHeightC] are both equal to 1, the following applies:

[0436] alfPicture[xCtbC+x][yCtbC+y]=recPicture L [h x ,v y ] (8-1243)

[0437] Otherwise (pcm_loop_filter_disabled_flag is equal to 0 or pcm_flag[x][y] is equal to 0), the following applies:

[0438] alfPicture[xCtbC+x][yCtbC+y]=Clip3(0,(1< <BitDepth C )-1,sum) (8-1244)

[0439] 2.11 CTU Processing Example

[0440] According to the current VVC design, if the bottom boundary of a CTB is the bottom boundary of a stripe / brick, the ALF virtual boundary handling method will be disabled. For example, Figure 19 As shown, a picture is divided into multiple CTUs and 2 slices.

[0441] Assuming the CTU size is M×M (e.g., M=64), then according to the virtual boundary definition, the last 4 rows in the CTB are processed as below the virtual boundary. In hardware implementation, the following conditions apply:

[0442] - If the bottom boundary of the CTB is the bottom boundary of a picture (e.g., CTU-D), it will process a (M+4)×M block consisting of 4 rows from the CTU row above and all rows in the current CTU.

[0443] Otherwise, if the bottom boundary of the CTB is the bottom boundary of a slice (or brick) (e.g., CTU-C), and loop_filter_across_slice_enabled_flag (or loop_filter_across_bricks_enabled_flag) is equal to 0, then process an (M+4)×M block consisting of 4 rows from the CTU row above and all rows in the current CTU.

[0444] - Otherwise, if the CTU / CTB is located in the first CTU row of a slice / brick / slice (e.g., CTU-A), then process the M×(M-4) block excluding the last 4 rows.

[0445] Otherwise, if the CTU / CTB is not in the first CTU row of a slice / brick / slice (e.g., CTU-B) nor in the last CTU row of a slice / brick / slice, process the M×M block that includes the 4 rows above the CTU row but does not include the last 4 rows of the current CTU.

[0446] Figure 19 An example of processing of CTUs in a picture is shown.

[0447] 2.12 360-degree video encoding and decoding

[0448] Horizontal surround motion compensation in VTM5 is a 360-degree-specific codec tool designed to improve the visual quality of 360-degree video reconstructed in equirectangular (ERP) projection formats. In traditional motion compensation, when a motion vector refers to a sample outside the picture boundaries of a reference image, padding is applied to derive the value of the sample by copying the nearest neighbor on the corresponding picture boundary. This padding approach is not suitable for 360-degree video and can cause visual artifacts known as "seam artifacts" in the reconstructed viewport video. Because 360-degree video is captured on a sphere and has no inherent "boundaries," reference samples that extend beyond the reference picture boundaries in the projection domain can always be derived from neighboring samples in the spherical domain. For general projection formats, deriving corresponding neighboring samples in the spherical domain can be difficult due to the 2D-to-3D and 3D-to-2D coordinate conversions and sample interpolation at fractional sample positions. For the left and right boundaries of the ERP projection format, this problem is much simpler, because the spherical neighbors outside the left image boundary can be obtained from the samples inside the right image boundary, and vice versa.

[0449] Figure 20 An example of horizontal surround motion compensation in VVC is shown.

[0450] Horizontal surround motion compensation processing such as Figure 20 When a part of the reference block is outside the left (or right) boundary of the reference picture in the projection domain, instead of repeated padding, the "outside boundary" part is taken from the corresponding spherical neighbor of the reference picture, towards the right (or left) boundary in the projection domain. Figure 20As shown, horizontal surround motion compensation can be combined with non-canonical padding methods commonly used in 360-degree video codecs. In VVC, this is achieved by signaling a high-level syntax element to indicate a surround offset, which should be set to the ERP picture width before padding; the syntax is used to adjust the position of the horizontal surround accordingly. This syntax is independent of the specific amount of padding at the left and right picture boundaries, and therefore naturally supports asymmetric padding of ERP pictures, such as when the left and right padding amounts are different. Horizontal surround motion compensation provides more meaningful information for motion compensation when the reference samples are outside the left and right reference picture boundaries.

[0451] For projection formats consisting of multiple faces, no matter what compact frame packing arrangement is used, discontinuities will appear between two or more adjacent faces in the frame packing picture. For example, considering Figure 24 In the 3×2 frame packing configuration depicted in [1], the three faces in the upper half are continuous in 3D geometry, and the three faces in the lower half are continuous in 3D geometry. However, the upper and lower halves of the frame packing image are discontinuous in 3D geometry. If loop filtering is performed at this discontinuity, face seam artifacts may appear in the reconstructed video.

[0452] To mitigate seam artifacts, loop filtering can be disabled across discontinuities in frame-packed pictures. Syntax is proposed to signal vertical and / or horizontal virtual boundaries at which loop filtering is disabled. Compared to using two slices (one for each set of contiguous slices) and disabling loop filtering across slices, the proposed signaling method is more flexible because it does not require the size of the slice to be a multiple of the CTU size.

[0453] 2.13 Example of independent regions with motion constraints based on sub-images

[0454] In some embodiments, the following features are included.

[0455] 1) A picture can be divided into sub-pictures.

[0456] 2) Indicate the existence of sub-pictures in the SPS, as well as other sequence-level information of the sub-pictures.

[0457] 3) Whether a sub-picture is processed as a picture in the decoding process (excluding loop filtering operations) can be controlled by the bitstream.

[0458] 4) Whether to disable loop filtering across sub-picture boundaries can be controlled by the bitstream of each sub-picture. Update DBF, SAO and ALF processing to control loop filtering operations across sub-picture boundaries.

[0459] 5) For simplicity, as a starting point, the sub-picture width, height, horizontal offset and vertical offset are signaled in units of luma samples in the SPS. The sub-picture boundaries are constrained to be slice boundaries.

[0460] 6) By slightly updating the coding_tree_unit() syntax, it is specified that the sub-picture is treated as a picture in the decoding process (excluding the loop filtering operation), and the following decoding process is updated.

[0461] ο(Advanced) Derivation of temporal luminance motion vector prediction.

[0462] ο Bilinear interpolation of luminance samples.

[0463] ο Luminance sample point 8-tap interpolation filtering processing.

[0464] ο Chroma sample interpolation processing.

[0465] 7) The sub-picture ID is explicitly specified in the SPS and included in the slice group header to enable extraction of the sub-picture sequence without changing the VCL NAL unit.

[0466] Output Sub-Picture Sets (OSPS) are proposed to specify canonical extraction and conformance points of sub-pictures and their groups.

[0467] 3. Technical issues addressed by the solutions provided in this document.

[0468] The current VVC design has the following problems.

[0469] 1. The current setting of enabling ALF virtual border depends on whether the bottom border of CTB is the bottom border of the picture. If true, disable ALF virtual border, such as Figure 19 However, it is possible that the bottom boundary of the CTB is outside the bottom boundary of the picture, such as a 256×240 picture is divided into four 128×128 CTUs. In this case, the ALF virtual boundary may be incorrectly set to true for the last two CTUs whose samples are outside the bottom boundary of the picture.

[0470] 2. Disable ALF virtual boundary processing at the bottom picture boundary and stripe / slice / brick boundaries. Disabling VB along stripe / brick boundaries may cause pipeline bubbles, or require processing 68 lines per virtual pipeline data unit (VPDU, 64×64 in VVC), assuming an LCU size of 64×64. For example:

[0471] a. For decoders that don't know slice / brick / slice boundaries upfront (e.g., low-latency applications), the ALF line buffer needs to be recovered. Whether the contents of the line buffer are used by the ALF filter depends on whether the current CTU is also a slice / brick / slice boundary CTU. However, this information is unknown until the next slice / brick / slice is decoded.

[0472] b. For decoders with upfront knowledge of stripe / brick / slice boundaries, either the decoder needs to tolerate pipeline bubbles (very unlikely) or run ALF at a constant 68 lines per 64x64 VDPU (overfeed) to avoid using the ALF line buffer.

[0473] 3. Different treatments for virtual boundaries and video unit boundaries, for example, there are different filling methods. At the same time, when a line is at multiple boundaries, more than one filling method can be performed on it.

[0474] a. In one example, if the bottom boundary of a block is a 360-degree virtual boundary and the ALF virtual boundary is also applied to the block, in this case, the filling method of the 360-degree virtual boundary can be applied first to generate virtual samples below the 360-degree virtual boundary. Afterwards, these virtual samples below the 360-degree virtual boundary are considered available. Figure 16A -C, you can further apply the ALF double-sided padding method. Figure 25 An example is described in .

[0475] 4. The way virtual boundaries are handled may be suboptimal because padding samples are used, which may reduce efficiency.

[0476] 5. When nonlinear ALF is disabled, MALF and both-side padding methods will produce the same results for filtering samples that need to cross virtual boundaries to access them. However, when nonlinear ALF is enabled, the two methods will produce different results. It would be beneficial to unify these two cases.

[0477] 6. The strips can be rectangular or non-rectangular, e.g. Figure 28 As shown. In this case, for a CTU, it may not coincide with any boundary (e.g., picture / slice / slice / brick). However, it may be necessary to access samples outside the current slice. If filtering across slice boundaries is disabled (e.g., loop_filter_across_slices_enabled_flag is false), it is unknown how ALF classification and filtering are performed.

[0478] 7. A sub-picture is a rectangular region of one or more strips within a picture. A sub-picture contains one or more strips that together cover a rectangular region of the picture. The syntax table has been modified as follows to include the concept of a sub-picture (bold, italic, and underlined).

[0479] 7.3.2.3 Sequence Parameter Set RBSP Syntax

[0480]

[0481] Note that enabling filtering across sub-pictures is controlled per sub-picture. However, enabling filtering across slices / slices / tiles is controlled at the picture level, which is signaled once to control all slices / slices / tiles within a picture.

[0482] 8.ALF classification is performed in 4×4 units, that is, all samples within a 4×4 unit share the same classification result. However, to be more precise, the gradients of the samples in the 8×8 window containing the current 4×4 block need to be calculated. In this case, Figure 30 As shown, 10×10 samples need to be accessed. If some samples are located in different video units (e.g., different strips / slices / tiles / sub-pictures / above, left, right, or bottom of the "360 virtual boundary" / above or below the "ALF virtual boundary"), it is necessary to define how to calculate the classification.

[0483] 9. In the VVC design, four boundary positions are identified (e.g., left vertical / right vertical / horizontal above / horizontal below). If a sample point is within the four boundary positions, it is marked as available. However, in VVC, a strip may cover a non-rectangular area, such as Figure 28 By checking these four boundary locations, erroneous availability results may be flagged. For example, Figure 28 The blue position (top left point) in the current block is in a different slice than the current block (e.g., the block covering the red position (bottom right point)), and if loop_filter_across_slices_enabled_flag is false (e.g., cross-slice samples are not allowed in ALF), then the blue position should be marked as "unavailable". However, by only checking the left vertical / right vertical / horizontal above / horizontal below boundaries of the current block, the blue position will be incorrectly marked as "available".

[0484] 4. Examples of Techniques and Embodiments

[0485] The following list should be considered as examples to explain the general concept. The listed techniques should not be interpreted narrowly. In addition, these techniques can be combined in any way.

[0486] The padding method for the ALF virtual boundary can be expressed as "pad on both sides", where if a sample located at (i, j) is padded, as shown in Figures 12-13, the corresponding sample located at (m, n) sharing the same filter coefficient is also padded, even if the sample is available.

[0487] The padding method used for picture boundaries / 360-degree video virtual boundaries, normal boundaries (such as top and bottom boundaries) can be expressed as "one-sided padding", where if a sample to be used is outside the boundary, it is copied from the available samples within the picture.

[0488] The padding method used for the left and right boundaries of a 360-degree video can be expressed as "wrapping-base padding", where if a sample to be used is outside the boundary, it is copied using the motion compensation result.

[0489] In the following discussion, a sample "at the boundary of a video unit" may mean that the distance between the sample and the boundary of the video unit is less than or not greater than a threshold. A "row" may refer to samples at the same horizontal position or samples at the same vertical position. (For example, samples in the same row and / or samples in the same column). The function Abs(x) is defined as follows:

[0490]

[0491] In the following discussion, "virtual samples" refer to generated samples that may be different from the reconstructed samples (possibly through deblocking and / or SAO processing). Virtual samples can be used to perform ALF on another sample. Virtual samples can be generated by padding.

[0492] An "ALF processing unit" is a unit defined by two horizontal boundaries and two vertical boundaries. The two horizontal boundaries may include two ALF virtual boundaries or one ALF virtual boundary and one picture boundary. The two vertical boundaries may include two vertical CTU boundaries or one vertical CTU boundary and one picture boundary. Figures 32A-32D An example is shown in .

[0493] A "narrow ALF processing unit" is a unit bounded by two horizontal boundaries and two vertical boundaries. One horizontal boundary may include an ALF virtual boundary or a 360-degree virtual boundary, and the other horizontal boundary may include a slice / brick / slice / sub-picture boundary or a 360-degree virtual boundary or a picture boundary. The vertical boundary may be a CTU boundary or a picture boundary or a 360-degree virtual boundary. Figure 34 An example is shown in .

[0494] "ALF virtual boundary processing is enabled for a block" may mean that applyVirtualBoundary in the specification is set to true. "Enabling virtual boundary" may indicate that the current block is divided into at least two parts by a virtual boundary, and samples in one part are not allowed to be used in the other part during filtering (e.g., ALF). The virtual boundary can be K rows above the bottom boundary of a block.

[0495] In the following description, neighboring samples may be those required for filter classification and / or filtering processing.

[0496] In this disclosure, a neighboring sample is "unavailable" if it is outside the current picture, current sub-picture, current slice, current slice, or current tile, or current CTU or current processing unit (such as an ALF processing unit or narrow ALF processing unit), or any other current video unit.

[0497] 1. Replace the determination of “the bottom boundary of the current codec tree block is the bottom boundary of the picture” with “the bottom boundary of the current codec tree block is the bottom boundary of the picture or outside the picture”.

[0498] a. Alternatively, in this case, the ALF virtual boundary handling method can be disabled.

[0499] 2. Whether to enable the use of virtual samples in the loop filtering process (eg, whether to enable virtual boundaries (eg, setting applyVirtualBoundary to true or false)) may depend on the CTB size.

[0500] a. In one example, for a given CTU / CTB size, such as a CTU / CTB size equal to K×L (eg, K=L=4), applyVirtualBoundary is always set to false.

[0501] b. In one example, for certain CTU / CTB sizes not larger or smaller than K×L (eg, K=L=8), applyVirtualBoundary is always set to false.

[0502] c. Alternatively, ALF is disabled for certain CTU / CTB sizes (such as 4x4, 8x8).

[0503] 3. Whether to allow the use of virtual samples (e.g., padding from reconstructed samples) in the loop filtering process (e.g., ALF) may depend on whether the bottom boundary of the block is the bottom boundary of a video unit, which has a finer granularity than a picture (e.g., a slice / tile / brick) or a virtual boundary.

[0504] a. In one example, an ALF virtual boundary handling method may be enabled for a codec tree block (CTB) (eg, set applyVirtualBoundary to true) if the bottom boundary of the CTB is a boundary of a video unit or a virtual boundary.

[0505] i. Alternatively, furthermore, if the bottom border is not the bottom picture border, or if the bottom border is outside the picture, the above method is enabled.

[0506] b. When the bottom boundary of the current codec treeblock is one of the bottom virtual boundaries of the picture and pps_loop_filter_across_virtual_boundaries_disabled_flag is equal to 1, the ALF virtual boundary handling method can still be enabled (eg, set applyVirtualBoundary to true).

[0507] c. In one example, whether the ALF virtual boundary processing method is enabled for a CTB (eg, the value of applyVirtualBoundary) may depend solely on the relationship between the bottom boundary of the CTB and the bottom boundary of the picture.

[0508] i. In one example, applyVirtualBoundary is set to false only when the bottom boundary of the CTB is the bottom boundary of the picture containing the CTB or the bottom boundary is outside the picture.

[0509] ii. In one example, when the bottom boundary of the CTB is not the bottom boundary of the picture containing the CTB, applyVirtualBoundary is set to true.

[0510] d. In one example, when decoding Figures 18A-18B When filtering a CTU-C in the virtual boundary, the M×M samples may be filtered using K rows from above the CTU and excluding K rows below the virtual boundary.

[0511] 4. Even when the control usage flag (eg, loop_filter_across_bricks_enabled_flag / loop_filter_across_slices_enabled_flag) of the signaling of the loop filter across brick / slice boundaries is true, it is recommended to disable the use of samples across brick / slice boundaries in the filtering process (eg, ALF).

[0512] a. Alternatively, in addition, the signaled loop_filter_across_bricks_enabled_flag / loop_filter_across_slices_enabled_flag may control only the filtering process of the deblocking filter and the SAO in addition to the AOF.

[0513] b. In one example, a virtual sample point may be used to replace the reconstructed sample point at the corresponding position to perform ALF on another sample point.

[0514] 5. When a block (e.g., CTB) contains samples located at a video unit boundary (e.g., a slice / brick / tile / 360-degree video virtual or normal boundary / picture boundary), how to generate virtual samples inside or outside the video unit (e.g., padding method) for loop filtering such as ALF can be unified for different types of boundaries.

[0515] a. Alternatively, furthermore, a virtual boundary method (eg, a two-side padding method) may be applied to the block to process samples at the boundary for loop filtering.

[0516] b. Alternatively, furthermore, when the block contains samples located at the bottom boundary of the video unit, the above method can be applied.

[0517] c. In one example, when decoding K lines of a block, if the K lines below the virtual boundary of the block (e.g., Figures 17A-17B When the last K rows in the CTU-B of the block are the last K rows in the CTU-B of the video unit and the bottom boundary of the block is the bottom boundary of the video unit, virtual samples can be generated in the ALF classification / filtering process to avoid using samples other than these K rows. For example, a "double-side padding" method can be applied.

[0518] i. Alternatively, ALF can be disabled for those last K rows.

[0519] d. In one example, when a block is located on multiple boundaries, the pixels / samples used for ALF classification can be restricted to not cross any of these multiple boundaries.

[0520] i. In one example, for a point, if one of its neighboring points is "unavailable" (eg, straddling any of a number of boundaries), it may not be possible to compute individual or all kinds of gradients / directivities for that point.

[0521] 1. In one example, the gradient of a sample point can be considered to be zero.

[0522] 2. In one example, the gradient of a sample point may be considered "unavailable" and may not be added to the activity obtained in the ALF classification process (eg, as defined in equation (8) of Section 2.6.1.1).

[0523] ii. In one example, when only some of the samples used in the ALF classification process are "available" (eg, not crossing any of these boundaries), the activity / directionality derived in the ALF classification process can be scaled by a factor.

[0524] iii. In one example, for a boundary block, assuming that gradients / directivities need to be calculated for N samples in the ALF classification process, and gradients can only be calculated for M samples (e.g., if some of a sample's neighbors are not "available", the gradient cannot be calculated for it), then the activity can be multiplied by N / M.

[0525] 1. Alternatively, you can multiply by a factor that depends on N / M. For example, the number could be M N

[0526] (N is an integer), for example, M=2.

[0527] e. In one example, the gradients of some samples in an M×N (eg, M=N=8, M columns and N rows in the current design) window can be used for classification.

[0528] i. For example, for the current N1*N2 (N1=N2=4 in the current design) block, M*N is centered around the N1*N2 block.

[0529] ii. In one example, a gradient of sample points may be used that does not require visiting the sample points across any boundaries.

[0530] 1. Alternatively, furthermore, when calculating the gradient of a sample point located at one or more boundaries, if some neighboring samples of the current sample point are “unavailable”, padding (eg, one-sided padding) may be performed.

[0531] 2. Alternatively, furthermore, if the current sample is located at the top boundary of a video unit (eg, a slice / brick / slice / 360-degree video virtual boundary or an ALF virtual boundary), the upper K (eg, K=1, 2) unavailable rows may be filled.

[0532] 3. Alternatively, if the current sample is located at the left boundary of the video unit, the left K (eg, K=1, 2) unavailable columns may be filled.

[0533] 4. Alternatively, if the current sample is located at the right boundary of the video unit, the right K (eg, K=1, 2) unavailable columns may be filled.

[0534] 5. Alternatively, if the current sample is located at the bottom boundary of the video unit, the bottom K (eg, K=1, 2) unavailable rows may be filled.

[0535] 6. Alternatively, in addition, if the current sample point is located at the top boundary and the left boundary of the video unit, the upper K1 (for example, K1=1, 2) unavailable rows can be filled first to generate an M*(N+K1) window, and then the left K2 (for example, K2=1, 2) unavailable columns can be filled to generate a (M+K2)*(N+K1) window.

[0536] a. Alternatively, the left K2 (e.g., K2=1, 2) unavailable columns can be filled first to generate an (M+K2)*N window, and then the top K1 (e.g., K1=1, 2) unavailable rows can be filled to generate an (M+K2)*(N+K1) window.

[0537] 7. Alternatively, in addition, if the current sample point is located at the top boundary and the right boundary of the video unit, the upper K1 (for example, K1=1, 2) unavailable rows can be filled first to generate an M*(N+K1) window, and then the right K2 (for example, K2=1, 2) unavailable columns can be filled to generate a (M+K2)*(N+K1) window.

[0538] a. Alternatively, the right K2 (e.g., K2=1, 2) unavailable columns can be filled first to generate an (M+K2)*N window, and then the upper K1 (e.g., K1=1, 2) unavailable rows can be filled to generate an (M+K2)*(N+K1) window.

[0539] 8. Alternatively, in addition, if the current sample point is located at the bottom boundary and the right boundary of the video unit, the bottom K1 (for example, K1=1, 2) unavailable rows can be filled first to generate an M*(N+K1) window, and then the right K2 (for example, K2=1, 2) unavailable columns can be filled to generate a (M+K2)*(N+K1) window.

[0540] a. Alternatively, the right K2 (e.g., K2=1, 2) unavailable columns can be filled first to generate an (M+K2)*N window, and then the bottom K1 (e.g., K1=1, 2) unavailable rows can be filled to generate an (M+K2)*(N+K1) window.

[0541] 9. Alternatively, further, if the current sample point is located at the bottom boundary and the left boundary of the video unit, the bottom K1 (e.g., K1=1, 2) unavailable rows may be filled first to generate an M*(N+K1) window, and then the left K2 (e.g., K2=1, 2) unavailable columns may be filled to generate an (M+K2)*(N+K1) window.

[0542] a. Alternatively, the left K2 (e.g., K2=1, 2) unavailable columns can be filled first to generate an (M+K2)*N window, and then the bottom K1 (e.g., K1=1, 2) unavailable rows can be filled to generate an (M+K2)*(N+K1) window.

[0543] 10. Alternatively, in addition, fill samples can be used to calculate the gradient.

[0544] iii. In one example, for a block at the top / bottom boundary of a video unit (eg, a stripe / brick / slice / 360-degree video virtual boundary or an ALF virtual boundary), the gradient of samples in the M*(N–C1) window may be used for classification of the block.

[0545] 1. Alternatively, in addition, the gradients of the top / bottom C1 rows of the M*N window are not used in the classification.

[0546] iv. In one example, for a block at the left / right boundary of a video unit, the gradient of the samples in the (M–C1)*N window may be used for classification of the block.

[0547] 1. Alternatively, in addition, the gradients of the left / right C1 columns of the M*N window are not used in the classification.

[0548] v. In one example, for blocks at the top and bottom boundaries of a video unit, the gradient of samples in an M*(N–C1–C2) window may be used for classification of the block.

[0549] 1. Alternatively, in addition, the gradients of the top C1 row and the bottom C2 row of the M*N window are not used in the classification.

[0550] vi. In one example, for blocks at the top and left boundaries of a video unit, the gradient of samples in a (M–C1)*(N–C2) window may be used for classification of the block.

[0551] 1. Alternatively, in addition, the gradients of the top C1 rows and left C2 columns of the M*N window are not used in the classification.

[0552] vii. In one example, for blocks at the top and right boundaries of a video unit, the gradient of samples in a (M-C1)*(N-C2) window may be used for classification of the block.

[0553] 1. Alternatively, in addition, the gradients of the top C1 rows and right C2 columns of the M*N window are not used in the classification.

[0554] viii. In one example, for blocks at the bottom and left boundaries of a video unit, the gradient of samples in a (M-C1)*(N-C2) window may be used for classification of the block.

[0555] 1. Alternatively, in addition, the gradients of the bottom C1 rows and left C2 columns of the M*N window are not used in the classification.

[0556] ix. In one example, for blocks at the bottom and right boundaries of a video unit, the gradient of samples in a (M–C1)*(N–C2) window may be used for classification of the block.

[0557] 1. Alternatively, in addition, the gradients of the bottom C1 rows and right C2 columns of the M*N window are not used in the classification.

[0558] x. In one example, for blocks at the left and right boundaries of a video unit, the gradient of samples in a (M–C1–C2)*N window may be used for classification of the block.

[0559] 1. Alternatively, in addition, the gradients of the left C1 column and the right C2 column of the M*N window are not used in the classification.

[0560] xi. In one example, for blocks at the top boundary, bottom boundary, and left boundary of a video unit, the gradient of samples in the (M–C3)*(N–C1–C2) window may be used for classification of the block.

[0561] 1. Alternatively, in addition, the gradients of the top C1 row, bottom C2 row, and left C3 column of the M*N window are not used in the classification.

[0562] xii. In one example, for blocks at the top boundary, bottom boundary, and right boundary of a video unit, the gradient of samples in a (M–C3)*(N–C1–C2) window may be used for classification of the block.

[0563] 1. Alternatively, in addition, the gradients of the top C1 row, bottom C2 row, and right C3 column of the M*N window are not used in the classification.

[0564] xiii. In one example, for blocks at the left boundary, right boundary, and top boundary of a video unit, the gradient of samples in a (M–C1–C2)*(N–C3) window may be used for classification of the block.

[0565] 1. Alternatively, in addition, the gradients of the left C1 column, right C2 column, and top C3 row of the M*N window are not used in the classification.

[0566] xiv. In one example, for blocks at the left boundary, right boundary, and bottom boundary of a video unit, the gradient of samples in a (M–C1–C2)*(N–C3) window may be used for classification of the block.

[0567] 1. Alternatively, in addition, the gradients of the left C1 column, right C2 column and bottom C3 row of the M*N window are not used in classification.

[0568] xv. In one example, for blocks at the left boundary, right boundary, top boundary, and bottom boundary of a video unit, the gradient of samples in a (M–C1–C2)*(N–C3–C4) window may be used for classification of the block.

[0569] 1. Alternatively, in addition, the gradients of the left C1 column and the right C2 column, the top C3 row and the bottom C4 row of the M*N window are not used in the classification.

[0570] xvi. In one example, C1, C2, C3, and C4 are equal to 2.

[0571] xvii. In one example, the gradient of a sample that does not have any "unavailable" neighboring samples required in the gradient calculation may be used.

[0572] f. In one example, when a line is located at multiple boundaries (eg, the distance between the line and the boundaries is less than a threshold), the filling process is performed only once, regardless of how many boundaries the line may belong to.

[0573] i. Alternatively, furthermore, how many adjacent rows should be filled may depend on the position of the current row relative to all boundaries.

[0574] ii. For example, how much of the adjacent row should be filled may be determined by the distance between the current row and two borders, such as when the current row is within two borders and the two borders are above and below.

[0575] iii. For example, how many adjacent rows should be filled may be determined by the distance between the current row and the nearest border, such as when the current row is within two borders and the two borders are above and below.

[0576] iv. For example, the number of adjacent rows to be filled can be calculated independently for each boundary, and the largest one is selected as the final number of filled rows.

[0577] v. In one example, it may be determined for each side of the row (eg, top and bottom) how many adjacent rows to fill.

[0578] vi. In one example, for the two-side filling method, it can be jointly determined how many adjacent rows should be filled on both sides.

[0579] vii. Alternatively, in addition, the two-side padding method used by ALF is applied.

[0580] g. In one example, when a row is located at multiple boundaries and there is at least one boundary in each side (eg, upper and lower sides) of the row, ALF may be disabled for it.

[0581] h. In one example, when the number of padding rows required for the current row is greater than a threshold, ALF may be disabled for the current row.

[0582] i. In one example, when the number of fill lines in either side is greater than a threshold, ALF may be disabled for the current row.

[0583] ii. In one example, when the total number of padding lines on both sides is greater than a threshold, ALF may be disabled for the current line.

[0584] i. Alternatively, furthermore, when a block contains samples located at the bottom boundary of a video unit and loop filtering such as ALF is enabled for the block, the above method can be applied.

[0585] j. Alternatively, furthermore, the above method may be applied under certain conditions, such as when the block contains samples located at the bottom boundary of the video unit and filtering across boundaries is not allowed (eg, pps_loop_filter_across_virtual_boundaries_disabled_flag / loop_filter_across_slices_enabled_flag / loop_filter_across_slices is true).

[0586] k. The proposed method is also applicable to samples / blocks located at vertical boundaries.

[0587] 6. When the sample point is at least two boundaries of a block (for example, at least one boundary above the current row is an ALF virtual boundary, and the boundary below is another boundary), the number of rows to be filled is not determined purely by the distance of the current row relative to the ALF virtual boundary, but by the distance of the current row relative to both boundaries.

[0588] a. In one example, set the number of rows padded on each side to (M – min(D0, D1)).

[0589] b. In one example, set the number of rows padded on each side to (M – max(D0, D1)).

[0590] c. For the above example, D0 and D1 represent the distance between the current row and the upper / lower boundaries.

[0591] d. For the above example, M represents the number of rows from the bottom of a CTU to the ALF virtual boundary.

[0592] 7. At least two ways of selecting samples in ALF classification and / or ALF linear or nonlinear filtering processing can be defined, where one way selects samples before applying any loop filtering method; and the other way selects samples after applying one or more loop filtering methods but before applying ALF.

[0593] a. In one example, the selection of different methods may depend on the location of the sample points to be filtered.

[0594] b. In one example, when a sample at the bottom boundary of a video unit (eg, CTB) is used for another sample in the ALF, the first method can be used for selection. Otherwise (not at the boundary), the second method is selected.

[0595] 8. It is recommended to disable samples that cross VPDU boundaries (e.g. 64×64 regions) in the filtering process.

[0596] a. In one example, when a sample required for ALF classification processing is outside the VPDU boundary or below the virtual boundary, a virtual sample can be used instead, or the classification result of the sample can be copied from samples associated with other samples, such as filling from available samples.

[0597] b. In one example, when samples required for filtering are outside the VPDU boundary or below the virtual boundary, they can be replaced by virtual samples, such as filling with available samples.

[0598] c. In one example, if a block contains samples located at the boundary of a VPDU, the ALF virtual boundary handling method can be enabled for the block (eg, set applyVirtualBoundary to true).

[0599] d. Alternatively, samples that cross horizontal VPDU boundaries may be disabled in the filtering process.

[0600] i. In one example, when samples required for filtering are below a horizontal VPDU boundary or below a virtual boundary, they may be replaced by virtual samples, such as by filling with available samples.

[0601] e. Alternatively, samples that cross vertical VPDU boundaries may be prohibited from being used in the filtering process.

[0602] i. In one example, when samples required for filtering are outside a vertical VPDU boundary or below a virtual boundary, virtual samples may be used instead, such as by padding with available samples.

[0603] 9. It is recommended not to use padded samples (e.g., unavailable, above / below virtual boundaries, above / below video unit boundaries) in the ALF classification / filtering process, and instead use reconstructed samples before all loop filters.

[0604] a. Alternatively, the concept of two-sided padding is applied in addition by passing padding samples from the reconstruction samples before all loop filters.

[0605] i. In one example, if the samples in the filter support come from reconstructed samples before all loop filters, then the symmetric samples in the filter support (e.g., symmetric to the origin, such as the current sample) should also use reconstructed samples before all loop filters.

[0606] 1. Assume that the coordinates of the current sample point to be filtered are (0, 0), the sample point at (i, j) is the reconstructed sample point before all loop filters, and the sample point at (-i, -j) is the reconstructed sample point before all loop filters.

[0607] 2. Assuming that the coordinates of the current sample to be filtered are (x, y), the sample at (x+i, y+j) is the reconstructed sample before all loop filters, then the sample at (x–i, y–j) ​​is the reconstructed sample before all loop filters.

[0608] b. Alternatively, in addition, when loop reshaping (also called LMCS) is enabled, all reconstructed samples before the loop filter are samples in the original domain converted from the reshaped domain.

[0609] 10. It is recommended not to use padded samples (e.g., unavailable, above / below virtual boundaries, above / below video unit boundaries) in the ALF filtering process, and instead adopt different ALF filter support.

[0610] a. In one example, assuming that a sample needs to be padded in the above method, instead of performing the padding, the filter coefficients associated with the sample are set to zero.

[0611] i. In this case, the filter support can be modified by excluding samples that need to be filled.

[0612] ii. Alternatively, in addition, except for the current sample, the filter coefficients applied to other samples remain unchanged, but the filter coefficients applied to the current sample can be modified, e.g., ((1<<C_BD) — the sum of all filter coefficients applied to samples that do not require padding), where C_BD indicates the bit depth of the filter coefficients.

[0613] 1. Take Figures 18A-18B as an example. When filtering rows L and I, the filter coefficient c12 applied to the current sample is modified to ((1<<C_BD)–2*(c4 + c5 + c6 + c7 + c8 + c9 + c10 + c11)).

[0614] b. In one example, assume that in the above method, the sample (x1, y1) is filled from (x2, y2). Then, instead of performing the fill, the filter coefficients associated with (x1, y1) are added to the position (x2, y2), regardless of whether the non - linear filter is enabled or disabled.

[0615] i. Alternatively, in addition, the clipping parameters for (x2, y2) can be derived on - the - fly.

[0616] 1. In one example, it can be set equal to the decoded clipping parameters of (x2, y2).

[0617] 2. Alternatively, it can also be set as the return value of a function, with the decoded clipping parameters of (x1, y1) and (x2, y2) as inputs, such as the larger value or the smaller value.

[0618] 11. The selection of clipping parameters / filter coefficients / filter support may depend on whether accessing padded samples is required for filtering the samples (e.g., not available, above / below the virtual boundary, above / below the video unit boundary).

[0619] a. In one example, for samples with the same category index, different clipping parameters / filter coefficients / filter support can be used, but for some of them, accessing padded samples is required, while for others it is not.

[0620] b. In one example, the clipping parameters / filter coefficients / filter support for filtering samples that require accessing padded samples can be signaled at the CTU / region / strip / slice level.

[0621] c. In one example, the clipping parameters / filter coefficients / filter support for filtering samples that require accessing padded samples can be derived from the samples used for filtering samples that do not require accessing padded samples.

[0622] i. In one example, Example 9a or 9b can be applied.

[0623] 12. How samples are processed at boundaries for loop filtering (such as ALF) may depend on the color components and / or color format.

[0624] a. For example, the definition of "at the border" can be different for different color components. In one example, if the distance between a luma sample and the bottom border is less than T1, then the luma sample is at the bottom border; otherwise, if the distance between a chroma sample and the bottom border is less than T2, then the chroma sample is at the bottom border. T1 and T2 can be different.

[0625] i. In one example, when the color format is not 4:4:4, T1 and T2 can be different.

[0626] 13. When the bottom / top / left / right boundary of a CTU / VPDU is also the boundary of a slice / tile / brick / sub-region with independent codecs, multiple padding processes will be applied in a fixed order.

[0627] a. In one example, in the first step, a fill method for stripes / slices / bricks is first applied (e.g., one-sided fill). Afterwards, a fill method for handling ALF virtual boundaries is further applied during the second step (e.g., two-sided fill method). In this case, the fill samples after the first step are marked as available and can be used to decide how many rows to fill in the ALF virtual boundary handling. The same rules used to handle CTUs that are not at those boundaries (e.g., Figure 16A -C).

[0628] 14. The proposed method can be applied to one or more boundaries between two sub-images.

[0629] a. The boundary to which the proposed method is applied can be a horizontal boundary.

[0630] b. The boundary to which the proposed method is applied can be a vertical boundary.

[0631] 15. The above proposed method can be applied to samples / blocks at vertical boundaries.

[0632] 16. Whether or how to apply the proposed method at the “360° virtual boundary” may depend on the location of the “360° virtual boundary”.

[0633] a. In one example, when the “360° virtual boundary” coincides with the CTU boundary, the proposed method can be applied. For example, for the samples at the “360° virtual boundary”, only two-side padding is applied in the ALF.

[0634] b. In one example, when the “360° virtual boundary” does not coincide with the CTU boundary, the proposed method may not be applied. For example, for the samples at the “360° virtual boundary”, only one-side padding may be applied in the ALF.

[0635] c. In one example, the same padding method may be applied in the ALF for samples at the “360 virtual boundary” regardless of the location of the “360 virtual boundary”.

[0636] i. For example, for samples at the “360 virtual boundary”, one-sided padding can be applied in the ALF.

[0637] ii. For example, for samples at the “360 virtual boundary”, padding on both sides can be applied in the ALF.

[0638] d. In one example, for samples of the plurality of boundaries where at least one boundary among the plurality of boundaries is a “360 virtual boundary” and at least one of the “360 virtual boundaries” does not coincide with a CTU boundary, the proposed method may not be applied.

[0639] i. For example, samples crossing any of these multiple boundaries can be filled by one-sided filling.

[0640] 1. Alternatively, furthermore, if there is a "virtual border", two-side padding can be applied in the ALF after the one-side padding.

[0641] e. In one example, for samples located between two boundaries, if one of them is a "360 virtual boundary" and the other is not, padding is only called once in the ALF process.

[0642] i. In one example, a fill method (eg, a two-side fill method) for processing an ALF virtual boundary may be called.

[0643] 1. Alternatively, a padding method (e.g., one-sided padding) may be called to handle picture (or strip / slice / brick / sub-picture) boundaries.

[0644] ii. Alternatively, two or more filling processes may be applied sequentially.

[0645] 1. In one example, a padding method (e.g., one-sided padding) for processing picture (or strip / slice / brick / sub-picture) boundaries may be applied first, and then a padding method (e.g., two-sided padding method) for processing ALF virtual boundaries may be further called.

[0646] a. Alternatively, in addition, filling points after the first filling are considered available in the second filling process.

[0647] iii. In one example, for a sample located between two or more boundaries (e.g., stripe boundary / slice boundary / brick boundary / "360 virtual boundary" / "ALF virtual boundary" / sub-picture boundary), if only one boundary is a "360 virtual boundary" (e.g., Figure 24 As shown, the first boundary is

[0648] For example, for these samples, only two-side padding can be applied in ALF.

[0649] 1. Alternatively, if these various boundaries are “360 virtual boundaries” or picture boundaries, the proposed method may not be applied. For example, for these samples, only one-side padding can be applied in ALF.

[0650] f. In one example, for samples located between two or more boundaries, and if at least one of the boundaries is a “360 virtual boundary” and it does not coincide with a CTU boundary, the proposed method may not be applied.

[0651] i. In this case, processing samples only at the "360 virtual boundary" and not at other kinds of boundaries can be done according to the prior art.

[0652] ii. In one example, for these samples, only one-side padding may be applied in the ALF.

[0653] g. In one example, for a sample point located between two or more boundaries, and if at least one of the boundaries is a “360 virtual boundary”, the proposed method may not be applied.

[0654] i. In this case, processing samples only at the "360 virtual boundary" and not at other kinds of boundaries can be done according to the prior art.

[0655] ii. In one example, for these samples, only one-side padding may be applied in the ALF.

[0656] 17. When the reference samples required in the ALF filtering process (e.g. Figure 16CWhen P0i (where i is A / B / C / D) or / and ALF classification processing is "unavailable" when filtering the current sample P0, for example, because the sample is located in a different video unit (e.g., slice / brick / slice / sub-picture) from the current sample, and filtering using samples across video units (e.g., slice / brick / slice / sub-picture boundaries) is not allowed, the "unavailable" samples can be filled with "available" samples (e.g., samples in the same slice / brick / slice / sub-picture as the current sample).

[0657] a. In one example, when necessary, the "unavailable" reference sample points may be first clipped to their nearest "available" horizontal position, and then the "unavailable" reference sample points may be clipped to their nearest "available" vertical position.

[0658] b. In one example, when necessary, the “unavailable” reference sample points may be first clipped to their nearest “available” vertical position, and then the “unavailable” sample points may be clipped to their nearest “available” horizontal position.

[0659] c. In one example, the coordinates of an “unusable” reference sample are clipped horizontally to the coordinates of its nearest “usable” sample (eg, minimum distance).

[0660] i. In one example, for two sample points with coordinates (x1, y1) and (x2, y2), the horizontal distance between them can be calculated as Abs(x1-x2).

[0661] d. In one example, the coordinates of an “unusable” reference sample are clipped vertically to the coordinates of its nearest “usable” sample (eg, minimum distance).

[0662] i. In one example, for two sample points with coordinates (x1, y1) and (x2, y2), the vertical distance between them can be calculated as Abs(y1–y2).

[0663] e. In one example, "unusable" samples are clipped to their closest "usable" sample (eg, minimum distance).

[0664] i. In one example, for two sample points with coordinates (x1, y1) and (x2, y2), the distance between them can be calculated as (x1–x2)*(x1–x2)+(y1–y2)*(y1–y2).

[0665] ii. Alternatively, the distance between two pixels can be calculated as Abs(x1-x2)+Abs(y1-y2).

[0666] f. In one example, the "unavailable" points can be filled in a predefined order until an "available" point is found. Figure 31 An example is shown in , where Cur is the current block / CU / PU / CTU.

[0667] i. For example, the vertical "available" samples may be checked first, and then the horizontal "available" samples may be checked.

[0668] ii. For example, the horizontal "available" samples may be checked first, and then the vertical "available" samples may be checked.

[0669] iii. For example, for an "unavailable" upper left neighboring sample (e.g., in region "1"), first, the left neighboring samples of the current block / CU / PU / CTU are checked (e.g., in region "4"). If there are no "available" samples, the upper neighboring samples are checked (e.g., in region "2"). If there are no "available" samples in either the left neighboring samples or the upper neighboring samples, the top left sample of the current block / CU / PU / CTU is used to fill the "unavailable" samples.

[0670] 1. Alternatively, for "unavailable" upper left neighboring samples, check the upper neighboring samples of the current block / CU / PU / CTU. If there are no "available" samples, check the left neighboring samples. If there are no "available" samples in the upper neighboring samples and the left neighboring samples, the top left sample of the current block / CU / PU / CTU is used to fill the "unavailable" samples.

[0671] iv. For example, for an "unavailable" upper right neighboring sample (e.g., in region "3"), first, the right neighboring samples of the current block / CU / PU / CTU are checked (e.g., in region "5"). If there are no "available" samples, the upper neighboring samples are checked (e.g., in region "2"). If there are no "available" samples in either the right neighboring samples or the upper neighboring samples, the top right sample of the current block / CU / PU / CTU is used to fill the "unavailable" samples.

[0672] 1. Alternatively, for "unavailable" upper right neighboring samples, first, check the upper neighboring samples of the current block / CU / PU / CTU. If there are no "available" samples, then check the right neighboring samples. If there are no "available" samples in the upper neighboring samples and the right neighboring samples, the top right sample of the current block / CU / PU / CTU is used to fill the "unavailable" samples.

[0673] v. For example, for "unavailable" lower left neighboring samples (e.g., in region "6"), first, check the left neighboring samples of the current block / CU / PU / CTU (e.g., in region "4"). If there are no "available" samples, then check the lower neighboring samples (e.g., in region "7"). If there are no "available" samples in either the left neighboring samples or the lower neighboring samples, the bottom left sample of the current block / CU / PU / CTU is used to fill the "unavailable" samples.

[0674] 1. Alternatively, for "unavailable" lower left neighboring samples, first, check the lower neighboring samples of the current block / CU / PU / CTU. If there are no "available" samples, check the left neighboring samples. If there are no "available" samples in the lower neighboring samples and the left neighboring samples, the bottom left sample of the current block / CU / PU / CTU is used to fill the "unavailable" samples.

[0675] vi. For example, for "unavailable" bottom-right neighboring samples, first, check the right neighboring samples of the current block / CU / PU / CTU (e.g., in area "5"). If there are no "available" samples, then check the bottom neighboring samples (e.g., in area "7"). If there are no "available" samples in either the right neighboring samples or the bottom neighboring samples, the bottom right samples of the current block / CU / PU / CTU are used to fill the "unavailable" samples.

[0676] 1. For example, for "unavailable" lower right neighboring samples, first, check the lower neighboring samples of the current block / CU / PU / CTU. If there are no "available" samples, then check the right neighboring samples. If there are no "available" samples in the lower neighboring samples and the right neighboring samples, the bottom right samples of the current block / CU / PU / CTU are used to fill the "unavailable" samples.

[0677] vii. In one example, for each adjacent region, one or more sample points may be examined in sequence. Alternatively, only one sample point may be examined.

[0678] viii. Alternatively, furthermore, if no check can find a usable sample, the value of the current sample to be filtered may be used instead.

[0679] ix. In one example, for "unavailable" top-left / top-right / bottom-left / bottom-right neighboring samples, they may always be filled with samples within the current block / CU / PU / CTU.

[0680] 1. In one example, for the “unavailable” upper left neighboring sample point (e.g. Figure 31 ), the top left samples of the current block / CU / PU / CTU are used to fill the "unavailable" samples.

[0681] 2. In one example, for the “unavailable” upper right neighboring sample point (e.g. Figure 31 In area "3" in the , the top right samples of the current block / CU / PU / CTU are used to fill the "unavailable" samples.

[0682] 3. In one example, for the “unavailable” lower left neighboring sample point (e.g. Figure 31 In area "6" in the , the bottom left samples of the current block / CU / PU / CTU are used to fill the "unavailable" samples.

[0683] 4. In one example, for the “unavailable” lower right adjacent sample point (e.g. Figure 31 In area "8" in the , the bottom right samples of the current block / CU / PU / CTU are used to fill the "unavailable" samples.

[0684] g. Alternatively, disable filtering for the current sample.

[0685] h. Alternatively, the classification process in the ALF (eg, gradient calculation for the current sample) may be disabled to use unavailable reference samples.

[0686] 18. How to derive filler samples for unavailable reference samples may depend on whether the CTU coincides with any boundary.

[0687] a. In one example, when the current CTU does not coincide with any kind of boundary, but filtering processing (e.g., ALF classification / ALF filtering processing) for the current sample requires access to reference samples in a different video unit (e.g., a slice), the method described in Example No. 16 can be applied.

[0688] i. Alternatively, further, when the current CTU does not coincide with any kind of boundary, but filtering processing for the current sample (e.g., ALF classification / ALF filtering processing) requires access to reference samples in a different video unit (e.g., slice), and filtering across slice boundaries is not allowed, the method described in Example No. 16 can be applied.

[0689] ii. Alternatively, further, when the current CTU does not coincide with any kind of boundary, but filtering processing for the current sample (e.g., ALF classification / ALF filtering processing) requires access to reference samples in different video units (e.g., slices), and reference samples in the same video unit and filtering across slice boundaries are not allowed, the method described in Example No. 16 can be applied.

[0690] b. In one example, when the current CTU coincides with at least one boundary, a uniform padding method (eg, two-sided or one-sided padding) may be applied.

[0691] i. Alternatively, when the current CTU coincides with various boundaries and filtering across those boundaries is not allowed, a unified padding method (eg, 2-side or 1-side padding) may be applied.

[0692] c. In one example, the method described in Example No. 16 may be used to fill in “unusable” sample points that cannot be filled by filling on both sides or / and filling on one side.

[0693] 19. Whether filtering processes (e.g., deblocking, SAO, ALF, bilateral filtering, Hadamard transform filtering, etc.) can access samples across boundaries of video units (e.g., slice / brick / tile / sub-picture boundaries) can be controlled at different levels, such as by themselves rather than for all video units in a sequence / picture.

[0694] a. Alternatively, a syntax element can be signaled in the PPS / slice header to indicate whether filtering can cross the slice boundary of the slice.

[0695] b. Alternatively, a syntax element can be signaled in the PPS to indicate whether filtering can be performed across tile / slice boundaries.

[0696] c. In one example, a syntax element may be signaled in the SPS / PPS to indicate whether filtering can be performed across tile boundaries or / and slice boundaries or / and strip boundaries or / and “360-degree virtual boundaries” of a video / picture.

[0697] i. In one example, separate syntax elements may be signaled for different kinds of boundaries.

[0698] ii. In one example, one syntax element may be signaled for all kinds of boundaries.

[0699] iii. In one example, a syntax element may be signaled for a set boundary.

[0700] 1. For example, one syntax element may be signaled for both tile boundaries and slice boundaries.

[0701] d. In one example, a syntax element may be signaled in the SPS to indicate whether there is a PPS / slice level indication on the filtering process.

[0702] i. In one example, separate syntax elements may be signaled for different kinds of boundaries.

[0703] ii. In one example, one syntax element may be signaled for all kinds of boundaries.

[0704] iii. In one example, a syntax element may be signaled for a set boundary.

[0705] 1. For example, one syntax element may be signaled for both tile boundaries and slice boundaries.

[0706] iv. An indication as to whether filtering can cross slice / tile / slice / sub-picture boundaries may be signaled in the PPS / slice header only if the corresponding syntax element in the SPS is equal to a certain value.

[0707] 1. Alternatively, when the corresponding syntax element in the SPS is equal to certain values, the indication as to whether the filtering process can cross the slice / tile / slice / sub-picture boundary may not be signaled in the PPS / slice header.

[0708] a. In this case, if the indication in the SPS is equal to a certain value, filtering may not be allowed to cross slice / brick / slice / sub-picture boundaries.

[0709] b. In this case, if the indication in the SPS is equal to a certain value, the filtering process can cross the slice / brick / slice / sub-picture boundary.

[0710] 20. It is recommended to check whether the samples located in the upper left / upper right / lower left / lower right neighboring area of ​​the current block are in the same video unit (e.g., slice / brick / slice / sub-picture / 360 virtual boundary) as the current block in ALF processing (e.g., classification and / or filtering processing). Let (x0, y0) represent the top left sample of the current luma codec tree block relative to the top left sample of the current picture, and let ctbXSize and ctbYSize be the CTU width and CTU height, respectively.

[0711] a. In one example, a representative sample point located in the upper left region may be checked, for example (x0–offsetX0, y0–offsetY0).

[0712] i. In one example, (offsetX0, offsetY0) can be equal to (1, 1), (2, 1), or (1, 2).

[0713] b. In one example, a representative sample point located in the upper right region may be checked, for example (x0+offsetX1, y0-offsetY1).

[0714] i. In one example, (offsetX1, offsetY1) can be equal to (ctbXSize, 1), (ctbXSize+1, 1), or (ctbXSize, 2).

[0715] c. In one example, a representative sample point located in the lower left region may be checked, for example (x0-offsetX2, y0+offsetY2).

[0716] i. In one example, (offsetX2, offsetY2) can be equal to (1, ctbYSize), (2, ctbYSize), or (1, ctbYSize+1).

[0717] d. In one example, a representative sample point located in the lower right region may be checked, for example (x0+offsetX3, y0+offsetY3).

[0718] i. In one example, (offsetX2, offsetY2) may be equal to (ctbXSize, ctbYSize), (ctbXSize+1, ctbYSize), or (ctbXSize, ctbYSize+1).

[0719] e. In one example, if representative samples in a region are in different video units and filtering across different video units is not allowed, the samples to be accessed in the region are marked as unavailable.

[0720] i. In one example, if the representative samples in a region are in different slices and loop_filter_across_slices_enabled_flag is equal to 0, the samples to be accessed in the region are marked as unavailable.

[0721] ii. In one example, if representative samples in a region are in different bricks and loop_filter_across_bricks_enabled_flag is equal to 0, the samples to be accessed in the region are marked as unavailable.

[0722] iii. In one example, if the representative samples in the region are in different sub-pictures and loop_filter_across_subpic_enabled_flag[SubPicIdx] is equal to 0, the samples to be accessed in the region are marked as unavailable. In one example, SubPicIdx is the index of the current sub-picture that includes the current block.

[0723] f. In one example, if M of the K representative samples in the K regions are in a different video unit than the current CTU and filtering across different video units is not allowed, then the samples to be accessed in the K regions are all marked as unavailable.

[0724] i. In one example, M is equal to 1 and K is equal to 2.

[0725] 1. For example, the K regions may include an upper left adjacent region and an upper right adjacent region.

[0726] 2. For example, the K regions may include an upper left adjacent region and a lower left adjacent region.

[0727] 3. For example, the K regions may include an upper right adjacent region and a lower right adjacent region.

[0728] 4. For example, the K regions may include a lower left adjacent region and a lower right adjacent region.

[0729] 5. Alternatively, M can be equal to 2.

[0730] ii. In one example, M is equal to 1 and K is equal to 3.

[0731] 1. For example, the K regions may include the upper left, upper right, and lower left neighboring regions.

[0732] 2. For example, the K regions may include the upper left, upper right, and lower right neighboring regions.

[0733] 3. For example, the K regions may include the upper right, lower left, and lower right adjacent regions.

[0734] 4. For example, the K regions may include the upper left, lower left, and lower right neighboring regions.

[0735] 5. Alternatively, M can be equal to 2 or 3.

[0736] iii. In one example, M is equal to 1 and K is equal to 4. For example, the K regions may include upper left, upper right, lower left, and lower right neighboring regions.

[0737] 1. Alternatively, M can be equal to 1, 2 or 3.

[0738] iv. In one example, N of the K representative samples (M<=N<=K) are checked to determine whether the M of the N representative samples exist in a video unit different from the current CTU.

[0739] 1. In one example, N is equal to M, and only M predefined representative sample points are checked.

[0740] 2. For example, when M is equal to 1 and K is equal to 2, and the K regions may include an upper left adjacent region and an upper right adjacent region, only representative samples of the upper left adjacent region are checked.

[0741] a. Alternatively, only representative samples of the upper right neighboring area are examined.

[0742] 3. For example, when M is equal to 1 and K is equal to 2, and the K regions may include a lower left adjacent region and a lower right adjacent region, only representative samples of the lower left adjacent region are checked.

[0743] a. Alternatively, only representative samples of the lower right neighboring area are examined.

[0744] 4. For example, when M is equal to 1 and K is equal to 2, and the K regions are the upper left adjacent region and the lower left adjacent region, only the representative sample points of the upper left adjacent region are checked.

[0745] a. Alternatively, only representative samples of the lower left neighboring area are examined.

[0746] 5. For example, when M is equal to 1 and K is equal to 2, and the K regions are the upper right adjacent region and the lower right adjacent region, only the representative sample points of the upper right adjacent region are checked.

[0747] a. Alternatively, only representative samples of the lower right neighboring area are examined.

[0748] 6. For example, when M is equal to 1 and K is equal to 4, and the K regions are the upper left adjacent region, the upper right adjacent region, the lower left adjacent region, and the lower right adjacent region, only the representative sample points of the upper left adjacent region are checked.

[0749] a. Alternatively, only representative samples of the lower right neighboring area are examined.

[0750] b. Alternatively, only representative samples of the lower left neighboring area are examined.

[0751] c. Alternatively, only representative samples of the upper right neighboring area are examined.

[0752] v. In one example, if the representative samples in a region are in different slices and loop_filter_across_slices_enabled_flag is equal to 0, the samples to be accessed in the region are marked as unavailable.

[0753] vi. In one example, if representative samples in a region are in different bricks and loop_filter_across_bricks_enabled_flag is equal to 0, the samples to be accessed in the region are marked as unavailable.

[0754] vii. In one example, if the representative sample in a region is in a different sub-picture and loop_filter_across_subpic_enabled_flag[SubPicIdx] is equal to 0, the sample to be accessed in the region is marked as unavailable. In one example, SubPicIdx is the index of the current sub-picture that includes the current block.

[0755] g. In the above example, offsetXi / offsetYi (where i is 0…3) is an integer.

[0756] i. Alternatively, in addition, offsetXi / offsetYi (where i is 0...3) can be set equal to the CTU width / height.

[0757] h. In one example, the current block may be a CTU.

[0758] i. In one example, the current block may be an ALF processing unit.

[0759] j. In one example, the current block may be a narrow ALF processing unit.

[0760] 21. The “availability” of the upper left / upper right / lower left / lower right neighboring regions of a CTU may be determined independently of the upper / left / right / lower right neighboring regions of the CTU.

[0761] a. In the ALF filtering or / and classification process, whether to perform padding on the upper left adjacent region and whether to perform padding on the upper adjacent region may be determined differently.

[0762] i. In one example, if the upper neighboring area is marked as "unavailable" and the upper left neighboring area is marked as "available", then padding is performed on the upper neighboring area, but not on the upper left neighboring area.

[0763] ii. In one example, if the upper left neighboring area is marked as “unavailable” and the upper neighboring area is marked as “available”, then padding is performed on the upper left neighboring area, but not on the upper neighboring area.

[0764] b. In the ALF filtering or / and classification process, whether to perform padding on the upper right adjacent region and whether to perform padding on the upper adjacent region may be determined differently.

[0765] c. In the ALF filtering or / and classification process, whether to perform padding on the upper left adjacent region and whether to perform padding on the left adjacent region may be determined differently.

[0766] d. In the ALF filtering or / and classification process, whether to perform padding on the lower left adjacent region and whether to perform padding on the left adjacent region may be determined differently.

[0767] e. In the ALF filtering or / and classification process, whether to perform padding on the lower right adjacent region and whether to perform padding on the right adjacent region may be determined differently.

[0768] f. In the ALF filtering or / and classification process, whether to perform padding on the upper right adjacent region and whether to perform padding on the right adjacent region may be determined differently.

[0769] g. In the ALF filtering or / and classification process, whether to perform padding on the lower right adjacent region and whether to perform padding on the lower adjacent region may be determined differently.

[0770] h. In the ALF filtering or / and classification process, whether to perform padding on the lower left adjacent region and whether to perform padding on the lower adjacent region may be determined differently.

[0771] i. In one example, when samples in the upper left neighboring region or / and the upper right neighboring region are determined to be "unavailable," samples in the upper neighboring region may still be determined to be "available" (e.g., if they are in the same video unit as the current CTU). For example, during ALF filtering and / or classification processing, sample padding may not be performed on the upper neighboring region.

[0772] j. In one example, when samples in the lower left neighboring region or / and the lower right neighboring region are determined to be "unavailable," samples in the lower neighboring region may still be determined to be "available" (e.g., if they are in the same video unit as the current CTU). For example, during ALF filtering and / or classification processing, sample padding may not be performed on the lower neighboring region.

[0773] k. In one example, when samples in the upper left neighboring region and / or the lower left neighboring region are determined to be "unavailable," samples in the left neighboring region may still be determined to be "available" (e.g., if they are in the same video unit as the current CTU). For example, sample padding may not be performed on the left neighboring region during ALF filtering and / or classification processing.

[0774] 1. In one example, when samples in the upper right neighboring region and / or the lower right neighboring region are determined to be "unavailable," samples in the right neighboring region may still be determined to be "available" (e.g., if they are in the same video unit as the current CTU). For example, sample padding may not be performed on the right neighboring region during ALF filtering and / or classification processing.

[0775] m. In one example, the “availability” checking method may be applied to the ALF processing unit.

[0776] n. In one example, the “availability” checking method can be applied to narrow ALF processing units.

[0777] 22. It is suggested that only samples within the current processing unit (e.g., an ALF processing unit bounded by two ALF virtual boundaries, where an example of an ALF processing unit is depicted in FIG32 , or a “narrow ALF processing unit”) may be used to fill in “unavailable” neighboring samples of the current CTU / block / processing unit.

[0778] a. In one example, if samples within the current processing unit are "unavailable" (eg, within a different "video unit" than the current CTU), they may be filled with samples within the current CTU.

[0779] b. In one example, if the left / right neighboring samples are “unavailable”, the left / right columns of the current processing unit can be used to fill them.

[0780] c. In one example, if the neighboring samples above / below are “unavailable”, the top / bottom row of the current processing unit can be used to fill them.

[0781] d. In one example, if the top left / top right / bottom left / bottom right neighboring samples are "unavailable", the top left / top right / bottom left / bottom right corner samples of the current processing unit can be used to fill them.

[0782] e. In one example, if the upper left adjacent sample of the current CTU is "unavailable" and the upper adjacent sample of the current CTU is "available", the upper adjacent sample can be used to fill the upper left adjacent sample, such as Figure 32A shown.

[0783] f. In one example, if both the upper left neighboring sample and the upper neighboring sample of the current CTU are "unavailable", the samples in the top row of the current CTU can be used to fill the upper neighboring samples first, and then the filled upper neighboring samples can be used to fill the upper left neighboring samples, such as Figure 32C shown.

[0784] g. In one example, if the upper left neighboring sample, the upper neighboring sample, and the left neighboring sample are all "unavailable", the samples in the top row of the current CTU can be used to fill the upper neighboring sample first, and then the filled upper neighboring sample can be used to fill the upper left neighboring sample, such as Figure 32D The left column of the current CTU can be used to fill the left adjacent sample points.

[0785] h. How padding is applied to loop filtering (eg, ALF) may depend on the unavailability of required sample locations relative to a processing unit (eg, an ALF processing unit or a narrow ALF processing unit).

[0786] i. In one example, the filling of unavailable neighboring samples of the current processing unit depends only on the samples within the current processing unit. In other words, the filling process of unavailable neighboring samples of the current processing unit is decoupled from any samples outside the current processing unit.

[0787] ii. In one example, if the upper left adjacent sample of the current processing unit is “unavailable”, the top left sample of the current processing unit can be used to fill such “unavailable” sample, as shown in Figures 33a-d and Figure 34 An example is shown.

[0788] iii. In one example, if the upper right adjacent sample point of the current processing unit is “unavailable”, the top right sample point of the current processing unit can be used to fill such “unavailable” sample point, as shown in Figures 33a-d and Figure 34 shown.

[0789] iv. In one example, if the lower left adjacent sample of the current processing unit is “unavailable”, the bottom left sample of the current processing unit can be used to fill such “unavailable” sample, as shown in Figures 33a-d and Figure 34 shown.

[0790] v. In one example, if the lower right adjacent sample point of the current processing unit is "unavailable", the bottom right sample point of the current processing unit can be used to fill such "unavailable" sample point, as shown in Figures 33a-d and Figure 34 shown.

[0791] i. In one example, if a neighboring sample is in a different slice / tile / sub-picture / 360 video virtual boundary, or outside the picture boundary, it is marked as unavailable.

[0792] i. Alternatively, the determination of sample point unavailability may be defined using the above bullet points.

[0793] j. In one example, the above method may be applied during the filtering process, but not during the classification process.

[0794] i. Alternatively, the above method may be applied during the filtering process and the classification process.

[0795] ii. Alternatively, the above method may be applied during the filtering process and the classification process.

[0796] 23. Unavailable neighboring samples of a processing unit (eg, ALF processing unit or / and narrow ALF processing unit or / and CTU) may be filled in a predefined order as follows.

[0797] a. If the adjacent sample points above the processing unit are not available, they can be filled with the top row of processing units.

[0798] i. Alternatively, in addition, the top left neighboring sample point can be filled with the top left sample point of the processing unit.

[0799] ii. Alternatively, in addition, the top right adjacent sample point can be filled with the top right sample point of the processing unit.

[0800] b. If the adjacent sample points below the processing unit are not available, they can be filled with the bottom row of processing units.

[0801] i. Alternatively, in addition, the bottom left adjacent sample point can be filled with the bottom left sample point of the processing unit.

[0802] ii. Alternatively, in addition, the bottom right adjacent sample point can be filled with the bottom right sample point of the processing unit.

[0803] c. If the left neighboring samples of the processing unit are not available, they can be filled with the left column of the processing unit.

[0804] d. If the right neighboring samples of the processing unit are not available, they can be filled with the right column of the processing unit.

[0805] e. If the left adjacent sample point and the upper adjacent sample point of the processing unit are available, and the upper left adjacent sample point of the processing unit is not available, the upper left adjacent sample point can be filled with the top left sample point of the processing unit.

[0806] f. If the right adjacent sample point and the lower adjacent sample point of the processing unit are available, and the bottom right adjacent sample point of the processing unit is not available, the bottom right adjacent sample point of the processing unit may be used to fill the lower right adjacent sample point.

[0807] g. A processing unit may include N (N is an integer, e.g., N=4) rows of CTUs, indicated by ctuUp, and CtbSize-M (M is an integer, e.g., M=N) rows of CTUs, indicated by ctuDown below ctuUp. ctuDown may be used when checking whether adjacent samples of a processing unit are available.

[0808] i. In one example, a neighboring sample is considered "unavailable" if it is in a different video unit than the ctuDown (e.g., the neighboring sample and the ctuDown belong to different bricks / slices / strips / sub-pictures, or they are on different sides of the 360 ​​virtual boundary) and filtering is not allowed across such video units.

[0809] ii. Alternatively, ctuUp can be used to check the availability of neighboring samples.

[0810] h. Repeat padding can be applied to all boundaries except ALF virtual boundaries.

[0811] i. Alternatively, repeating fills can be applied to all borders.

[0812] ii. Alternatively, repeating fill may be applied to all horizontal borders.

[0813] iii. Alternatively, repeating fill may be applied to all vertical boundaries.

[0814] iv. Alternatively, a mirror fill can be applied to all horizontal borders.

[0815] v. Alternatively, a mirror fill can be applied to all vertical borders.

[0816] i. In one example, a processing unit may be split (horizontally and / or vertically) into multiple processing units if the processing unit is crossed by one or more brick / strip / slice / sub-picture boundaries and / or 360 virtual boundaries, and filtering is not allowed across such boundaries.

[0817] i. Alternatively, further, the partitioning process may be performed covertly until no processing unit is crossed by any brick / slice / slice / sub-picture boundary or / and 360° virtual boundary or ALF virtual boundary, wherein filtering processing is not allowed across such boundaries, e.g., such boundaries can only be processing unit boundaries. Such a processing unit is hereinafter referred to as a "basic ALF processing unit."

[0818] ii. Alternatively, further, ALF processing is performed after such division processing is completed, for example, ALF processing is performed on a "basic ALF processing unit".

[0819] iii. Alternatively, furthermore, the above filling method may be performed on a "basic ALF processing unit".

[0820] 24. The method proposed above can be applied not only to ALF, but also to other types of filtering methods that need to access samples outside the current block.

[0821] a. Alternatively, the above proposed method can be applied to other codec tools (non-filtering methods) that need to access samples outside the current block.

[0822] b. The above-proposed method can be applied to CC-ALF (cross-component adaptive loop filter).

[0823] 25. Filtering processes such as CC-ALF or / and ALF or / and SAO or / and DB (deblocking) may be applied only to samples in the considered picture (or output picture / consistency window / scaling window) rather than samples in the entire picture.

[0824] a. In one example, those samples outside the considered picture (or output picture / consistency window / scaling window) may not be allowed to be filtered, regardless of the signaled value of the filter on / off flag.

[0825] 26. Whether and / or how the above methods should be applied may be determined by:

[0826] a. Messages signaled in DPS / SPS / VPS / PPS / APS / picture header / slice header / slice group header / largest codec unit (LCU) / codec unit (CU) / LCU row / LCU group / TU / PU block / video codec unit

[0827] b. Location of CU / PU / TU / block / video codec unit

[0828] c. Block dimensions of the current block and / or its neighboring blocks

[0829] d. Block shape of the current block and / or its neighboring blocks

[0830] e. Codec information of the current block and / or its adjacent blocks

[0831] f. Indication of color format (e.g., 4:2:0, 4:4:4)

[0832] g. Codec tree structure

[0833] h. Slice / slice group type and / or picture type

[0834] i. Color component (for example, can only be applied to chroma components or luma components)

[0835] j. Time domain layer ID

[0836] k. Standard profiles / levels / grades

[0837] 5. Examples

[0838] In the following sections, some examples of how the current version of the VVC standard can be modified to accommodate some embodiments of the disclosed technology are described. New additions are indicated by bold, italic, and underlined text. Deleted sections are indicated by [[ ]].

[0839] 5.1 Example #1

[0840] loop_filter_across_bricks_enabled_flag is equal to 1, indicating that loop filtering operations can be performed across brick boundaries in pictures that reference PPS. loop_filter_across_bricks_enabled_flag is equal to 0, indicating that loop filtering operations are not performed across brick boundaries in pictures that reference PPS. Loop filtering operations include deblocking filter, sample adaptive offset filter [[and adaptive loop filter]] operations. When not present, the value of loop_filter_across_bricks_enabled_flag is inferred to be equal to 1.

[0841] loop_filter_across_slices_enabled_flag is equal to 1, indicating that loop filtering operations can be performed across slice boundaries in pictures that reference PPS. loop_filter_across_slice_enabled_flag is equal to 0, indicating that loop filtering operations are not performed across slice boundaries in pictures that reference PPS. Loop filtering operations include deblocking filter, sample adaptive offset filter [[and adaptive loop filter]] operations. When not present, the value of loop_filter_across_slices_enabled_flag is inferred to be equal to 0.

[0842] 5.2 Example #2

[0843] Figure 21 The processing of CTUs in a picture is shown. Figure 19 The differences compared are highlighted with dashed lines.

[0844] 5.3 Example #3

[0845] 8.8.5.2 Codec Tree Block Filtering for Luma Samples

[0846] The inputs to this process are:

[0847] - Before adaptive loop filtering, the reconstructed luminance picture sample array recPicture L ,

[0848] -Filtered and reconstructed brightness picture sample array alfPictureL ,

[0849] -Specifies the luma position (xCtb, yCtb) of the top left sample of the current luma codec treeblock relative to the top left sample of the current picture.

[0850] The output of this process is the modified filtered reconstructed luminance picture sample array alfPicture L .

[0851] Use the position (xCtb, yCtb) and the reconstructed luminance picture sample array recPicture L The filter index derivation process of clause 8.8.5.3 is called with as input, and filtIdx[x][y] and transposeIdx[x][y] with x,y=0..CtbSizeY-1 as output.

[0852] For the filtered reconstructed luminance sample alfPicture L [x][y], each reconstructed luminance sample in the current luminance codec tree block recPictureL[x][y] is filtered as follows (x, y = 0..CtbSizeY-1).

[0853] - The luma filter coefficient array f[j] and luma clipping value array c[j] corresponding to the filter specified by filtIdx[x][y] are derived as follows, j = 0...11:

[0854] -…

[0855] - The luma filter coefficients and shear value index dx derived from transposeIdx[x][y] are as follows:

[0856] -…

[0857] - The position (h) of each corresponding luma sample in a given luma sample array recPicture (where i, j = -3...3) x+i ,v y+j ) is derived as follows:

[0858] -…

[0859] -The variable applyVirtualBoundary is derived as follows:

[0860] - If [[one or more]] of the following conditions are true, set applyVirtualBoundary to 0.

[0861] [[-The bottom boundary of the current codec tree block is the bottom boundary of the picture.

[0862] - The bottom boundary of the current codec tree block is the bottom boundary of the brick, and loop_filter_across_bricks_enabled_flag is equal to 0.

[0863] - The bottom boundary of the current codec treeblock is the bottom boundary of the slice, and loop_filter_across_slices_enabled_flag is equal to 0.

[0864] - The bottom boundary of the current codec treeblock is one of the bottom virtual boundaries of the picture, and pps_loop_filter_across_virtual_boundaries_disabled_flag is set equal to 1. ]]

[0865] Otherwise, applyVirtualBoundary is set equal to 1.

[0866] - Based on the horizontal luma sample position y and applyVirtualBoundary, the reconstructed sample offsets r1, r2, and r3 are specified in Table 8-22.

[0867] -…

[0868] 8.8.5.4 Codec Tree Block Filtering for Chroma Samples

[0869] The inputs to this process are:

[0870] -reconstructed chroma picture sample array recPicture before adaptive loop filtering,

[0871] -Filtered and reconstructed chrominance picture sample array alfPicture,

[0872] -Specifies the chroma position (xCtbC, yCtbC) of the top left sample of the current chroma codec tree block relative to the top left sample of the current picture.

[0873] The output of this process is the modified filtered reconstructed chroma picture sample array alfPicture.

[0874] The width and height ctbWidthC and ctbHeightC of the current chroma codec tree block are derived as follows:

[0875] ctbWidthC=CtbSizeY / SubWidthC (8-1230)

[0876] ctbHeightC=CtbSizeY / SubHeightC (8-1231)

[0877] To derive the filtered reconstructed chroma samples alfPicture[x][y], each reconstructed chroma sample in the current chroma codec tree block recPicture[x][y] is filtered as follows, where x=0...ctbWidthC-1, y=0...ctbHeightC-1.

[0878] - In a given chroma sample array recPicture (where i, j = -2...2), the position (h) of each corresponding chroma sample (x, y) x+i ,v y+j ) is derived as follows:

[0879] If pps_loop_filter_across_virtual_boundaries_disabled_flag is equal to 1 for any n=0...pps_num_ver_virtual_boundaries-1, and xCtbC+x-PpsVirtualBoundariesPosX[n] / SubWidthC is greater than or equal to 0 and less than 2, the following applies:

[0880] h x+i =Clip3(PpsVirtualBoundariesPosX[n] / SubWidthC,

[0881] pic_width_in_luma_samples / SubWidthC-1,xCtbC+x+i)(8-1232)

[0882] Otherwise, if pps_loop_filter_across_virtual_boundaries_disabled_flag is equal to 1 for any n=0...pps_num_ver_virtual_boundaries-1, and PpsVirtualBoundariesPosX[n] / SubWidthC-xCtbC-x is greater than 0 and less than 3, the following applies:

[0883] h x+i =Clip3(0,PpsVirtualBoundariesPosX[n] / SubWidthC-1,xCtbC+x+i)(8-1233)

[0884] - Otherwise, the following applies:

[0885] h x+i=Clip3(0,pic_width_in_luma_samples / SubWidthC-1,xCtbC+x+i) (8-1234)

[0886] If pps_loop_filter_across_virtual_boundaries_disabled_flag is equal to 1, and yCtbC+y-PpsVirtualBoundariesPosY[n] / SubHeightC is greater than or equal to 0 and less than 2, the following applies:

[0887] v y+j =Clip3(PpsVirtualBoundariesPosY[n] / SubHeightC,pic_height_in_luma_samples / SubHeightC-1,yCtbC+y+j) (8-1235)

[0888] Otherwise, if pps_loop_filter_across_virtual_boundaries_disabled_flag is equal to 1 for any n = 0 ... pps_num_hor_virtual_boundaries - 1, and PpsVirtualBoundariesPosY[n] / SubHeightC-yCtbC-y is greater than 0 and less than 3, the following applies:

[0889] v y+j =Clip3(0,PpsVirtualBoundariesPosY[n] / SubHeightC-1,yCtbC+y+j)(8-1236)

[0890] - Otherwise, the following applies:

[0891] v y+j =Clip3(0,

[0892] pic_height_in_luma_samples / SubHeightC-1,yCtbC+y+j) (8-1237)

[0893] -The variable applyVirtualBoundary is derived as follows:

[0894] - If [[one or more]] of the following conditions are true, set applyVirtualBoundary to 0.

[0895] - The bottom boundary of the current codec tree block is the bottom boundary of the picture.

[0896] -[[The bottom boundary of the current codec tree block is the bottom boundary of the brick, and loop_filter_across_bricks_enabled_flag is equal to 0.

[0897] - The bottom boundary of the current codec treeblock is the bottom boundary of the slice, and loop_filter_across_slices_enabled_flag is equal to 0.

[0898] - The bottom boundary of the current codec treeblock is one of the bottom virtual boundaries of the picture, and pps_loop_filter_across_virtual_boundaries_disabled_flag is set equal to 1. ]]

[0899] Otherwise, applyVirtualBoundary is set equal to 1.

[0900] - Based on the horizontal luma sample position y and applyVirtualBoundary, the reconstructed sample offsets r1 and r2 are specified in Table 8-22.

[0901] …

[0902] Alternatively, the condition “the bottom boundary of the current codec treeblock is the bottom boundary of the picture” may be replaced with “the bottom boundary of the current codec treeblock is the bottom boundary of the picture or outside the picture”.

[0903] 5.4 Example #4

[0904] This embodiment shows an example in which samples below the VPDU area are not allowed to be used in the ALF classification process (corresponding to bullet point 7 in Section 4).

[0905] 8.8.5.3 Derivation of filter index and ALF transpose for luma samples

[0906] The inputs to this process are:

[0907] – luma position (xCtb, yCtb), which specifies the top left sample of the current luma codec treeblock relative to the top left sample of the current picture,

[0908] – Before adaptive loop filtering, the reconstructed luminance picture sample array recPictureL .

[0909] The output of this processing is:

[0910] – Classification filter index array filtIdx[x][y], where x, y = 0..CtbSizeY–1.

[0911] – Transpose index array transposeIdx[x][y], where x, y = 0..CtbSizeY-1.

[0912] The position (h) of each corresponding luma sample (x, y) in a given array recPicture of luma samples with i, j = -2..5 x+i ,v y+j ) is obtained as follows:

[0913] – If pps_loop_filter_across_virtual_boundaries_disabled_flag is equal to 1 for any n = 0..pps_num_ver_virtual_boundaries-1, and xCtb+x-PpsVirtualBoundariesPosX[n] is greater than or equal to 0 and less than 2, the following applies:

[0914] h x+i =Clip3(PpsVirtualBoundariesPosX[n],pic_width_in_luma_samples-1,xCtb+x+i) (8-1193)

[0915] – Otherwise, if pps_loop_filter_across_virtual_boundaries_disabled_flag is equal to 1 for any n = 0..pps_num_ver_virtual_boundaries-1, and PpsVirtualBoundariesPosX[n]-xCtb-x is greater than 0 and less than 6, the following applies: x+i =Clip3(0,PpsVirtualBoundariesPosX[n]-1,xCtb+x+i) (8-1194)

[0916] – Otherwise, the following applies:

[0917] h x+i =Clip3(0,pic_width_in_luma_samples-1,xCtb+x+i) (8-1195)

[0918] – If pps_loop_filter_across_virtual_boundaries_disabled_flag is equal to 1 for any n = 0..pps_num_hor_virtual_boundaries–1, and yCtb+y−PpsVirtualBoundariesPosY[n] is greater than or equal to 0 and less than 2, the following applies: v y+j =Clip3(PpsVirtualBoundariesPosY[n],pic_height_in_luma_samples-1,yCtb+y+j) (8-1196)

[0919] – Otherwise, if pps_loop_filter_across_virtual_boundaries_disabled_flag is equal to 1 for any n = 0..pps_num_hor_virtual_boundaries-1, and PpsVirtualBoundariesPosY[n]-yCtb-y is greater than 0 and less than 6, the following applies:

[0920] v y+j =Clip3(0,PpsVirtualBoundariesPosY[n]-1,yCtb+y+j) (8-1197)

[0921] – Otherwise, the following applies:

[0922] – If yCtb+CtbSizeY is greater than or equal to pic_height_in_luma_samples, the following applies:

[0923] v y+j =Clip3(0,pic_height_in_luma_samples-1,yCtb+y+j) (8-1198)

[0924] – Otherwise, if y is less than CtbSizeY-4, the following applies:

[0925] v y+j =Clip3(0,yCtb+CtbSizeY-5,yCtb+y+j) (8-1199)

[0926] – Otherwise, the following applies:

[0927] v y+j=Clip3(yCtb+CtbSizeY-4,pic_height_in_luma_samples-1,yCtb+y+j)(8-1200)

[0928] The classification filter index array filtIdx and the transpose index array transposeIdx are obtained by the following sequential steps:

[0929] 1. The variables filtH[x][y], filtV[x][y], filtD0[x][y], and filtD1[x][y] (with x, y = -2..CtbSizeY+1) are derived as follows:

[0930] – If both x and y are even or both x and y are uneven, then the following applies:

[0931] filtH[x][y]=Abs((recPicture[h x ,v y ]<<1)-recPicture[h x-1 ,v y ]-recPicture[h x+1 ,v y ]) (8-1201)

[0932] filtV[x][y]=Abs((recPicture[h x ,v y ]<<1)-recPicture[h x ,v y-1 ]-recPicture[h x ,v y+1 ]) (8-1202)

[0933] filtD0[x][y]=Abs((recPicture[h x ,v y ]<<1)-recPicture[h x-1 ,v y-1 ]-recPicture[h x+1 ,v y+1 ]) (8-1203)

[0934] filtD1[x][y]=Abs((recPicture[h x ,v y ]<<1)-recPicture[h x+1 ,v y-1 ]-recPicture[h x-1,v y+1 ]) (8-1204)

[0935] – Otherwise, set filtH[x][y], filtV[x][y], filtD0[x][y], and filtD1[x][y] equal to 0.

[0936] 2. The variables minY, maxY, and ac are derived as follows:

[0937] – If (y<<2) is equal to [[(CtbSizeY-8)]], and (yCtb+CtbSizeY) is less than pic_height_in_luma_samples–1, then set minY equal to -2, maxY equal to 3, and ac equal to 96.

[0938] – Otherwise, if (y<<2) is equal to [[(CtbSizeY-4)]], and (yCtb+CtbSizeY) is less than pic_height_in_luma_samples–1, then set minY equal to 0, maxY equal to 5, and ac equal to 96.

[0939] – Otherwise, set minY equal to -2, maxY equal to 5, and ac equal to 64.

[0940] 3. The variables varTempH1[x][y], varTempV1[x][y], varTempD01[x][y], varTempD11[x][y], and varTemp[x][y] with x,y=0..(CtbSizeY-1)>>2 are derived as follows:

[0941] sumH[x][y]=Σ i Σ j filtH[h (x<<2)+i -xCtb][v (y<<2)+j -yCtb], where i = -2..5, j = minY..maxY (8-1205)

[0942] sumV[x][y]=Σ i ∑ j filtV[h (x<<2)+i -xCtb][v (y<<2)+j -yCtb], where i = -2..5, j = minY..maxY (8-1206)

[0943] sumD0[x][y]=∑ i Σ j filtD0[h (x<<2)+i -xCtb][v (y<<2)+j -yCtb], where i = -2..5, j = minY..maxY (8-1207)

[0944] sumD1[x][y]=Σ i Σ j filtD1[h (x<<2)+i -xCtb][v (y<<2)+j -yCtb], where i = -2..5, j = minY..maxY (8-1208)

[0945] sumOfHV[x][y]=sumH[x][y]+sumV[x][y] (8-1209)

[0946] 4. The variables dir1[x][y], dir2[x][y] and dirS[x][y] with x,y = 0..CtbSizeY–1 are derived as follows:

[0947] – The variables hv1, hv0 and dirHV are derived as follows:

[0948] …

[0949] – The variables d1, d0 and dirD are derived as follows:

[0950] …

[0951] 5. The derivation of the variable avgVar[x][y] with x,y=0..CtbSizeY-1 is as follows:

[0952] varTab[ ]={0,1,2,2,2,2,2,3,3,3,3,3,3,3,3,4} (8-1227)

[0953] avgVar[x][y]=varTab[Clip3(0,15,(sumOfHV[x>>2][y>>2]*ac)>>(3+BitDepth Y ))] (8-1228)

[0954] 6. The classification filter index array filtIdx[x][y] and the transpose index array transposeIdx[x][y] with x=y=0..CtbSizeY–1 are derived as follows:

[0955] transposeTable[]={0,1,0,2,2,3,1,3}

[0956] transposeIdx[x][y]=transposeTable[dir1[x][y]*2+(dir2[x][y]>>1)]

[0957] filtIdx[x][y]=avgVar[x][y]

[0958] When dirS[x][y] is not equal to 0, filtIdx[x][y] is modified as follows:

[0959] filtIdx[x][y]+=(((dir1[x][y]&0x1)<<1)+dirS[x][y])*5 (8-1229)

[0960] 5.5 Example #5

[0961] For samples located at various borders (e.g., strip / brick borders, 360-degree virtual borders), the fill process is called only once. How many rows are filled on each side depends on the position of the current sample relative to the border.

[0962] In one example, the ALF two-side filling method is applied. Alternatively, in addition, in the symmetric two-side filling method, when the sample point is at two boundaries, for example, one boundary on the upper side and one boundary on the lower side, as shown in FIG. Figure 27 As shown in , how many samples are filled is determined by the closer boundary. At the same time, when deriving classification information, only Figure 27 4 rows between the two boundaries.

[0963] Figure 26 An example of a filling method is shown if 4 rows of samples have two boundaries. In one example, Figure 26 The first boundary in may be an ALF virtual boundary; Figure 25 The second boundary in can be a strip / slice / brick boundary or a 360-degree virtual boundary.

[0964] 5.6 Example #6

[0965] 8.8.5.2 Codec Tree Block Filtering for Luma Samples

[0966] The inputs to this process are:

[0967] - Before adaptive loop filtering, the reconstructed luminance picture sample array recPicture L .

[0968] -Filtered and reconstructed brightness picture sample array alfPicture L .

[0969] -Specifies the luma position (xCtb, yCtb) of the top left sample of the current luma codec treeblock relative to the top left sample of the current picture.

[0970] The output of this process is the modified filtered reconstructed luminance picture sample array alfPicture L .

[0971] Use the position (xCtb, yCtb) and the reconstructed luminance picture sample array recPicture L The filter index derivation process of clause 8.8.5.3 is called with as input, and filtIdx[x][y] and transposeIdx[x][y] with x,y=0..CtbSizeY-1 as output.

[0972] - The position (h) of each corresponding luma sample in a given luma sample array recPicture (where i, j = -3...3) x+i ,v y+j ) is derived as follows:

[0973] If pps_loop_filter_across_virtual_boundaries_disabled_flag is equal to 1 for any n=0...pps_num_ver_virtual_boundaries-1, and xCtb+x-PpsVirtualBoundariesPosX[n] is greater than or equal to 0 and less than 3, the following applies:

[0974] h x+i =Clip3(PpsVirtualBoundariesPosX[n],pic_width_in_luma_samples-1,xCtb+x+i) (8-1197)

[0975] Otherwise, if pps_loop_filter_across_virtual_boundaries_disabled_flag is equal to 1 for any n = 0..pps_num_ver_virtual_boundaries–1, and PpsVirtualBoundariesPosX[n]-xCtb-x is greater than 0 and less than 4, the following applies:

[0976] h x+i =Clip3(0,PpsVirtualBoundariesPosX[n]-1,xCtb+x+i) (8-1198)

[0977] - Otherwise, the following applies:

[0978] h x+i =Clip3(0,pic_width_in_luma_samples-1,xCtb+x+i) (8-1199)

[0979] -[[If pps_loop_filter_across_virtual_boundaries_disabled_flag is equal to 1 for any n=0..pps_num_hor_virtual_boundaries–1, and yCtb+y−PpsVirtualBoundariesPosY[n] is greater than or equal to 0 and less than 3, the following applies:

[0980] v y+j =Clip3(PpsVirtualBoundariesPosY[n],pic_height_in_luma_samples-1,yCtb+y+j) (8-1200)

[0981] Otherwise, if pps_loop_filter_across_virtual_boundaries_disabled_flag is equal to 1 for any n = 0..pps_num_hor_virtual_boundaries–1, and PpsVirtualBoundariesPosY[n]-yCtb-y is greater than 0 and less than 4, the following applies:

[0982] v y+j =Clip3(0,PpsVirtualBoundariesPosY[n]-1,yCtb+y+j) (8-1201)]]

[0983] -[[Otherwise,]] the following applies:

[0984] v y+j =Clip3(0,pic_height_in_luma_samples-1,yCtb+y+j) (8-1202)

[0985] -[[The variable applyVirtualBoundary is derived as follows:

[0986] - If one or more of the following conditions are true, applyVirtualBoundary is set to 0.

[0987] - The bottom boundary of the current codec tree block is the bottom boundary of the picture.

[0988] - The bottom boundary of the current codec tree block is the bottom boundary of the brick, and loop_filter_across_bricks_enabled_flag is equal to 0.

[0989] - The bottom boundary of the current codec treeblock is the bottom boundary of the slice, and loop_filter_across_slices_enabled_flag is equal to 0.

[0990] - The bottom boundary of the current codec treeblock is one of the bottom virtual boundaries of the picture, and pps_loop_filter_across_virtual_boundaries_disabled_flag is set equal to 1.

[0991] - Otherwise, applyVirtualBoundary is set equal to 1. ]]

[0992]

[0993]

[0994] condition r1 r2 r3 (y==CtbSizeY-5||y==CtbSizeY-4)&&(applyVirtualBoundary==1) 0 0 0 (y==CtbSizeY-6||y==CtbSizeY-3)&&(applyVirtualBoundary==1) 1 1 1 (y==CtbSizeY-7||y==CtbSizeY-2)&&(applyVirtualBoundary==1) 1 2 2 other 1 2 3

[0995]

[0996] 8.8.5.3 Derivation of filter index and ALF transpose for luma samples

[0997] The inputs to this process are:

[0998] – luma position (xCtb, yCtb), which specifies the top left sample of the current luma codec treeblock relative to the top left sample of the current picture,

[0999] – Before adaptive loop filtering, the reconstructed luminance picture sample array recPicture L .

[1000] The output of this processing is:

[1001] – Classification filter index array filtIdx[x][y], where x, y = 0..CtbSizeY–1.

[1002] – Transpose index array transposeIdx[x][y], where x, y = 0..CtbSizeY-1.

[1003] The position (h) of each corresponding luma sample (x, y) in a given array recPicture of luma samples with i, j = -2..5 x+i ,v y+j ) is obtained as follows:

[1004] – If pps_loop_filter_across_virtual_boundaries_disabled_flag is equal to 1 for any n = 0..pps_num_ver_virtual_boundaries-1, and xCtb+x-PpsVirtualBoundariesPosX[n] is greater than or equal to 0 and less than 2, the following applies:

[1005] h x+i =Clip3(PpsVirtualBoundariesPosX[n],pic_width_in_luma_samples-1,xCtb+x+i) (8-1208)

[1006] Otherwise, if pps_loop_filter_across_virtual_boundaries_disabled_flag is equal to 1 for any n = 0..pps_num_ver_virtual_boundaries-1, and PpsVirtualBoundariesPosX[n]-xCtb-x is greater than 0 and less than 6, the following applies:

[1007] h x+i =Clip3(0,PpsVirtualBoundariesPosX[n]-1,xCtb+x+i) (8-1209)

[1008] – Otherwise, the following applies:

[1009] h x+i =Clip3(0,pic_width_in_luma_samples-1,xCtb+x+i) (8-1210)

[1010] [[– If pps_loop_filter_across_virtual_boundaries_disabled_flag is equal to 1 for any n = 0..pps_num_hor_virtual_boundaries–1, and yCtb+y−PpsVirtualBoundariesPosY[n] is greater than or equal to 0 and less than 2, the following applies:

[1011] v y+j =Clip3(PpsVirtualBoundariesPosY[n],pic_height_in_luma_samples-1,yCtb+y+j) (8-1211)

[1012] – Otherwise, if pps_loop_filter_across_virtual_boundaries_disabled_flag is equal to 1 for any n = 0..pps_num_hor_virtual_boundaries-1, and PpsVirtualBoundariesPosY[n]-yCtb-y is greater than 0 and less than 6, the following applies:

[1013] v y+j =Clip3(0,PpsVirtualBoundariesPosY[n]-1,yCtb+y+j) (8-1212)

[1014] – Otherwise, the following applies:

[1015] – If yCtb+CtbSizeY is greater than or equal to pic_height_in_luma_samples, the following applies:

[1016] v y+j =Clip3(0,pic_height_in_luma_samples-1,yCtb+y+j) (8-1213)

[1017] – Otherwise, if y is less than CtbSizeY-4, the following applies:

[1018] v y+j =Clip3(0,yCtb+CtbSizeY-5,yCtb+y+j) (8-1214)

[1019] – Otherwise, the following applies:

[1020] v y+j =Clip3(yCtb+CtbSizeY-4,pic_height_in_luma_samples-1,yCtb+y+j)(8-1215)]]

[1021]

[1022] The classification filter index array filtIdx and the transpose index array transposeIdx are obtained by the following sequential steps:

[1023] 1. The variables filtH[x][y], filtV[x][y], filtD0[x][y], and filtD1[x][y] (with x, y = -2..CtbSizeY+1) are derived as follows:

[1024] – If both x and y are even or both x and y are uneven, then the following applies:

[1025] filtH[x][y]=Abs((recPicture[h x ,v y ]<<1)-recPicture[h x-1 ,v y ]-recPicture[h x+1 ,v y ]) (8-1216)

[1026] filtV[x][y]=Abs((recPicture[h x ,v y ]<<1)-recPicture[h x ,v y-1 ]-recPicture[h x ,v y+1 ]) (8-1217)

[1027] filtD0[x][y]=Abs((recPicture[h x ,v y ]<<1)-recPicture[h x-1 ,v y-1 ]-recPicture[h x+1 ,v y+1 ])

[1028] (8-1218)

[1029] filtD1[x][y]=Abs((recPicture[h x ,v y ]<<1)-recPicture[h x+1 ,v y-1 ]-recPicture[h x-1 ,v y+1 ])

[1030] (8-1219)

[1031] – Otherwise, set filtH[x][y], filtV[x][y], filtD0[x][y], and filtD1[x][y] equal to 0.

[1032] 2. The variables minY, maxY, and ac are derived as follows:

[1033] – If (y<<2) is equal to (CtbSizeY-8) and (yCtb+CtbSizeY) is less than pic_height_in_luma_samples-1, then minY is set equal to -2, maxY is set equal to 3, and ac is set equal to 96.

[1034] – Otherwise, if (y<<2) is equal to (CtbSizeY-4) and (yCtb+CtbSizeY) is less than pic_height_in_luma_samples-1, then minY is set equal to 0, maxY is set equal to 5, and ac is set equal to 96.

[1035]

[1036]

[1037] –[[Otherwise, set minY equal to -2, maxY equal to 5, and ac equal to 64. ]]

[1038] 3. The variables sumH[x][y], sumV[x][y], sumD0[x][y], sumD1[x][y], and sumOfHV[x][y] (where x, y = 0 ... (CtbSizeY-1)>>2) are derived as follows:

[1039] sumH[x][y]=Σ i Σ j filtH[h (x<<2)+i -xCtb][v (y<<2)+j -yCtb], where i = -2..5, j = minY..maxY (8-1220)

[1040] sumV[x][y]=Σ i ∑ j filtV[h (x<<2)+i -xCtb][v (y<<2)+j -yCtb], where i = -2..5, j = minY..maxY (8-1221)

[1041] sumD0[x][y]=∑ i ∑ jfiltD0[h (x<<2)+i -xCtb][v (y<<2)+j -yCtb], where i = -2..5, j = minY..maxY (8-1222)

[1042] sumD1[x][y]=∑ i ∑ j filtD1[h (x<<2)+i -xCtb][v (y<<2)+j -yCtb], where i = -2..5, j = minY..maxY (8-1223)

[1043] sumOfHV[x][y]=sumH[x][y]+sumV[x][y] (8-1224) ...

[1044] 8.8.5.4 Codec Tree Block Filtering for Chroma Samples

[1045] The input to this process is

[1046] -reconstructed chroma picture sample array recPicture before adaptive loop filtering,

[1047] -Filtered and reconstructed chrominance picture sample array alfPicture,

[1048] -Specifies the chroma position (xCtbC, yCtbC) of the top left sample of the current chroma codec tree block relative to the top left sample of the current picture.

[1049] The output of this process is the modified filtered reconstructed chroma picture sample array alfPicture.

[1050] The width and height ctbWidthC and ctbHeightC of the current chroma codec tree block are derived as follows:

[1051] ctbWidthC=CtbSizeY / SubWidthC (8-1245)

[1052] ctbHeightC=CtbSizeY / SubHeightC (8-1246)

[1053] To derive the filtered reconstructed chroma samples alfPicture[x][y], each reconstructed chroma sample in the current chroma codec tree block recPicture[x][y] is filtered as follows, where x=0...ctbWidthC-1, y=0...ctbHeightC-1.

[1054] - In a given chroma sample array recPicture (where i, j = -2...2), the position (h) of each corresponding chroma sample (x, y) x+i ,v y+j ) is derived as follows:

[1055] If pps_loop_filter_across_virtual_boundaries_disabled_flag is equal to 1 for any n=0...pps_num_ver_virtual_boundaries-1, and xCtbC+x-PpsVirtualBoundariesPosX[n] / SubWidthC is greater than or equal to 0 and less than 2, the following applies:

[1056] h x+i =Clip3(PpsVirtualBoundariesPosX[n] / SubWidthC,pic_width_in_luma_samples / SubWidthC-1,xCtbC+x+i) (8-1247)

[1057] Otherwise, if pps_loop_filter_across_virtual_boundaries_disabled_flag is equal to 1 for any n=0...pps_num_ver_virtual_boundaries-1, and PpsVirtualBoundariesPosX[n] / SubWidthC-xCtbC-x is greater than 0 and less than 3, the following applies:

[1058] h x+i =Clip3(0,PpsVirtualBoundariesPosX[n] / SubWidthC-1,xCtbC+x+i) (8-1248)

[1059] - Otherwise, the following applies:

[1060] h x+i =Clip3(0,pic_width_in_luma_samples / SubWidthC-1,xCtbC+x+i) (8-1249)

[1061] -[[If, for any n=0...pps_num_hor_virtual_boundaries−1, pps_loop_filter_across_virtual_boundaries_disabled_flag is equal to 1, and yCtbC+y−PpsVirtualBoundariesPosY[n] / SubHeightC is greater than or equal to 0 and less than 2, the following applies:

[1062] v y+j =Clip3(PpsVirtualBoundariesPosY[n] / SubHeightC,

[1063] pic_height_in_luma_samples / SubHeightC-1,yCtbC+y+j) (8-1250)

[1064] Otherwise, if pps_loop_filter_across_virtual_boundaries_disabled_flag is equal to 1 for any n = 0 ... pps_num_hor_virtual_boundaries - 1, and PpsVirtualBoundariesPosY[n] / SubHeightC-yCtbC-y is greater than 0 and less than 3, the following applies:

[1065] v y+j =Clip3(0,PpsVirtualBoundariesPosY[n] / SubHeightC-1,yCtbC+y+j)(8-1251)

[1066] Otherwise, the following applies:

[1067] v y+j =Clip3(0,pic_height_in_luma_samples / SubHeightC-1,yCtbC+y+j)(8-1252)

[1068] -[[The variable applyVirtualBoundary is derived as follows:

[1069] - If one or more of the following conditions are true, applyVirtualBoundary is set to 0.

[1070] - The bottom boundary of the current codec tree block is the bottom boundary of the picture.

[1071] - The bottom boundary of the current codec tree block is the bottom boundary of the brick, and loop_filter_across_bricks_enabled_flag is equal to 0.

[1072] - The bottom boundary of the current codec treeblock is the bottom boundary of the slice, and loop_filter_across_slices_enabled_flag is equal to 0.

[1073] - The bottom boundary of the current codec treeblock is one of the bottom virtual boundaries of the picture, and pps_loop_filter_across_virtual_boundaries_disabled_flag is set equal to 1.

[1074] - Otherwise, applyVirtualBoundary is set equal to 1. ]]

[1075] – The variables boundaryPos1 and boundaryPos2 are derived by invoking the vertical boundary position derivation process for luma samples as specified in 8.8.5.5, with yCtb equal to yCtb and y equal to y.

[1076] – Set the variable boundaryPos1 equal to boundaryPos1 / SubWidthC.

[1077] – Set the variable boundaryPos2 equal to boundaryPos2 / SubWidthC.

[1078] – Depending on the horizontal luma sample position y and applyVirtualBoundary, the reconstructed sample offsets r1 and r2 are specified in Table 8-24.

[1079] –The variable curr is derived as follows:

[1080] curr=recPicture[h x ,v y ] (8-1253)

[1081] – The array f[j] of chroma filter coefficients and the array c[j] of chroma clipping values ​​are derived as follows, where j = 0..5:

[1082] f[j]=AlfCoeff C [slice_alf_aps_id_chroma][j] (8-1254)

[1083] c[j]=AlfClip C[slice_alf_aps_id_chroma][j] (8 - 1255)

[1084] – The sum of variables is derived as follows:

[1085] sum = f[0] * (Clip3(-c[0], c[0], recPicture[h x , v y+r2 - curr) + Clip3(-c[0], c[0], recPicture[h x , v y-r2 - curr)) + f[1] * (Clip3(-c[1], c[1], recPicture[h x+1 , v y+r1 - curr) + Clip3(-c[1], c[1], recPicture[h x-1 , v y-r1 - curr)) + f[2] * (Clip3(-c[2], c[2], recPicture[h x , v y+r1 - curr) + Clip3(-c[2], c[2], recPicture[h x , v y-r1 - curr)) + f[3] * (Clip3(-c[3], c[3], recPicture[h x-1 , v y+r1 - curr) + Clip3(-c[3], c[3], recPicture[h x+1 , v y-r1 - curr)) + f[4] * (Clip3(-c[4], c[4], recPicture[h x+2 , v y - curr) + Clip3(-c[4], c[4], recPicture[h x-2 , v y - curr)) + f[5] * (Clip3(-c[5], c[5], recPicture[h x+1 , v y - curr) + Clip3(-c[5], c[5], recPicture[h x-1 , v y - curr)) (8 - 1256)

[1086] sum = curr + ((sum + 64) >> 7) (8 - 1257)

[1087] – The modified filtered reconstructed chrominance picture sample alfPicture[xCtbC+x][yCtbC+y] is derived as follows:

[1088] – If pcm_loop_filter_disabled_flag and pcm_flag[(xCtbC+x)*SubWidthC][(yCtbC+y)*SubHeightC] are both equal to 1, the following applies:

[1089] alfPicture[xCtbC+x][yCtbC+y]=recPicture L [h x ,v y ] (8-1258)

[1090] – Otherwise (pcm_loop_filter_disabled_flag is equal to 0 or pcm_flag[x][y] is equal to 0), the following applies:

[1091] alfPicture[xCtbC+x][yCtbC+y]=Clip3(0,(1< <BitDepth C )-1,sum) (8-1259)

[1092] [[Table 8-25 - Specification of r1 and r2 according to horizontal luma sample position y and applyVirtualBoundary]]

[1093] [[condition r1 r2 (y==ctbHeightC-2||y==ctbHeightC-3)&&(applyVirtualBoundary==1) 0 0 (y==ctbHeightC-1||y==ctbHeightC-4)&&(applyVirtualBoundary==1) 1 1 otherwise 1 2]]

[1094]

[1095]

[1096]

[1097]

[1098]

[1099] 5.7 Example #7

[1100] For a CTU, it may not coincide with any boundary (e.g., picture / slice / slice / brick / sub-picture boundary). However, it may be necessary to access samples outside the current unit (e.g., picture / slice / slice / brick / sub-picture). If filtering across slice boundaries is disabled (e.g., loop_filter_across_slices_enabled_flag is false), samples need to be padded outside the current unit.

[1101] For example, for Figure 28 The sample point 2801 (for example, the luma sample point) in Figure 29 Fill the samples used in the ALF filtering process as shown in .

[1102] 5.8 Example #8

[1103] In this embodiment, the following main ideas are applied:

[1104] When ALF virtual boundary is enabled:

[1105] - For CTUs that are not in the last row of CTUs in a picture (for example, the bottom boundary of the CTU is not the bottom boundary of the picture or exceeds the bottom boundary of the picture), enable ALF virtual boundaries, that is, a CTU can be divided into two or more parts, and the samples in one part are not allowed to use the samples in the other part.

[1106] - For a CTU located in the last row of CTUs in a picture (for example, the bottom boundary of the CTU is the bottom boundary of the picture or exceeds the bottom boundary of the picture), enable ALF virtual boundary, that is, a CTU may be divided into two or more parts, and the samples in one part are not allowed to use the samples in the other part.

[1107] When filling borders (including ALF virtual borders, strip / slice / brick / sub-picture borders, and "360 virtual borders") during classification processing:

[1108] For samples located at one (or more) boundaries, when adjacent samples across the boundaries are not allowed to be used, single-side filling is performed to fill such adjacent samples.

[1109] When padding boundaries (including ALF virtual boundaries, strip / slice / brick / sub-picture boundaries, and "360 virtual boundaries") during ALF filtering:

[1110] When cross-boundary neighboring samples are not allowed, for samples located at one (or more) boundaries (slice / slice / brick / sub-picture boundaries or "360 virtual boundaries") that coincide with CTU boundaries, perform side padding to fill in such neighboring samples.

[1111] When cross-boundary neighboring samples are not allowed, for samples located at one (or more) boundaries (picture boundaries or “360° virtual boundaries”) that do not coincide with CTU boundaries, one-sided padding is performed to fill such neighboring samples.

[1112] 8.8.5.2 Codec Tree Block Filtering for Luma Samples

[1113] The inputs to this process are:

[1114] - Before adaptive loop filtering, the reconstructed luminance picture sample array recPicture L ,

[1115] -Filtered and reconstructed brightness picture sample array alfPicture L ,

[1116] -Specifies the luma position (xCtb, yCtb) of the top left sample of the current luma codec treeblock relative to the top left sample of the current picture.

[1117] The output of this process is the modified filtered reconstructed luminance picture sample array alfPicture L .

[1118] Use the position (xCtb, yCtb) and the reconstructed luminance picture sample array recPicture L The filter index derivation process of clause 8.8.5.3 is called with as input, and filtIdx[x][y] and transposeIdx[x][y] with x,y=0..CtbSizeY-1 as output.

[1119] For the filtered reconstructed luminance sample alfPicture L [x][y], each reconstructed luma sample in the current luma codec tree block recPictureL[x][y] is filtered as follows (x, y = 0..CtbSizeY-1):

[1120] - The luma filter coefficient array f[j] and luma clipping value array c[j] corresponding to the filter specified by filtIdx[x][y] are derived as follows, j = 0...11:

[1121] -If AlfCtbFiltSetIdxY[xCtb>>Log2CtbSize][yCtb>>Log2CtbSize]

[1122] If less than 16, the following applies:

[1123] i=AlfCtbFiltSetIdxY[xCtb>>Log2CtbSize][yCtb>>Log2CtbSize] (8-1187)

[1124] f[j]=AlfFixFiltCoeff[AlfClassToFiltMap[i][filtidx]][j] (8-1188)

[1125] c[j]=2 BitdepthY (8-1189)

[1126] Otherwise (AlfCtbFiltSetIdxY[xCtb>>Log2CtbSize][yCtb>>Log2CtbSize] is greater than or equal to 16), the following applies:

[1127] i=slice_alf_aps_id_luma[AlfCtbFiltSetIdxY[xCtb>>Log2CtbSize][yCtb>>Log2CtbSize]-16] (8-1190)

[1128] f[j]=AlfCoeff L [i][filtIdx[x][y]][j] (8-1191)

[1129] c[j]=AlfClip L [i][filtIdx[x][y]][j] (8-1192)

[1130] - The luma filter coefficients and shear value index dx derived from transposeIdx[x][y] are as follows:

[1131] If transposeIndex[x][y] is equal to 1, then the following applies:

[1132] idx[ ]={9,4,10,8,1,5,11,7,3,0,2,6} (8-1193)

[1133] Otherwise, if transposeIndex[x][y] is equal to 2, the following applies:

[1134] idx[ ]={0,3,2,1,8,7,6,5,4,9,10,11} (8-1194)

[1135] Otherwise, if transposeIndex[x][y] is equal to 3, then the following applies:

[1136] idx[ ]={9,8,10,4,3,7,11,5,1,0,2,6} (8-1195)

[1137] - Otherwise, the following applies:

[1138] idx[ ]={0,1,2,3,4,5,6,7,8,9,10,11} (8-1196)

[1139] - The position (h) of each corresponding luma sample in a given luma sample array recPicture (where i, j = -3...3) x+i ,v y+j ) is derived as follows:

[1140] – if pps_loop_filter_across_virtual_boundaries_disabled_flag is equal to 1 for any n = 0..pps_num_ver_virtual_boundaries–1, And xCtb+x-PpsVirtualBoundariesPosX[n] is greater than or equal to 0 and less than 3, then the following applies:

[1141] h x+i =Clip3(PpsVirtualBoundariesPosX[n],pic_width_in_luma_samples-1,xCtb+x+i) (8-1197)

[1142] – Otherwise, if pps_loop_filter_across_virtual_boundaries_disabled_flag is equal to 1 for any n = 0..pps_num_ver_virtual_boundaries–1, And PpsVirtualBoundariesPosX[n]-xCtb-x is greater than 0 and less than 4, then the following applies:

[1143] h x+i =Clip3(0,PpsVirtualBoundariesPosX[n]-1,xCtb+x+i) (8-1198)

[1144] – Otherwise, the following applies:

[1145] h x+i=Clip3(0,pic_width_in_luma_samples-1,xCtb+x+i) (8-1199)

[1146] –[[When in position (h x ,v y ) contains luma samples on a sub-picture, the following applies:

[1147] h x+i =Clip3(SubPicLeftBoundaryPos,SubPicRightBoundaryPos,h x+i )(8-1184)]]

[1148] – if pps_loop_filter_across_virtual_boundaries_disabled_flag is equal to 1 for any n = 0..pps_num_hor_virtual_boundaries–1, And yCtb+y-PpsVirtualBoundariesPosY[n] is greater than or equal to 0 and less than 3, then the following applies:

[1149] v y+j =Clip3(PpsVirtualBoundariesPosY[n],pic_height_in_luma_sample

[1150] s-1,yCtb+y+j)(8-1200)

[1151] – Otherwise, if pps_loop_filter_across_virtual_boundaries_disabled_flag is equal to 1 for any n = 0..pps_num_hor_virtual_boundaries–1, And PpsVirtualBoundariesPosY[n]-yCtb-y is greater than 0 and less than 4, then the following applies:

[1152] v y+j =Clip3(0,PpsVirtualBoundariesPosY[n]-1,yCtb+y+j) (8-1201)

[1153] – Otherwise, the following applies:

[1154] v y+j =Clip3(0,pic_height_in_luma_samples-1,yCtb+y+j) (8-1202)

[1155] –[[When in position (h x ,v y ) contains luma samples on a sub-picture, the following applies:

[1156] v y+j =Clip3(SubPicTopBoundaryPos,SubPicBotBoundaryPos,v y+j ) (8-1184)

[1157] -The variable applyVirtualBoundary is derived as follows:

[1158] - If one or more of the following conditions are true, applyVirtualBoundary is set to 0.

[1159] - The bottom boundary of the current codec tree block is the bottom boundary of the picture.

[1160] - The bottom boundary of the current codec tree block is the bottom boundary of the brick, and loop_filter_across_bricks_enabled_flag is equal to 0.

[1161] - The bottom boundary of the current codec treeblock is the bottom boundary of the slice, and loop_filter_across_slices_enabled_flag is equal to 0.

[1162] – The bottom boundary of the current codec treeblock is the bottom boundary of the sub-picture, and the loop_filter_across_sub_pic_enabled_flag[SubPicIdx] of the sub-picture containing the luma sample at position (hx,vy) is equal to 0.

[1163] - The bottom boundary of the current codec treeblock is one of the bottom virtual boundaries of the picture, and pps_loop_filter_across_virtual_boundaries_disabled_flag is set equal to 1.

[1164] - Otherwise, applyVirtualBoundary is set equal to 1. ]]

[1165]

[1166] –According to the horizontal brightness sample position y and [[applyVirtualBoundary]] The reconstructed sample point offsets r1, r2 and r3 are specified in Table 8-24.

[1167]

[1168] -The variable curr is derived as follows:

[1169] curr=recPicture L [h x ,v y ] (8-1203)

[1170] -The derivation of the variable sum is as follows:

[1171] sum=f[idx[0]]*(Clip3(-c[idx[0]],c[idx[0]],recPicture L [h x ,v y+r3 ]-curr)+Clip3(-c[idx[0]],c[idx[0]],recPicture L [h x ,v y-r3 ]-curr))+f[idx[1]]*(Clip3(-c[idx[1]],c[idx[1]],recPicture L [h x+ c 1,v y+r2 ]-curr)+Clip3(-c[idx[1]],c[idx[1]],recPicture L [h x- c 1,v y-r2 ]-curr))+f[idx[2]]*(Clip3(-c[idx[2]],c[idx[2]],recPicture L [h x ,v y+r2 ]-curr)+Clip3(-c[idx[2]],c[idx[2]],recPicture L [h x ,vy-r2 ]-curr))+f[idx[3]]*(Clip3(-c[idx[3]],c[idx[3]],recPicture L [h x- c 1,v y+r2 ]-curr)+Clip3(-c[idx[3]],c[idx[3]],recPicture L [h x+ c 1,v y-r2 ]-curr))+f[idx[4]]*(Clip3(-c[idx[4]],c[idx[4]],recPicture L [h x+ c 2,v y+r1 ]-curr)+Clip3(-c[idx[4]],c[idx[4]],recPicture L [h x- c 2,v y-r1 ]-curr))+f[idx[5]]*(Clip3(-c[idx[5]],c[idx[5]],recPicture L [h x+ c 1,v y+r1 ]-curr)+Clip3(-c[idx[5]],c[idx[5]],recPicture L [h x- c 1,v y-r1 ]-curr))+f[idx[6]]*(Clip3(-c[idx[6]],c[idx[6]],recPicture L [h x ,v y+r1 ]-curr)+Clip3(-c[idx[6]],c[idx[6]],recPicture L [h x ,v y-r1 ]-curr))+f[idx[7]]*(Clip3(-c[idx[7]],c[idx[7]],recPicture L [h x- c 1,v y+r1]-curr)+Clip3(-c[idx[7]],c[idx[7]],recPicture L [h x+ c 1,v y-r1 ]-curr))+f[idx[8]]*(Clip3(-c[idx[8]],c[idx[8]],recPicture L [h x- c 2,v y+r1 ]-curr)+Clip3(-c[idx[8]],c[idx[8]],recPicture L [h x+ c 2,v y-r1 ]-curr))+f[idx[9]]*(Clip3(-c[idx[9]],c[idx[9]],recPicture L [h x+ c 3,v y ]-curr)+Clip3(-c[idx[9]],c[idx[9]],recPicture L [h x- c 3,v y ]-curr))+f[idx

[10] ]*(Clip3(-c[idx

[10] ],c[idx

[10] ],recPicture L [h x+ c 2,v y ]-curr)+Clip3(-c[idx

[10] ],c[idx

[10] ],recPicture L [h x- c 2,v y ]-curr))+f[idx

[11] ]*(Clip3(-c[idx

[11] ],c[idx

[11] ],recPicture L [h x+ c 1,v y ]-curr)+Clip3(-c[idx

[11] ],c[idx

[11] ],recPicture L [h x- c 1,v y ]-curr)) (8-1204)

[1172] sum=curr+((sum+64)>>7) (8-1205)

[1173] - The derivation of the modified filtered reconstructed luminance picture sample alfPictureL[xCtb+x][yCtb+y] is as follows:

[1174] - If pcm_loop_filter_disabled_flag and pcm_flag[xCtb+x][yCtb+y] are both equal to 1, the following applies:

[1175] alfPicture L [xCtb+x][yCtb+y]=recPicture L [h x ,v y ] (8-1206)

[1176] Otherwise (pcm_loop_filter_disabled_flag is equal to 0 or pcm_flag[x][y] is equal to 0), the following applies:

[1177] alfPicture L [xCtb+x][yCtb+y]=Clip3(0,(1< <BitDepth Y )-1,sum) (8-1207)

[1178] Table 8-24 – According to the horizontal luminance sample position y, Specification of r1, r2, and r3 for [[and applyVirtualBoundary]]

[1179] [[condition r1 r2 r3 (y==CtbSizeY-5||y==CtbSizeY-4)&&(applyVirtualBoundary==1) 0 0 0 (y==CtbSizeY-6||y==CtbSizeY-3)&&(applyVirtualBoundary==1) 1 1 1 (y==CtbSizeY-7||y==CtbSizeY-2)&&(applyVirtualBoundary==1) 1 2 2 other 1 2 3]]

[1180]

[1181]

[1182]

[1183] 8.8.5.3 Derivation of filter index and ALF transpose for luma samples

[1184] The inputs to this process are:

[1185] – luma position (xCtb, yCtb), which specifies the top left sample of the current luma codec treeblock relative to the top left sample of the current picture,

[1186] – Before adaptive loop filtering, the reconstructed luminance picture sample array recPicture L .

[1187] The output of this processing is:

[1188] – Classification filter index array filtIdx[x][y], where x, y = 0..CtbSizeY–1.

[1189] – Transpose index array transposeIdx[x][y], where x, y = 0..CtbSizeY-1.

[1190] The position (h) of each corresponding luma sample (x, y) in a given array recPicture of luma samples with i, j = -2..5 x+i ,v y+j ) is obtained as follows:

[1191] – If for any n = 0..pps_num_ver_virtual_boundaries-1, If pps_loop_filter_across_virtual_boundaries_disabled_flag is equal to 1, and xCtb+x-PpsVirtualBoundariesPosX[n] is greater than or equal to 0 and less than 2, the following applies: x+i =Clip3(PpsVirtualBoundariesPosX[n],pic_width_in_luma_samples-1,xCtb+x+i) (8-1208)

[1192] – Otherwise, if pps_loop_filter_across_virtual_boundaries_disabled_flag is equal to 1 for any n = 0..pps_num_ver_virtual_boundaries-1, And PpsVirtualBoundariesPosX[n]-xCtb-x is greater than 0 and less than 6, then the following applies: x+i =Clip3(0,PpsVirtualBoundariesPosX[n]-1,xCtb+x+i) (8-1209)

[1193] – Otherwise, the following applies:

[1194] h x+i=Clip3(0,pic_width_in_luma_samples-1,xCtb+x+i) (8-1210)

[1195] –[[When the included position (h x ,v y ) is equal to 0 for the sub-picture of the luma sample at position , the following applies:

[1196] h x+i =Clip3(SubPicLeftBoundaryPos,SubPicRightBoundaryPos,h x+i ) (8-1184)]]

[1197] – if pps_loop_filter_across_virtual_boundaries_disabled_flag is equal to 1 for any n = 0..pps_num_hor_virtual_boundaries–1, and yCtb+y–PpsVirtualBoundariesPosY[n] is greater than or equal to 0 and less than 2, then the following applies:

[1198] v y+j =Clip3(PpsVirtualBoundariesPosY[n],pic_height_in_luma_samples-1,yCtb+y+j) (8-1211)

[1199] – Otherwise, if pps_loop_filter_across_virtual_boundaries_disabled_flag is equal to 1 for any n = 0..pps_num_hor_virtual_boundaries-1, And PpsVirtualBoundariesPosY[n]–yCtb-y is greater than 0 and less than 6, then the following applies:

[1200] v y+j =Clip3(0,PpsVirtualBoundariesPosY[n]-1,yCtb+y+j) (8-1212)

[1201] – Otherwise, the following applies:

[1202] – If yCtb+CtbSizeY is greater than or equal to pic_height_in_luma_samples, the following applies:

[1203] v y+j =Clip3(0,pic_height_in_luma_samples-1,yCtb+y+j) (8-1213)

[1204] –[[Otherwise, if y is less than CtbSizeY-4, the following applies:

[1205] v y+j =Clip3(0,yCtb+CtbSizeY-5,yCtb+y+j) (8-1214)

[1206] – Otherwise, the following applies:

[1207] v y+j =Clip3(yCtb+CtbSizeY-4,

[1208] pic_height_in_luma_samples-1,yCtb+y+j) (8-1215)

[1209] – When the included position (h x ,v y ) is equal to 0 for the sub-picture of the luma sample at position , the following applies:

[1210] v y+j =Clip3(SubPicTopBoundaryPos,SubPicBotBoundaryPos,v y+j ) (8-1184)]]

[1211]

[1212]

[1213] The classification filter index array filtIdx and the transpose index array transposeIdx are obtained by the following sequential steps:

[1214] 1. The variables filtH[x][y], filtV[x][y], filtD0[x][y], and filtD1[x][y] (with x, y = -2..CtbSizeY+1) are derived as follows:

[1215] – If both x and y are even or both x and y are uneven, then the following applies:

[1216] filtH[x][y]=Abs((recPicture[h x ,v y ]<<1)-recPicture[h x-1 ,v y ]-recPicture[h x+1 ,v y ]) (8-1216)

[1217] filtV[x][y]=Abs((recPicture[h x ,v y ]<<1)-recPicture[h x ,v y-1 ]-recPicture[h x ,v y+1 ]) (8-1217)

[1218] filtD0[x][y]=Abs((recPicture[h x ,v y ]<<1)-recPicture[h x-1 ,v y-1 ]-recPicture[h x+1 ,v y+1 ]) (8-1218)

[1219] filtD1[x][y]=Abs((recPicture[h x ,v y ]<<1)-recPicture[h x+1 ,v y-1 ]-recPicture[h x-1 ,v y+1 ]) (8-1219)

[1220] – Otherwise, set filtH[x][y], filtV[x][y], filtD0[x][y], and filtD1[x][y] equal to 0.

[1221] 2.[[The variables minY, maxY, and ac are derived as follows:

[1222] – If (y<<2) is equal to (CtbSizeY-8) and (yCtb+CtbSizeY) is less than pic_height_in_luma_samples-1, then minY is set equal to -2, maxY is set equal to 3, and ac is set equal to 96.

[1223] – Otherwise, if (y<<2) is equal to (CtbSizeY-4) and (yCtb+CtbSizeY) is less than pic_height_in_luma_samples-1, then minY is set equal to 0, maxY is set equal to 5, and ac is set equal to 96. ]]

[1224] 3. The variables sumH[x][y], sumV[x][y], sumD0[x][y], sumD1[x][y], and sumOfHV[x][y] (where x, y = 0 ... (CtbSizeY-1)>>2) are derived as follows:

[1225]

[1226]

[1227]

[1228] sumH[x][y]=Σ i ∑ j filtH[h (x<<2)+i -xCtb][v (y<<2)+j -yCtb], where i = [[-2..5]],j=minY..maxY (8-1220)

[1229] sumV[x][y]=Σ i Σ j filtV[h (x<<2)+i -xCtb][v (y<<2)+j -yCtb], where i = [[-2..5]],j=minY..maxY (8-1221)

[1230] sumD0[x][y]=Σ i Σ j filtD0[h (x<<2)+i -xCtb][v (y<<2)+j -yCtb], where i = [[-2..5]],j=minY..maxY (8-1222)

[1231] sumD1[x][y]=Σ i ∑ j filtD1[h (x<<2)+i -xCtb][v (y<<2)+j -yCtb], where i = [[-2..5]],j=minY..maxY (8-1223)

[1232] sumOfHV[x][y]=sumH[x][y]+sumV[x][y] (8-1224)

[1233] 4. When x, y = 0..CtbSizeY-1, the variables dir1[x][y], dir2[x][y], and dirS[x][y] are derived as follows:

[1234] – The variables hv1, hv0 and dirHV are exported as follows:

[1235] – If sumV[x>>2][y>>2] is greater than sumH[x>>2][y>>2], then the following applies:

[1236] hv1=sumV[x>>2][y>>2] (8-1225)

[1237] hv0=sumH[x>>2][y>>2] (8-1226)

[1238] dirHV=1 (8-1227)

[1239] – Otherwise, the following applies:

[1240] hv1=sumH[x>>2][y>>2] (8-1228)

[1241] hv0=sumV[x>>2][y>>2] (8-1229)

[1242] dirHV=3 (8-1230)

[1243] – The variables d1, d0 and dirD are derived as follows:

[1244] – If sumD0[x>>2][y>>2] is greater than sumD1[x>>2][y>>2], then the following applies:

[1245] d1=sumD0[x>>2][y>>2] (8-1231)

[1246] d0=sumD1[x>>2][y>>2] (8-1232)

[1247] dirD=0 (8-1233)

[1248] – Otherwise, the following applies:

[1249] d1=sumD1[x>>2][y>>2] (8-1234)

[1250] d0=sumD0[x>>2][y>>2] (8-1235)

[1251] dirD=2 (8-1236)

[1252] – The variables hvd1 and hvd0 are exported as follows:

[1253] hvd1=(d1*hv0>hv1*d0)? d1:hv1 (8-1237)

[1254] hvd0=(d1*hv0>hv1*d0)? d0:hv0 (8-1238)

[1255] – The variables dirS[x][y], dir1[x][y] and dir2[x][y] are derived as follows:

[1256] dir1[x][y]=(d1*hv0>hv1*d0)? dirD: dirHV (8-1239)

[1257] dir2[x][y]=(d1*hv0>hv1*d0)? dirHV: dirD (8-1240)

[1258] dirS[x][y]=(hvd1>2*hvd0)? 1: (((hvd1*2>9*hvd0)?2:0) (8-1241)

[1259] 5. The variable avgVar[x][y] with x, y = 0..CtbSizeY-1 is derived as follows:

[1260] varTab[]={0,1,2,2,2,2,2,3,3,3,3,3,3,3,3,4} (8-1242)

[1261] avgVar[x][y]=varTab[Clip3(0,15,(sumOfHV[x>>2][y>>2]*ac)>>(3+BitDepthY))] (8-1243)

[1262] 6. The classification filter index array filtIdx[x][y] and the transpose index array transposeIdx[x][y] of x=y=0..CtbSizeY-1 are derived as follows:

[1263] transposeTable[]={0, 1, 0, 2, 2, 3, 1, 3}

[1264] transposeIdx[x][y]=transposeTable[dir1[x][y]*2+(dir2[x][y]>>1)]

[1265] filtIdx[x][y]=avgVar[x][y]

[1266] When dirS[x][y] is not equal to 0, filtIdx[x][y] is modified as follows:

[1267] filtIdx[x][y]+=(((dir1[x][y]&0x1)<<1)+dirS[x][y])*5(8-1244)

[1268] 8.8.5.4 Codec Tree Block Filtering for Chroma Samples

[1269] The inputs to this process are:

[1270] -The reconstructed chroma picture sample array recPicture before adaptive loop filtering processing.

[1271] -Filtered reconstructed chroma picture sample array alfPicture.

[1272] -Specifies the chroma position (xCtbC, yCtbC) of the top left sample of the current chroma codec tree block relative to the top left sample of the current picture.

[1273] The output of this process is the modified filtered reconstructed chroma picture sample array alfPicture.

[1274] The width and height ctbWidthC and ctbHeightC of the current chroma codec tree block are derived as follows:

[1275] ctbWidthC=CtbSizeY / SubWidthC (8-1245)

[1276] ctbHeightC=CtbSizeY / SubHeightC (8-1246)

[1277] To derive the filtered reconstructed chroma samples alfPicture[x][y], each reconstructed chroma sample in the current chroma codec tree block recPicture[x][y] is filtered as follows, where x=0...ctbWidthC-1, y=0...ctbHeightC-1.

[1278] - In a given chroma sample array recPicture (where i, j = -2...2), the position (h) of each corresponding chroma sample (x, y) x+i ,v y+j ) is derived as follows:

[1279] – if pps_loop_filter_across_virtual_boundaries_disabled_flag is equal to 1 for any n = 0..pps_num_ver_virtual_boundaries–1, AND xCtbC+x-PpsVirtualBoundariesPosX[n] / SubWidthC is greater than or equal to 0 and less than 2, then the following applies:

[1280] h x+i =Clip3(PpsVirtualBoundariesPosX[n] / SubWidthC,

[1281] pic_width_in_luma_samples / SubWidthC-1,xCtbC+x+i) (8-1247)

[1282] – Otherwise, if pps_loop_filter_across_virtual_boundaries_disabled_flag is equal to 1 for any n = 0..pps_num_ver_virtual_boundaries–1, And PpsVirtualBoundariesPosX[n] / SubWidthC-xCtbC–x is greater than 0 and less than 3, then the following applies:

[1283] h x+i =Clip3(0,PpsVirtualBoundariesPosX[n] / SubWidthC-1,xC tbC+x+i)(8-1248)

[1284] – Otherwise, the following applies:

[1285] h x+i =Clip3(0,pic_width_in_luma_samples / SubWidthC-1,xCtbC+x+i) (8-1249)

[1286] –[[When in position (h x ,v y ) contains luma samples on a sub-picture, the following applies:

[1287] h x+i =Clip3(SubPicLeftBoundaryPos / SubWidthC,SubPicRightBoundaryPos / SubWidthC,h x+i ) (8-1184)]]

[1288] – if pps_loop_filter_across_virtual_boundaries_disabled_flag is equal to 1 for any n = 0..pps_num_hor_virtual_boundaries-1, And yCtbC+y-PpsVirtualBoundariesPosY[n] / SubHeightC is greater than or equal to 0 and less than 2, then the following applies:

[1289] v y+j =Clip3(PpsVirtualBoundariesPosY[n] / SubHeightC,pic_height_in_luma_samples / SubHeightC-1,yCtbC+y+j) (8-1250)

[1290] – Otherwise, if pps_loop_filter_across_virtual_boundaries_disabled_flag is equal to 1 for any n = 0..pps_num_hor_virtual_boundaries–1, And PpsVirtualBoundariesPosY[n] / SubHeightC-yCtbC-y is greater than 0 and less than 3, then the following applies:

[1291] v y+j=Clip3(0,PpsVirtualBoundariesPosY[n] / SubHeightC-1,yCtbC+y+j)(8-1251)

[1292] – Otherwise, the following applies:

[1293] v y+j =Clip3(0,pic_height_in_luma_samples / SubHeightC-1,yCtbC+y+j) (8-1252)

[1294] –[[When in position (h x ,v y ) contains luma samples on a sub-picture, the following applies:

[1295] v y+j =Clip3(SubPicTopBoundaryPos / SubWidthC,SubPicBotBoundaryPos / SubWidthC,v y+j ) (8-1184)

[1296] -The variable applyVirtualBoundary is derived as follows:

[1297] - If one or more of the following conditions are true, applyVirtualBoundary is set to 0.

[1298] - The bottom boundary of the current codec tree block is the bottom boundary of the picture.

[1299] - The bottom boundary of the current codec tree block is the bottom boundary of the brick, and loop_filter_across_bricks_enabled_flag is equal to 0.

[1300] - The bottom boundary of the current codec treeblock is the bottom boundary of the slice, and loop_filter_across_slices_enabled_flag is equal to 0.

[1301] – The bottom boundary of the current codec tree block is the bottom boundary of the sub-picture and is at position (h x ,v y ) contains luma samples on the sub-picture loop_filter_across_sub_pic_enabled_flag[SubPicIdx] is equal to 0.

[1302] - The bottom boundary of the current codec treeblock is one of the bottom virtual boundaries of the picture, and pps_loop_filter_across_virtual_boundaries_disabled_flag is set equal to 1.

[1303] - Otherwise, applyVirtualBoundary is set equal to 1. ]]

[1304]

[1305] –According to the horizontal brightness sample position y [[and applyVirtualBoundary]] specify the reconstructed sample offsets r1 and r2 in Table 8-27.

[1306]

[1307] –The variable curr is derived as follows:

[1308] curr=recPicture[h x ,v y ] (8-1253)

[1309] – The chroma filter coefficient array f[j] and the chroma clipping value array c[j] are derived as follows, where j = 0..5:

[1310] f[j]=AlfCoeffC[slice_alf_aps_id_chroma][j] (8-1254)

[1311] c[j]=AlfClipC[slice_alf_aps_id_chroma][j] (8-1255)

[1312] –The variable sum is derived as follows:

[1313] sum=f[0]*(Clip3(-c[0],c[0],recPicture[h x ,v y+r2 ]-curr)+Clip3(-c[0],c[0],recPicture[h x ,v y-r2 ]-curr))+f[1]*(Clip3(-c[1],c[1],recPicture[h x+ c 1,v y+r1 ]-curr)+Clip3(-c[1],c[1],recPicture[hx- c 1,v y-r1 ]-curr))+f[2]*(Clip3(-c[2],c[2],recPicture[h x ,v y+r1 ]-curr)+Clip3(-c[2],c[2],recPicture[h x ,v y-r1 ]-curr))+f[3]*(Clip3(-c[3],c[3],recPicture[h x- c 1,v y+r1 ]-curr)+Clip3(-c[3],c[3],recPicture[h x+ c 1,v y-r1 ]-curr))+f[4]*(Clip3(-c[4],c[4],recPicture[h x+ c 2,v y ]-curr)+Clip3(-c[4],c[4],recPicture[h x- c 2,v y ]-curr))+f[5]*(Clip3(-c[5],c[5],recPicture[h x+ c 1,v y ]-curr)+Clip3(-c[5],c[5],recPicture[h x- c 1,v y ]-curr)) (8-1256)

[1314] sum=curr+(sum+64)>>7) (8-1257)

[1315] - The derivation of the modified filtered reconstructed chrominance picture sample alfPicture[xCtbC+x][yCtbC+y] is as follows:

[1316] - If pcm_loop_filter_disabled_flag and pcm_flag[(xCtbC+x)*SubWidthC][(yCtbC+y)*SubHeightC] are both equal to 1, the following applies:

[1317] alfPicture[xCtbC+x][yCtbC+y]=recPicture L [h x ,v y ] (8-1258)

[1318] Otherwise (pcm_loop_filter_disabled_flag is equal to 0 or pcm_flag[x][y] is equal to 0), the following applies:

[1319] alfPicture[xCtbC+x][yCtbC+y]=Clip3(0,(1< <BitDepth C )-1,sum)(8-1259)

[1320] Table 8-27 – Horizontal luminance sample position y Specification of r1 and r2 for [[and applyVirtualBoundary]]

[1321] [[condition r1 r2 (y==ctbHeightC-2||y==ctbHeightC-3)&&(applyVirtualBoundary==1) 0 0 (y==ctbHeightC-1||y==ctbHeightC-4)&&(applyVirtualBoundary==1) 1 1 otherwise 1 2]]

[1322]

[1323]

[1324]

[1325]

[1326]

[1327]

[1328]

[1329] The specific value -128 used in the above embodiments may be replaced by other values, such as -K, where, for example, K is greater than or not less than the number of rows offset from the bottom boundary of the CTU (eg, K=-5).

[1330] Alternatively, the conditional check of “PpsVirtualBoundariesPosY[n] is not equal to pic_height_in_luma_samples–1 or 0” may be further deleted based on PpsVirtualBoundariesPosY[n] being in the range of 1 to Ceil(pic_height_in_luma_samples÷8)−1 (including 1 and Ceil(pic_height_in_luma_samples÷8)−1).

[1331] Alternatively, if each sample is located at a video unit boundary, a flag may be used to mark whether each sample needs to be processed differently.

[1332] 5.9 Example #9

[1333] In this embodiment, the following main ideas are applied:

[1334] When ALF virtual boundary is enabled:

[1335] - For CTUs that are not in the last row of CTUs in a picture (for example, the bottom boundary of the CTU is not the bottom boundary of the picture or exceeds the bottom boundary of the picture), enable ALF virtual boundaries, that is, a CTU can be divided into two or more parts, and the samples in one part are not allowed to use the samples in the other part.

[1336] - For a CTU located in the last row of CTUs in a picture (for example, the bottom boundary of the CTU is the bottom boundary of the picture or exceeds the bottom boundary of the picture), enable ALF virtual boundary, that is, a CTU may be divided into two or more parts, and the samples in one part are not allowed to use the samples in the other part.

[1337] When filling borders (including ALF virtual borders, strip / slice / brick / sub-picture borders, and "360 virtual borders") during classification processing:

[1338] For samples located at one (or more) boundaries, when adjacent samples across the boundaries are not allowed to be used, single-side filling is performed to fill such adjacent samples.

[1339] When padding boundaries (including ALF virtual boundaries, strip / slice / brick / sub-picture boundaries, and "360 virtual boundaries") during ALF filtering:

[1340] When cross-boundary neighboring samples are not allowed, for samples located at one (or more) boundaries (slice / slice / brick / sub-picture boundaries or "360 virtual boundaries") that coincide with CTU boundaries, perform side padding to fill in such neighboring samples.

[1341] When cross-boundary neighboring samples are not allowed, for samples located at one (or more) boundaries (picture boundaries or “360° virtual boundaries”) that do not coincide with CTU boundaries, one-sided padding is performed to fill such neighboring samples.

[1342] 8.8.5.2 Codec Tree Block Filtering for Luma Samples

[1343] The inputs to this process are:

[1344] - Before adaptive loop filtering, the reconstructed luminance picture sample array recPicture L ,

[1345] -Filtered and reconstructed brightness picture sample array alfPicture L ,

[1346] -Specifies the luma position (xCtb, yCtb) of the top left sample of the current luma codec treeblock relative to the top left sample of the current picture.

[1347] The output of this process is the modified filtered reconstructed luminance picture sample array alfPicture L .

[1348] Use the position (xCtb, yCtb) and the reconstructed luminance picture sample array recPicture L The filter index derivation process of clause 8.8.5.3 is called with as input, and filtIdx[x][y] and transposeIdx[x][y] with x,y=0..CtbSizeY-1 as output.

[1349] For the filtered reconstructed luminance sample alfPicture L [x][y], each reconstructed luma sample in the current luma codec tree block recPictureL[x][y] is filtered as follows (x, y = 0..CtbSizeY-1):

[1350] - The luma filter coefficient array f[j] and luma clipping value array c[j] corresponding to the filter specified by filtIdx[x][y] are derived as follows, j = 0...11:

[1351] -If AlfCtbFiltSetIdxY[xCtb>>Log2CtbSize][yCtb>>Log2CtbSize]

[1352] If less than 16, the following applies:

[1353] i=AlfCtbFiltSetIdxY[xCtb>>Log2CtbSize][yCtb>>Log2CtbSize] (8-1187)

[1354] f[j]=AlfFixFiltCoeff[AlfClassToFiltMap[i][filtidx]][j] (8-1188)

[1355] c[j]=2 BitdepthY(8-1189)

[1356] Otherwise (AlfCtbFiltSetIdxY[xCtb>>Log2CtbSize][yCtb>>Log2CtbSize] is greater than or equal to 16), the following applies:

[1357] i=slice_alf_aps_id_luma[AlfCtbFiltSetIdxY[xCtb>>Log2CtbSize][yCtb>>Log2CtbSize]-6] (8-1190)

[1358] f[j]=AlfCoeff L [i][filtIdx[x][y]][j] (8-1191)

[1359] c[j]=AlfClip L [i][filtIdx[x][y]][j] (8-1192)

[1360] - The luma filter coefficients and shear value index dx derived from transposeIdx[x][y] are as follows:

[1361] If transposeIndex[x][y] is equal to 1, then the following applies:

[1362] idx[ ]={9,4,10,8,1,5,11,7,3,0,2,6} (8-1193)

[1363] Otherwise, if transposeIndex[x][y] is equal to 2, the following applies:

[1364] idx[ ]={0,3,2,1,8,7,6,5,4,9,10,11} (8-1194)

[1365] Otherwise, if transposeIndex[x][y] is equal to 3, then the following applies:

[1366] idx[ ]={9,8,10,4,3,7,11,5,1,0,2,6} (8-1195)

[1367] - Otherwise, the following applies:

[1368] idx[ ]={0,1,2,3,4,5,6,7,8,9,10,11} (8-1196)

[1369] - The position (h) of each corresponding luma sample in a given luma sample array recPicture (where i, j = -3...3) x+i ,v y+j ) is derived as follows:

[1370] – if pps_loop_filter_across_virtual_boundaries_disabled_flag is equal to 1 for any n = 0..pps_num_ver_virtual_boundaries–1, And xCtb+x-PpsVirtualBoundariesPosX[n] is greater than or equal to 0 and less than 3, then the following applies:

[1371] h x+i =Clip3(PpsVirtualBoundariesPosX[n],pic_width_in_luma_samples-1,xCtb+x+i) (8-1197)

[1372] – Otherwise, if pps_loop_filter_across_virtual_boundaries_disabled_flag is equal to 1 for any n = 0..pps_num_ver_virtual_boundaries–1, And PpsVirtualBoundariesPosX[n]-xCtb-x is greater than 0 and less than 4, then the following applies:

[1373] h x+i =Clip3(0,PpsVirtualBoundariesPosX[n]-1,xCtb+x+i) (8-1198)

[1374] – Otherwise, the following applies:

[1375] h x+i =Clip3(0,pic_width_in_luma_samples-1,xCtb+x+i) (8-1199)

[1376] –[[When in position (h x ,v y ) contains luma samples on a sub-picture, the following applies:

[1377] h x+i =Clip3(SubPicLeftBoundaryPos,SubPicRightBoundaryPos,h x+i ) (8-1184)]]

[1378] – if pps_loop_filter_across_virtual_boundaries_disabled_flag is equal to 1 for any n = 0..pps_num_hor_virtual_boundaries–1, And yCtb+y-PpsVirtualBoundariesPosY[n] is greater than or equal to 0 and less than 3, then the following applies:

[1379] v y+j =Clip3(PpsVirtualBoundariesPosY[n],pic_height_in_luma_sample s-1,yCtb+y+j) (8-1200)

[1380] – Otherwise, if pps_loop_filter_across_virtual_boundaries_disabled_flag is equal to 1 for any n = 0..pps_num_hor_virtual_boundaries–1, And PpsVirtualBoundariesPosY[n]-yCtb-y is greater than 0 and less than 4, then the following applies:

[1381] v y+j =Clip3(0,PpsVirtualBoundariesPosY[n]-1,yCtb+y+j) (8-1201)

[1382] – Otherwise, the following applies:

[1383] v y+j =Clip3(0,pic_height_in_luma_samples-1,yCtb+y+j) (8-1202)

[1384] –[[When in position (h x ,v y) contains luma samples on a sub-picture, the following applies:

[1385] v y+j =Clip3(SubPicTopBoundaryPos,SubPicBotBoundaryPos,v y+j ) (8-1184)

[1386] -The variable applyVirtualBoundary is derived as follows:

[1387] - If one or more of the following conditions are true, applyVirtualBoundary is set to 0.

[1388] - The bottom boundary of the current codec tree block is the bottom boundary of the picture.

[1389] - The bottom boundary of the current codec tree block is the bottom boundary of the brick, and loop_filter_across_ bricks_enabled_flag is equal to 0.

[1390] - The bottom boundary of the current codec tree block is the bottom boundary of the slice, and loop_filter_across_ slices_enabled_flag is equal to 0.

[1391] - The bottom boundary of the current codec tree block is the bottom boundary of the sub-picture and is at position (h x ,v y ) contains bright The loop_filter_across_sub_pic_enabled_flag[SubPicIdx] of the sub-picture with the same sample is equal to 0.

[1392] - The bottom boundary of the current codec tree block is one of the bottom virtual boundaries of the picture, and pps_loop_ filter_across_virtual_boundaries_disabled_flag is set equal to 1.

[1393] - Otherwise, applyVirtualBoundary is set equal to 1. ]]

[1394]

[1395] -according to Luminance sample position y The reconstructed sample offsets r1, r2 and r3 are specified in Table 8-24.

[1396]

[1397] -The variable curr is derived as follows:

[1398] curr=recPicture L [h x ,v y ] (8-1203)

[1399] -The derivation of the variable sum is as follows:

[1400] sum=f[idx[0]]*(Clip3(-c[idx[0]],c[idx[0]],recPicture L [hx ,v y+r3 ]-curr)+Clip3(-c[idx[0]],c[idx[0]],recPicture L [h x ,v y-r3 ]-curr))+f[idx[1]]*(Clip3(-c[idx[1]],c[idx[1]],recPicture L [h x+ c 1,v y+r2 ]-curr)+Clip3(-c[idx[1]],c[idx[1]],recPicture L [h x- c 1,v y-r2 ]-curr))+f[idx[2]]*(Clip3(-c[idx[2]],c[idx[2]],recPicture L [h x ,v y+r2 ]-curr)+Clip3(-c[idx[2]],c[idx[2]],recPicture L [h x ,v y-r2 ]-curr))+f[idx[3]]*(Clip3(-c[idx[3]],c[idx[3]],recPicture L [h x- c 1,v y+r2 ]-curr)+Clip3(-c[idx[3]],c[idx[3]],recPicture L [h x+ c 1,v y-r2 ]-curr))+f[idx[4]]*(Clip3(-c[idx[4]],c[idx[4]],recPicture L [h x+ c 2,v y+r1 ]-curr)+Clip3(-c[idx[4]],c[idx[4]],recPicture L [h x- c 2,v y-r1]-curr))+f[idx[5]]*(Clip3(-c[idx[5]],c[idx[5]],recPicture L [h x+ c 1,v y+r1 ]-curr)+Clip3(-c[idx[5]],c[idx[5]],recPicture L [h x- c 1,v y-r1 ]-curr))+f[idx[6]]*(Clip3(-c[idx[6]],c[idx[6]],recPicture L [h x ,v y+r1 ]-curr)+Clip3(-c[idx[6]],c[idx[6]],recPicture L [h x ,v y-r1 ]-curr))+f[idx[7]]*(Clip3(-c[idx[7]],c[idx[7]],recPicture L [h x- c 1,v y+r1 ]-curr)+Clip3(-c[idx[7]],c[idx[7]],recPicture L [h x+ c 1,v y-r1 ]-curr))+f[idx[8]]*(Clip3(-c[idx[8]],c[idx[8]],recPicture L [h x- c 2,v y+r1 ]-curr)+Clip3(-c[idx[8]],c[idx[8]],recPicture L [h x+ c 2,v y-r1 ]-curr))+f[idx[9]]*(Clip3(-c[idx[9]],c[idx[9]],recPicture L [h x+ c 3,v y ]-curr)+Clip3(-c[idx[9]],c[idx[9]],recPictureL [h x- c 3,v y ]-curr))+f[idx

[10] ]*(Clip3(-c[idx

[10] ],c[idx

[10] ],recPicture L [h x+ c 2,v y ]-curr)+Clip3(-c[idx

[10] ],c[idx

[10] ],recPicture L [h x- c 2,v y ]-curr))+f[idx

[11] ]*(Clip3(-c[idx

[11] ],c[idx

[11] ],recPicture L [h x+ c 1,v y ]-curr)+Clip3(-c[idx

[11] ],c[idx

[11] ],recPicture L [h x- c 1,v y ]-curr)) (8-1204)

[1401] sum=curr+((sum+64)>>7) (8-1205)

[1402] - The derivation of the modified filtered reconstructed luminance picture sample alfPictureL[xCtb+x][yCtb+y] is as follows:

[1403] - If pcm_loop_filter_disabled_flag and pcm_flag[xCtb+x][yCtb+y]

[1404] are all equal to 1, then the following applies:

[1405] alfPicture L [xCtb+x][yCtb+y]=recPicture L [h x ,v y ] (8-1206)

[1406] Otherwise (pcm_loop_filter_disabled_flag is equal to 0 or pcm_flag[x][y] is equal to 0), the following applies:

[1407] alfPicture L [xCtb+x][yCtb+y]=Clip3(0,(1< <BitDepth Y )-1,sum) (8-1207)

[1408] Table 8-24 – According to [[Level] Luminance sample position y, Specification of r1, r2, and r3 for [[and applyVirtualBoundary]]

[1409] [[condition r1 r2 r3 (y==CtbSizeY-5||y==CtbSizeY-4)&&(applyVirtualBoundary==1) 0 0 0 (y==CtbSizeY-6||y==CtbSizeY-3)&&(applyVirtualBoundary==1) 1 1 1 (y==CtbSizeY-7||y==CtbSizeY-2)&&(applyVirtualBoundary==1) 1 2 2 otherwise 1 2 3]]

[1410]

[1411]

[1412]

[1413] 8.8.5.3 Derivation of filter index and ALF transpose for luma samples

[1414] The inputs to this process are:

[1415] – luma position (xCtb, yCtb), which specifies the top left sample of the current luma codec treeblock relative to the top left sample of the current picture,

[1416] – Before adaptive loop filtering, the reconstructed luminance picture sample array recPicture L .

[1417] The output of this processing is:

[1418] – Classification filter index array filtIdx[x][y], where x, y = 0..CtbSizeY–1.

[1419] – Transpose index array transposeIdx[x][y], where x, y = 0..CtbSizeY-1.

[1420] The position (h) of each corresponding luma sample (x, y) in a given array recPicture of luma samples with i, j = -2..5 x+i ,v y+j ) is obtained as follows:

[1421] – If for any n = 0..pps_num_ver_virtual_boundaries-1, If pps_loop_filter_across_virtual_boundaries_disabled_flag is equal to 1, and xCtb+x-PpsVirtualBoundariesPosX[n] is greater than or equal to 0 and less than 2, the following applies: x+i =Clip3(PpsVirtualBoundariesPosX[n],pic_width_in_luma_samples-1,xCtb+x+i) (8-1208)

[1422] – Otherwise, if pps_loop_filter_across_virtual_boundaries_disabled_flag is equal to 1 for any n = 0..pps_num_ver_virtual_boundaries-1, And PpsVirtualBoundariesPosX[n]-xCtb-x is greater than 0 and less than 6, then the following applies:

[1423] h x+i =Clip3(0,PpsVirtualBoundariesPosX[n]-1,xCtb+x+i) (8-1209)

[1424] – Otherwise, the following applies:

[1425] h x+i =Clip3(0,pic_width_in_luma_samples-1,xCtb+x+i) (8-1210)

[1426] –[[When the included position (h x ,v y ) is equal to 0 for the sub-picture of the luma sample at position , the following applies:

[1427] h x+i =Clip3(SubPicLeftBoundaryPos,SubPicRightBoundaryPos,h x+i ) (8-1184)]]

[1428] – if pps_loop_filter_across_virtual_boundaries_disabled_flag is equal to 1 for any n = 0..pps_num_hor_virtual_boundaries–1, and yCtb+y–PpsVirtualBoundariesPosY[n] is greater than or equal to 0 and less than 2, then the following applies:

[1429] v y+j =Clip3(PpsVirtualBoundariesPosY[n],pic_height_in_luma_samples-1,yCtb+y+j) (8-1211)

[1430] – Otherwise, if pps_loop_filter_across_virtual_boundaries_disabled_flag is equal to 1 for any n = 0..pps_num_hor_virtual_boundaries-1, And PpsVirtualBoundariesPosY[n]–yCtb-y is greater than 0 and less than 6, then the following applies:

[1431] v y+j =Clip3(0,PpsVirtualBoundariesPosY[n]-1,yCtb+y+j) (8-1212)

[1432] – Otherwise, the following applies:

[1433] –[[If yCtb+CtbSizeY is greater than or equal to pic_height_in_luma_samples, then the following applies:]]

[1434] v y+j =Clip3(0,pic_height_in_luma_samples-1,yCtb+y+j) (8-1213)

[1435] –[[Otherwise, if y is less than CtbSizeY-4, the following applies:

[1436] v y+j =Clip3(0,yCtb+CtbSizeY-5,yCtb+y+j) (8-1214)

[1437] – Otherwise, the following applies:

[1438] v y+j =Clip3(yCtb+CtbSizeY-4,pic_height_in_luma_samples-1,yCtb+y+j)(8-1215)

[1439] – When the included position (h x ,v y ) is equal to 0 for the sub-picture of the luma sample at position , the following applies:

[1440] v y+j =Clip3(SubPicTopBoundaryPos,SubPicBotBoundaryPos,v y+j ) (8-1184)]]

[1441]

[1442]

[1443] The classification filter index array filtIdx and the transpose index array transposeIdx are obtained by the following sequential steps:

[1444] 1. The variables filtH[x][y], filtV[x][y], filtD0[x][y], and filtD1[x][y] (with x, y = -2..CtbSizeY+1) are derived as follows:

[1445] – If both x and y are even or both x and y are uneven, then the following applies:

[1446] filtH[x][y]=Abs((recPicture[h x ,v y ]<<1)-recPicture[h x-1 ,v y ]-recPicture[h x+1 ,v y ]) (8-1216)

[1447] filtV[x][y]=Abs((recPicture[h x ,v y ]<<1)-recPicture[h x ,v y-1 ]-recPicture[h x,v y+1 ]) (8-1217)

[1448] filtD0[x][y]=Abs((recPicture[h x ,v y ]<<1)-recPicture[h x-1 ,v y-1 ]-recPicture[h x+1 ,v y+1 ]) (8-1218)

[1449] filtD1[x][y]=Abs((recPicture[h x ,v y ]<<1)-recPicture[h x+1 ,v y-1 ]-recPicture[h x-1 ,v y+1 ]) (8-1219)

[1450] – Otherwise, set filtH[x][y], filtV[x][y], filtD0[x][y], and filtD1[x][y] equal to 0.

[1451] 2.[[The variables minY, maxY, and ac are derived as follows:

[1452] – If (y<<2) is equal to (CtbSizeY-8) and (yCtb+CtbSizeY) is less than pic_height_in_luma_samples-1, then minY is set equal to -2, maxY is set equal to 3, and ac is set equal to 96.

[1453] – Otherwise, if (y<<2) is equal to (CtbSizeY-4) and (yCtb+CtbSizeY) is less than pic_height_in_luma_samples-1, then minY is set equal to 0, maxY is set equal to 5, and ac is set equal to 96. ]]

[1454]

[1455]

[1456] –[[The variables sumH[x][y], sumV[x][y], sumD0[x][y], sumD1[x][y], and sumOfHV[x][y] (where x, y = 0..(CtbSizeY-1)>>2) are derived as follows:]]

[1457] sumH[x][y]=∑ i Σ j filtH[h (x<<2)+i -xCtb][v (y<<2)+j -yCtb], where i = [[-2..5]],j=minY..maxY (8-1220)

[1458] sumV[x][y]=Σ i Σ j filtV[h (x<<2)+i -xCtb][v (y<<2)+j -yCtb], where i = [[-2..5]],j=minY..maxY (8-1221)

[1459] sumD0[x][y]=Σ i Σ j filtD0[h (x<<2)+i -xCtb][v (y<<2)+j -yCtb], where i = [[-2..5]],j=minY..maxY (8-1222)

[1460] sumD1[x][y]=∑ i ∑ j filtD1[h (x<<2)+i -xCtb][v (y<<2)+j -yCtb], where i = [[-2..5]],j=minY..maxY (8-1223)

[1461] sumOfHV[x][y]=sumH[x][y]+sumV[x][y] (8-1224)

[1462] 4. When x, y = 0..CtbSizeY-1, the variables dir1[x][y], dir2[x][y], and dirS[x][y] are derived as follows:

[1463] – The variables hv1, hv0 and dirHV are exported as follows:

[1464] – If sumV[x>>2][y>>2] is greater than sumH[x>>2][y>>2], then the following applies:

[1465] hv1=sumV[x>>2][y>>2] (8-1225)

[1466] hv0=sumH[x>>2][y>>2] (8-1226)

[1467] dirHV=1 (8-1227)

[1468] – Otherwise, the following applies:

[1469] hv1=sumH[x>>2][y>>2] (8-1228)

[1470] hv0=sumV[x>>2][y>>2] (8-1229)

[1471] dirHV=3(8-1230)

[1472] – The variables d1, d0 and dirD are derived as follows:

[1473] – If sumD0[x>>2][y>>2] is greater than sumD1[x>>2][y>>2], then the following applies:

[1474] d1=sumD0[x>>2][y>>2] (8-1231)

[1475] d0=sumD1[x>>2][y>>2] (8-1232)

[1476] dirD=0 (8-1233)

[1477] – Otherwise, the following provisions apply:

[1478] d1=sumD1[x>>2][y>>2] (8-1234)

[1479] d0=sumD0[x>>2][y>>2] (8-1235)

[1480] dirD=2 (8-1236)

[1481] – The variables hvd1 and hvd0 are exported as follows:

[1482] hvd1=(d1*hv0>hv1*d0)? d1:hv1 (8-1237)

[1483] hvd0=(d1*hv0>hv1*d0)? d0:hv0 (8-1238)

[1484] – The variables dirS[x][y], dir1[x][y] and dir2[x][y] are derived as follows:

[1485] dir1[x][y]=(d1*hv0>hv1*d0)? dirD: dirHV (8-1239)

[1486] dir2[x][y]=(d1*hv0>hv1*d0)? dirHV: dirD (8-1240)

[1487] dirS[x][y]=(hvd1>2*hvd0)? 1: (((hvd1*2>9*hvd0)?2:0) (8-1241)

[1488] 5. The variable avgVar[x][y] with x, y = 0..CtbSizeY-1 is derived as follows:

[1489] varTab[]={0, 1, 2, 2, 2, 2, 2, 3, 3, 3, 3, 3, 3, 3, 3, 4} (8-1242)

[1490] avgVar[x][y]=varTab[Clip3(0,15,(sumOfHV[x>>2][y>>2]*ac)>>(3+BitDepthY))] (8-1243)

[1491] 6. The classification filter index array filtIdx[x][y] and the transpose index array transposeIdx[x][y] of x=y=0..CtbSizeY-1 are derived as follows:

[1492] transposeTable[]={0, 1, 0, 2, 2, 3, 1, 3}

[1493] transposeIdx[x][y]=transposeTable[dir1[x][y]*2+(dir2[x][y]>>1)]

[1494] filtIdx[x][y]=avgVar[x][y]

[1495] When dirS[x][y] is not equal to 0, filtIdx[x][y] is modified as follows:

[1496] filtIdx[x][y]+=(((dir1[x][y]&0x1)<<1)+dirS[x][y])*5 (8-1244)

[1497] 8.8.5.4 Codec Tree Block Filtering for Chroma Samples

[1498] The inputs to this process are:

[1499] -reconstructed chroma picture sample array recPicture before adaptive loop filtering,

[1500] -Filtered and reconstructed chrominance picture sample array alfPicture,

[1501] -Specifies the chroma position (xCtbC, yCtbC) of the top left sample of the current chroma codec tree block relative to the top left sample of the current picture.

[1502] The output of this process is the modified filtered reconstructed chroma picture sample array alfPicture.

[1503] The width and height ctbWidthC and ctbHeightC of the current chroma codec tree block are derived as follows:

[1504] ctbWidthC=CtbSizeY / SubWidthC (8-1245)

[1505] ctbHeightC=CtbSizeY / SubHeightC (8-1246)

[1506] To derive the filtered reconstructed chroma samples alfPicture[x][y], each reconstructed chroma sample in the current chroma codec tree block recPicture[x][y] is filtered as follows, where x=0...ctbWidthC-1, y=0...ctbHeightC-1.

[1507] - In a given chroma sample array recPicture (where i, j = -2...2), the position (h) of each corresponding chroma sample (x, y) x+i ,v y+j ) is derived as follows:

[1508] – if pps_loop_filter_across_virtual_boundaries_disabled_flag is equal to 1 for any n = 0..pps_num_ver_virtual_boundaries–1, AND xCtbC+x-PpsVirtualBoundariesPosX[n] / SubWidthC is greater than or equal to 0 and less than 2, then the following applies:

[1509] h x+i=Clip3(PpsVirtualBoundariesPosX[n] / SubWidthC,pic_width_in_luma_samples / SubWidthC-1,xCtbC+x+i) (8-1247)

[1510] – Otherwise, if pps_loop_filter_across_virtual_boundaries_disabled_flag is equal to 1 for any n = 0..pps_num_ver_virtual_boundaries–1, And PpsVirtualBoundariesPosX[n] / SubWidthC-xCtbC–x is greater than 0 and less than 3, then the following applies:

[1511] h x+i =Clip3(0,PpsVirtualBoundariesPosX[n] / SubWidthC-1,xCtbC+x+i)(8-1248)

[1512] – Otherwise, the following applies:

[1513] h x+i =Clip3(0,pic_width_in_luma_samples / SubWidthC-1,xCtbC+x+i) (8-1249)

[1514] –[[When in position (h x ,v y ) contains luma samples on a sub-picture, the following applies:

[1515] h x+i =Clip3(SubPicLeftBoundaryPos / SubWidthC,SubPicRightBoundaryPos / SubWidthC,h x+i ) (8-1184)]]

[1516] – if pps_loop_filter_across_virtual_boundaries_disabled_flag is equal to 1 for any n = 0..pps_num_hor_virtual_boundaries-1, And yCtbC+y-PpsVirtualBoundariesPosY[n] / SubHeightC is greater than or equal to 0 and less than 2, then the following applies:

[1517] v y+j =Clip3(PpsVirtualBoundariesPosY[n] / SubHeightC,pic_height_in_luma_samples / SubHeightC-1,yCtbC+y+j) (8-1250)

[1518] – Otherwise, if pps_loop_filter_across_virtual_boundaries_disabled_flag is equal to 1 for any n = 0..pps_num_hor_virtual_boundaries–1, And PpsVirtualBoundariesPosY[n] / SubHeightC-yCtbC-y is greater than 0 and less than 3, then the following applies:

[1519] v y+j =Clip3(0,PpsVirtualBoundariesPosY[n] / SubHeightC-1,yCtbC+y+j) (8-1251)

[1520] – Otherwise, the following applies:

[1521] v y+j =Clip3(0,pic_height_in_luma_samples / SubHeightC-1,yCtbC+y+j)(8-1252)

[1522] –[[When in position (h x ,v y ) contains luma samples on a sub-picture, the following applies:

[1523] v y+j =Clip3(SubPicTopBoundaryPos / SubWidthC,SubPicBotBoundaryPos / SubWidthC,v y+j ) (8-1184)

[1524] -The variable applyVirtualBoundary is derived as follows:

[1525] - If one or more of the following conditions are true, applyVirtualBoundary is set to 0.

[1526] - The bottom boundary of the current codec tree block is the bottom boundary of the picture.

[1527] - The bottom boundary of the current codec tree block is the bottom boundary of the brick, and loop_filter_across_bricks_enabled_flag is equal to 0.

[1528] - The bottom boundary of the current codec treeblock is the bottom boundary of the slice, and loop_filter_across_slices_enabled_flag is equal to 0.

[1529] – The bottom boundary of the current codec treeblock is the bottom boundary of the sub-picture, and the loop_filter_across_sub_pic_enabled_flag[SubPicIdx] of the sub-picture containing the luma sample at position (hx,vy) is equal to 0.

[1530] - The bottom boundary of the current codec treeblock is one of the bottom virtual boundaries of the picture, and pps_loop_filter_across_virtual_boundaries_disabled_flag is set equal to 1.

[1531] - Otherwise, applyVirtualBoundary is set equal to 1. ]]

[1532]

[1533] -according to Sample point position y, clipTopPos and clipBottomPos The reconstructed sample offsets r1 and r2 are specified in Table 8-27.

[1534]

[1535] –The variable curr is derived as follows:

[1536] curr=recPicture[h x ,v y ] (8-1253) – The chroma filter coefficient array f[j] and the chroma clipping value array c[j] are derived as follows, where j = 0..5:

[1537] f[j] = AlfCoeffC[slice_alf_aps_id_chroma][j] (8 - 1254)

[1538] c[j] = AlfClipC[slice_alf_aps_id_chroma][j] (8 - 1255)

[1539] - The derivation of the variable sum is as follows:

[1540] sum = f[0] * (Clip3(-c[0], c[0], recPicture[hx, vy + r2] - curr) + Clip3(-c[0], c[0], recPicture[h x , v y-r2 - curr)) + f[1] * (Clip3(-c[1], c[1], recPicture[h x+ c 1, v y+r1 - curr) + Clip3(-c[1], c[1], recPicture[h x- c 1, v y-r1 - curr)) + f[2] * (Clip3(-c[2], c[2], recPicture[h x , v y+r1 - curr) + Clip3(-c[2], c[2], recPicture[h x , v y-r1 - curr)) + f[3] * (Clip3(-c[3], c[3], recPicture[h x- c 1, v<s y+r1 - curr) + Clip3(-c[3], c[3], recPicture[h<s x+ <000s c 1, v y-r1 - curr)) + f[4] * (Clip()3(-c[4], c[4], recPicture[h x+ c 2, v[[ID=​​​​​​​ c 1,v y ]-curr)+Clip3(-c[5],c[5],recPicture[h x- c 1,v y ]-curr)) (8-1256)

[1541] sum=curr+(sum+64)>>7) (8-1257)

[1542] - The derivation of the modified filtered reconstructed chrominance picture sample alfPicture[xCtbC+x][yCtbC+y] is as follows:

[1543] - If pcm_loop_filter_disabled_flag and pcm_flag[(xCtbC+x)*SubWidthC][(yCtbC+y)*SubHeightC] are both equal to 1, the following applies:

[1544] alfPicture[xCtbC+x][yCtbC+y]=recPicture L [h x ,v y ]

[1545] (8-1258)

[1546] Otherwise (pcm_loop_filter_disabled_flag is equal to 0 or pcm_flag[x][y] is equal to 0), the following applies:

[1547] alfPicture[xCtbC+x][yCtbC+y]=Clip3(0,(1< <BitDepth C )-1,sum) (8-1259)

[1548] Table 8-27 – According to Sample point position y, Specification of r1 and r2 for [[and applyVirtualBoundary]]

[1549]

[1550]

[1551]

[1552]

[1553]

[1554]

[1555] The specific value -128 used in the above embodiments may be replaced by other values, such as -K, where, for example, K is greater than or not less than the number of rows offset from the bottom boundary of the CTU (eg, K=-5).

[1556] Alternatively, if each sample is located at a video unit boundary, a flag may be used to mark whether each sample needs to be processed differently.

[1557] 5.10 Example #10

[1558] 8.8.5.2 Codec Tree Block Filtering for Luma Samples

[1559] The inputs to this process are:

[1560] - Before adaptive loop filtering, the reconstructed luminance picture sample array recPicture L，

[1561] -Filtered and reconstructed brightness picture sample array alfPicture L，

[1562] -Specifies the luma position (xCtb, yCtb) of the top left sample of the current luma codec treeblock relative to the top left sample of the current picture.

[1563] The output of this process is the modified filtered reconstructed luminance picture sample array alfPicture L .

[1564] The filter index derivation process of Section 8.8.5.3 is performed using the position (xCtb, yCtb) and the reconstructed luma picture sample array recPicture L Called with filtIdx[x][y] as input and transposeIdx[x][y] (where x,y = 0..CtbSizeY-1) as output.

[1565] For the filtered reconstructed luminance sample alfPicture L [x][y], each reconstructed luma sample in the current luma codec tree block recPictureL[x][y] is filtered as follows (where x, y = 0..CtbSizeY-1):

[1566] …

[1567] - The position (h) of each corresponding luma sample (x, y) within the given array recPicture of luma samples x+i ,v y+j ) (where i, j = -3...3) is derived as follows:

[1568] If, for any n = 0..pps_num_ver_virtual_boundaries–1, pps_loop_filter_across_virtual_boundaries_disabled_flag is equal to 1, and PpsVirtualBoundariesPosX[n] % CtbSizeY is not equal to 0, and xCtb+x−PpsVirtualBoundariesPosX[n] is greater than or equal to 0 and less than 3, the following applies:

[1569] h x+i =Clip3(PpsVirtualBoundariesPosX[n],pic_width_in_luma_samples-1,xCtb+x+i) (8-1229)

[1570] Otherwise, if pps_loop_filter_across_virtual_boundaries_disabled_flag is equal to 1, and PpsVirtualBoundariesPosX[n] % CtbSizeY is not equal to 0, and PpsVirtualBoundariesPosX[n]-xCtb-x is greater than 0 and less than 4, the following applies:

[1571] h x+i =Clip3(0,PpsVirtualBoundariesPosX[n]-1,xCtb+x+i) (8-1230)

[1572] - Otherwise, the following applies:

[1573] h x+i =Clip3(0,pic_width_in_luma_samples-1,xCtb+x+i) (8-1231)

[1574] If, for any n = 0..pps_num_hor_virtual_boundaries–1, pps_loop_filter_across_virtual_boundaries_disabled_flag is equal to 1, and PpsVirtualBoundariesPosY[n] % CtbSizeY is not equal to 0, and yCtb+y−PpsVirtualBoundariesPosY[n] is greater than or equal to 0 and less than 3, the following applies:

[1575] v y+j =Clip3(PpsVirtualBoundariesPosY[n],pic_height_in_luma_samples-1,yCtb+y+j) (8-1232)

[1576] Otherwise, if pps_loop_filter_across_virtual_boundaries_disabled_flag is equal to 1, and PpsVirtualBoundariesPosX[n] % CtbSizeY is not equal to 0, and PpsVirtualBoundariesPosY[n]-yCtb-y is greater than 0 and less than 4, the following applies:

[1577] v y+j =Clip3(0,PpsVirtualBoundariesPosY[n]-1,yCtb+y+j) (8-1233)

[1578] - Otherwise, the following applies:

[1579] v y+j =Clip3(0,pic_height_in_luma_samples-1,yCtb+y+j) (8-1234)

[1580] - Derives the variables clipLeftPos, clipRightPos, clipTopPos, clipBottomPos, by calling the ALF boundary position derivation process specified in clause 8.8.5.5 with (xCtb, yCtb) and (x, y) as input

[1581]

[1582]

[1583] – The vertical sample position offsets y1, y2, and y3 are specified in Table 8-20 according to the vertical luma sample position y, clipLeftPos, and clipRightPos.

[1584] – The horizontal sample position offsets x1, x2, and x3 are specified in Table 8-21 according to the horizontal luma sample position x, clipLeftPos, and clipRightPos.

[1585] …

[1586] 8.8.5.3 Derivation of filter index and ALF transpose for luma samples

[1587] The inputs to this process are:

[1588] – specifies the luma position (xCtb, yCtb) of the top left sample of the current luma codec tree block relative to the top left sample of the current picture,

[1589] – Before adaptive loop filtering, the reconstructed luminance picture sample array recPicture L .

[1590] The output of this processing is:

[1591] – Classification filter index array filtIdx[x][y], where x, y = 0..CtbSizeY–1.

[1592] – Transpose index array transposeIdx[x][y], where x, y = 0..CtbSizeY-1.

[1593] The position (h) of each corresponding luma sample (x, y) within a given array of luma samples recPicture x+i ,v y+j ) (where i, j = -2..5) is derived as follows:

[1594] – If, for any n = 0..pps_num_ver_virtual_boundaries-1, PpsVirtualBoundariesPosX[n] % CtbSizeY is not equal to 0, pps_loop_filter_across_virtual_boundaries_disabled_flag is equal to 1, and xCtb+x-PpsVirtualBoundariesPosX[n] is greater than or equal to 0 and less than 2, the following applies:

[1595] h x+i=Clip3(PpsVirtualBoundariesPosX[n],pic_width_in_luma_samples-1,xCtb+x+i) (8-1239)

[1596] Otherwise, if for any n = 0..pps_num_ver_virtual_boundaries-1, pps_loop_filter_across_virtual_boundaries_disabled_flag is equal to 1, PpsVirtualBoundariesPosX[n] % CtbSizeY is not equal to 0, and PpsVirtualBoundariesPosX[n]-xCtb-x is greater than 0 and less than 6, the following applies:

[1597] h x+i =Clip3(0,PpsVirtualBoundariesPosX[n]-1,xCtb+x+i) (8-1240)

[1598] – Otherwise, the following applies:

[1599] h x+i =Clip3(0,pic_width_in_luma_samples-1,xCtb+x+i) (8-1241)

[1600] – If, for any n = 0..pps_num_hor_virtual_boundaries–1, pps_loop_filter_across_virtual_boundaries_disabled_flag is equal to 1, PpsVirtualBoundariesPosY[n] % CtbSizeY is not equal to 0, and yCtb+y–PpsVirtualBoundariesPosY[n] is greater than or equal to 0 and less than 2, the following applies:

[1601] v y+j =Clip3(PpsVirtualBoundariesPosY[n],pic_height_in_luma_samples-1,yCtb+y+j) (8-1242)

[1602] – Otherwise, if for any n = 0..pps_num_hor_virtual_boundaries-1, pps_loop_filter_across_virtual_boundaries_disabled_flag is equal to 1, PpsVirtualBoundariesPosY[n] % CtbSizeY is not equal to 0, and PpsVirtualBoundariesPosY[n]–yCtb-y is greater than 0 and less than 6, the following applies:

[1603] v y+j =Clip3(0,PpsVirtualBoundariesPosY[n]-1,yCtb+y+j) (8-1243)

[1604] – Otherwise, the following applies:

[1605] v y+j =Clip3(0,pic_height_in_luma_samples-1,yCtb+y+j) (8-1244)

[1606] – Derives the variables clipLeftPos, clipRightPos, clipTopPos, clipBottomPos, by calling the ALF boundary position derivation process specified in clause 8.8.5.5 with (xCtb, yCtb) and (x, y) as input

[1607] – If clipTopPos is not equal to -128, the following applies:

[1608] v y+j =Clip3(clipTopPos,pic_height_in_luma_samples-1,yCtb+y+j) (8-1245)

[1609] – If clipBottomPos is not equal to -128, the following applies:

[1610] v y+j =Clip3(0,clipBottomPos-1,yCtb+y+j) (8-1246)

[1611] – If clipLeftPos is not equal to -128, the following applies:

[1612] h x+i=Clip3(clipLeftPos,pic_width_in_luma_samples-1,xCtb+x+i) (8-1247)

[1613] – If clipRightPos is not equal to -128, the following applies:

[1614] h x+i =Clip3(0,clipRightPos-1,xCtb+x+i) (8-1248)

[1615]

[1616] The classification filter index array filtIdx and the transpose index array transposeIdx are derived by the following sequential steps:

[1617] 1. The variables filtH[x][y], filtV[x][y], filtD0[x][y], and filtD1[x][y] (where x, y = -2..CtbSizeY+1) are derived as follows:

[1618] – If both x and y are even or both x and y are uneven, then the following applies:

[1619] filtH[x][y]=Abs((recPicture[h x ,v y ]<<1)-recPicture[h x-1 ,v y ]-recPict ure[h x+1 ,v y ]) (8-1249)

[1620] filtV[x][y]=Abs((recPicture[h x ,v y ]<<1)-recPicture[h x ,v y-1 ]-recPicture[h x ,v y+1 ]) (8-1250)

[1621] filtD0[x][y]=Abs((recPicture[h x ,v y ]<<1)-recPicture[h x-1 ,v y-1 ]-recPicture[h x+1 ,vy+1 ]) (8-1251)

[1622] filtD1[x][y]=Abs((recPicture[h x ,v y ]<<1)-recPicture[h x+1 ,v y-1 ]-recPicture[h x-1 ,v y+1 ]) (8-1252)

[1623] – Otherwise, filtH[x][y], filtV[x][y], filtD0[x][y], and filtD1[x][y] are set equal to 0.

[1624] …

[1625] 8.8.5.4 Codec Tree Block Filtering for Chroma Samples

[1626] The inputs to this process are:

[1627] -recPicture, the reconstructed chroma picture sample array before adaptive loop filtering.

[1628] -Filtered reconstructed chroma picture sample array alfPicture.

[1629] - specifies the chroma position (xCtbC, yCtbC) of the top left sample of the current chroma codec tree block relative to the top left sample of the current picture,

[1630] - Alternative chroma filter index altIdx.

[1631] The output of this process is the modified filtered reconstructed chroma picture sample array alfPicture.

[1632] The width ctbWidthC and height ctbHeightC of the current chroma codec tree block are derived as follows:

[1633] ctbWidthC=CtbSizeY / SubWidthC (8-1278)

[1634] ctbHeightC=CtbSizeY / SubHeightC (8-1279)

[1635] To filter the derivation of the reconstructed chroma samples alfPicture[x][y], each reconstructed chroma sample in the current chroma codec tree block recPicture[x][y] is filtered as follows, where x=0...ctbWidthC-1, y=0...ctbHeightC-1.

[1636] - the position (h) of each corresponding chroma sample (x,y) within the given array recPicture of chroma samples x+i ,v y+j )(where i, j = -2...2) is derived as follows:

[1637] If, for any n = 0..pps_num_ver_virtual_boundaries–1, pps_loop_filter_across_virtual_boundaries_disabled_flag is equal to 1, PpsVirtualBoundariesPosX[n] % CtbSizeY is not equal to 0, and xCtbC + x-PpsVirtualBoundariesPosX[n] / SubWidthC is greater than or equal to 0 and less than 2, the following applies:

[1638] h x+i =Clip3(PpsVirtualBoundariesPosX[n] / SubWidthC,pic_width_in_luma_samples / SubWidthC-1,xCtbC+x+i) (8-1280)

[1639] - Otherwise, if for any n=0..pps_num_ver_virtual_boundaries-1, pps_loop_filter_across_virtual_boundaries_disabled_flag is equal to 1, PpsVirtualBoundariesPosX[n]%CtbSizeY is not equal to 0, and PpsVirtualBoundariesPosX[n] / SubWidthC-xCtbC-x is greater than 0 and less than 3,

[1640] The following applies:

[1641] h x+i =Clip3(0,PpsVirtualBoundariesPosX[n] / SubWidthC-1,xCtbC+x+i)(8-1281)

[1642] - Otherwise, the following applies:

[1643] h x+i =Clip3(0,pic_width_in_luma_samples / SubWidthC-1,xCtbC+x+i)(8-1282)

[1644] If, for any n = 0..pps_num_hor_virtual_boundaries–1, pps_loop_filter_across_virtual_boundaries_disabled_flag is equal to 1, PpsVirtualBoundariesPosY[n] % CtbSizeY is not equal to 0, and yCtbC+y−PpsVirtualBoundariesPosY[n] / SubHeightC is greater than or equal to 0 and less than 2, the following applies:

[1645] v y+j =Clip3(PpsVirtualBoundariesPosY[n] / SubHeightC,pic_height_in_luma_samples / SubHeightC-1,yCtbC+y+j) (8-1283)

[1646] Otherwise, if for any n = 0..pps_num_hor_virtual_boundaries-1, pps_loop_filter_across_virtual_boundaries_disabled_flag is equal to 1, PpsVirtualBoundariesPosY[n] % CtbSizeY is not equal to 0, and PpsVirtualBoundariesPosY[n] / SubHeightC-yCtbC-y is greater than 0 and less than 3, the following applies:

[1647] v y+j =Clip3(0,PpsVirtualBoundariesPosY[n] / SubHeightC-1,yCtbC+y+j) (8-1284)

[1648] - Otherwise, the following applies:

[1649] v y+j=Clip3(0,pic_height_in_luma_samples / SubHeightC-1,yCtbC+y+j) (8-1285)

[1650] - Derives the variables clipLeftPos, clipRightPos, clipTopPos, clipBottomPos, by calling the ALF boundary position derivation process specified in clause 8.8.5.5 with (xCtbC*SubWidthC,yCtbC*SubHeightC) and (x*SubWidthC,y*SubHeightC) as input.

[1651]

[1652]

[1653] …

[1654] 8.5.5.5 ALF boundary position derivation and processing

[1655] The inputs to this process are:

[1656] – specifies the luma position (xCtb, yCtb) of the top left sample of the current luma codec tree block relative to the top left sample of the current picture,

[1657] – Specifies the luma position (x, y) of the current sample relative to the top left sample of the current luma codec treeblock.

[1658] The output of this processing is:

[1659] – left vertical border position clipLeftPos,

[1660] – right vertical border position clipRightPos,

[1661] – The horizontal border position above clipTopPos,

[1662] – The bottom horizontal border position clipBottomPos.

[1663]

[1664] Set the variables clipLeftPos, clipRightPos, clipTopPos, and clipBottomPos to -128.

[1665]

[1666] The variable clipTopPos is modified as follows:

[1667] – If the bottom boundary of the current codec tree block is not the bottom boundary of the picture, and y-(CtbSizeY-4) is greater than or equal to 0, then set the variable clipTopPos equal to yCtb+CtbSizeY-4.

[1668] Otherwise, if, for any n = 0..pps_num_hor_virtual_boundaries-1, pps_loop_filter_across_virtual_boundaries_disabled_flag is equal to 1, PpsVirtualBoundariesPosY[n] % CtbSizeY is equal to 0, and yCtb+y-PpsVirtualBoundariesPosY[n] is greater than or equal to 0 and less than 3, the following applies:

[1669] clipTopPos=PpsVirtualBoundariesPosY[n] (8-1292)

[1670] – Otherwise, if y is less than 3, and the top boundary of the current codec tree block is not the top boundary of the picture, and one or more of the following conditions are true, then set the variable clipTopPos equal to yCtb:

[1671] – If the top boundary of the current codec tree block is the top boundary of a brick and loop_filter_across_bricks_enabled_flag is equal to 0.

[1672] – If the top boundary of the current codec treeblock is the top boundary of a slice and loop_filter_across_slices_enabled_flag is equal to 0.

[1673] – If the top boundary of the current codec treeblock is the top boundary of a sub-picture and loop_filter_across_subpic_enabled_flag[SubPicIdx] is equal to 0.

[1674] The variable clipBottomPos is modified as follows:

[1675] – If the bottom boundary of the current codec tree block is not the bottom boundary of the picture, and CtbSizeY-4-y is greater than 0 and less than 4, then set the variable clipBottomPos equal to yCtb+CtbSizeY-4.

[1676] – Otherwise, if for any n = 0..pps_num_hor_virtual_boundaries-1, pps_loop_filter_across_virtual_boundaries_disabled_flag is equal to 1, PpsVirtualBoundariesPosY[n] % CtbSizeY is equal to 0, PpsVirtualBoundariesPosY[n] is not equal to pic_height_in_luma_samples-1 or 0, and PpsVirtualBoundariesPosY[n]-yCtb–y is greater than 0 and less than 4, then the following applies:

[1677] clipBottomPos=PpsVirtualBoundariesPosY[n] (8-1293)

[1678] Otherwise, if CtbSizeY-y is less than 4, and the bottom boundary of the current codec tree block is not the bottom boundary of the picture, and one or more of the following conditions are true, then set the variable clipBottomPos equal to yCtb+CtbSizeY:

[1679] – If the bottom boundary of the current codec tree block is the bottom boundary of a brick and loop_filter_across_bricks_enabled_flag is equal to 0.

[1680] – If the bottom boundary of the current codec treeblock is the bottom boundary of a slice and loop_filter_across_slices_enabled_flag is equal to 0.

[1681] – If the bottom boundary of the current codec treeblock is the bottom boundary of the sub-picture and loop_filter_across_subpic_enabled_flag[SubPicIdx] is equal to 0.

[1682] The variable clipLeftPos is modified as follows:

[1683] – If, for any n = 0..pps_num_ver_virtual_boundaries-1, pps_loop_filter_across_virtual_boundaries_disabled_flag is equal to 1, PpsVirtualBoundariesPosX[n] % CtbSizeY is equal to 0, and xCtb+x-PpsVirtualBoundariesPosX[n] is greater than or equal to 0 and less than 3, the following applies:

[1684] clipLeftPos=PpsVirtualBoundariesPosX[n] (8-1294)

[1685] – Otherwise, if x is less than 3, the left boundary of the current codec tree block is not the left boundary of the picture, and one or more of the following conditions are true, then set the variable clipLeftPos equal to xCtb:

[1686] – If the left border of the current codec tree block is the left border of a brick and loop_filter_across_bricks_enabled_flag is equal to 0.

[1687] – If the left boundary of the current codec treeblock is the left boundary of a slice, and loop_filter_across_slices_enabled_flag is equal to 0.

[1688] – If the left boundary of the current codec treeblock is the left boundary of a sub-picture and loop_filter_across_subpic_enabled_flag[SubPicIdx] is equal to 0.

[1689] The variable clipRightPos is modified as follows:

[1690] – If, for any n = 0..pps_num_ver_virtual_boundaries-1, pps_loop_filter_across_virtual_boundaries_disabled_flag is equal to 1, PpsVirtualBoundariesPosX[n] % CtbSizeY is equal to 0, and PpsVirtualBoundariesPosX[n]-xCtb-x is greater than 0 and less than 4, the following applies:

[1691] clipRightPos=PpsVirtualBoundariesPosX[n] (8-1295)

[1692] Otherwise, if CtbSizeY-x is less than 4, and the right boundary of the current codec tree block is not the right boundary of the picture, and one or more of the following conditions are true, then set the variable clipRightPos equal to xCtb+CtbSizeY:

[1693] – If the right border of the current codec tree block is the right border of a brick, and loop_filter_across_bricks_enabled_flag is equal to 0.

[1694] – If the right boundary of the current codec treeblock is the right boundary of a slice, and loop_filter_across_slices_enabled_flag is equal to 0.

[1695] – If the right border of the current codec treeblock is the right border of the sub-picture and loop_filter_across_subpic_enabled_flag[SubPicIdx] is equal to 0.

[1696]

[1697]

[1698]

[1699] Example 11

[1700] In this embodiment, the following method is implemented as follows:

[1701] - Only use the samples in the current ALF processing unit for filling

[1702] - Apply duplicate padding to all boundaries except ALF virtual boundaries (apply mirror padding)

[1703] - When the picture height is an integer multiple of the CTU size, apply ALF VB to the last CTU row.

[1704] 8.5.5.2 Codec Tree Block Filtering for Luma Samples

[1705] The inputs to this process are:

[1706] – Reconstructed luminance picture sample array recPicture before adaptive loop filtering processing L ,

[1707] –Filtered and reconstructed brightness picture sample array alfPicture L ,

[1708] –Specifies the luma position (xCtb, yCtb) of the top left sample of the current luma codec tree block relative to the top left sample of the current picture.

[1709] The output of this process is the modified filtered reconstructed luminance picture sample array alfPicture L .

[1710] Use the position (xCtb, yCtb) and the reconstructed luminance picture sample array recPicture L The derivation process of filter index clause 8.8.5.3 is called with filtIdx[x][y] as input and transposeIdx[x][y] (where x,y = 0..CtbSizeY-1) as output.

[1711] In order to filter the derivation of the reconstructed luma sample alfPictureL[x][y], for each reconstructed luma sample recPicture inside the current luma codec tree block L [x][y] is filtered as follows, where x, y = 0..CtbSizeY-1:

[1712] The luma filter coefficient array f[j] and luma clipping value array c[j] corresponding to the filter specified by filtIdx[x][y] are derived as follows (where j = 0..11):

[1713] – If AlfCtbFiltSetIdxY[xCtb>>CtbLog2SizeY][yCtb>>CtbLog2SizeY] is less than 16, the following applies:

[1714] i=AlfCtbFiltSetIdxY[xCtb>>CtbLog2SizeY][yCtb>>CtbLog2SizeY] (8-1219)

[1715] f[j]=AlfFixFiltCoeff[AlfClassToFiltMap[i][filtIdx[x][y]]][j] (8-1220)

[1716] c[j]=2 BitdepthY (8-1221)

[1717] – Otherwise, (AlfCtbFiltSetIdxY[xCtb>>CtbLog2SizeY][yCtb>>CtbLog2SizeY] is greater than or equal to 16), the following applies:

[1718] i=slice_alf_aps_id_luma[AlfCtbFiltSetIdxY[xCtb>>CtbLog2SizeY][yCtb>>CtbLog2SizeY]-16] (8-1222)

[1719] f[j]=AlfCoeff L [i][filtIdx[x][y]][j] (8-1223)

[1720] c[j]=AlfClip L [i][filtIdx[x][y]][j] (8-1224)

[1721] – Derived luma filter coefficients and clipping value index idx depending on transposeIdx[x][y] as follows:

[1722] – If transposeIndex[x][y] is equal to 1, the following applies:

[1723] idx[]={9,4,10,8,1,5,11,7,3,0,2,6} (8-1225)

[1724] – Otherwise, if transposeIndex[x][y] is equal to 2, the following applies:

[1725] idx[]={0,3,2,1,8,7,6,5,4,9,10,11} (8-1226)

[1726] – Otherwise, if transposeIndex[x][y] is equal to 3, the following applies:

[1727] idx[]={9,8,10,4,3,7,11,5,1,0,2,6} (8-1227)

[1728] – Otherwise, the following applies:

[1729] idx[]={0,1,2,3,4,5,6,7,8,9,10,11} (8-1228)

[1730] – The position (h) of each corresponding luminance sample (x, y) within the given array RecPicture of luminance samples x+i ,v y+j ) (where i, j = -3..3) is derived as follows:

[1731] –[[If for any n=0..pps_num_ver_virtual_boundaries–1, pps_loop_filter_across_virtual_boundaries_disabled_flag is equal to 1, PpsVirtualBoundariesPosX[n]%CtbSizeY is not equal to 0, and xCtb+x-

[1732] If PpsVirtualBoundariesPosX[n] is greater than or equal to 0 and less than 3, the following applies:

[1733] –h x+i =Clip3(PpsVirtualBoundariesPosX[n],pic_width_in_luma_samples-1,xCtb+x+i) (8-1229)

[1734] – Otherwise, if for any n = 0..pps_num_ver_virtual_boundaries–1, pps_loop_filter_across_virtual_boundaries_disabled_flag is equal to 1, PpsVirtualBoundariesPosX[n] % CtbSizeY is not equal to 0, and PpsVirtualBoundariesPosX[n]-xCtb-x is greater than 0 and less than 4, the following applies:

[1735] h x+i =Clip3(0,PpsVirtualBoundariesPosX[n]-1,xCtb+x+i) (8-1230)

[1736] – Otherwise, the following applies:]]

[1737] h x+i =Clip3(0,pic_width_in_luma_samples-1,xCtb+x+i) (8-1231)

[1738] –[[If, for any n = 0..pps_num_hor_virtual_boundaries–1, pps_loop_filter_across_virtual_boundaries_disabled_flag is equal to 1, PpsVirtualBoundariesPosY[n] % CtbSizeY is not equal to 0, and yCtb+y-PpsVirtualBoundariesPosY[n] is greater than or equal to 0 and less than 3, the following applies:

[1739] v y+j =Clip3(PpsVirtualBoundariesPosY[n],pic_height_in_luma_samples-1,yCtb+y+j) (8-1232)

[1740] – Otherwise, if for any n = 0..pps_num_hor_virtual_boundaries-1, pps_loop_filter_across_virtual_boundaries_disabled_flag is equal to 1, PpsVirtualBoundariesPosY[n] % CtbSizeY is not equal to 0, and PpsVirtualBoundariesPosY[n]–yCtb-y is greater than 0 and less than 4, the following applies:

[1741] v y+j =Clip3(0,PpsVirtualBoundariesPosY[n]-1,yCtb+y+j) (8-1233)

[1742] – Otherwise, the following applies:]]

[1743] v y+j =Clip3(0,pic_height_in_luma_samples-1,yCtb+y+j) (8-1234)

[1744] – derive the variables clipLeftPos, clipRightPos, clipTopPos, clipBottomPos, by calling the ALF boundary position derivation process specified in clause 8.8.5.5 with (xCtb, yCtb) and (x, y) as input .

[1745]

[1746] –[[Vertical sample position offsets y1, y2, and y3 are specified in Table 8-20 based on the vertical luma sample position y, clipLeftPos, and clipRightPos.

[1747] – Horizontal sample position offsets x1, x2, and x3 are specified in Table 8-21 based on the horizontal luma sample position x, clipLeftPos, and clipRightPos.

[1748] –The variable curr is derived as follows:

[1749] curr=recPicture L [h x ,v y ] (8-1235)

[1750]

[1751] -The variable sum is derived as follows:

[1752] sum=f[idx[0]]*(Clip3(-c[idx[0]],c[idx[0]],recPicture L [h x ,v y+y3 ]-curr)+Clip3(-c[idx[0]],c[idx[0]],recPicture L [h x ,v y-y3 ]-curr))+f[idx[1]]*(Clip3(-c[idx[1]],c[idx[1]],recPicture L [h x+[[x]]1 ,v y+y2 ]-curr)+Clip3(-c[idx[1]],c[idx[1]],recPicture L [h x-[[x]]1 ,v y-y2 ]-curr))+f[idx[2]]*(Clip3(-c[idx[2]],c[idx[2]],recPicture L [h x ,v y+y2 ]-curr)+Clip3(-c[idx[2]],c[idx[2]],recPicture L [h x ,v y-y2]-curr))+f[idx[3]]*(Clip3(-c[idx[3]],c[idx[3]],recPicture L [h x-[[x]]1 ,v y+y2 ]-curr)+Clip3(-c[idx[3]],c[idx[3]],recPicture L [h x+[[x]]1 ,v y-y2 ]-curr))+f[idx[4]]*(Clip3(-c[idx[4]],c[idx[4]],recPicture L [h x+[[x]]2 ,v y+y1 ]-curr)+Clip3(-c[idx[4]],c[idx[4]],recPicture L [h x-[[x]]2 ,v y-y1 ]-curr))+f[idx[5]]*(Clip3(-c[idx[5]],c[idx[5]],recPicture L [h x+[[x]]1 ,v y+y1 ]-curr)+Clip3(-c[idx[5]],c[idx[5]],recPicture L [h x-[[x]]1 ,v y-y1 ]-curr))+f[idx[6]]*(Clip3(-c[idx[6]],c[idx[6]],recPicture L [h x ,v y+y1 ]-curr)+Clip3(-c[idx[6]],c[idx[6]],recPicture L [h x ,v y-y1 ]-curr))+f[idx[7]]*(Clip3(-c[idx[7]],c[idx[7]],recPicture L [h x-[[x]]1 ,v y+y1 ]-curr)+Clip3(-c[idx[7]],c[idx[7]],recPicture L [h x+[[x]]1 ,v y-y1 ]-curr))+f[idx[8]]*(Clip3(-c[idx[8]],c[idx[8]],recPictureL [h x-[[x]]2 ,v y+y1 ]-curr)+Clip3(-c[idx[8]],c[idx[8]],recPicture L [h x+[[x]]2 ,v y-y1 ]-curr))+f[idx[9]]*(Clip3(-c[idx[9]],c[idx[9]],recPicture L [h x+[[x]]3 ,v y ]-curr)+Clip3(-c[idx[9]],c[idx[9]],recPicture L [h x-[[x]]3 ,v y ]-curr))+f[idx

[10] ]*(Clip3(-c[idx

[10] ],c[idx

[10] ],recPicture L [h x+[[x]]2 ,v y ]-curr)+Clip3(-c[idx

[10] ],c[idx

[10] ],recPicture L [h x-[[x]]2 ,v y ]-curr))+f[idx

[11] ]*(Clip3(-c[idx

[11] ],c[idx

[11] ],recPicture L [h x+[[x]]1 ,v y ]-curr)+Clip3(-c[idx

[11] ],c[idx

[11] ],recPicture L [h x-[[x]]1 ,v y ]-curr)) (8-1236)

[1753] sum=curr+((sum+64)>>7) (8-1237)

[1754] -Modified filtered reconstructed luminance picture samples alfPicture L [xCtb+x][yCtb+y] is derived as follows:

[1755] alfPicture L [xCtb+x][yCtb+y]=Clip3(0,(1< <BitDepth Y )-1,sum) (8-1238)

[1756]

[1757] condition y1 y2 y3 (y==CtbSizeY-5||y==CtbSizeY-4)&&(applyVirtualBoundary==1) 0 0 0 (y==CtbSizeY-6||y==CtbSizeY-3)&&(applyVirtualBoundary==1) 1 1 1 (y==CtbSizeY-7||y==CtbSizeY-2)&&(applyVirtualBoundary==1) 1 2 2 other 1 2 3 [[

[1759]

[1760]

[1761] condition x1 x2 x3 (xCtb+x==clipLeftPos||xCtb+x==clipRightPos-1) 0 0 0 (xCtb+x==clipLeftPos+1||xCtb+x==clipRightPos-2) 1 1 1 (xCtb+x==clipLeftPos+2||xCtb+x==clipRightPos-3) 1 2 2 other 1 2 3 ]]

[1763] 8.5.5.3 Derivation of ALF transpose and filter index for luma samples

[1764] The inputs to this process are:

[1765] – specifies the luma position (xCtb, yCtb) of the top left sample of the current luma codec tree block relative to the top left sample of the current picture,

[1766] – Reconstructed luminance picture sample array recPicture before adaptive loop filtering processing L .

[1767] The output of this processing is

[1768] – Classification filter index array filtIdx[x][y], where x, y = 0..CtbSizeY-1

[1769] – Transpose index array transposeIdx[x][y], where x, y = 0..CtbSizeY-1.

[1770]

[1771] [[Position(h x+i ,v y+j )f]] for each corresponding luma [[samples]] block (denoted by (x4, y4)) within a given array recPicture of luma samples (where [[i, j = -2..5]] ) is derived as follows:

[1772]

[1773] –[[If, for any n = 0..pps_num_ver_virtual_boundaries–1, pps_loop_filter_across_virtual_boundaries_disabled_flag is equal to 1, PpsVirtualBoundariesPosX[n] % CtbSizeY is not equal to 0, and xCtb+x-PpsVirtualBoundariesPosX[n] is greater than or equal to 0 and less than 2, the following applies:

[1774] h x+i =Clip3(PpsVirtualBoundariesPosX[n],pic_width_in_luma_samples-1,xCtb+x+i) (8-1239)

[1775] – Otherwise, if for any n = 0..pps_num_ver_virtual_boundaries–1, pps_loop_filter_across_virtual_boundaries_disabled_flag is equal to 1, PpsVirtualBoundariesPosX[n] % CtbSizeY is not equal to 0, and PpsVirtualBoundariesPosX[n]–xCtb-x is greater than 0 and less than 6, the following applies:

[1776] h x+i =Clip3(0,PpsVirtualBoundariesPosX[n]-1,xCtb+x+i) (8-1240)

[1777] – Otherwise, the following applies:]]

[1778] h x+i =Clip3(0,pic_width_in_luma_samples-1,xCtb+x+i) (8-1241)

[1779] – [[If, for any n = 0..pps_num_hor_virtual_boundaries-1, pps_loop_filter_across_virtual_boundaries_disabled_flag is equal to 1, PpsVirtualBoundariesPosY[n] % CtbSizeY is not equal to 0, and yCtb+y-PpsVirtualBoundariesPosY[n] is greater than or equal to 0 and less than 2, the following applies:

[1780] v y+j =Clip3(PpsVirtualBoundariesPosY[n],pic_height_in_luma_samples-1,yCtb+y+j) (8-1242)

[1781] Otherwise, if for any n = 0..pps_num_hor_virtual_boundaries-1, pps_loop_filter_across_virtual_boundaries_disabled_flag is equal to 1, PpsVirtualBoundariesPosY[n] % CtbSizeY is not equal to 0, and PpsVirtualBoundariesPosY[n]-yCtb-y is greater than 0 and less than 6, the following applies:

[1782] v y+j =Clip3(0,PpsVirtualBoundariesPosY[n]-1,yCtb+y+j) (8-1243)

[1783] – Otherwise, the following applies:]]

[1784] v y+j =Clip3(0,pic_height_in_luma_samples-1,yCtb+y+j) (8-1244)

[1785]

[1786] – [[The variables clipLeftPos, clipRightPos, clipTopPos, and clipBottomPos are derived by invoking the ALF boundary position derivation process specified in clause 8.8.5.5 with (xCtb, yCtb) and (x, y) as input.

[1787] – If clipTopPos is not equal to -128, the following applies:

[1788] v y+j =Clip3(clipTopPos,pic_height_in_luma_samples-1,yCtb+y+j) (8-1245)

[1789] – If clipBottomPos is not equal to -128, the following applies:

[1790] v y+j =Clip3(0,clipBottomPos-1,yCtb+y+j) (8-1246)

[1791] – If clipLeftPos is not equal to -128, the following applies:

[1792] h x+i =Clip3(clipLeftPos,pic_width_in_luma_samples-1,xCtb+x+i) (8-1247)

[1793] – If clipRightPos is not equal to -128, the following applies:

[1794] h x+i =Clip3(0,clipRightPos-1,xCtb+x+i) (8-1248)

[1795] The classification filter index array filtIdx and the transpose index array transposeIdx are derived by the following ordered steps:]]

[1796] 1. Variable filtH[x ][y ]、filtV[x ][y ]、filtD0[x ][y ] and filtD1[x ][y ](where [[x,y=-2..CtbSizeY+1]] The derivation is as follows:

[1797] – If [[x]] and [[y]] are all even, or [[x]] and [[y]] If both are uneven numbers, the following applies:

[1798] [[filtH[x][y]=Abs((recPicture[h x ,v y ]<<1)-recPicture[h x-1 ,v y ]-recPicture[h x+1 ,v y ]) (8-1249)

[1799] filtV[x][y]=Abs((recPicture[h x ,v y ]<<1)-recPicture[h x ,v y-1 ]-recPicture[h x ,v y+1 ]) (8-1250)

[1800] filtD0[x][y]=Abs((recPicture[h x ,v y ]<<1)-recPicture[h x-1 ,v y-1 ]-recPicture[h x+1 ,v y+1 ]) (8-1251)

[1801] filtD1[x][y]=Abs((recPicture[h x ,v y ]<<1)-recPicture[h x+1 ,v y-1 ]-recPicture[h x-1 ,v y+1 ])]] (8-1252)

[1802]

[1803] – Otherwise, filtH[x ][y ]、filtV[x ][y ]、filtD0[x ][y ] and filtD1[x ][y ] is set to equal to 0.

[1804] 2.[[Variables ac[x][y], sumH[x][y], sumV[x][y], sumD0[x][y], sumD1[x][y] and sumOfHV[x][y] (x, y = 0..(CtbSizeY-1)>>2) are derived as follows:

[1805] – The variables minY, maxY and ac are derived as follows:

[1806] – If clipTopPos is not equal to -128, clipBottomPos is equal to -128 and (y<<2) is equal to clipTopPos, then set minY equal to 0 and maxY equal to 5.

[1807] – Otherwise, if clipTopPos is equal to -128 and clipBottomPos is not equal to -128 and (y<<2) is equal to (clipBottomPos-4), then set minY equal to -2 and maxY equal to 3.

[1808] – Otherwise, if clipTopPos is not equal to -128 and clipBottomPos is not equal to -128, then set minY equal to 0 and maxY equal to 3.

[1809] – Otherwise, set minY equal to -2 and maxY equal to 5.

[1810] – The variables minX, maxX and ac are derived as follows:

[1811] – If clipLeftPos is not equal to -128 and (x<<2) is equal to clipLeftPos, then set minX equal to 0 and maxX equal to 5.

[1812] – Otherwise, if clipRightPos is not equal to -128 and (x<<2) is equal to (clipRightPos-4), then set minX equal to -2 and maxX equal to 3.

[1813] – Otherwise, set minX equal to -2 and maxX equal to 5.

[1814] – The variable ac[x][y] is specified in Table 8-22 according to minX, maxX, minY, and maxY.

[1815] – Derivation of variables sumH[x][y], sumV[x][y], sumD0[x][y], sumD1[x][y] and sumOfHV[x][y] as follows:

[1816] sumH[x][y]=∑ i ∑ j filter[h (x<<2)+i -xCtb][v (y<<2)+j -yCtb] (8-1253)

[1817] where i = minX..maxX, j = minY..maxY

[1818] asV[x][y]=Σ i Σ j filtV[h (x<<2)+i -xCtb][v (y<<2)+j -yCtb] (8-1254)

[1819] where i = minX..maxX, j = minY..maxY

[1820] sumD0[x][y]=Σ i Σ j filtD0[h (x<<2)+i -xCtb][v (y<<2)+j -yCtb] (8-1255)

[1821] where i = minX..maxX, j = minY..maxY

[1822] sumD1[x][y]=Σ i S j filterD1[h (x<<2)+i -xCtb][v (y<<2)+j -yCtb] (8-1256)

[1823] where i = minX..maxX, j = minY..maxY

[1824] sumOfHV[x][y]=sumH[x][y]+sumV[x][y] (8-1257)]]

[1825]

[1826] The classification filter index array filtIdx and the transpose index array transposeIdx are derived by the following steps:

[1827] 3. The variables dir1[x][y], dir2[x][y], and dirS[x][y] (where x, y = 0..CtbSizeY-1) are derived as follows:

[1828] – The variables hv1, hv0, and dirHV are derived as follows:

[1829] – If sumV[x>>2][y>>2] is greater than sumH[x>>2][y>>2], then the following applies:

[1830] hv1=sumV[x>>2][y>>2] (8-1258)

[1831] hv0=sumH[x>>2][y>>2] (8-1259)

[1832] dirHV=1 (8-1260)

[1833] – Otherwise, the following applies:

[1834] hv1=sumH[x>>2][y>>2] (8-1261)

[1835] hv0=sumV[x>>2][y>>2] (8-1262)

[1836] dirHV=3 (8-1263)

[1837] –The variables d1, d0 and dirD are derived as follows:

[1838] – If sumD0[x>>2][y>>2] is greater than sumD1[x>>2][y>>2], the following applies:

[1839] d1=sumD0[x>>2][y>>2](8-1264)

[1840] d0=sumD1[x>>2][y>>2] (8-1265)

[1841] dirD=0 (8-1266)

[1842] – Otherwise, the following applies:

[1843] d1=sumD1[x>>2][y>>2] (8-1267)

[1844] d0=sumD0[x>>2][y>>2] (8-1268)

[1845] dirD=2 (8-1269)

[1846] –The variables hvd1 and hvd0 are derived as follows:

[1847] hvd1=(d1*hv0>hv1*d0)? d1:hv1 (8-1270)

[1848] hvd0=(d1*hv0>hv1*d0)? d0:hv0 (8-1271)

[1849] – The variables dirS[x][y], dir1[x][y] and dir2[x][y] are derived as follows:

[1850] dir1[x][y]=(d1*hv0>hv1*d0)? dirD:dirHV (8-1272)

[1851] dir2[x][y]=(d1*hv0>hv1*d0)? dirHV:dirD (8-1273)

[1852] dirS[x][y]=(hvd1>2*hvd0)? 1:((hvd1*2>9*hvd0)?2:0) (8-1274)

[1853] 4. The variable avgVar[x][y] (where x, y = 0..CtbSizeY-1) is derived as follows:

[1854] varTab[]={0,1,2,2,2,2,2,3,3,3,3,3,3,3,3,4} (8-1275)

[1855] avgVar[x][y]=varTab[Clip3(0,15,(sumOfHV[x>>2][y>>2]*ac[x ][y ])>>(3+BitDepth Y ))] (8-1276)

[1856] 5. The classification filter index array filtIdx[x][y] and the transpose index array transposeIdx[x][y] (where x=y=0..CtbSizeY-1) are derived as follows:

[1857] transposeTable[]={0,1,0,2,2,3,1,3}

[1858] transposeIdx[x][y]=transposeTable[dir1[x][y]*2+(dir2[x][y]>>1)]

[1859] filtIdx[x][y]=avgVar[x][y]

[1860] When dirS[x][y] is not equal to 0, filtIdx[x][y] is modified as follows:

[1861] filtIdx[x][y]+=(((dir1[x][y]&0x1)<<1)+dirS[x][y])*5(8-1277)

[1862]

[1863] ]]

[1865] 8.5.5.3 Codec Tree Block Filtering for Chroma Samples

[1866] The inputs to this process are:

[1867] – the reconstructed chroma picture sample array recPicture before adaptive loop filtering processing,

[1868] –Filtered and reconstructed chrominance picture sample array alfPicture,

[1869] – specifies the chroma position (xCtbC, yCtbC) of the top left sample of the current chroma codec tree block relative to the top left sample of the current picture,

[1870] – Alternative chroma filter index altIdx.

[1871] The output of this process is the modified filtered reconstructed chroma picture sample array alfPicture.

[1872] The width ctbWidthC and height ctbHeightC of the current chroma codec tree block are derived as follows:

[1873] ctbWidthC=CtbSizeY / SubWidthC

[1874] (8-1278)

[1875] ctbHeightC=CtbSizeY / SubHeightC

[1876] (8-1279)

[1877] To filter the derivation of the reconstructed chroma samples alfPicture[x][y], each reconstructed chroma sample within the current chroma codec treeblock recPicture[x][y] is filtered as follows, where x = 0..ctbWidthC-1 and y = 0..ctbHeightC-1:

[1878] – The position (h) of each corresponding chroma sample (x, y) within the given array recPicture of filtered samples x+i ,v y+j )(where i,j = -2..2) is derived as follows:

[1879] –[[If, for any n = 0..pps_num_ver_virtual_boundaries–1, pps_loop_filter_across_virtual_boundaries_disabled_flag is equal to 1, PpsVirtualBoundariesPosX[n] % CtbSizeY is not equal to 0, and xCtbC + x-PpsVirtualBoundariesPosX[n] / SubWidthC is greater than or equal to 0 and less than 2, the following applies:

[1880] h x+i =Clip3(PpsVirtualBoundariesPosX[n] / SubWidthC,

[1881] pic_width_in_luma_samples / SubWidthC-1,xCtbC+x+i) (8-1280)

[1882] – Otherwise, if for any n = 0..pps_num_ver_virtual_boundaries–1, pps_loop_filter_across_virtual_boundaries_disabled_flag is equal to 1, PpsVirtualBoundariesPosX[n] % CtbSizeY is not equal to 0, and PpsVirtualBoundariesPosX[n] / SubWidthC-xCtbC–x is greater than 0 and less than 3, the following applies:

[1883] h x+i =Clip3(0,PpsVirtualBoundariesPosX[n] / SubWidthC-1,xCtbC+x+i) (8-1281)

[1884] – Otherwise, the following applies:]]

[1885] h x+i =Clip3(0,pic_width_in_luma_samples / SubWidthC-1,xCtbC+x+i) (8-1282)

[1886] –[[If, for any n = 0..pps_num_hor_virtual_boundaries–1, pps_loop_filter_across_virtual_boundaries_disabled_flag is equal to 1, PpsVirtualBoundariesPosY[n] % CtbSizeY is not equal to 0, and yCtbC+y-PpsVirtualBoundariesPosY[n] / SubHeightC is greater than or equal to 0 and less than 2, the following applies:

[1887] v y+j =Clip3(PpsVirtualBoundariesPosY[n] / SubHeightC,pic_height_in_luma_samples / SubHeightC-1,yCtbC+y+j) (8-1283)

[1888] Otherwise, if for any n = 0..pps_num_hor_virtual_boundaries-1, pps_loop_filter_across_virtual_boundaries_disabled_flag is equal to 1, PpsVirtualBoundariesPosY[n] % CtbSizeY is not equal to 0, and PpsVirtualBoundariesPosY[n] / SubHeightC-yCtbC-y is greater than 0 and less than 3, the following applies:

[1889] v y+j =Clip3(0,PpsVirtualBoundariesPosY[n] / SubHeightC-1,yCtbC+y+j)(8-1284)

[1890] – Otherwise, the following applies:]]

[1891] v y+j =Clip3(0,pic_height_in_luma_samples / SubHeightC-1,yCtbC+y+j) (8-1285)

[1892] – Derives the variables clipLeftPos, clipRightPos, clipTopPos, clipBottomPos, by calling the ALF boundary position derivation process specified in clause 8.5.5.5 with (xCtbC*SubWidthC,yCtbC*SubHeightC) and (x*SubWidthC,y*SubHeightC) as input.

[1893]

[1894] –[[Set the variable clipLeftPos equal to clipLeftPos / SubWidthC.

[1895] –Set the variable clipRightPos equal to clipRightPos / SubWidthC.

[1896] –Set the variable clipTopPos equal to clipTopPos / SubHeightC.

[1897] – Set the variable clipBottomPos equal to clipBottomPos / SubHeightC.

[1898] – The vertical sample position offsets y1 and y2 are specified in Table 8-20 according to the vertical chroma sample position y, clipTopPos and clipBottomPos.

[1899] – The horizontal sample position offsets x1 and x2 are specified in Table 8-24 according to the horizontal chroma sample position x, clipLeftPos and clipRightPos. ]]

[1900]

[1901] –The variable curr is derived as follows:

[1902] curr=recPicture[h x ,v y ] (8-1286)

[1903] – The chroma filter coefficient array f[j] and the chroma clipping value array c[j] (where j = 0..5) are derived as follows:

[1904] f[j]=AlfCoeff C [slice_alf_aps_id_chroma][altIdx][j] (8-1287)

[1905] c[j]=AlfClip C [slice_alf_aps_id_chroma][altIdx][j] (8-1288)

[1906] -The variable sum is derived as follows:

[1907] sum=f[0]*(Clip3(-c[0],c[0],recPicture[h x ,v y+y2 ]-curr)+Clip3(-c[0],c[0],recPicture[h x ,v y-y2 ]-curr))+f[1]*(Clip3(-c[1],c[1],recPicture[h x+[[x]]1 ,v y+y1 ]-curr)+Clip3(-c[1],c[1],recPicture[h x-[[x]]1 ,v y-y1 ]-curr))+f[2]*(Clip3(-c[2],c[2],recPicture[h x ,v y+y1]-curr)+Clip3(-c[2],c[2],recPicture[h x ,v y-y1 ]-curr))+f[3]*(Clip3(-c[3],c[3],recPicture[h x-[[x]]1 ,v y+y1 ]-curr)+Clip3(-c[3],c[3],recPicture[h x+[[x]]1 ,v y-y1 ]-curr))+f[4]*(Clip3(-c[4],c[4],recPicture[h x+[[x]]2 ,v y ]-curr)+Clip3(-c[4],c[4],recPicture[h x-[[x]]2 ,v y ]-curr))+f[5]*(Clip3(-c[5],c[5],recPicture[h x+[[x]]1 ,v y ]-curr)+Clip3(-c[5],c[5],recPicture[h x-[[x]]1 ,v y ]-curr)) (8-1289)

[1908] sum=curr+(sum+64)>>7) (8-1290)

[1909] -The modified filtered reconstructed chrominance picture sample alfPicture[xCtbC+x][yCtbC+y] is derived as follows:

[1910] alfPicture[xCtbC+x][yCtbC+y]=Clip3(0,(1< <BitDepth C )-1,sum) (8-1291)

[1911]

[1912] [[

[1914]

[1915]

[1916] condition x1 x2 (xCtbC+x==clipLeftPos||xCtbC+x==clipRightPos-1) 0 0 (xCtbC+x==clipLeftPos+1||xCtbC+x==clipRightPos-2) 1 1 other 1 2 ]]

[1918] 8.8.5.3 ALF boundary position derivation processing

[1919] The inputs to this process are:

[1920] – specifies the luma position (xCtb, yCtb) of the top left sample of the current luma codec tree block relative to the top left sample of the current picture,

[1921] – Specifies the luma position (x, y) of the current sample relative to the top left sample of the current luma codec treeblock.

[1922] The output of this processing is:

[1923] – left vertical border position clipLeftPos,

[1924] – right vertical border position clipRightPos,

[1925] – The horizontal border position above clipTopPos,

[1926] – The bottom horizontal border position clipBottomPos.

[1927]

[1928] Set the variables clipLeftPos, clipRightPos, clipTopPos, and clipBottomPos to -128.

[1929]

[1930] The variable clipTopPos is modified as follows:

[1931] – If [[the bottom boundary of the current codec tree block is not the bottom boundary of the picture, and]]y-(CtbSizeY-4) is greater than or equal to 0, then set the variable clipTopPos equal to yCtb+CtbSizeY-4.

[1932] Otherwise, if, for any n = 0..pps_num_hor_virtual_boundaries–1, pps_loop_filter_across_virtual_boundaries_disabled_flag is equal to 1 [[, PpsVirtualBoundariesPosY[n] % CtbSizeY is equal to 0,]] and yCtb+y−PpsVirtualBoundariesPosY[n] is greater than or equal to 0 and less than 3, then the following applies: clipTopPos=PpsVirtualBoundariesPosY[n] (8-1292)

[1933] – Otherwise, if y is less than 3[[, and the top boundary of the current codec tree block is not the top boundary of the picture,]] and one or more of the following conditions are true, then set the clipTopPos variable equal to yCtb:

[1934] – If the top boundary of the current codec tree block is the top boundary of a brick and loop_filter_across_bricks_enabled_flag is equal to 0.

[1935] – If the top boundary of the current codec treeblock is the top boundary of a slice and loop_filter_across_slices_enabled_flag is equal to 0.

[1936] – If the top boundary of the current codec treeblock is the top boundary of a sub-picture and loop_filter_across_subpic_enabled_flag[SubPicIdx] is equal to 0.

[1937] The variable clipBottomPos is modified as follows:

[1938] – If [[the bottom boundary of the current codec tree block is not the bottom boundary of the picture, and]] CtbSizeY-4-y is greater than 0 and less than 4, then set the variable clipBottomPos equal to yCtb+CtbSizeY-4.

[1939] – Otherwise, if for any n = 0..pps_num_hor_virtual_boundaries–1, pps_loop_filter_across_virtual_boundaries_disabled_flag is equal to 1 [[, PpsVirtualBoundariesPosY[n] % CtbSizeY is equal to 0]], PpsVirtualBoundariesPosY[n] is not equal to pic_height_in_luma_samples-1 or 0, and PpsVirtualBoundariesPosY[n]-yCtb–y is greater than 0 and less than 4, then the following applies:

[1940] clipBottomPos=PpsVirtualBoundariesPosY[n] (8-1293)

[1941] – Otherwise, if CtbSizeY-y is less than 4[[, and the bottom boundary of the current codec tree block is not the bottom boundary of the picture,]] and one or more of the following conditions are true, then set the variable clipBottomPos equal to yCtb+CtbSizeY:

[1942] – If the bottom boundary of the current codec tree block is the bottom boundary of a brick and loop_filter_across_bricks_enabled_flag is equal to 0.

[1943] – If the bottom boundary of the current codec treeblock is the bottom boundary of a slice and loop_filter_across_slices_enabled_flag is equal to 0.

[1944] – If the bottom boundary of the current codec treeblock is the bottom boundary of the sub-picture and loop_filter_across_subpic_enabled_flag[SubPicIdx] is equal to 0.

[1945] The variable clipLeftPos is modified as follows:

[1946] – If, for any n = 0..pps_num_ver_virtual_boundaries-1, pps_loop_filter_across_virtual_boundaries_disabled_flag is equal to 1 [[, PpsVirtualBoundariesPosX[n] % CtbSizeY is equal to 0,]] and xCtb+x-PpsVirtualBoundariesPosX[n] is greater than or equal to 0 and less than 3, the following applies:

[1947] clipLeftPos=PpsVirtualBoundariesPosX[n] (8-1294)

[1948] – Otherwise, if x is less than 3 [[, the left boundary of the current codec tree block is not the left boundary of the picture]] and one or more of the following conditions are true, then set the variable clipLeftPos equal to xCtb:

[1949] – If the left border of the current codec tree block is the left border of a brick and loop_filter_across_bricks_enabled_flag is equal to 0.

[1950] – If the left boundary of the current codec treeblock is the left boundary of a slice, and loop_filter_across_slices_enabled_flag is equal to 0.

[1951] – If the left boundary of the current codec treeblock is the left boundary of a sub-picture and loop_filter_across_subpic_enabled_flag[SubPicIdx] is equal to 0.

[1952] The variable clipRightPos is modified as follows:

[1953] – If, for any n = 0..pps_num_ver_virtual_boundaries-1, pps_loop_filter_across_virtual_boundaries_disabled_flag is equal to 1 [[, PpsVirtualBoundariesPosX[n] % CtbSizeY is equal to 0,]] and PpsVirtualBoundariesPosX[n]-xCtb-x is greater than 0 and less than 4, the following applies:

[1954] clipRightPos=PpsVirtualBoundariesPosX[n] (8-1295)

[1955] – Otherwise, if CtbSizeY-x is less than 4[[, and the right boundary of the current codec tree block is not the right boundary of the picture,]] and one or more of the following conditions are true, then set the variable clipRightPos equal to xCtb+CtbSizeY:

[1956] – If the right border of the current codec tree block is the right border of a brick, and loop_filter_across_bricks_enabled_flag is equal to 0.

[1957] – If the right boundary of the current codec treeblock is the right boundary of a slice, and loop_filter_across_slices_enabled_flag is equal to 0.

[1958] – If the right border of the current codec treeblock is the right border of the sub-picture and loop_filter_across_subpic_enabled_flag[SubPicIdx] is equal to 0.

[1959]

[1960]

[1961]

[1962]

[1963]

[1964] Figure 22 is a block diagram of a video processing device 2200. The device 2200 can be used to implement one or more methods described herein. The device 2200 can be embodied in a smartphone, a tablet computer, a computer, an Internet of Things (IoT) receiver, etc. The device 2200 may include one or more processors 2202, one or more memories 2204, and video processing hardware 2206. The processor 2202 can be configured to implement one or more methods described in this document. The memory(s) 2204 can be used to store data and code for implementing the methods and techniques described herein. The video processing hardware 2206 can be used to implement some of the techniques described in this document in hardware circuitry. In some embodiments, the video processing hardware 2206 can be internal to or partially internal to the processor 2202 (e.g., a graphics processor unit).

[1965] In some embodiments, the video encoding and decoding method may use Figure 22 The described apparatus is implemented on a hardware platform.

[1966] Figure 23 23 is a flow chart of an example method 2300 for video processing. The method includes, for a conversion between a current video block of a video and a bitstream representation of the current video block, determining (2302) one or more interpolation filters to use during the conversion, wherein the one or more interpolation filters are from a plurality of interpolation filters for the video, and performing (2304) the conversion using the one or more interpolation filters.

[1967] Use solution lists to further describe the various solutions and embodiments described in this document.

[1968] Section 4, item 1 provides further examples of the following solutions.

[1969] 1. A method of video processing, comprising: performing conversion between video blocks of a video picture and a bitstream representation thereof, wherein the video blocks are processed using logical groupings of codec tree blocks, wherein the codec tree blocks are processed based on whether a bottom boundary of a bottom codec tree block is outside a bottom boundary of the video picture.

[1970] 2. The method according to solution 1, wherein processing the codec treeblock comprises: performing adaptive loop filtering on sample values ​​of the codec treeblock by using samples within the codec treeblock.

[1971] 3. The method according to solution 1, wherein processing the codec tree block includes: performing adaptive loop filtering on the sample values ​​of the codec tree block by disabling the division of the codec tree block into two parts according to a virtual boundary.

[1972] Section 4, item 2 provides further examples of the following solutions.

[1973] 4. A method of video processing, comprising: determining a usage status of virtual samples during loop filtering based on a condition of a codec tree block of a current video block; and performing conversion between the video block and a bitstream representation of the video block consistent with the usage status of the virtual samples.

[1974] 5. The method according to solution 4, wherein the logical true value of the usage state indicates that the current video block is divided into at least two parts by a virtual boundary, and filtering of samples in one part to utilize information of another part is not allowed.

[1975] 6. The method of solution 4, wherein a logical true value of the usage state indicates use of virtual samples during in-loop filtering, and wherein the in-loop filtering is performed using values ​​of the modified reconstructed samples of the current video block.

[1976] 7. The method of solution 4, wherein a logical false value of the usage state indicates that filtering of samples in a block is allowed to utilize information in the same block.

[1977] 8. The method of solution 4, wherein the logical true value of the usage state indicates that loop filtering is performed on the reconstructed samples of the current video block without further modifying the reconstructed samples.

[1978] 9. The method according to any one of solutions 4-8, wherein the condition specifies setting the usage state to a logical false value because the codec tree block has a specific size.

[1979] 10. The method according to any one of solutions 4-8, wherein the condition specifies setting the usage state to a logical false value because the size of the codec tree block is greater than a specific size.

[1980] 11. According to the method of any one of solutions 4-8, the codec tree block has a size smaller than a specific size.

[1981] Section 4, item 3 provides further examples of the following solutions.

[1982] 12. The method according to solution 5, wherein the condition depends on whether the bottom boundary of the current video block is smaller than the bottom boundary of the video unit of the video picture, or the bottom boundary of the current video block is a virtual boundary.

[1983] 13. The method of solution 12, wherein the condition depends on whether the bottom boundary of the current video block is a bottom boundary of a slice or a tile boundary.

[1984] 14. The method of solution 12, wherein the condition specifies setting the usage state to a logical true value when a bottom boundary of the current video block is a bottom boundary of a slice or a tile boundary.

[1985] 15. The method of solution 4-12, wherein the condition specifies setting the usage state to a logical false value when the bottom boundary of the current video block is at or outside the bottom boundary of the picture boundary.

[1986] Section 4, item 4 provides further examples of the following solutions.

[1987] 16. A method of video processing, comprising: during a conversion between a video picture logically grouped into one or more video slices or video bricks and a bitstream representation of the video picture, determining to disable use of samples in another slice or brick in adaptive loop filter processing; and performing the conversion consistent with the determination.

[1988] Section 4, item 5 provides further examples of the following solutions.

[1989] 17. A method of video processing, comprising: during conversion between a current video block of a video picture and a bitstream representation of the current video block, determining that the current video block includes samples located at a boundary of a video unit of the video picture; and performing the conversion based on the determination, wherein performing the conversion includes generating virtual samples for loop filtering processing using a unified method that is the same for all boundary types in the video picture.

[1990] 18. The method of solution 17, wherein the video unit is a strip or a slice or a 360-degree video.

[1991] 19. The method of solution 17, wherein the loop filtering comprises adaptive loop filtering.

[1992] 20. The method according to any one of solutions 17-19, wherein the unified method is a two-sided filling method.

[1993] 21. A method according to any one of solutions 17 to 20, wherein the unified method is that when access to samples below the first row is not allowed and padding is used to generate virtual samples for the samples below the first row, access to samples above the second row is also set to be not allowed and padding is used to generate virtual samples for the samples above the second row.

[1994] 22. A method according to any one of solutions 17 to 20, wherein the unified method is that when access to samples above the first row is not allowed and padding is used to generate virtual samples for the samples above the first row, access to samples below the second row is also set to be not allowed and padding is used to generate virtual samples for the samples below the second row.

[1995] 23. The method according to any one of solutions 21-22, wherein the distance between the first row and the current row where the current sample to be filtered is located is equal to the distance between the second row and the first row.

[1996] Section 4, item 6 provides further examples of the following solutions.

[1997] 24. A method of video processing, comprising: during a transition between a current video block of a video picture and a bitstream representation thereof, determining to apply one of a plurality of adaptive loop filter (ALF) sample selection methods available for the video picture during the transition; and performing the transition by applying the one of the plurality of ALF sample selection methods.

[1998] 25. The method of solution 24, wherein the multiple ALF sample selection methods include: a first method in which samples are selected during conversion and before a loop filter is applied to the current video block; and a second method in which samples are selected during conversion and after the loop filter is applied to the current video block.

[1999] Section 4, item 7 provides further examples of the following solutions.

[2000] 26. A method of video processing, comprising: performing a loop filtering operation on samples of a current video block of a video picture during a conversion between the current video block and a bitstream representation of the current video block based on a boundary rule; wherein the boundary rule prohibits using samples of a virtual pipeline data unit (VPDU) that crosses the video picture, and using a result of the loop filtering operation to perform the conversion.

[2001] 27. The method of solution 26, wherein the VPDU corresponds to a region of a video picture having a fixed size.

[2002] 28. The method according to any of solutions 26-27, wherein the boundary rule further specifies the use of virtual samples instead of forbidden samples for loop filtering.

[2003] 29. The method according to solution 28, wherein the virtual sample points are generated by filling.

[2004] Section 4, item 8 provides further examples of the following solutions.

[2005] 30. A method of video processing, comprising: performing a loop filtering operation on samples of a current video block of a video picture during a conversion between the current video block and a bitstream representation of the current video block based on a boundary rule; wherein the boundary rule specifies that for positions of the current video block that cross a video unit boundary, samples generated without padding are used; and using a result of the loop filtering operation to perform the conversion.

[2006] 31. The method according to solution 30, wherein the sample points are generated using a two-side filling technique.

[2007] 32. The method of solution 30, wherein the loop filtering operation includes using the same virtual sample generation technique for positionally symmetrical samples during the loop filtering operation.

[2008] 33. The method according to any one of solutions 30-32, wherein performing a loop filtering operation on the samples of the current video block comprises: reshaping the samples of the current video block before applying the loop filtering.

[2009] Section 4, item 9 provides further examples of the following solutions.

[2010] 34. A method of video processing, comprising: performing a loop filtering operation on samples of a current video block of a video picture during a conversion between the current video block and a bitstream representation of the current video block based on a boundary rule; wherein the boundary rule specifies selecting a filter for the loop filtering operation, the dimension of the filter should be such that the samples of the current video block used during the loop filtering do not cross a boundary of a video unit of the video picture; and using the result of the loop filtering operation to perform the conversion.

[2011] Section 4, item 10 provides further examples of the following solutions.

[2012] 35. A method of video processing, comprising: performing a loop filtering operation on samples of a current video block of a video picture during a conversion between the current video block and a bitstream representation of the current video block based on a boundary rule; wherein the boundary rule specifies selecting a cropping parameter or filter coefficient for the loop filtering operation based on whether the loop filtering requires padding samples; and using a result of the loop filtering operation to perform the conversion.

[2013] 36. The method of solution 35, wherein the cropping parameters or filter coefficients are included in the bitstream representation.

[2014] Section 4, item 11 provides further examples of the following solutions.

[2015] 37. A method of video processing, comprising: performing a loop filtering operation on samples of a current video block of a video picture during a conversion between the current video block and a bitstream representation of the current video block based on a boundary rule; wherein the boundary rule depends on a color component identity of the current video block; and using a result of the loop filtering operation to perform the conversion.

[2016] 38. The method of solution 37, wherein the boundary rules are different for brightness and / or different color components.

[2017] 39. The method of any one of solutions 1-38, wherein the converting comprises encoding the current video block into the bitstream representation.

[2018] 40. The method according to any of the solutions 1-38, wherein said converting c...

Claims

1. A method for processing video data, comprising: For conversion between a current block of a video and a bitstream of the video, determining that one or more top-left neighboring samples outside the current block are unavailable to a codec tool applied to the current block; In a case where the left neighboring samples outside the current block are available and the upper neighboring samples outside the current block are available, filling the one or more upper left neighboring samples with the upper neighboring samples outside the current block; as well as performing the conversion based on the padded one or more upper left neighboring samples, The one or more upper left adjacent sample points filled are used for a classification operation in an adaptive loop filter process of the codec tool, wherein the one or more upper left adjacent sample points filled are used to determine a classification filter index in the classification operation, and the classification filter index is used to determine a filter coefficient set, wherein during the classification operation, a classification of an M×N region of the current block is determined based on a gradient of a subset of samples of a P×Q region, wherein the P×Q region includes the M×N region, the current block is a codec tree block, the M×N region is a 4×4 region, the P×Q region is an 8×8 region, and When there are samples in the 4×4 block located at the left boundary of the current block, the two leftmost columns of samples in the 8×8 area are filled; or, when there are samples in the 4×4 block located at the top boundary of the current block, the two topmost rows of samples in the 8×8 area are filled.

2. The method according to claim 1, wherein: The one or more upper left neighboring samples are unavailable because the one or more upper left neighboring samples are located in a different video unit from the current block and the codec tool is not allowed to use samples across video units.

3. The method according to claim 1, wherein: The left neighboring samples and the upper neighboring samples are available because the left neighboring samples and the upper neighboring samples are located in the same video unit as the current block.

4. The method according to claim 2, wherein: The video unit is a slice.

5. The method according to claim 1, wherein: The padded one or more upper left neighboring samples and the set of filter coefficients are used in filter computation of the adaptive loop filter process to derive filtered reconstructed samples of the current block.

6. The method according to claim 1, wherein: The coding tool includes an adaptive loop filtering (ALF) process.

7. The method according to claim 1, wherein: The codec tool includes a cross-component adaptive loop filtering CC-ALF process.

8. The method according to claim 1, wherein: The converting includes encoding the video into the bitstream.

9. The method according to claim 1, wherein: The converting includes decoding the video from the bitstream.

10. An apparatus for processing video data, comprising a processor and a non-transitory memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to: For conversion between a current block of a video and a bitstream of the video, determining that one or more top-left neighboring samples outside the current block are unavailable to a codec tool applied to the current block; In a case where the left neighboring samples outside the current block are available and the upper neighboring samples outside the current block are available, filling the one or more upper left neighboring samples with the upper neighboring samples outside the current block; as well as performing the conversion based on the padded one or more upper left neighboring samples, The one or more upper left adjacent sample points filled are used for a classification operation in an adaptive loop filter process of the codec tool, wherein the one or more upper left adjacent sample points filled are used to determine a classification filter index in the classification operation, and the classification filter index is used to determine a filter coefficient set, wherein during the classification operation, a classification of an M×N region of the current block is determined based on a gradient of a subset of samples of a P×Q region, wherein the P×Q region includes the M×N region, the current block is a codec tree block, the M×N region is a 4×4 region, the P×Q region is an 8×8 region, and When there are samples in the 4×4 block located at the left boundary of the current block, the two leftmost columns of samples in the 8×8 area are filled; or, when there are samples in the 4×4 block located at the top boundary of the current block, the two topmost rows of samples in the 8×8 area are filled.

11. The device according to claim 10, wherein: The one or more upper left neighboring samples are unavailable because the one or more upper left neighboring samples are located in a different video unit than the current block and the codec is not allowed to use samples across video units; The left neighboring sample point and the upper neighboring sample point are available because the left neighboring sample point and the upper neighboring sample point are located in the same video unit as the current block; and The video unit is a strip.

12. The device according to claim 10, wherein: The padded one or more upper left neighboring samples and the set of filter coefficients are used in filter computation of the adaptive loop filter process to derive filtered reconstructed samples of the current block.

13. The device according to claim 10, wherein: The codec tool comprises an adaptive loop filtering (ALF) process, and the codec tool comprises a cross-component adaptive loop filtering (CC-ALF) process.

14. A non-transitory computer-readable storage medium storing instructions that cause a processor to: For conversion between a current block of a video and a bitstream of the video, determining that one or more top-left neighboring samples outside the current block are unavailable to a codec tool applied to the current block; In a case where the left neighboring samples outside the current block are available and the upper neighboring samples outside the current block are available, filling the one or more upper left neighboring samples with the upper neighboring samples outside the current block; as well as performing the conversion based on the padded one or more upper left neighboring samples, The one or more upper left adjacent sample points filled are used for a classification operation in an adaptive loop filter process of the codec tool, wherein the one or more upper left adjacent sample points filled are used to determine a classification filter index in the classification operation, and the classification filter index is used to determine a filter coefficient set, wherein during the classification operation, a classification of an M×N region of the current block is determined based on a gradient of a subset of samples of a P×Q region, wherein the P×Q region includes the M×N region, the current block is a codec tree block, the M×N region is a 4×4 region, the P×Q region is an 8×8 region, and When there are samples in the 4×4 block located at the left boundary of the current block, the two leftmost columns of samples in the 8×8 area are filled; or, when there are samples in the 4×4 block located at the top boundary of the current block, the two topmost rows of samples in the 8×8 area are filled.

15. The non-transitory computer-readable storage medium of claim 14, wherein: The one or more upper left neighboring samples are unavailable because the one or more upper left neighboring samples are located in a different video unit than the current block and the codec is not allowed to use samples across video units; The left neighboring sample point and the upper neighboring sample point are available because the left neighboring sample point and the upper neighboring sample point are located in the same video unit as the current block; and The video unit is a strip.

16. The non-transitory computer-readable storage medium of claim 14, wherein: The padded one or more upper left neighboring samples and the set of filter coefficients are used in filter computation of the adaptive loop filter process to derive filtered reconstructed samples of the current block; as well as The codec tool comprises an adaptive loop filtering (ALF) process, and the codec tool comprises a cross-component adaptive loop filtering (CC-ALF) process.

17. A method for storing a bitstream of a video, comprising: For a current block of a video, determining that one or more top-left neighboring samples outside the current block are unavailable to a codec tool applied to the current block; In a case where the left neighboring samples outside the current block are available and the upper neighboring samples outside the current block are available, filling the one or more upper left neighboring samples with the upper neighboring samples outside the current block; generating the bitstream based on the padded one or more upper left adjacent samples; as well as storing the bitstream in a non-transitory computer-readable recording medium, The one or more upper left adjacent sample points filled are used for a classification operation in an adaptive loop filter process of the codec tool, wherein the one or more upper left adjacent sample points filled are used to determine a classification filter index in the classification operation, and the classification filter index is used to determine a filter coefficient set, wherein during the classification operation, a classification of an M×N region of the current block is determined based on a gradient of a subset of samples of a P×Q region, wherein the P×Q region includes the M×N region, the current block is a codec tree block, the M×N region is a 4×4 region, the P×Q region is an 8×8 region, and When there are samples in the 4×4 block located at the left boundary of the current block, the two leftmost columns of samples in the 8×8 area are filled; or, when there are samples in the 4×4 block located at the top boundary of the current block, the two topmost rows of samples in the 8×8 area are filled.

18. The method for storing a bit stream of a video according to claim 17, wherein: The one or more upper left neighboring samples are unavailable because the one or more upper left neighboring samples are located in a different video unit than the current block and the codec is not allowed to use samples across video units; The left neighboring sample point and the upper neighboring sample point are available because the left neighboring sample point and the upper neighboring sample point are located in the same video unit as the current block; and The video unit is a strip.

19. The method for storing a bit stream of a video according to claim 17, wherein: The padded one or more upper left neighboring samples and the set of filter coefficients are used in filter computation of the adaptive loop filter process to derive filtered reconstructed samples of the current block; as well as The codec tool comprises an adaptive loop filtering (ALF) process, and the codec tool comprises a cross-component adaptive loop filtering (CC-ALF) process.

Images