Sample filling for cross-component adaptive loop filtering

Abstract

A video processing method is described. The method includes: for a conversion between a video unit and a bitstream representation of the video, determining whether to enable a mirror fill process for filling unavailable luminance samples during the application of a loop filtering tool for the video unit; and performing the conversion based on the determination.

Description

[0001] Cross-references to related applications

[0002] This application is based on International Patent Application No. PCT / CN2020 / 135134, filed on December 10, 2020, which claims priority and interest in International Patent Application No. PCT / CN2019 / 124481, filed on December 11, 2019. All of the aforementioned patent applications are incorporated herein by reference in their entirety. Technical Field

[0003] This application document relates to the encoding and decoding of images and videos. Background Technology

[0004] Digital video accounts for the largest share of bandwidth usage on the internet and other digital communication networks. As the number of connected user devices capable of receiving and displaying video increases, the bandwidth demand for digital video is expected to continue to grow. Summary of the Invention

[0005] This document discloses techniques that can be used by video encoders and decoders during video encoding or decoding to perform cross-component adaptive loop filtering.

[0006] In one example aspect, a method for video processing is disclosed. The method includes: a conversion between video units and a bitstream representation of the video; determining whether a mirror fill process for filling unavailable luminance samples is enabled for the video units during the application of a loop filtering tool; and performing the conversion based on the determination.

[0007] In another example aspect, a method for video processing is disclosed. The method includes: for a conversion between video units and a bitstream representation of the video; determining, based on encoding / decoding information of the video units, whether to apply a repeat padding process and / or a mirror padding process to pad samples located at virtual boundaries; and performing the conversion based on the determination.

[0008] In yet another example aspect, a video encoder device is disclosed. The video encoder includes a processor configured to implement the methods described above.

[0009] In yet another example aspect, a video decoder device is disclosed. The video decoder includes a processor configured to implement the methods described above.

[0010] In yet another example, a computer-readable medium having code stored thereon is disclosed. The code implements one of the methods described herein in the form of processor-executable code.

[0011] These and other features are described throughout this document. Attached Figure Description

[0012] Figure 1 The nominal vertical and horizontal positions of the 4:2:2 luminance and chrominance samples in the image are shown.

[0013] Figure 2 This is an example of an encoder block diagram.

[0014] Figure 3 The image shown has an 18x12 brightness CTU divided into 12 slices and 3 raster scan strips.

[0015] Figure 4 The image shown has an 18x12 luminance CTU divided into 24 segments and 9 rectangular stripes.

[0016] Figure 5 The image shows a panel divided into four pieces and four rectangular strips.

[0017] Figure 6 The image is shown as being divided into 15 segments, 24 strips, and 24 sub-images.

[0018] Figures 7A-7C It is shown that: Figure 7A It is the CTB that spans the bottom image boundary; Figure 7B It is the CTB that crosses the right edge of the image; Figure 7C It is the CTB that crosses the right bottom edge of the image.

[0019] Figure 8 It is an illustration of image samples on an 8×8 grid, horizontal and vertical block boundaries, and non-overlapping blocks of 8×8 samples that can be de-blocked in parallel.

[0020] Figure 9 The pixels involved in the filter on / off decision and strong / weak filter selection are shown.

[0021] Figure 10 Four 1-D orientation patterns for EO sample point classification are shown: horizontal (EO classification = 0), vertical (EO classification = 1), 135° diagonal (EO classification = 2), and 45° diagonal (EO classification = 3).

[0022] Figure 11 An example of the ALF filter shape is shown (chroma: 5×5 rhombus, luminance: 7×7 rhombus).

[0023] Figure 12A The location of the subsamples of the vertical gradient is shown. Figure 12B The locations of subsamples of the horizontal gradient are shown. Figure 12C The locations of subsamples of the diagonal gradient are shown, and Figure 12D The location of the subsamples of the diagonal gradient is shown.

[0024] Figure 13An example of the loop filter line buffer requirements for the luminance component is shown in VTM-4.0.

[0025] Figure 14 An example of the loop filter line buffer requirements for the chroma component is shown in VTM-4.0.

[0026] Figure 15 An example of modified block classification at virtual boundaries is shown.

[0027] Figure 16 An example of modified ALF filtering for the luminance component at the virtual boundary is shown.

[0028] Figure 17A This shows the required line (on each side) that needs to be filled above / below VB. Figure 17B This shows the two required lines (on each side) that need to be filled above / below VB, and Figure 17C The diagram shows the three required lines (on each side) that need to be filled above / below VB.

[0029] Figure 18 An example of repeated padding of luminance ALF filtering at the boundaries of an image / sub-image / strip / piece is shown.

[0030] Figure 19 An example of horizontal winding motion compensation in VVC is shown.

[0031] Figure 20 An image of the HEC in a 3×2 layout is shown.

[0032] Figure 21A The arrangement of CC-ALF with respect to other loop filters is shown. Figure 21B A diamond-shaped filter is shown.

[0033] Figure 22 An example of repeated padding at the virtual boundary of CC-ALF in JVET-P0080 is shown.

[0034] Figure 23 A 3×4 diamond filter with 8 unique coefficients is shown.

[0035] Figure 24 This is an example of repeated padding at the ALF virtual boundary of CC-ALF in JVET-P1008.

[0036] Figure 25 The 8-coefficient CC-ALF filter shape of JVET-P0106 ​​is shown.

[0037] Figure 26 The shape of the 6-coefficient CC-ALF filter in JVET-P0173 is shown.

[0038] Figure 27 The shape of the 6-coefficient CC-ALF filter in JVET-P0251 is shown.

[0039] Figure 28 An example of the JC-CCALF process is shown.

[0040] Figure 29 Example locations of sample points to be filled are shown for a CC-ALF filtering method with an 8-tap 4×3 filter shape.

[0041] Figure 30 An example of mirror fill method 1 is shown.

[0042] Figure 31 An example of mirror fill method 2 is shown.

[0043] Figure 32 This is a block diagram of an example video processing system that can implement the disclosed technology.

[0044] Figure 33 This is a block diagram of an example hardware platform used for video processing.

[0045] Figure 34 This is a flowchart of an example video processing method.

[0046] Figure 35 This is a block diagram illustrating a video encoding / decoding system according to some embodiments of the present disclosure.

[0047] Figure 36 This is a block diagram illustrating an encoder according to some embodiments of the present disclosure.

[0048] Figure 37 This is a block diagram illustrating a decoder according to some embodiments of the present disclosure.

[0049] Figure 38A and 38B This is a flowchart of an example video processing method based on some implementations of the disclosed technology. Detailed Implementation

[0050] This document uses section headings for ease of understanding and does not limit the applicability of the technologies and embodiments disclosed in each section to that section. Additionally, H.266 terminology is used in some descriptions solely for ease of understanding and does not limit the scope of the disclosed technologies. Therefore, the technologies described herein are also applicable to other video codec protocols and designs.

[0051] 1. Overview

[0052] This document relates to video codec technologies. Specifically, it covers image / sub-image / strip / piece boundaries, as well as 360-degree video virtual boundaries and ALF virtual boundaries codecs, particularly for Cross-Component Adaptive Loop Filtering (CC-ALF) and other codec tools in image / video codecs. It can be applied to existing video codec standards such as HEVC, or to standards yet to be finalized (multi-functional video codecs). It can also be applied to future video codec standards or video codecs.

[0053] 2. Introduction to Video Encoding and Decoding

[0054] Video codec standards have evolved primarily through the development of well-known ITU-T and ISO / IEC standards. ITU-T produced H.261 and H.263, ISO / IEC produced MPEG-1 and MPEG-4 Visual, and the two organizations jointly produced the H.262 / MPEG-2 Video, H.264 / MPEG-4 Advanced Video Coding (AVC), and H.265 / HEVC standards. Since H.262, video codec standards have been based on a hybrid video codec architecture, employing temporal prediction plus transform coding. To explore future video codec technologies beyond HEVC, the Joint Video Exploration Group (JVET) was jointly established by VCEG and MPEG in 2015. Since then, many new methods have been adopted by JVET and incorporated into reference software called the Joint Exploration Model (JEM). In April 2018, a Joint Video Experts Group (JVET) was created between VCEG (Q6 / 16) and ISO / IEC JTC1 SC29 / WG11 (MPEG) to study the VVC standard, with the goal of reducing the bit rate by 50% compared to HEVC.

[0055] 2.1. Color Space and Chroma Subsampling

[0056] A color space (also known as a color model (or color system)) is an abstract mathematical model that simply describes the range of colors as tuples of numbers, typically 3 or 4 values ​​or color components (e.g., RGB). Essentially, a color space is a refinement of a coordinate system and its subspaces.

[0057] For video compression, the most commonly used color spaces are YCbCr and RGB.

[0058] YCbCr, Y'CbCr, or Y Pb / Cb Pr / Cr (also written as YCBCR or Y'CBCR) are color space families used as part of the color image pipeline in video and digital photography systems. Y' is the luminance component, and CB and CR are the blue and red difference chromaticity components. Y' (with an apostrophe) is different from Y; it is luminance, meaning the light intensity is based on a gamma-corrected non-linear encoding of the RGB primary colors.

[0059] Chroma subsampling is a practice of encoding images by achieving chromaticity information with a lower resolution than luminance information, taking advantage of the human visual system's sensitivity to color differences that are lower than luminance. 2.1.1.4:4:4

[0061] Each of the three Y'CbCr components has the same sampling rate, so there is no chromaticity subsampling. This scheme is sometimes used in high-end film scanners and film post-production. 2.1.2.4:2:2

[0063] The two chroma components are sampled at half the luminance sampling rate: the horizontal chroma resolution is halved while the vertical chroma resolution remains unchanged. This reduces the bandwidth of the uncompressed video signal by one-third with almost no visual difference. An example of the nominal vertical and horizontal positions for the 4:2:2 color format is shown in the VVC working draft. Figure 1 As shown in the image. 2.1.3.4:2:0

[0065] In 4:2:0, the horizontal sampling is doubled compared to 4:1:1, but the vertical resolution is half because the Cb and Cr channels are sampled only per alternating line in this scheme. Therefore, the data rate is the same. Cb and Cr are subsampled horizontally and vertically, each in a factor of 2. There are three variations of the 4:2:0 scheme with different horizontal and vertical settings.

[0066] In MPEG-2, Cb and Cr are horizontally co-located. Cb and Cr are vertically located between pixels (gap-filling positioning).

[0067] • In JPEG / JFIF, H.261 and MPEG-1, Cb and Cr are interstitially located between alternating luminance samples.

[0068] In a 4:2:0 DV configuration, Cb and Cr are co-located in the horizontal direction. In the vertical direction, they are co-located on alternating lines.

[0069] Table 2-1. SubWidthC and SubHeightC values ​​derived from chroma_format_idc and separate_colour_plane_flag

[0070]

[0071]

[0072] 2.2. Encoding / decoding streams of typical video codecs

[0073] Figure 2 An example of a VVC encoder block diagram is shown, containing three loop filtering blocks: Deblocking Filter (DF), Sample Adaptive Offset (SAO), and ALF. Unlike DF, which uses predefined filters, SAO and ALF reduce the mean square error between the original and reconstructed samples by adding an offset and applying a Finite Impulse Response (FIR) filter to the original samples of the current image, respectively. Side information signaling from the encoder / decoder informs the offset and filter coefficients. ALF is located at the final processing stage of each image and can be viewed as a tool attempting to capture and resolve artifacts produced by previous stages.

[0074] 2.3. Example Definition of Video Unit

[0075] The image is divided into one or more slice rows and one or more slice columns. A slice is a sequence of CTUs that covers a rectangular area of ​​the image. The CTUs in a slice are scanned in raster scan order within that slice.

[0076] A strip consists of an integer number of consecutive complete CTU lines within an integer number of complete slices or images.

[0077] Two stripe modes are supported: raster scan stripe mode and rectangular stripe mode. In raster scan stripe mode, the stripe contains a sequence of complete slices in a sheet raster scan of the image. In rectangular stripe mode, the stripe contains several complete slices collectively forming a rectangular region of the image, or several consecutive complete CTU rows of a slice collectively forming a rectangular region of the image. Slices within a rectangular stripe are scanned in sheet raster scan order within the rectangular region corresponding to that stripe.

[0078] The sub-image contains one or more stripes that collectively cover a rectangular area of ​​the image.

[0079] Figure 3 An example of raster scan strip segmentation of an image is shown, where the image is divided into 12 slices and 3 raster scan strips.

[0080] Figure 4 The VVC specification shows an example of rectangular strip segmentation for an image, where the image is divided into 24 segments (6 columns and 4 rows) and 9 rectangular strips.

[0081] Figure 4 The image has an 18x12 luminance CTU, which is divided into 24 panels and 9 rectangular stripes (informative).

[0082] Figure 5 An example of an image divided into slices and rectangular strips is shown, where the image is divided into 4 slices (2 columns and 2 rows) and 4 rectangular strips.

[0083] Figure 6 An example of sub-image segmentation of an image is shown, where the image is segmented into 15 pieces, covering 4x4 CTUs of varying dimensions, 24 strips, and 24 sub-images.

[0084] 2.3.1. CTU / CTB Dimensions

[0085] In VVC, the CTU size, which is notified by the syntax element log2_ctu_size_minus2 signaling in SPS, can be as small as 4×4.

[0086] 7.3.2.3 Sequence Parameter Set (RBSP) Syntax

[0087]

[0088]

[0089]

[0090] The increment of 2 in log2_ctu_size_minus2 specifies the size of the luminance codec tree block for each CTU.

[0091] log2_min_luma_coding_block_size_minus2 plus 2 specifies the minimum luma encoding / decoding block size.

[0092] The variables CtbLog2SizeY, CtbSizeY, MinCbLog2SizeY, MinCbSizeY, MinTbLog2SizeY, MaxTbLog2SizeY, MinTbSizeY, MaxTbSizeY, PicWidthInCtbsY, PicHeightInCtbsY, PicSizeInCtbsY, PicWidthInMinCbsY, PicHeightInMinCbsY, PicSizeInMinCbsY, PicSizeInSamplesY, PicWidthInSamplesC, and PicHeightInSamplesC are derived as follows:

[0093] CtbLog2SizeY=log2_ctu_size_minus2+2 (7-9)

[0094] CtbSizeY=1<<CtbLog2SizeY (7-10)

[0095] MinCbLog2SizeY=log2_min_luma_coding_block_size_minus2+2 (7-11)

[0096] MinCbSizeY=1<<MinCbLog2SizeY (7-12)

[0097] MinTbLog2SizeY=2 (7-13)

[0098] MaxTbLog2SizeY=6 (7-14)

[0099] MinTbSizeY=1<<MinTbLog2SizeY (7-15)

[0100] MaxTbSizeY=1<<MaxTbLog2SizeY (7-16)

[0101] PicWidthInCtbsY=Ceil(pic_width_in_luma_samples÷CtbSizeY) (7-17)

[0102] PicHeightInCtbsY=Ceil(pic_height_in_luma_samples÷CtbSizeY) (7-18)

[0103] PicSizeInCtbsY=PicWidthInCtbsY*PicHeightInCtbsY (7-19)

[0104] PicWidthInMinCbsY=pic_width_in_luma_samples / MinCbSizeY (7-20)

[0105] PicHeightInMinCbsY=pic_height_in_luma_samples / MinCbSizeY (7-21)

[0106] PicSizeInMinCbsY=PicWidthInMinCbsY*PicHeightInMinCbsY (7-22)

[0107] PicSizeInSamplesY = pic_width_in_luma_samples * pic_height_in_luma_samples (7-23)

[0108] PicWidthInSamplesC = pic_width_in_luma_samples / SubWidthC (7-24)

[0109] PicHeightInSamplesC = pic_height_in_luma_samples / SubHeightC (7-25)

[0110] 2.3.2. CTUs in the Picture

[0111] Assume that the CTB / LCU size is indicated by M×N (typically M equals N, as defined in HEVC / VVC), and for a CTB located at the picture (or slice or strip or other type of category, the picture boundary is taken as an example) boundary, K×L samples are within the picture boundary, where K < M or L < N. For those CTBs shown in Figures 7A-7C the CTB size is still equal to M×N, however, the bottom boundary / right boundary of the CTB is outside the picture.

[0112] Figure 7 shows examples of CTBs across the picture boundary, (a) K = M, L < N; (b) K < M, L = N; (c) K < M, L < N. And, (a) CTB across the bottom picture boundary, (b) CTB across the right picture boundary, (c) CTB across the right-bottom picture boundary.

[0113] 2.4. Deblocking Filter (DB)

[0114] The input to the DB is the reconstructed samples before loop filtering.

[0115] The vertical edges in the picture are filtered first. Then the horizontal edges in the picture are filtered with the samples modified by the vertical edge filtering process as input. The vertical and horizontal edges in the CTB of each CTU are processed separately based on the coding / decoding unit. The vertical edges of the coding / decoding blocks in the coding / decoding unit are filtered, starting from the edge on the left-hand side of the coding / decoding block and continuing through the edges in their geometric order towards the right-hand side of the coding / decoding block. The horizontal edges of the coding / decoding blocks in the coding / decoding unit are filtered, starting from the edge on the top of the coding / decoding block and continuing through the edges in their geometric order towards the bottom of the coding / decoding block.

[0116] Figure 8It is a diagram of image samples and horizontal and vertical block boundaries on an 8×8 grid, as well as non-overlapping blocks of 8×8 samples, which can be de-blocked in parallel.

[0117] 2.4.1. Boundary Decision

[0118] The filter is applied to the 8×8 block boundaries. Furthermore, it must be a transform block boundary or an encoder / decoder sub-block boundary (e.g., ATMVP due to the use of affine motion prediction). For those boundaries that are not such boundaries, the filter is disabled.

[0119] 2.4.2. Boundary Strength Calculation

[0120] For transform block boundaries / encoder / decoder sub-block boundaries, if they are located in an 8×8 grid, they can be filtered, and the settings for bS[xDi][yDj] (where [xDi][yDj] refers to coordinates) of this edge are defined in Tables 2-2 and 2-3, respectively.

[0121] Table 2-2. Boundary Strength (when SPS IBC is disabled)

[0122]

[0123] Table 2-3. Boundary Strength (when SPS IBC is enabled)

[0124]

[0125] 2.4.3. Deblocking decision for the luminance component

[0126] The decision-making process for removing blocks is described in this subsection.

[0127] A wider, stronger luminance filter is used only when all conditions 1, 2, and 3 are true (TRUE).

[0128] Condition 1 is the "large block condition". This condition detects whether the samples on the P-side and Q-side belong to a large block, which are represented by the variables bSidePisLargeBlk and bSideQisLargeBlk, respectively. bSidePisLargeBlk and bSideQisLargeBlk are defined as follows.

[0129] bSidePisLargeBlk = ((edge ​​type is vertical and p0 belongs to CU with width >= 32) || (edge ​​type is horizontal and p0 belongs to CU with height >= 32)) ? TRUE: FALSE

[0130] bSideQisLargeBlk = ((edge ​​type is vertical and q0 belongs to CU with width >= 32) || (edge ​​type is horizontal and q0 belongs to CU with height >= 32)) ? TRUE: FALSE

[0131] Based on bSidePisLargeBlk and bSideQisLargeBlk, condition 1 is defined as follows.

[0132] Condition1=(bSidePisLargeBlk||bSidePisLargeBlk)? TRUE:FALSE

[0133] Next, if Condition1 is true, Condition2 will be further checked. First, derive the following variables:

[0134] –dp0, dp3, dq0, dq3 are derived first, as in HEVC.

[0135] –if (p side is greater than or equal to 32)

[0136] dp0=(dp0+Abs(p50-2*p40+p30)+1)>>1

[0137] dp3=(dp3+Abs(p53-2*p43+p33)+1)>>1

[0138] –if (q side is greater than or equal to 32)

[0139] dq0=(dq0+Abs(q50-2*q40+q30)+1)>>1

[0140] dq3=(dq3+Abs(q53-2*q43+q33)+1)>>1

[0141] Condition2=(d<β)? TRUE:FALSE

[0142] Where d = dp0 + dq0 + dp3 + dq3.

[0143] If conditions 1 and 2 are valid, whether any block uses sub-blocks is further examined:

[0144]

[0145]

[0146] Finally, if both conditions 1 and 2 are valid, the proposed deblocking method will check condition 3 (large block strong filtering condition), which is defined as follows.

[0147] In condition 3 StrongFilterCondition, derive the following variables:

[0148] dpq is derived, as in HEVC.

[0149] sp3 = Abs(p3 - p0) is derived, as in HEVC.

[0150] if (p side is greater than or equal to 32)

[0151] if (Sp == 5)

[0152] sp3=(sp3+Abs(p5-p3)+1)>>1

[0153] else

[0154] sp3=(sp3+Abs(p7-p3)+1)>>1

[0155] The expression sq3 = Abs(q0 - q3) is derived, as shown in HEVC.

[0156] if (q side is greater than or equal to 32)

[0157] If (Sq == 5)

[0158] sq3=(sq3+Abs(q5-q3)+1)>>1

[0159] else

[0160] sq3=(sq3+Abs(q7-q3)+1)>>1

[0161] In HEVC, StrongFilterCondition = (dpq < (β >> 2), sp3 + sq3 < (3 * β >> 5), and Abs(p0 - q0) < (5 * t). C +1)>>1)? TRUE:FALSE.

[0162] 2.4.4. Stronger deblocking filter for brightness (designed for larger blocks)

[0163] Bilinear filtering is used when samples on either side of the boundary belong to a large block. Samples belonging to a large block are defined as having a vertical edge width >= 32 and a horizontal edge height >= 32.

[0164] Bilinear filtering is listed below.

[0165] Block boundary samples for i = 0 to Sp-1 p i and q for j = 0 to Sq-1 i(In the above HEVC deblocking, pi and qi are the i-th sample point in the row of the filtered vertical edge, or the i-th sample point in the column of the filtered horizontal edge) are replaced by linear interpolation as follows:

[0166] —p i ′=(f i *Middle s,t +(64-f i )*P s +32)>>6), trim to p i ±tcPD i

[0167] —q j ′=(g j *Middle s,t +(64-g j )*Q s +32)>>6), trim to q j ±tcPD j

[0168] tcPD i and tcPD j The terminology is the position-related clipping described in section 2.4.7, and g j f i Middle s,t P s and Q s The following is given.

[0169] 2.4.5. Color deblocking control

[0170] Strong chroma filtering is applied to both sides of the block boundary. Here, the chroma filter is selected when both sides of the chroma edge are greater than or equal to 8 (chroma position), and the following decisions with three conditions are met: the first is the boundary strength and the block size decision. The proposed filter can be applied when the block width or height orthogonally across the block edge is equal to or greater than 8 in the chroma sample domain. The second and third are essentially the same as the HEVC luminance deblocking decision, which are the on / off decision and the strong filtering decision, respectively.

[0171] In the first decision, the boundary strength (bS) is modified for chroma filtering and the conditions are checked sequentially. If a condition is met, the remaining conditions with lower priority are skipped.

[0172] Chromatic deblocking is performed when bS equals 2, or when bS equals 1 when a large block boundary is detected.

[0173] The second and third conditions are essentially the same as the following HEVC luminance strong filtering decision.

[0174] In the second condition:

[0175] d is derived, such as the brightness deblocking in HEVC.

[0176] The second condition will be TRUE when d is less than β.

[0177] In the third condition, StrongFilterCondition is derived as follows:

[0178] dpq is derived, as in HEVC.

[0179] sp3 = Abs(p3 - p0) is derived, as in HEVC.

[0180] The expression sq3 = Abs(q0 - q3) is derived, as shown in HEVC.

[0181] In HEVC design, StrongFilterCondition = (dpq < (β >> 2), sp3 + sq3 < (β >> 3), and Abs(p0 - q0) < (5 * t) C +1)>>1)

[0182] 2.4.6. Strong deblocking filtering of chroma

[0183] The following strong deblocking filter for chroma is defined as:

[0184] p2′=(3*p3+2*p2+p1+p0+q0+4)>>3

[0185] p1′=(2*p3+p2+2*p1+p0+q0+q1+4)>>3

[0186] p0′=(p3+p2+p1+2*p0+q0+q1+q2+4)>>3

[0187] The proposed chromaticity filtering performs deblocking on a 4×4 chromaticity sample grid.

[0188] 2.4.7. Position-related clipping

[0189] Position-dependent clipping (tcPD) is applied to the output samples of the luminance filtering process involving strong and long filters at the 7, 5, and 3 samples at the modification boundaries. Assuming the quantization error distribution, increasing the clipping value of the samples is proposed, which is expected to have higher quantization noise, and therefore is expected to have a higher deviation between the reconstructed sample values ​​and the true sample values.

[0190] For each P or Q boundary filtered by an asymmetric filter, based on the results of the decision-making process in Section 2.4.2, a table of location-related thresholds is selected from two tables (i.e., Tc7 and Tc3, listed below), which are provided to the decoder as edge information:

[0191] Tc7={6,5,4,3,2,1,1}; Tc3={6,4,2};

[0192] tcPD=(Sp==3)? Tc3:Tc7;

[0193] tcQD=(Sq==3)? Tc3:Tc7;

[0194] For P or Q boundaries filtered using a short symmetric filter, apply a low-amplitude position-dependent threshold:

[0195] Tc3 = {3, 2, 1};

[0196] After defining the threshold, the filter's p' i and q' i Sample values ​​are clipped based on tcP and tcQ clipping values:

[0197] p” i =Clip3(p' i +tcP i ,p' i –tcP i ,p' i );

[0198] q” j =Clip3(q' j +tcQ j ,q' j –tcQ j ,q' j );

[0199] Where p' i and q' i These are the filtered sample values, p” i and q" j It refers to the output sample values ​​after cropping, and tcP i tcP i From VVC tc parameters as well as tcPD and tcQ D The derived clipping threshold. The function Clip3 is the clipping function, as specified in VVC.

[0200] 2.4.8. Sub-block Removal and Adjustment

[0201] To enable parallel-friendly deblocking using long filters and sub-block deblocking, the long filter is limited to modifying a maximum of 5 samples (AFFINE, ATMVP, or DMVR) on the side using sub-block deblocking, as shown in the long filter's brightness control. Furthermore, sub-block deblocking is adjusted such that sub-block boundaries on the 8×8 grid near the CU or implicit TU boundaries are limited to modifying a maximum of two samples on each side.

[0202] The following applies to sub-block boundaries that are not aligned with the CU boundary.

[0203]

[0204] An edge value of 0 corresponds to the CU boundary, while an edge value of 2 or orthogonalLength-2 corresponds to 8 sample points from the sub-block boundary of the CU boundary. If implicit partitioning using TU is used, then implicit TU is true.

[0205] 2.5. Sample Adaptive Migration (SAO)

[0206] The input to SAO is the reconstructed samples after DB. The concept of SAO is to reduce the average sample distortion of a region by first classifying the region samples into multiple categories with selected classifiers, obtaining the offset for each category, and then adding the offset to each sample within each category. The classifier index and the region's offset are encoded in the bitstream. In HEVC and VVC, the region (the unit for SAO parameter signaling notification) is defined as a CTU.

[0207] Two SAO types that meet the low complexity requirements are used in HEVC. These two types are Edge Offset (EO) and Band Offset (BO), which are discussed in more detail below. The index of the SAO type is encoded and decoded (it is in the range [0, 2]. For EO, ​​the sample classification is based on a comparison between the current sample and its neighboring samples according to a 1-D directional pattern (horizontal, vertical, 135° diagonal, and 45° diagonal).

[0208] Figure 10 A-10D shows four 1-D orientation patterns for EO sample point classification: horizontal (EO classification = 0), vertical (EO classification = 1), 135° diagonal (EO classification = 2), and 45° diagonal (EO classification = 3).

[0209] For a given EO category, each sample point within the CTB is classified into one of five categories. The current sample point value (labeled "c") is compared along the selected 1-D pattern with its two adjacent sample point values ​​(labeled "a" and "b"). The classification rules for each sample point are summarized in Tables 2-4. Categories 1 and 4 are associated with local valleys and local peaks along the selected 1-D pattern, respectively. Categories 2 and 3 are associated with concave and convex corners along the selected 1-D pattern, respectively. If the current sample point does not belong to EO categories 1–4, it is category 0 and SAO is not applied.

[0210] Table 2-4. Sampling classification rules for edge offset

[0211] category condition 1 c < a and c < b 2 (c<a&&c==b)||(c==a&&c<b) 3 (c>a&&c==b)||(c==a&&c>b) 4 c>a&&c>b 5 None of the above

[0212] 2.6. Adaptive Loop Filtering (ALF)

[0213] In VVC, an adaptive loop filter (ALF) with block-based filter adaptation is applied. For the luminance component, one of 25 filters is selected for each 4×4 block based on the direction and activity of the local gradient.

[0214] 2.6.1. Filter Shape

[0215] Use two diamond filter shapes (such as...) Figure 11 (As described in the text). A 7×7 rhombus shape is applied to the luminance component, and a 5×5 rhombus shape is applied to the chrominance component.

[0216] Figure 11 The shape of the ALF filter is shown (chroma: 5×5 rhombus, luminance: 7×7 rhombus).

[0217] 2.6.2. Block Classification

[0218] For the luminance component, each 4×4 block is classified into one of 25 categories, based on its directionality (D) and activity. The quantization value is used to derive the classification index C as follows:

[0219]

[0220] To calculate D and The gradients in the horizontal, vertical, and two diagonal directions are first calculated using the 1-D Laplacian operator:

[0221]

[0222]

[0223]

[0224]

[0225] The indices i and j refer to the coordinates of the top left sample point within the 4×4 block, and R(i,j) indicates the reconstructed sample point at coordinates (i,j).

[0226] To reduce the complexity of block classification, a 1-D Laplace operator for subsampling is applied for computation. Figure 12 illustrates the Laplace computation for subsampling. Figure 12A The location of the subsamples of the vertical gradient is shown. Figure 12B The locations of subsamples of the horizontal gradient are shown. Figure 12C The locations of subsamples of the diagonal gradient are shown, and Figure 12D The location of the subsamples of the diagonal gradient is shown.

[0227] The maximum and minimum values ​​of the gradient D in the horizontal and vertical directions are then set as follows:

[0228]

[0229] The maximum and minimum values ​​of the gradients in the two diagonal directions are set as follows:

[0230]

[0231] To derive the values ​​of directionality D, these values ​​are compared to each other and to two thresholds t1 and t2:

[0232] Step 1. If and If true, then D is set to 0.

[0233] Step 2. If If so, continue from step 3; otherwise, continue from step 4.

[0234] Step 3. If Then D is set to 2; otherwise, D is set to 1.

[0235] Step 4. If Then D is set to 4; otherwise, D is set to 3.

[0236] The activity value A is calculated as:

[0237]

[0238] A is further quantized to the range of 0 to 4 (inclusive), and the quantized value is designated as...

[0239] For the chromaticity components in the image, no classification method is applied; that is, a single set of ALF coefficients is applied to each chromaticity component.

[0240] 2.6.3. Geometric Transformation of Filter Coefficients and Clipping Values

[0241] Before filtering each 4×4 luma block, geometric transformations (such as rotation or diagonal and vertical flips) are applied to the filter coefficients f(k, l) and the corresponding filter clipping value c(k, l), which depends on the gradient value calculated for that block. This is equivalent to applying these transformations to samples in the filter support region. The idea is to make the different blocks to which ALF is applied more similar by aligning their orientations. Three geometric transformations are introduced, including diagonal, vertical flip, and rotation:

[0242] Diagonal: f D (k, l) = f(l, k), c D (k, l) = c(l, k),

[0243] (2-9)

[0244] Vertical flip: f V (k, l) = f(k, Kl-1), c V (k, l) = c(k, Kl-1)

[0245] (2-10)

[0246] Rotation: f R (k, l) = f(Kl-1, k), c R (k, l) = c(Kl-1, k)

[0247] (2-11)

[0248] Where K is the size of the filter, and 0 ≤ k, l ≤ K⁻¹ are the coefficient coordinates, such that position (0, 0) is located in the upper left corner, and position (K⁻¹, K⁻¹) is located in the lower right corner. The transform is applied to the filter coefficients f(k, l) and to the clipping value c(k, l), which depends on the gradient value calculated for that block. The relationship between the transform and the four gradients in the four directions is summarized in the following table.

[0249] Table 2-5. Gradient and transformation mapping for a block's computation.

[0250] gradient value Transformation <![CDATA[g d2 <g d1 And g h <g v ]]> No transformation <![CDATA[g d2 <g d1 And g v <g h ]]> diagonal <![CDATA[g d1 <g d2 And g h <g v ]]> Vertical flip <![CDATA[g d1 <g d2 And g v <g h ]]> Rotation

[0251] 2.6.4. Filter Parameter Signaling Notification

[0252] ALF filtering parameters are signaled in the Adaptive Parameter Set (APS). Within an APS, up to 25 sets of luma filter coefficients and clipping value indices, and up to eight sets of chroma filter coefficients and clipping value indices can be signaled. To reduce bit overhead, filter coefficients for different categories of the luma component can be merged. In the strip header, the index of the APS used for the current strip is signaled.

[0253] The clipping value index decoded from the APS allows a table of clipping values ​​to determine the clipping value for both the luminance and chrominance components. These clipping values ​​depend on the internal bit depth. More precisely, the clipping values ​​are obtained by the following formula:

[0254] AlfClip=(round(2 B-a*n for n∈[0..N-1]}

[0255] (2-12)

[0256] Where B equals the internal bit depth, a is a predefined constant value equal to 2.35, and N equals 4, which is the number of clipped values ​​allowed in VVC.

[0257] In the stripe header, up to seven APS indices can be signaled to specify the luma filter banks used for the current stripe. The filtering process can be further controlled at the CTB level. Flags are always signaled to indicate whether the ALF is applied to the luma CTB. The luma CTB can select from 16 fixed filter banks and filter banks from the APS. Filter bank indices are signaled for the luma CTB to indicate which filter bank is applied. The 16 fixed filter banks are predefined and hard-coded in the encoder and decoder.

[0258] For chroma components, the APS index is signaled in the stripe header to indicate the chroma filter group being used for the current stripe. At the CTB level, if there is more than one chroma filter group in the APS, the filter index is signaled for each chroma CTB.

[0259] The filter coefficients are quantized, where the norm is equal to 128. To limit the multiplication complexity, bitstream compliance is applied, ensuring that coefficients outside the center position should have values ​​in the range of -2. 7 Up to 2 7 The range is -1 (inclusive). The central position coefficient is not signaled in the bitstream and is considered equal to 128.

[0260] 2.6.5. Filtering Process

[0261] On the decoder side, when ALF is enabled for CTB, each sample R(i,j) within the CU is filtered to obtain the sample value R′(i,j), as shown below.

[0262] R′(i,j)=R(i,j)+((∑ k≠0 ∑ l≠0 f(k,l)×K(R(i+k,j+l)-R(i,j),c(k,l))+64)>>7) (2-13)

[0263] Where f(k, l) represents the filtering coefficients for decoding, K(x, y) is the pruning function, and c(k, l) represents the pruning parameters for decoding. Variables k and l in... and The values ​​vary between L and L, where L represents the filter length. The clipping function K(x,y) = min(y,max(-y,x)), which corresponds to the function Clip3(-y,y,x).

[0264] 2.6.6. Virtual Boundary Filtering Process with Reduced Line Buffers

[0265] In both hardware and embedded software, image-based processing is practically unacceptable due to its high image buffer requirements. Using on-chip image buffers is very expensive, and using off-chip image buffers significantly increases external memory access, power consumption, and data access latency. Therefore, DF, SAO, and ALF will shift from image-based to LCU-based decoding in real-world products. When LCU-based processing is used in DF, SAO, and ALF, the entire decoding process can be performed in parallel on multiple LCUs in a pipelined manner during raster scanning, one LCU at a time. In this case, DF, SAO, and ALF require line buffers because processing one LCU row requires pixels from the LCU rows above. If off-chip line buffers (e.g., DRAM) are used, external memory bandwidth and power consumption will increase; if on-chip line buffers (e.g., SRAM) are used, chip area will increase. Therefore, although line buffers are already much smaller than image buffers, reducing line buffers is still desirable.

[0266] In VTM-4.0, such as Figure 13As described, the total number of line buffers required for the luminance component is 11.25 lines. The line buffer requirement is explained as follows: Horizontal edges where deblocking overlaps with CTU edges cannot be performed because decision-making and filtering require lines K, L, M, M from the first CTU and lines O, P from the bottom CTU. Therefore, horizontal edges where deblocking overlaps with CTU boundaries are postponed until the lower CTU arrives. Thus, for lines K, L, M, N, the reconstructed luminance samples must be stored in the line buffer (4 lines). SAO filtering can then be performed on lines A through J. Line J can be SAO filtered because deblocking does not change the samples in line K. For SAO filtering of line K, the edge offset classification decision is only stored in the line buffer (which is 0.25 luminance lines). ALF filtering can be performed only on line AF. Figure 13 As shown, ALF classification is performed on each 4×4 block. Each 4×4 block classification requires an 8×8 active window, which in turn requires a 9×9 window to compute the 1d Laplacian operator to determine the gradient.

[0267] Therefore, for block classification of 4×4 blocks overlapping with lines G, H, I, and J, SAO-filtered samples below the virtual boundary are required. Furthermore, for ALF classification, SAO-filtered samples of lines D, E, and F are required. Additionally, ALF filtering of line G requires SAO-filtered samples of lines D, E, and F from the lines above. Therefore, the total line buffer requirements are as follows:

[0268] – Line KN (Horizontal DF Pixel): 4 lines

[0269] – Line DJ (pixels filtered by SAO): 7 lines

[0270] – SAO edge offset classifier value between lines J and K: 0.25 lines

[0271] Therefore, the total number of brightness lines required is 7 + 4 + 0.25 = 11.25.

[0272] Similarly, the line buffer for the chroma component requires... Figure 14 As shown in the figure, the line buffer requirement for the chroma component is to be evaluated as 6.25 lines.

[0273] Figure 13 The loop filter line buffer requirements in VTM-4.0 for the luminance component are shown.

[0274] Figure 14 The loop filter line buffer requirements for the chromaticity component in VTM-4.0 are shown.

[0275] To eliminate the line buffer requirements of SAO and ALF, the concept of Virtual Boundary (VB) is introduced to reduce the line buffer requirements of ALF in the latest VVC. Modified block classification and filtering are used for samples close to the horizontal CTU boundary. Figure 13 As shown, the VB is shifted upwards by N pixels from the horizontal LCU boundary. For each LCU, SAO and ALF can process pixels above the VB before the lower LCU arrives, but cannot process pixels below the VB until the lower LCU arrives, due to DF. Considering the cost of hardware implementation, the proposed space between the VB and the horizontal LCU boundary is set to four pixels for the luma component (i.e., ...). Figure 13 or Figure 15 In the N=4) and for the chroma component it is set to two pixels (i.e. N=2).

[0276] Figure 15 The modified block classification at the virtual boundary is shown.

[0277] The modified block classification was applied to the luminance component, such as Figure 16 As shown in the diagram. For the 1D Laplacian operator gradient computation of a 4×4 block above the virtual boundary, only samples above the virtual boundary are used. Similarly, for the 1D Laplacian operator gradient computation of a 4×4 block below the virtual boundary, only samples below the virtual boundary are used. The quantization of the activity value A is scaled accordingly to account for the reduced number of samples used in the 1D Laplacian operator gradient computation.

[0278] For the filtering process, the mirror (symmetric) fill operation at the virtual boundary is used for both the luminance and chrominance components. For example... Figure 16 As shown, when the sample being filtered is below the virtual boundary, the adjacent sample above the virtual boundary is filled. At the same time, the corresponding sample on the other side is also filled symmetrically.

[0279] Figure 16 The modified ALF filter for the luminance component at the virtual boundary is shown.

[0280] For another example, if a sample point located at (i, j) (e.g., Figure 17B If the dashed line P0A in the middle is filled, then the position in (m, n) (for example, Figure 17B Samples corresponding to P3B (with dashed lines) that share the same filter coefficients are also filled in, even if the samples are available. Figures 17A-17C As shown in the image.

[0281] Figure 17A This shows one required line (on each side) that needs to be filled above / below VB.

[0282] Figure 17BThe diagram shows the two required lines (on each side) that need to be filled above / below VB.

[0283] Figure 17C The diagram shows the three required lines (on each side) that need to be filled above / below VB.

[0284] Figure 27 An example of modified luminance ALF filtering at virtual boundaries is shown.

[0285] Unlike the mirror (symmetric) fill method used at horizontal CTU boundaries, a repeated (one-sided) fill process is applied to strip, patch, and sub-image boundaries when cross-boundary filters are disabled. The repeated (one-sided) fill process is also applied at image boundaries. The filled samples are used in both the classification and filtering processes. Figure 18 An example of a repeating padding method for luminance ALF filtering at the boundaries of an image / sub-image / strip / piece is shown.

[0286] Figure 18 An example of repeated padding of luminance ALF filtering at the boundaries of an image / sub-image / strip / piece is shown.

[0287] 2.7.360-degree video encoding

[0288] Horizontal roll motion compensation in VTM5 is a 360-specific codec tool designed to improve the visual quality of reconstructed 360-degree video in equirectangular projection (ERP) formats. In conventional motion compensation, when a motion vector refers to a sample point beyond the boundary of a reference image, a repetition padding is applied to derive the value of the sample point outside the boundary by copying the nearest neighboring image from the corresponding image boundary. This repetition padding method is unsuitable for 360-degree video and can result in visual artifacts known as "seam artifacts" in the reconstructed viewport video. Because 360-degree video is captured on a sphere and inherently has no "boundaries," reference samples outside the boundary of the reference image in the projection domain can always be obtained from neighboring samples in the spherical domain. For general projection formats, deriving the corresponding neighboring samples in the spherical domain can be difficult because it involves 2D-to-3D and 3D-to-2D coordinate transformations, as well as sample interpolation of fractional sample positions. This problem is simpler for the left and right boundaries of the ERP projection format because the sphere neighborhood outside the left image boundary can be obtained from the sample points inside the right image boundary, and vice versa.

[0289] Figure 19 An example of horizontal winding motion compensation in VVC is shown.

[0290] The horizontal winding motion compensation process is in Figure 19As shown in the diagram. When a portion of the reference block is outside the left (or right) boundary of the reference image within the projection domain, no repetition padding is applied; the portion "outside the boundary" is taken from the corresponding sphere neighborhood within the reference image facing the right (or left) boundary within the projection domain. Repetition padding is only applied to the top and bottom image boundaries. For example, in... Figure 19 As shown, horizontal roll motion compensation can be combined with the non-canonical padding method frequently used in 360-degree video encoding and decoding. In VVC, this is achieved by signaling higher-level syntax elements to indicate the roll offset, which should be set to the ERP picture width before padding; this syntax is then used to adjust the position of the horizontal roll accordingly. This syntax is unaffected by the specific amount of padding on the left and right picture boundaries, and therefore naturally supports asymmetric padding of the ERP picture, i.e., when the left and right paddings are different. Horizontal roll motion compensation provides more meaningful information for motion compensation when the reference sample is outside the left and right boundaries of the reference picture.

[0291] For projection formats consisting of multiple faces, regardless of the compact frame packing arrangement used, discontinuities occur between two or more adjacent faces in the frame-packed image. For example, consider in Figure 20 The 3×2 frame packing configuration shown has three faces in the upper half that are continuous in 3D geometry, and three faces in the lower half that are continuous in 3D geometry. However, the upper and lower halves of the frame packing image are discontinuous in 3D geometry. If a loop filtering operation is performed across this discontinuity, face seam artifacts may become visible in the reconstructed video.

[0292] To mitigate face seam artifacts, loop filtering across discontinuities can be disabled in frame-packed images. A proposed syntax signals vertical and / or horizontal virtual boundaries, across which loop filtering is disabled. Compared to using two slices, one for each group of consecutive faces, and disabling loop filtering across slices, the proposed signaling notification method is more flexible because it does not require face sizes to be multiple CTU dimensions.

[0293] Figure 20 An image of the HEC in a 3×2 layout is shown.

[0294] 2.8. JVET-P0080: CE5-2.1, CE5-2.2: Transcomponent Adaptive Loop Filtering

[0295] Figure 21A The arrangement of CC-ALF[1] with respect to other loop filters is shown. CC-ALF uses linear, diamond-shaped filters. Figure 21B The operation is performed on the luminance channel of each chromaticity component, and is expressed as follows:

[0296]

[0297] in

[0298] (x, y) is the position of the i-th chromaticity component after positive refinement.

[0299] (x C ,y C () is the brightness position based on (x,y).

[0300] S i It supports filtering of chrominance component i in luminance.

[0301] c i (x0, y0) represents the filter coefficients.

[0302] (2-14)

[0303] Figure 21A An example arrangement of CC-ALF with respect to other loop filters is shown. Figure 21B A diamond-shaped filter is shown.

[0304] Supported area around its centered brightness position (x C ,y C The CC-ALF filter coefficients are calculated based on the spatial scaling factor between the luma and chroma planes. All filter coefficients are transmitted in the APS and have an 8-bit dynamic range. The APS can be referenced in the strip header. The CC-ALF coefficients for each chroma component of the strip are also stored in a buffer corresponding to the temporal sublayer. The reuse of these groups of temporal sublayer filter coefficients facilitates the use of strip-level flags. The application of the CC-ALF filter is controlled on the variable block size (i.e., 16×16, 32×32, 64×64, 128×128) and is notified by context-coded flag signaling received for each block of samples. The block size, along with the CC-ALF enable flag, is received at the strip level for each chroma component. Boundary padding for horizontal virtual boundaries utilizes repetition. For the remaining boundaries, the same type of padding is used, similar to regular ALF.

[0305] 2.8.1. CC-ALF Specification in JVET-P0080

[0306] Cross-component filtering process for xxxx chroma samples

[0307] The input to this process is:

[0308] –Reconstructing the luminance image sample array recPicture L Before the brightness adaptive loop filtering process,

[0309] – Filtered reconstruction of chroma image sample array alfPicture C ,

[0310] – Chroma position (xC, yC), specifies the left top-left chroma point of the current block relative to the left top-left chroma point of the current image.

[0311] – Width of the chroma sample block ccAlfWidth

[0312] – Height of the chromaticity sample block ccAlfHeight

[0313] – Cross-component filter coefficients CcAlfCoeff[j], where j = 0..13

[0314] The output of this process is a modified filtered reconstructed chroma image sample array ccAlfPicture.

[0315] The derivation of the brightness position (xCtb, yCtb) of the codec tree block is as follows:

[0316] xCtb=(((xC*SubWidthC)>>CtbLog2SizeY)< <CtbLog2SizeY (8-1229)

[0317] yCtb=(((yC*SubHeightC)>>CtbLog2SizeY)< <CtbLog2SizeY (8-1229)

[0318] For the derivation of the reconstructed chromaticity sample ccAlfPicture[xC+x][yC+y], the sample alfPicture C Each reconstructed chroma sample within the current chroma block [xC+x][yC+y], where x = 0..ccAlfWidth-1 and y = 0..ccAlfHeight-1, is filtered as follows:

[0319] – The brightness position (xL, yL) of the current chromaticity sample point at the chromaticity position (xC+x, yC+y) is set to equal to ((xC+x)*SubWidthC, (yC+y)*SubHeightC).

[0320] – array recPicture L Brightness position within (h) xL+i v yL+j ), where i = -2..2, j = -2..3,

[0321] The derivation is as follows:

[0322] – If pps_loop_filter_across_virtual_boundaries_disabled_flag equals 1, and PpsVirtualBoundariesPosX[n]%CtbSizeY is not equal to 0, and for any n = 0..pps_num_ver_virtual_boundaries–1, xL-PpsVirtualBoundariesPosX[n] is greater than or equal to 0 and less than 3, the following applies:

[0323] h xL+i =Clip3(PpsVirtualBoundariesPosX[n],pic_width_in_luma_samples-1,xL+i) (8-1229)

[0324] – Otherwise, if pps_loop_filter_across_virtual_boundaries_disabled_flag equals 1, and PpsVirtualBoundariesPosX[n]%CtbSizeY is not equal to 0, and for any n = 0..pps_num_ver_virtual_boundaries–1, PpsVirtualBoundariesPosX[n]-xL is greater than 0 and less than 4, the following applies:

[0325] h x+i =Clip3(0,PpsVirtualBoundariesPosX[n]-1,xL+i) (8-1230)

[0326] –Otherwise, the following applies:

[0327] h x+i =Clip3(0,pic_width_in_luma_samples-1,xL+i) (8-1231)

[0328] – If pps_loop_filter_across_virtual_boundaries_disabled_flag equals 1, and PpsVirtualBoundariesPosY[n]%CtbSizeY is not equal to 0, and for any n = 0..pps_num_hor_virtual_boundaries–1, yL - PpsVirtualBoundariesPosY[n] is greater than or equal to 0 and less than 3, the following applies:

[0329] vy+j =Clip3(PpsVirtualBoundariesPosY[n], pic_height_in_luma_samples-1, yL+j) (8-1232)

[0330] – Otherwise, if pps_loop_filter_across_virtual_boundaries_disabled_flag equals 1, and PpsVirtualBoundariesPosY[n]%CtbSizeY is not equal to 0, and for any n = 0..pps_num_hor_virtual_boundaries–1, PpsVirtualBoundariesPosY[n]-yL is greater than 0 and less than 4, the following applies:

[0331] v y+j =Clip3(0,PpsVirtualBoundariesPosY[n]-1,yL+j) (8-1233)

[0332] –Otherwise, the following applies:

[0333] v y+j =Clip3(0,pic_height_in_luma_samples-1,yL+j) (8-1234)

[0334] – The variables clipLeftPos, clipRightPos, clipTopPos, and clipBottomPos are derived by calling the ALF boundary position derivation procedure as specified in Clause 8.8.5.5, with (xCtb, yCtb) and (xL–xCtb, yL–yCtb) as input.

[0335] – The vertical sample position offsets yM2, yM1, yP1, yP2, and yP3 are specified in Table 2-6 based on the vertical luminance sample positions yL, clipLeftPos, and clipRightPos.

[0336] – The horizontal sample position offsets xM1, xM2, xP1, and xP2 are specified in Table 2-7 based on the horizontal brightness sample positions xL, clipLeftPos, and clipRightPos.

[0337] – The derivation of variable curr is as follows:

[0338] curr = alfPicture C [xC+x,yC+y] (8-1286)

[0339] – The array of cross-component filter coefficients f[j] is derived as follows, where j = 0..13:

[0340] f[j]=CcAlfCoeff[j] (8-1287)

[0341] – The derivation of variable sum is as follows:

[0342]

[0343]

[0344] sum=curr+(sum+64)>>7) (8-1290)

[0345] – The derivation of the modified filtered reconstructed chroma image sample array ccAlfPicture[xC+x][yC+y] is as follows:

[0346] ccAlfPicture[xC+x][yC+y]=Clip3(0, (1< <BitDepth C (8-1291)

[0347] Table 2-6. Specifications of yM1, yM2, yP1, yP2, and yP3 based on the vertical brightness sample point positions yL, clipTopPos, and clipBottomPos

[0348] condition yM2 yM1 yP1 yP2 yP3 yL==clipTopPos+1 -1 -1 1 2 3 yL==clipTopPos 0 0 1 2 3 yL==clipBottomPos-1 -2 -1 0 0 0 yL==clipBottomPos-2 -2 -1 1 1 1 yL==clipBottomPos-3 -2 -1 1 2 2 otherwise -2 -1 1 2 3

[0349] Table 2-7. Specifications of xM1, xM2, xP1, and xP2 based on the horizontal luminance sample point positions xL, clipLeftPos, and clipRightPos

[0350] condition xM2 xM1 xP1 xP2 xL==clipLeftPos+1 –1 –1 1 2 xL==clipLeftPos 0 0 1 2 xL==clipRightPos-1 –2 –1 0 0 xL==clipRightPos–2 –2 –1 1 1 otherwise –2 –1 1 2

[0351] 2.8.2. Filling method at virtual boundaries in JVET-P0080

[0352] Similar to the luminance ALF / chrominance ALF, the CC-ALF in JVET-P0080 uses repeated padding at the virtual boundaries of the ALF. For example... Figure 22 As shown, if brightness samples above or below the ALF virtual boundary are unavailable, the line closest to the sample is used for filling. Detailed filling methods are also shown in Table 2-6.

[0353] 2.9. JVET-P1008: CE5 Related: Design of CC-ALF

[0354] In JVET-O0636[1] and CE5-2.1[2], cross-component adaptive loop filtering (CC-ALF) was introduced and studied. The filter uses a linear filter to filter the luminance sample values ​​and generates residual corrections for the chrominance channel from the output of the co-located filter. The filter is designed to operate in parallel with the existing luminance ALF.

[0355] A CC-ALF design is proposed, which simplifies and better aligns with existing ALFs. This design uses a 3×4 rhombus shape with 8 unique coefficients. This reduces the number of multiplications by 43% compared to the 5×6 design studied in CE5-2.1. When the limits for chroma ALF or CC-ALF are enabled for the chroma components of the CTU, the multiplier count per pixel is limited to 16 (compared to 15 for the current ALF). The dynamic range of the filter coefficients is limited to 6 bits (signed). Illustrations of the proposed filter and the filter from the CE5-2.1 scheme are shown in [the provided text]. Figure 23 middle.

[0356] To better align with existing ALF designs, the filter coefficients are signaled in the APS. Up to four filters are supported, and filter selection is indicated at the CTU level. Symmetry line selection is used at virtual boundaries for further harmonization with the ALF. Finally, to limit the amount of storage required for the corrected output, the CC-ALF residual output is clipped to -2. BitDepthC-1 Up to 2 BitDepthC-1 -1 (including the end value).

[0357] The CC-ALF specification in JVET-P1008.

[0358] Cross-component filtering process for xxxx chroma samples

[0359] The input to this process is:

[0360] – Reconstructed brightness image sample array recPicture L Before the brightness adaptive loop filtering process,

[0361] – Filtered reconstruction of chroma image sample array alfPicture C ,

[0362] – Chroma position (xCtbC, yCtbC), specifies the top-left sample of the current chroma codec block relative to the left-left sample of the current image.

[0363] – Width of the chroma sample block ccAlfWidth

[0364] – Height of the chromaticity sample block ccAlfHeight

[0365] – Cross-component filter coefficients CcAlfCoeff[j], where j = 0.7

[0366] The output of this process is a modified filtered, reconstructed chroma image sample array ccAlfPicture.

[0367] The luminance positions (xCtb, yCtb) of the codec tree block are derived as follows:

[0368] xCtb=(((xCtbC*SubWidthC)>>CtbLog2SizeY)< <CtbLog2SizeY (8-1229)

[0369] yCtb=(((yCtbC*SubHeightC)>>CtbLog2SizeY)< <CtbLog2SizeY (8-1229)

[0370] The derivation of the reconstructed chromaticity sample ccAlfPicture[xCtbC+x][yCtbC+y] for filtering involves each reconstructed chromaticity sample within the current chromaticity block of sample alfPictureC[xCtbC+x][yCtbC+y], where x = 0..ccAlfWidth-1 and y = 0..ccAlfHeight-1, being filtered as follows:

[0371] – The brightness position (xL, yL) of the current chromaticity sample point at the chromaticity position (xCtbC+x, yCtbC+y) is set to equal to ((xCtbC+x)*SubWidthC, (yCtbC+y)*SubHeightC).

[0372] – array recPicture L Brightness position within (h) xL+i v yL+j ), where i = -1..1, j = -1..2,

[0373] The derivation is as follows:

[0374] – If pps_loop_filter_across_virtual_boundaries_disabled_flag equals 1, and PpsVirtualBoundariesPosX[n]%CtbSizeY is not equal to 0, and for any n = 0..pps_num_ver_virtual_boundaries–1, xL-PpsVirtualBoundariesPosX[n] is greater than or equal to 0 and less than 3, the following applies:

[0375] hxL+i =Clip3(PpsVirtualBoundariesPosX[n],pic_width_in_luma_samples-1,xL+i) (8-1229)

[0376] – Otherwise, if pps_loop_filter_across_virtual_boundaries_disabled_flag equals 1, and PpsVirtualBoundariesPosX[n]%CtbSizeY is not equal to 0, and for any n = 0..pps_num_ver_virtual_boundaries–1, PpsVirtualBoundariesPosX[n]-xL is greater than 0 and less than 4, the following applies:

[0377] h x+i =Clip3(0,PpsVirtualBoundariesPosX[n]-1,xL+i) (8-1230)

[0378] –Otherwise, the following applies:

[0379] h x+i =Clip3(0,pic_width_in_luma_samples-1,xL+i) (8-1231)

[0380] – If pps_loop_filter_across_virtual_boundaries_disabled_flag equals 1, and PpsVirtualBoundariesPosY[n]%CtbSizeY is not equal to 0, and for any n = 0..pps_num_hor_virtual_boundaries-1, yL - PpsVirtualBoundariesPosY[n] is greater than or equal to 0 and less than 3, the following applies:

[0381] v y+j =Clip3(PpsVirtualBoundariesPosY[n], pic_height_in_luma_samples-1, yL+j) (8-1232)

[0382] – Otherwise, if pps_loop_filter_across_virtual_boundaries_disabled_flag equals 1, and PpsVirtualBoundariesPosY[n]%CtbSizeY is not equal to 0, and for any n = 0..pps_num_hor_virtual_boundaries-1, PpsVirtualBoundariesPosY[n]-yL is greater than 0 and less than 4, the following applies:

[0383] v y+j =Clip3(0,PpsVirtualBoundariesPosY[n]-1,yL+j) (8-1233)

[0384] –Otherwise, the following applies:

[0385] v y+j =Clip3(0,pic_height_in_luma_samples-1,yL+j) (8-1234)

[0386] – The variables clipLeftPos, clipRightPos, clipTopPos, and clipBottomPos are derived by calling the ALF boundary position derivation procedure as specified in Clause 8.8.5.5, with (xCtb, yCtb) and (xL–xCtb, yL–yCtb) as input.

[0387] – The vertical sample point position offsets yM1, yP1, and yP2 are specified in Table 2-8 based on the vertical brightness sample point positions yL, clipLeftPos, and clipRightPos.

[0388] – The horizontal sample position offsets xM1 and xP1 are specified in Table 2-9 based on the horizontal brightness sample positions xL, clipLeftPos, and clipRightPos.

[0389] – The derivation of variable curr is as follows:

[0390] curr = alfPicture C [xCtbC+x,yCtbC+y] (8-1286)

[0391] – The array of cross-component filter coefficients f[j] is derived as follows, where j = 0..7:

[0392] f[j]=CcAlfCoeff[j] (8-1287)

[0393] – The derivation of variable sum is as follows:

[0394]

[0395] sum = Clip3(-(1<<(BitDepth)) C -1)), (1<<(BitDepth) C -1))-1,sum) (8-1290)

[0396] sum=curr+(sum+64)>>(7+(BitDepth Y -BitDepth C (8-1290)

[0397] – The derivation of the modified filtered reconstructed chroma image sample array ccAlfPicture[xCtbC+x][yCtbC+y] is as follows:

[0398] ccAlfPicture[xCtbC+x][yCtbC+y]=Clip3(0,(1< <BitDepth C (8-1291)

[0399] Table 2-8. Specifications of yM1, yP1, and yP2 based on the vertical brightness sample point positions yL, clipTopPos, and clipBottomPos

[0400] condition yM1 yP1 yP2 yL==clipTopPos+1 -1 1 1 yL==clipTopPos 0 0 1 yL==clipBottomPos–1 0 0 1 yL==clipBottomPos–2 -1 1 1 otherwise -1 1 2

[0401] Table 2-9. Specifications of xM1 and xP1 based on the horizontal luminance sample point positions xL, clipLeftPos, and clipRightPos

[0402] condition xM1 xP1 xL==clipLeftPos 0 0 xL==clipRightPos-1 0 0 xL==clipRightPos–2 –1 1 otherwise –1 1

[0403] 2.9.1. Filling method at virtual boundaries in JVET-P1008

[0404] For CC-ALF in JVET-P1008, mirror (symmetric) fill is used at the ALF virtual boundary. For example... Figure 24 As shown, if the brightness samples above or below the ALF virtual boundary are unavailable, the nearest sample line is used for filling, and the corresponding sample also needs to be filled. Detailed filling methods are shown in Table 2-9.

[0405] 2.10. Simplified method of CC-ALF in JVET-P2025

[0406] 2.10.1. Alternative Filter Shapes

[0407] The CC-ALF filter shape is modified to have 8 or 6 coefficients, as shown in the figure below.

[0408] Figure 25 The shape of an 8-coefficient CC-ALF filter in JVET-P0106 ​​is shown.

[0409] Figure 26 The shape of a 6-coefficient CC-ALF filter in JVET-P0173 is shown.

[0410] Figure 27 The shape of a 6-coefficient CC-ALF filter in JVET-P0251 is shown.

[0411] 2.10.2. Joint Chromaticity Cross-Component Adaptive Filtering

[0412] Joint Chromaticity Cross-Component Adaptive Loop Filtering (JC-CCALF) uses only a set of CCALF filter coefficients trained on the encoder to generate a filtered output as a refinement signal, which is directly added to the Cb component and then appropriately weighted and added to the Cr component. The filter is indicated at the CTU level or by block size, which is communicated with each band-ground signaling.

[0413] Supported chroma block sizes range from the minimum chroma CTU size to the current chroma CTU size. The minimum chroma CTU size is the minimum of the minimum possible width and height of the chroma CTU, i.e., Min(32 / SubWidthC, 32 / SubHeightC), while the current chroma CTU size is the minimum of the width and height of the current chroma CTU, i.e., Min(CtbWidthC, CtbHeightC). For example, if the CTU size is set to a maximum of 128×128, the JC-CCALF chroma block size for a stripe will be one of 32×32, 64×64, and 128×128 for 4:4:4 video, or one of 16×16, 32×32, and 64×64 for 4:2:0 and 4:2:2 video.

[0414] Figure 28 The JC-CCALF process is shown.

[0415] 3. The technical problem solved by the technical solution described in this paper.

[0416] The current design of CC-ALF boundary padding has the following problems:

[0417] 1. The filling method at the ALF virtual boundary in CC-ALF may be suboptimal because the use of filled samples may be inefficient.

[0418] 2. Different ways of handling ALF virtual boundaries and video unit boundaries (e.g., picture / subpicture / strip / piece boundaries) and 360-degree virtual boundaries, i.e., different filling methods exist.

[0419] 3. In ALF, mirror fill is applied, where the distance to the current sample is calculated to determine which corresponding sample needs to be filled. However, in CC-ALF, specifically for 4:2:0, multiple luminance samples are involved in filtering one chroma sample. How to determine which corresponding sample needs to be filled is unknown.

[0420] 4. Examples of techniques and embodiments

[0421] The following list should be considered as examples to illustrate the overall concept. These items should not be interpreted in a narrow way. Furthermore, these items can be combined in any way.

[0422] In some embodiments described in this disclosure, the term 'CC-ALF' refers to an encoding / decoding tool that refines samples in a first color component (e.g., Cb) using sample values ​​from a second color component (e.g., Y) or multiple color components (e.g., both Y and Cr). It is not limited to the CC-ALF techniques described in [1]-[4]. The term "corresponding filtered sample set" can be used to refer to those samples contained in the filter support; for example, for CC-ALF, "corresponding filtered sample set" can be used to refer to co-occurring luminance samples of chrominance samples and adjacent luminance samples of co-occurring luminance samples, which are used to derive the refinement / offset of chrominance samples.

[0423] The padding method used for ALF virtual boundaries can be referred to as 'mirror padding', in which the first unavailable sample located at (i, j) is padded, and the second sample in the filter support of ALF, defined by the 'sample corresponding to the first sample' (e.g., the sample corresponding to (m, n) that shares the same distance as the current luminance sample), is also padded, even if the second sample is available.

[0424] In one example, vertical fill is used, such as a sample point located at (x, y1) that needs to be filled is set to be equal to a sample point located at (x, y2), where y1 refers to the y coordinate of the sample point or the corresponding sample point, and y2 refers to the y coordinate of the sample point used for filling.

[0425] In one example, horizontal fill is used, such as a sample point located at (x1, y) that needs to be filled is set to be equal to a sample point located at (x2, y), where x1 refers to the x-coordinate of the sample point or the corresponding sample point, and x2 refers to the x-coordinate of the sample point used for filling.

[0426] The filling method used for image / sub-image / strip / piece boundary / 360-degree video virtual boundary, and normal boundary (e.g., top and bottom boundary) can be referred to as 'repeated fill', where if a sample to be used is outside the boundary, it can be copied from an available sample inside the boundary.

[0427] In this disclosure, adjacent (adjacent or non-adjacent) samples are “unavailable” if they are located in a different video processing unit (e.g., one of the following: current picture, or current subpicture, or current slice, or current strip, or current brick, or current CTU, or current processing unit (such as ALF processing unit or narrow ALF processing unit), or any other current video unit) or are not reconstructed or are not allowed across a filtered video processing unit.

[0428] Handling ALF virtual boundaries in CC-ALF

[0429] 1. For unavailable luminance samples that need to be filled at the virtual boundary of an ALF sample, mirror filling can be used to derive unavailable luminance samples and one or more corresponding luminance samples for filtering in CC-ALF. That is, at least one corresponding luminance sample of an unavailable sample needs to be filled, even if it is available.

[0430] a. In one example, the luminance sample corresponding to a luminance sample that is determined to be unavailable can be filled using the mirror fill method.

[0431] b. In one example, whether a luminance sample (in the corresponding set of filtered samples) is determined to be an unusable sample can depend on the distance of the sample relative to the representative luminance sample and / or the distance of the unusable sample relative to the representative luminance sample. Let C be the designation of the center row where the representative luminance sample is located. Assume a K×L filter shape is used in CC-ALF, which employs samples in K rows and L columns.

[0432] i. In one example, a representative luminance sample is defined as the co-bit luminance sample of the current chrominance sample that needs to be filtered.

[0433] 1) In one example, the position of the co-bit luminance sample of the current chroma sample can depend on the color format.

[0434] a) In one example, the co-bit luminance sample of the chrominance sample at (x, y) is defined in the 4:2:0 chrominance format as the sample at (2x, 2y).

[0435] b) In one example, the co-occurring luminance sample of the chrominance sample at (x, y) is defined in the 4:2:2 chrominance format as the sample at (2x, y).

[0436] c) In one example, the co-located luma sample of the chroma sample at (x, y) is defined as the sample at (x, y) in the 4:4:4 chroma format.

[0437] ii. In one example, the distance may refer to the vertical distance between the row containing the luma sample and the row containing the representative luma sample. For example, the distance can be calculated as the absolute y-coordinate difference between the luma sample and the representative luma sample.

[0438] 1) As Figure 29 shown, the center row where the representative luma sample is located, the row of unavailable samples, and the row of corresponding samples are respectively designated as C, M, and N, and M is not equal to N. Designate d(x, y) as the absolute y-coordinate difference between x and y, which means the distance between row x and row y.

[0439] iii. In one example, the determination of the corresponding samples to be filled in the mirror padding may depend on how many rows of samples will be adopted by the filter shape.

[0440] iv. In one example, when d(C, M) = d(N, C), if the unavailable sample is in row M (for example, M < C < N or M > C > N), the sample in row N is determined as the corresponding sample to be filled.

[0441] 1) In one example, if the value K (for example, the K×L CC-ALF filter shape) is odd, the mirror padding method of ALF (for example, Figure 16 ) can be used for CC-ALF, where the center luma sample is selected as the representative luma sample.

[0442] a) In one example, assume K = 5 and designate yM2 = -2, yM1 = -1, yL = 0, yP1 = 1, yP2 = 2 as the y-coordinates of five sample rows, as shown in Table 4-5. The ALF virtual boundary is equal to CtbSizeY – 4.

[0443] i. In one example, when the ALF virtual boundary is above the representative luma sample, the unavailable sample can be filled with the closest row below the ALF virtual boundary. At the same time, the corresponding sample can be filled with the closest row above the row where the corresponding sample is located.

[0444] 1. In one example, when yL is equal to CtbSizeY – 3 and the sample at row yM2 is unavailable, the sample (x, yM2) at row yM2 can be filled with the sample (x, yM1) at row yM1. At the same time, the sample (x, yP2) at the corresponding row yP2 can be filled with the sample (x, yP1) at row yP1.

[0445] 2. In one example, when yL equals CtbSizeY – 4 and rows yM2 and yM1 are unavailable, the samples (x, yM2) and (x, yM1) at rows yM2 and yM1 can be filled with the sample (x, yL) at row yL. Meanwhile, the corresponding samples (x, yP2) and (x, yP1) at rows yP2 and yP1 can be filled with the sample (x, yL) at row yL.

[0446] ii. In one example, when the ALF virtual boundary is below a representative brightness sample, the unavailable sample can be filled with the nearest row above the ALF virtual boundary. Meanwhile, the corresponding sample can be filled with the nearest row below the row containing the corresponding sample.

[0447] 1. In one example, when yL equals CtbSizeY – 6 and row yP2 is unavailable, the sample (x, yP2) at row yP2 can be filled with the sample (x, yP1) at row yP1. At the same time, the corresponding sample (x, yM2) at row yM2 can be filled with the sample (x, yM1) at row yM1.

[0448] 2. In one example, when yL equals CtbSizeY – 5 and rows yP2 and yP1 are unavailable, the samples (x, yP2) and (x, yP1) at rows yP2 and yP1 can be filled with the sample (x, yL) at row yL. Simultaneously, the samples (x, yM2) and (x, yM1) at the corresponding rows yM2 and yM1 can be filled with the sample (x, yL) at row yL.

[0449] 2) In one example, if the value K (e.g., the shape of the K×L CC-ALF filter) is even, it can be used Figure 30 The mirror fill method is defined in [the document]. When unavailable samples are located above (below) the ALF virtual boundary in row M(N), they are filled from the nearest sample row below (above) the ALF virtual boundary. It is proposed that corresponding samples located below (above) the ALF virtual boundary in row N(M) can be filled from the nearest sample row above (below) the ALF virtual boundary.

[0450] a) In one example, assume K = 2 and yL = 0 and yP1 = 1 refer to the y coordinates of two sample rows, as shown in Table 4-1. The ALF virtual boundary is equal to CtbSizeY – 4.

[0451] i. In one example, when the ALF virtual boundary is above a representative brightness sample, the unavailable sample can be filled with the nearest row below the ALF virtual boundary. Meanwhile, the corresponding sample can be filled with the nearest row above the row containing the corresponding sample.

[0452] 1. In one example, when yL equals CtbSizeY – 4 and the rows above yL are unavailable, the sample point (x, yP1) at the corresponding row yP1 can be filled with the sample point (x, yL) at the row yL.

[0453] ii. In one example, when the ALF virtual boundary is below a representative brightness sample, the unavailable sample can be filled with the nearest row above the ALF virtual boundary. Meanwhile, the corresponding sample can be filled with the nearest row below the row containing the corresponding sample.

[0454] 1. In one example, when yL equals CtbSizeY – 5 and row yP1 is unavailable, the sample point (x, yP1) at row yP1 can be filled with the sample point (x, yL) at row yL.

[0455] b) In one example, assume K = 4 and yM1 = -1, yL = 0, yP1 = 1, yP2 = 2 represent the y-coordinates of the four sample rows, as shown in Table 4-3. The ALF virtual boundary is equal to CtbSizeY – 4.

[0456] i. In one example, when the ALF virtual boundary is above a representative brightness sample, the unavailable sample can be filled with the nearest row below the ALF virtual boundary. Meanwhile, the corresponding sample can be filled with the nearest row above the row containing the corresponding sample.

[0457] 1. In one example, when yL equals CtbSizeY – 3 and the row above yM1 is unavailable, the sample point (x, yP2) at the corresponding row yP2 can be filled with the sample point (x, yP1) at the row yP1.

[0458] 2. In one example, when yL equals CtbSizeY – 4 and the row above yM1 and yM1 are unavailable, the sample (x, yM1) at row yM1 can be filled with the sample (x, yL) at row yL. At the same time, the corresponding samples (x, yP2) and (x, yP1) at rows yP2 and yP1 can be filled with the sample (x, yL) at row yL.

[0459] ii. In one example, when the ALF virtual boundary is below a representative brightness sample, the unavailable sample can be filled with the nearest row above the ALF virtual boundary. Meanwhile, the corresponding sample can be filled with the nearest row below the row containing the corresponding sample.

[0460] 1. In one example, when yL equals CtbSizeY – 6 and row yP2 is unavailable, the sample (x, yP2) at row yP2 can be filled with the sample (x, yP1) at row yP1.

[0461] 2. In one example, when yL equals CtbSizeY – 5 and rows yP2 and yP1 are unavailable, the samples (x, yP2) and (x, yP1) at rows yP2 and yP1 can be filled with the sample (x, yL) at row yL. Simultaneously, the sample (x, yM1) at the corresponding row yM1 can be filled with the sample (x, yL) at row yL.

[0462] c) In one example, assume K = 6 and yM2 = -2, yM1 = -1, yL = 0, yP1 = 1, yP2 = 2, yP3 = 3 represent the y-coordinates of the six sample rows, as shown in Table 4-6. The ALF virtual boundary is equal to CtbSizeY – 4.

[0463] i. In one example, when the ALF virtual boundary is above a representative brightness sample, the unavailable sample can be filled with the nearest row below the ALF virtual boundary. Meanwhile, the corresponding sample can be filled with the nearest row above the row containing the corresponding sample.

[0464] 1. In one example, when yL equals CtbSizeY–2 and the rows above yM2 are unavailable, the sample point (x, yP3) at the corresponding row yP3 can be filled with the sample point (x, yP2) at the row yP2.

[0465] 2. In one example, when yL equals CtbSizeY – 3 and the row above yM2 and yM2 are unavailable, the sample (x, yM2) at row yM2 can be filled with the sample (x, yM1) at row yM1. ​​Meanwhile, the corresponding samples (x, yP3) and (x, yP2) at rows yP3 and yP2 can be filled with the sample (x, yP1) at row yP1.

[0466] 3. In one example, when yL equals CtbSizeY – 4 and the rows above yM2, yM2, and yM1 are unavailable, the samples (x, yM2) and (x, yM1) at rows yM2 and yM1 can be filled with the sample (x, yL) at row yL. Simultaneously, the samples (x, yP3), (x, yP2), and (x, yP1) at the corresponding rows yP3, yP2, and yP1 can be filled with the sample (x, yL) at row yL.

[0467] ii. In one example, when the ALF virtual boundary is below a representative brightness sample, the unavailable sample can be filled with the nearest row above the ALF virtual boundary. Meanwhile, the corresponding sample can be filled with the nearest row below the row containing the corresponding sample.

[0468] 1. In one example, when yL is equal to CtbSizeY – 7 and the row yP3 is unavailable, the sample (x, yP3) at row yP3 can be filled with the sample (x, yP2) at row yP2.

[0469] 2. In one example, when yL is equal to CtbSizeY – 6 and the rows yP3 and yP2 are unavailable, the samples (x, yP3) and (x, yP2) at rows yP3 and yP2 can be filled with the sample (x, yP1) at the row. Meanwhile, the corresponding sample (x, yM2) at row yM2 can be filled with the sample (x, yM1) at row yM1.

[0470] 3. In one example, when yL is equal to CtbSizeY – 5 and the rows yP3, yP2, and yP1 are unavailable, the samples (x, yP3), (x, yP2), and (x, yP1) at rows yP3, yP2, and yP1 can be filled with the sample (x, yL) at row yL. Meanwhile, the corresponding samples (x, yM2) and (x, yM1) at rows yM2 and yM1 can be filled with the sample (x, yL) at that row.

[0471] v. In one example, if the unavailable sample is located in row M (e.g., M < C), then when d(C, M) = d(N, C) – offset (where offset is an integer value, e.g., equal to 1) or d(C, M) < d(N, C), the sample located in row N is determined as the corresponding sample to be filled.

[0472] 1) In one example, if the unavailable sample is located in row M (e.g., M > C), then when d(M, C) = d(C, N) – offset (where offset is an integer value, e.g., equal to 1) or d(C, M) < d(N, C), the sample located in row N is regarded as the corresponding sample to be filled.

[0473] 2) In one example, the mirror filling method defined in Figure 31 can be adopted. When the unavailable samples are in row M (N) above (below) the ALF virtual boundary and they are filled from the closest sample row below (above) the ALF virtual boundary. It is proposed that the corresponding samples in row N (M) below (above) the ALF virtual boundary can be filled from the closest sample row above (below) row N (M).

[0474] a) In one example, assuming K = 2 and yL = 0 and yP1 = 1 are designated as the y - coordinates of two sample rows, as shown in Table 4 - 2. The ALF virtual boundary is equal to CtbSizeY – 4.

[0475] i. In one example, when the ALF virtual boundary is below a representative brightness sample, the unavailable sample can be filled with the nearest row above the ALF virtual boundary. Meanwhile, the corresponding sample can be filled with the nearest row below the row containing the corresponding sample.

[0476] 1. In one example, when yL equals CtbSizeY–5 and row yP1 is unavailable, the sample point (x, yP1) at row yP1 can be filled with the sample point (x, yL) at row yL. b) In one example, assume K = 4 and yM1 = -1, yL = 0, yP1 = 1, yP2 = 2 represent the y-coordinates of the four sample point rows, as shown in Table 4-4. The ALF virtual boundary equals CtbSizeY–4.

[0477] i. In one example, when the ALF virtual boundary is above a representative brightness sample, the unavailable sample can be filled with the nearest row below the ALF virtual boundary. Meanwhile, the corresponding sample can be filled with the nearest row above the row containing the corresponding sample.

[0478] 1. In one example, when yL equals CtbSizeY – 4 and the row above yM1 and yM1 are unavailable, the sample (x, yM1) at row yM1 can be filled with the sample (x, yL) at row yL. At the same time, the sample (x, yP2) at the corresponding row yP2 can be filled with the sample (x, yP1) at row yP1.

[0479] ii. In one example, when the ALF virtual boundary is below a representative brightness sample, the unavailable sample can be filled with the nearest row above the ALF virtual boundary. Meanwhile, the corresponding sample can be filled with the nearest row below the row containing the corresponding sample.

[0480] 1. In one example, when yL equals CtbSizeY – 6 and row yP2 is unavailable, the sample (x, yP2) at row yP2 can be filled with the sample (x, yP1) at row yP1.

[0481] 2. In one example, when yL equals CtbSizeY – 5 and rows yP2 and yP1 are unavailable, the samples (x, yP2) and (x, yP1) at rows yP2 and yP1 can be filled with the sample (x, yL) at row yL. Simultaneously, the sample (x, yM1) at the corresponding row yM1 can be filled with the sample (x, yL) at row yL.

[0482] c) In one example, assume K = 6 and yM2 = -2, yM1 = -1, yL = 0, yP1 = 1, yP2 = 2, yP3 = 3 represent the y-coordinates of the six sample rows, as shown in Table 4-7. The ALF virtual boundary is equal to CtbSizeY – 4.

[0483] i. In one example, when the ALF virtual boundary is above a representative brightness sample, the unavailable sample can be filled with the nearest row below the ALF virtual boundary. Meanwhile, the corresponding sample can be filled with the nearest row above the row containing the corresponding sample.

[0484] 1. In one example, when yL equals CtbSizeY – 3 and the row above yM2 and yM2 are unavailable, the sample (x, yM2) at row yM2 can be filled with the sample (x, yM1) at row yM1. ​​At the same time, the sample (x, yP3) at the corresponding row yP3 can be filled with the sample (x, yP2) at row yP2.

[0485] 2. In one example, when yL equals CtbSizeY – 4 and the rows above yM2, yM2, and yM1 are unavailable, the samples (x, yM2) and (x, yM1) at rows yM2 and yM1 can be filled with the sample (x, yL) at row yL. Meanwhile, the samples (x, yP3) and (x, yP2) at the corresponding rows yP3 and yP2 can be filled with the sample (x, yP1) at row yP1.

[0486] ii. In one example, when the ALF virtual boundary is below a representative brightness sample, the unavailable sample can be filled with the nearest row above the ALF virtual boundary. Meanwhile, the corresponding sample can be filled with the nearest row below the row containing the corresponding sample.

[0487] 1. In one example, when yL equals CtbSizeY – 7 and row yP3 is unavailable, the sample (x, yP3) at row yP3 can be filled with the sample (x, yP2) at row yP2. At the same time, the corresponding sample (x, yM2) at row yM2 can be filled with the sample (x, yM1) at row yM1.

[0488] 2. In one example, when yL equals CtbSizeY – 6 and rows yP3 and yP2 are unavailable, the samples (x, yP3) and (x, yP2) at rows yP3 and yP2 can be filled with the sample (x, yP1) at row yP1. Meanwhile, the samples (x, yM2) and (x, yM1) at the corresponding rows yM2 and yM1 can be filled with the sample (x, yL) at row yL.

[0489] 3. In one example, when yL equals CtbSizeY – 5 and rows yP3, yP2, and yP1 are unavailable, the samples (x, yP3), (x, yP2), and (x, yP1) at rows yP3, yP2, and yP1 can be filled with the sample (x, yL) at yL. Simultaneously, the samples (x, yM2) and (x, yM1) at the corresponding rows yM2 and yM1 can be filled with the sample (x, yL) at yL.

[0490] c. Figure 29 This shows when the filter is located at (X) c Y c This is an example of the location of the current chroma sample point (above the ALF virtual boundary, referred to by C0) and its corresponding sample point (referred to by C7).

[0491] d. In one example, whether mirror padding is enabled or disabled for CC-ALF / chroma ALF / luminance ALF / other types of filtering methods at the ALF virtual boundary can be notified at the sequence level / picture level / strip level / piece group level signaling, such as in the sequence header / picture header / SPS / VPS / DPS / PPS / APS / strip header / piece group header.

[0492] e. In one example, whether to enable or disable repeat padding and / or mirror padding at ALF virtual boundaries may depend on the encoding / decoding information.

[0493] i. In one example, the encoding / decoding information could refer to the block size, such as the CTU / CTB size.

[0494] 1) In one example, when the CTU / CTB size is greater than or equal to T, such as T = 32 / 64 / 128, mirror fill can be used at the ALF virtual boundary.

[0495] 2) In one example, when the CTU / CTB size is less than or equal to T, such as T = 4 / 8 / 16, repeated padding can be used at the ALF virtual boundary.

[0496] 2. In the above bullet points, vertical fill can be replaced by horizontal fill.

[0497] a. Alternatively, the fill direction (vertical or horizontal) to be used may depend on whether the boundary is a horizontal or vertical boundary.

[0498] b. Alternatively, vertical distance can be replaced by horizontal distance.

[0499] 3. The mirror fill method in bullet point 1 can be used for picture / subpicture / strip / piece boundaries and / or 360-degree boundaries.

[0500] Overall Solution

[0501] 4. Whether and / or how the disclosed methods can be applied in sequence-level / picture-level / strip-level / piece-group-level signaling notifications, such as in sequence headers / picture headers / SPS / VPS / DPS / PPS / APS / strip headers / piece-group headers.

[0502] 5. Whether and / or how the disclosed methods are applied may depend on information such as color format, single / dual tree splitting, and sample location (e.g., relative to CU / CTU).

[0503] Table 4-1. Specifications based on the vertical brightness sample point position yL and applyAlfLineBufBoundary's yP1

[0504] condition yP1 (yL==CtbSizeY-4)&&(applyAlfLineBufBoundary==1) 0 (yL==CtbSizeY-5)&&(applyAlfLineBufBoundary==1) 0 otherwise 1

[0505] Table 4-2. Specifications based on the vertical brightness sample location yL and applyAlfLineBufBoundary's yP1

[0506] condition yP1 (yL==CtbSizeY-4)&&(applyAlfLineBufBoundary==1) 1 (yL==CtbSizeY-5)&&(applyAlfLineBufBoundary==1) 0 otherwise 1

[0507] Table 4-3. Specifications of yM1, yP1, and yP2 based on the vertical brightness sample point position yL and applyAlfLineBufBoundary.

[0508] condition yM1 yP1 yP2 (yL==CtbSizeY-3)&&(applyAlfLineBufBoundary==1) -1 1 1 (yL == CtbSizeY - 4) && (applyAlfLineBufBoundary == 1) 0 0 0 (yL == CtbSizeY - 5) && (applyAlfLineBufBoundary == 1) 0 0 0 (yL == CtbSizeY - 6) && (applyAlfLineBufBoundary == 1) -1 1 1 Otherwise -1 1 2

[0509] Table 4-4. Specifications of yM1, yP1, and yP2 based on the vertical luminance sample point position yL and applyAlfLineBufBoundary.

[0510] Condition yM1 yP1 yP2 (yL == CtbSizeY - 4) && (applyAlfLineBufBoundary == 1) 0 1 1 (yL == CtbSizeY - 5) && (applyAlfLineBufBoundary == 1) 0 0 0 (yL == CtbSizeY - 6) && (applyAlfLineBufBoundary == 1) 0 1 1 Otherwise -1 1 2

[0511] Table 4-5. Specifications of yM2, yM1, yP1, and yP2 based on the vertical brightness sample point position yL and applyAlfLineBufBoundary.

[0512]

[0513] Table 4-6. Specifications of yM2, yM1, yP1, yP2, and yP3 based on the vertical luminance sample point position yL and applyAlfLineBufBoundary.

[0514]

[0515] Table 4-7. Specifications of yM2, yM1, yP1, yP2, and yP3 based on the vertical luminance sample point position yL and applyAlfLineBufBoundary.

[0516]

[0517] 5. Examples

[0518] The changes are highlighted by showing deletions and additions.

[0519] 5.1. Example #1

[0520] The working draft specified in JVET-P0080 can be modified as follows.

[0521] Cross-component filtering process for xxxx chroma samples

[0522] The input to this process is:

[0523] – Reconstructed brightness image sample array recPicture L Before the brightness adaptive loop filtering process,

[0524] – Filtered reconstruction of chroma image sample array alfPicture C ,

[0525] – Chroma position (xC, yC), specifies the left top-left chroma point of the current block relative to the left top-left chroma point of the current image.

[0526] – Width of the chroma sample block ccAlfWidth

[0527] – Height of the chromaticity sample block ccAlfHeight

[0528] – Cross-component filter coefficients CcAlfCoeff[j], where j = 0..13

[0529] The output of this process is a modified filtered, reconstructed chroma image sample array ccAlfPicture.

[0530] The luminance positions (xCtb, yCtb) of the codec tree block are derived as follows:

[0531] xCtb=(((xC*SubWidthC)>>CtbLog2SizeY)< <CtbLog2SizeY (8-1229)

[0532] yCtb=(((yC*SubHeightC)>>CtbLog2SizeY)< <CtbLog2SizeY (8-1229)

[0533] For the reconstructed chromaticity sample points ccAlfPicture[xC+x][yC+y] derived from the filter, and sample points alfPicture C Each reconstructed chroma sample within the current chroma block of [xC+x][yC+y], where x = 0..ccAlfWidth-1 and y = 0..ccAlfHeight-1, is filtered as follows:

[0534] – The brightness position (xL, yL) of the current chromaticity sample point at the chromaticity position (xC+x, yC+y) is set to equal to ((xC+x)*SubWidthC, (yC+y)*SubHeightC).

[0535] – array recPicture L Brightness position within (h) xL+i v yL+j ), where i = -2..2, j = -2..3,

[0536] The derivation is as follows:

[0537] – If pps_loop_filter_across_virtual_boundaries_disabled_flag equals 1, and PpsVirtualBoundariesPosX[n]%CtbSizeY is not equal to 0, and for any n = 0..pps_num_ver_virtual_boundaries–1, xL-PpsVirtualBoundariesPosX[n] is greater than or equal to 0 and less than 3, the following applies:

[0538] h xL+i =Clip3(PpsVirtualBoundariesPosX[n],pic_width_in_luma_samples-1,xL+i) (8-1229)

[0539] – Otherwise, if pps_loop_filter_across_virtual_boundaries_disabled_flag equals 1, and PpsVirtualBoundariesPosX[n]%CtbSizeY is not equal to 0, and for any n = 0..pps_num_ver_virtual_boundaries–1, PpsVirtualBoundariesPosX[n]-xL is greater than 0 and less than 4, the following applies:

[0540] h x+i=Clip3(0,PpsVirtualBoundariesPosX[n]-1,xL+i) (8-1230)

[0541] –Otherwise, the following applies:

[0542] h x+i =Clip3(0,pic_width_in_luma_samples-1,xL+i) (8-1231)

[0543] – If pps_loop_filter_across_virtual_boundaries_disabled_flag equals 1, and PpsVirtualBoundariesPosY[n]%CtbSizeY is not equal to 0, and for any n = 0..pps_num_hor_virtual_boundaries–1, yL - PpsVirtualBoundariesPosY[n] is greater than or equal to 0 and less than 3, the following applies:

[0544] v y+j =Clip3(PpsVirtualBoundariesPosY[n], pic_height_in_luma_samples-1, yL+j) (8-1232)

[0545] – Otherwise, if pps_loop_filter_across_virtual_boundaries_disabled_flag equals 1, and PpsVirtualBoundariesPosY[n]%CtbSizeY is not equal to 0, and for any n = 0..pps_num_hor_virtual_boundaries–1, PpsVirtualBoundariesPosY[n]-yL is greater than 0 and less than 4, the following applies:

[0546] v y+j =Clip3(0,PpsVirtualBoundariesPosY[n]-1,yL+j) (8-1233)

[0547] –Otherwise, the following applies:

[0548] v y+j =Clip3(0,pic_height_in_luma_samples-1,yL+j) (8-1234)

[0549] – The variables clipLeftPos, clipRightPos, clipTopPos, and clipBottomPos are derived by invoking the ALF boundary position derivation procedure specified in Clause 8.8.5.5, with (xCtb, yCtb) and (xL–xCtb, yL–yCtb) as inputs.

[0550] – The vertical sample position offsets yM2, yM1, yP1, yP2, and yP3 are specified in Table 4-1 according to the specifications of the vertical luminance sample position yL and yP1 of applyAlfLineBufBoundary.

[0551]

[0552] Table 4-2. Specifications based on the vertical brightness sample location yL and applyAlfLineBufBoundary's yP1

[0553]

[0554]

[0555] –Based on the vertical brightness sample point positions yL, clipLeftPos, and clipRightPos.

[0556] – The horizontal sample position offsets xM1, xM2, xP1, and xP2 are specified in the table y-yyyy based on the horizontal brightness sample positions xL, clipLeftPos, and clipRightPos.

[0557] – The derivation of variable curr is as follows:

[0558] curr = alfPicture C [xC+x,yC+y] (8-1286)

[0559] – The array of cross-component filter coefficients f[j] is derived as follows, where j = 0..13:

[0560] f[j]=CcAlfCoeff[j] (8-1287)

[0561] – The derivation of variable sum is as follows:

[0562]

[0563] sum=curr+(sum+64)>>7) (8-1290)

[0564] – The derivation of the modified filtered reconstructed chroma image sample array ccAlfPicture[xC+x][yC+y] is as follows:

[0565] ccAlfPicture[xC+x][yC+y]=Clip3(0, (1< <BitDepth C (8-1291)

[0566] Table x-xx – Specifications of yM1, yM2, yP1, yP2, and yP3 based on vertical luminance sample point positions yL, clipTopPos, and clipBottomPos

[0567] Delete the following table:

[0568] Condition yM2 yM1 yP1 yP2 yP3 yL == clipTopPos + 1 -1 -1 1 2 3 yL == clipTopPos 0 0 1 2 3 yL == clipBottomPos - 1 -2 -1 0 0 0 yL == clipBottomPos - 2 -2 -1 1 1 1 yL == clipBottomPos - 3 -2 -1 1 2 2 Otherwise -2 -1 1 2 3

[0569] Replace the above table with the following table:

[0570] Condition yM2 yM1 yP1 yP2 yP3 yL == clipTopPos + 1 -1 -1 1 2 2 yL == clipTopPos 0 0 1 1 1 yL == clipBottomPos - 1 0 0 0 0 0 yL == clipBottomPos - 2 0 0 1 1 1 yL == clipBottomPos - 3 -1 -1 1 2 2 Otherwise -2 -1 1 2 3

[0571] Table y-yy – Specifications of xM1, xM2, xP1, and xP2 based on the horizontal luminance sample point positions xL, clipLeftPos, and clipRightPos

[0572] Condition xM2 xM1 xP1 xP2 xL == clipLeftPos + 1 –1 –1 1 2 xL == clipLeftPos 0 0 1 2 xL == clipRightPos - 1 –2 –1 0 0 xL == clipRightPos – 2 –2 –1 1 1 Otherwise –2 –1 1 2

[0573] 5.2. Example #2

[0574] The working draft specified in JVET-P0080 can be modified as follows.

[0575] Cross-component filtering process for xxxx chroma samples

[0576] The input to this process is:

[0577] – Reconstructed brightness image sample array recPicture L Before the brightness adaptive loop filtering process,

[0578] – Filtered reconstruction of chroma image sample array alfPicture C ,

[0579] – Chroma position (xC, yC), specifies the left top-left chroma point of the current block relative to the left top-left chroma point of the current image.

[0580] – Width of the chroma sample block ccAlfWidth

[0581] – Height of the chromaticity sample block ccAlfHeight

[0582] – Cross-component filter coefficients CcAlfCoeff[j], where j = 0..13

[0583] The output of this process is a modified filtered, reconstructed chroma image sample array ccAlfPicture.

[0584] The luminance positions (xCtb, yCtb) of the codec tree block are derived as follows:

[0585] xCtb=(((xC*SubWidthC)>>CtbLog2SizeY)< <CtbLog2SizeY (8-1229)

[0586] yCtb=(((yC*SubHeightC)>>CtbLog2SizeY)< <CtbLog2SizeY (8-1229)

[0587] For the reconstructed chromaticity sample points ccAlfPicture[xC+x][yC+y] derived from the filter, and sample points alfPicture C Each reconstructed chroma sample within the current chroma block of [xC+x][yC+y], where x = 0..ccAlfWidth-1 and y = 0..ccAlfHeight-1, is filtered as follows:

[0588] – The brightness position (xL, yL) of the current chromaticity sample point at the chromaticity position (xC+x, yC+y) is set to equal to ((xC+x)*SubWidthC, (yC+y)*SubHeightC).

[0589] – array recPicture L Brightness position within (h) xL+i v yL+j ), where i = -2..2, j = -2..3,

[0590] The derivation is as follows:

[0591] – If pps_loop_filter_across_virtual_boundaries_disabled_flag equals 1, and PpsVirtualBoundariesPosX[n]%CtbSizeY is not equal to 0, and for any n = 0..pps_num_ver_virtual_boundaries–1, xL-PpsVirtualBoundariesPosX[n] is greater than or equal to 0 and less than 3, the following applies:

[0592] h xL+i=Clip3(PpsVirtualBoundariesPosX[n],pic_width_in_luma_samples-1,xL+i) (8-1229)

[0593] – Otherwise, if pps_loop_filter_across_virtual_boundaries_disabled_flag equals 1, and PpsVirtualBoundariesPosX[n]%CtbSizeY is not equal to 0, and for any n = 0..pps_num_ver_virtual_boundaries–1, PpsVirtualBoundariesPosX[n]-xL is greater than 0 and less than 4, the following applies:

[0594] h x+i =Clip3(0,PpsVirtualBoundariesPosX[n]-1,xL+i) (8-1230)

[0595] –Otherwise, the following applies:

[0596] h x+i =Clip3(0,pic_width_in_luma_samples-1,xL+i) (8-1231)

[0597] – If pps_loop_filter_across_virtual_boundaries_disabled_flag equals 1, and PpsVirtualBoundariesPosY[n]%CtbSizeY is not equal to 0, and for any n = 0..pps_num_hor_virtual_boundaries–1, yL - PpsVirtualBoundariesPosY[n] is greater than or equal to 0 and less than 3, the following applies:

[0598] v y+j =Clip3(PpsVirtualBoundariesPosY[n], pic_height_in_luma_samples-1, yL+j) (8-1232)

[0599] – Otherwise, if pps_loop_filter_across_virtual_boundaries_disabled_flag equals 1, and PpsVirtualBoundariesPosY[n]%CtbSizeY is not equal to 0, and for any n = 0..pps_num_hor_virtual_boundaries–1, PpsVirtualBoundariesPosY[n]-yL is greater than 0 and less than 4, the following applies:

[0600] v y+j =Clip3(0,PpsVirtualBoundariesPosY[n]-1,yL+j) (8-1233)

[0601] –Otherwise, the following applies:

[0602] v y+j =Clip3(0,pic_height_in_luma_samples-1,yL+j) (8-1234)

[0603] – The variables clipLeftPos, clipRightPos, clipTopPos, and clipBottomPos are derived by invoking the ALF boundary position derivation procedure specified in Clause 8.8.5.5, with (xCtb, yCtb) and (xL–xCtb, yL–yCtb) as inputs.

[0604] – The vertical sample position offsets yM2, yM1, yP1, yP2, and yP3 are specified in Table 4-1 according to the specifications of the vertical luminance sample position yL and yP1 of applyAlfLineBufBoundary.

[0605]

[0606] Table 4-2. Specifications based on the vertical brightness sample location yL and applyAlfLineBufBoundary's yP1

[0607]

[0608] –Based on the vertical brightness sample point positions yL, clipLeftPos, and clipRightPos.

[0609] – The horizontal sample position offsets xM1, xM2, xP1, and xP2 are specified in the table y-yyyy based on the horizontal brightness sample positions xL, clipLeftPos, and clipRightPos.

[0610] – The derivation of variable curr is as follows:

[0611] curr = alfPicture C [xC+x,yC+y] (8-1286)

[0612] – The array of cross-component filter coefficients f[j] is derived as follows, where j = 0..13:

[0613] f[j]=CcAlfCoeff[j] (8-1287)

[0614] – The derivation of variable sum is as follows:

[0615]

[0616] sum=curr+(sum+64)>>7) (8-1290)

[0617] – The derivation of the modified filtered reconstructed chroma image sample array ccAlfPicture[xC+x][yC+y] is as follows:

[0618] ccAlfPicture[xC+x][yC+y]=Clip3(0, (1< <BitDepth C (8-1291)

[0619] Table x-xx – Specifications of yM1, yM2, yP1, yP2, and yP3 based on vertical luminance sample point positions yL, clipTopPos, and clipBottomPos

[0620] Delete the following table:

[0621] Condition yM2 yM1 yP1 yP2 yP3 yL == clipTopPos + 1 -1 -1 1 2 3 yL == clipTopPos 0 0 1 2 3 yL == clipBottomPos - 1 -2 -1 0 0 0 [[ID=5*]]yL == clipBottomPos - 2 -2 -1 1 1 1 yL == clipBottomPos - 3 -2 -1 1 2 2 Otherwise -2 -1 1 2 3

[0622] Add the following table:

[0623] [[ID=*8]]Condition yM2 yM1 yP1 yP2 yP3 yL == clipTopPos + 2 -2 -1 1 2 2 yL == clipTopPos + 1 -1 -1 1 1 1 yL == clipTopPos 0 0 0 0 0 yL == clipBottomPos - 1 0 0 0 0 0 yL == clipBottomPos - 2 0 -1 1 1 1 yL == clipBottomPos - 3 -2 -1 1 2 2 Otherwise -2 -1 1 2 3

[0624] Table y-yy – Specifications of xM1, xM2, xP1, and xP2 based on the horizontal luminance sample point positions xL, clipLeftPos, and clipRightPos

[0625] Condition xM2 xM1 xP1 xP2 xL == clipLeftPos + 1 –1 –1 1 2 xL == clipLeftPos 0 0 1 2 xL == clipRightPos - 1 –2 –1 0 0 xL == clipRightPos – 2 –2 –1 1 1 Otherwise –2 –1 1 2

[0626] 5.3. Example #3

[0627] The working draft specified in JVET-P1008 can be modified as follows.

[0628] Cross-component filtering process for xxxx chroma samples

[0629] The input to this process is:

[0630] – Reconstructed brightness image sample array recPicture L Before the brightness adaptive loop filtering process,

[0631] – Filtered reconstruction of chroma image sample array alfPicture C ,

[0632] – Chroma position (xCtbC, yCtbC), specifies the top-left sample of the current chroma codec block relative to the left-left sample of the current image.

[0633] – Width of the chroma sample block ccAlfWidth

[0634] – Height of the chromaticity sample block ccAlfHeight

[0635] – Cross-component filter coefficients CcAlfCoeff[j], where j = 0.7

[0636] The output of this process is a modified filtered reconstructed chroma image sample array ccAlfPicture.

[0637] The luminance positions (xCtb, yCtb) of the codec tree block are derived as follows:

[0638] xCtb=(((xCtbC*SubWidthC)>>CtbLog2SizeY)< <CtbLog2SizeY (8-1229)

[0639] yCtb=(((yCtbC*SubHeightC)>>CtbLog2SizeY)< <CtbLog2SizeY (8-1229)

[0640] For the reconstructed chromaticity sample points ccAlfPicture[xCtbC+x][yCtbC+y] derived from the filter, and sample points alfPicture C Each reconstructed chroma sample within the current chroma block of [xCtbC+x][yCtbC+y], where x = 0..ccAlfWidth-1 and y = 0..ccAlfHeight-1, is filtered as follows:

[0641] – The brightness position (xL, yL) of the current chromaticity sample point at the chromaticity position (xCtbC+x, yCtbC+y) is set to equal to ((xCtbC+x)*SubWidthC, (yCtbC+y)*SubHeightC).

[0642] – array recPicture L Brightness position within (h) xL+i v yL+j ), where i = -1..1, j = -1..2,

[0643] The derivation is as follows:

[0644] – If pps_loop_filter_across_virtual_boundaries_disabled_flag equals 1, and PpsVirtualBoundariesPosX[n]%CtbSizeY is not equal to 0, and for any n = 0..pps_num_ver_virtual_boundaries–1, xL-PpsVirtualBoundariesPosX[n] is greater than or equal to 0 and less than 3, the following applies:

[0645] h xL+i =Clip3(PpsVirtualBoundariesPosX[n],pic_width_in_luma_samples-1,xL+i) (8-1229)

[0646] – Otherwise, if pps_loop_filter_across_virtual_boundaries_disabled_flag equals 1, and PpsVirtualBoundariesPosX[n]%CtbSizeY is not equal to 0, and for any n = 0..pps_num_ver_virtual_boundaries–1, PpsVirtualBoundariesPosX[n]-xL is greater than 0 and less than 4, the following applies:

[0647] h x+i =Clip3(0,PpsVirtualBoundariesPosX[n]-1,xL+i) (8-1230)

[0648] –Otherwise, the following applies:

[0649] h x+i=Clip3(0,pic_width_in_luma_samples-1,xL+i) (8-1231)

[0650] – If pps_loop_filter_across_virtual_boundaries_disabled_flag equals 1, and PpsVirtualBoundariesPosY[n]%CtbSizeY is not equal to 0, and for any n = 0..pps_num_hor_virtual_boundaries–1, yL - PpsVirtualBoundariesPosY[n] is greater than or equal to 0 and less than 3, the following applies:

[0651] v y+j =Clip3(PpsVirtualBoundariesPosY[n], pic_height_in_luma_samples-1, yL+j) (8-1232)

[0652] – Otherwise, if pps_loop_filter_across_virtual_boundaries_disabled_flag equals 1, and PpsVirtualBoundariesPosY[n]%CtbSizeY is not equal to 0, and for any n = 0..pps_num_hor_virtual_boundaries–1, PpsVirtualBoundariesPosY[n]-yL is greater than 0 and less than 4, the following applies:

[0653] v y+j =Clip3(0,PpsVirtualBoundariesPosY[n]-1,yL+j) (8-1233)

[0654] –Otherwise, the following applies:

[0655] v y+j =Clip3(0,pic_height_in_luma_samples-1,yL+j) (8-1234)

[0656] – The variables clipLeftPos, clipRightPos, clipTopPos, and clipBottomPos are derived by invoking the ALF boundary position derivation procedure specified in Clause 8.8.5.5, with (xCtb, yCtb) and (xL–xCtb, yL–yCtb) as inputs.

[0657] – The vertical sample position offsets yM1, yP1, and yP2 are specified in Table 4-1 according to the specifications of the vertical luminance sample position yL and yP1 of applyAlfLineBufBoundary.

[0658]

[0659] Table 4-2. Specifications based on the vertical brightness sample location yL and applyAlfLineBufBoundary's yP1

[0660]

[0661] –Based on the vertical brightness sample point positions yL, clipLeftPos, and clipRightPos.

[0662] – The horizontal sample position offsets xM1 and xP1 are specified in the table y-yyyy based on the horizontal brightness sample positions xL, clipLeftPos, and clipRightPos.

[0663] – The derivation of variable curr is as follows:

[0664] curr = alfPicture C [xCtbC+x,yCtbC+y] (8-1286)

[0665] – The array of cross-component filter coefficients f[j] is derived as follows, where j = 0..7:

[0666] f[j]=CcAlfCoeff[j] (8-1287)

[0667] – The derivation of variable sum is as follows:

[0668]

[0669]

[0670] sum = Clip3(-(1<<(BitDepth)) C -1)), (1<<(BitDepth) C -1))-1,sum) (8-1290)

[0671] sum=curr+(sum+64)>>(7+(BitDepth Y -BitDepth C (8-1290)

[0672] – Modified filter for reconstructing the chroma image sample array ccAlfPicture[xCtbC+x][yCtbC+y]

[0673] The derivation is as follows:

[0674] ccAlfPicture[xCtbC+x][yCtbC+y]=Clip3(0,(1< <BitDepth C (8-1291)

[0675] Table x-xx – Specification of yM1, yP1, and yP2 based on vertical luminance sample point positions yL, clipTopPos, and clipBottomPos

[0676] Delete the following table:

[0677] Condition yM1 yP1 yP2 ​ -1 1 1 ​ 0 0 1 ​ 0 0 1 ​ -1 1 1 ​ -1 1 2

[0678] Add the following table

[0679] ​ ​ ​ ​ yL==clipTopPos 0 1 1 yL==clipBottomPos-1 0 0 0 yL==clipBottomPos-2 0 1 1 otherwise -1 1 2

[0680] Table y-yy – Specification of xM1 and xP1 based on the horizontal luminance sample point positions xL, clipLeftPos, and clipRightPos

[0681] condition xM1 xP1 xL==clipLeftPos 0 0 xL==clipRightPos-1 0 0 xL==clipRightPos–2 –1 1 otherwise –1 1

[0682] 5.4. Example #4

[0683] The working draft specified in JVET-P1008 can be modified as follows.

[0684] Cross-component filtering process for xxxx chroma samples

[0685] The input to this process is:

[0686] – Reconstructed brightness image sample array recPicture L Before the brightness adaptive loop filtering process,

[0687] – Filtered reconstruction of chroma image sample array alfPicture C ,

[0688] – Chroma position (xCtbC, yCtbC), specifies the top-left sample of the current chroma codec block relative to the left-left sample of the current image.

[0689] – Width of the chroma sample block ccAlfWidth

[0690] – Height of the chromaticity sample block ccAlfHeight

[0691] – Cross-component filter coefficients CcAlfCoeff[j], where j = 0.7

[0692] The output of this process is a modified filtered reconstructed chroma image sample array ccAlfPicture.

[0693] The luminance positions (xCtb, yCtb) of the codec tree block are derived as follows:

[0694] xCtb=(((xCtbC*SubWidthC)>>CtbLog2SizeY)< <CtbLog2SizeY (8-1229)

[0695] yCtb=(((yCtbC*SubHeightC)>>CtbLog2SizeY)< <CtbLog2SizeY (8-1229)

[0696] For the reconstructed chromaticity sample points ccAlfPicture[xCtbC+x][yCtbC+y] derived from the filter, and sample points alfPicture C Each reconstructed chroma sample within the current chroma block of [xCtbC+x][yCtbC+y], where x = 0..ccAlfWidth-1 and y = 0..ccAlfHeight-1, is filtered as follows:

[0697] – The brightness position (xL, yL) of the current chromaticity sample point at the chromaticity position (xCtbC+x, yCtbC+y) is set to equal to ((xCtbC+x)*SubWidthC, (yCtbC+y)*SubHeightC).

[0698] – array recPicture L Brightness position within (h) xL+i v yL+j ), where i = -1..1, j = -1..2,

[0699] The derivation is as follows:

[0700] – If pps_loop_filter_across_virtual_boundaries_disabled_flag equals 1, and PpsVirtualBoundariesPosX[n]%CtbSizeY is not equal to 0, and for any n = 0..pps_num_ver_virtual_boundaries–1, xL-PpsVirtualBoundariesPosX[n] is greater than or equal to 0 and less than 3, the following applies:

[0701] h xL+i =Clip3(PpsVirtualBoundariesPosX[n],pic_width_in_luma_samples-1,xL+i) (8-1229)

[0702] – Otherwise, if pps_loop_filter_across_virtual_boundaries_disabled_flag equals 1, and PpsVirtualBoundariesPosX[n]%CtbSizeY is not equal to 0, and for any n = 0..pps_num_ver_virtual_boundaries–1, PpsVirtualBoundariesPosX[n]-xL is greater than 0 and less than 4, the following applies:

[0703] h x+i =Clip3(0,PpsVirtualBoundariesPosX[n]-1,xL+i) (8-1230)

[0704] –Otherwise, the following applies:

[0705] h x+i =Clip3(0,pic_width_in_luma_samples-1,xL+i) (8-1231)

[0706] – If pps_loop_filter_across_virtual_boundaries_disabled_flag equals 1, and PpsVirtualBoundariesPosY[n]%CtbSizeY is not equal to 0, and for any n = 0..pps_num_hor_virtual_boundaries–1, yL - PpsVirtualBoundariesPosY[n] is greater than or equal to 0 and less than 3, the following applies:

[0707] vy+j =Clip3(PpsVirtualBoundariesPosY[n], pic_height_in_luma_samples-1, yL+j) (8-1232)

[0708] – Otherwise, if pps_loop_filter_across_virtual_boundaries_disabled_flag equals 1, and PpsVirtualBoundariesPosY[n]%CtbSizeY is not equal to 0, and for any n = 0..pps_num_hor_virtual_boundaries–1, PpsVirtualBoundariesPosY[n]-yL is greater than 0 and less than 4, the following applies:

[0709] v y+j =Clip3(0,PpsVirtualBoundariesPosY[n]-1,yL+j) (8-1233)

[0710] –Otherwise, the following applies:

[0711] v y+j =Clip3(0,pic_height_in_luma_samples-1,yL+j) (8-1234)

[0712] – The variables clipLeftPos, clipRightPos, clipTopPos, and clipBottomPos are derived by invoking the ALF boundary position derivation procedure specified in Clause 8.8.5.5, with (xCtb, yCtb) and (xL–xCtb, yL–yCtb) as inputs.

[0713] – The vertical sample position offsets yM1, yP1, and yP2 are specified in Table 4-1 according to the specifications of the vertical luminance sample position yL and yP1 of applyAlfLineBufBoundary.

[0714]

[0715] Table 4-2. Specifications based on the vertical brightness sample location yL and applyAlfLineBufBoundary's yP1

[0716]

[0717] –Based on the vertical brightness sample point positions yL, clipLeftPos, and clipRightPos.

[0718] – The horizontal sample position offsets xM1 and xP1 are specified in the table y-yyyy based on the horizontal brightness sample positions xL, clipLeftPos, and clipRightPos.

[0719] – The derivation of variable curr is as follows:

[0720] curr = alfPicture C [xCtbC+x,yCtbC+y] (8-1286)

[0721] – The array of cross-component filter coefficients f[j] is derived as follows, where j = 0..7:

[0722] f[j]=CcAlfCoeff[j] (8-1287)

[0723] – The derivation of variable sum is as follows:

[0724]

[0725] sum = Clip3(-(1<<(BitDepth)) C -1)), (1<<(BitDepth) C -1))-1,sum) (8-1290)

[0726] sum=curr+(sum+64)>>(7+(BitDepth Y -BitDepth C (8-1290)

[0727] – Modified filter for reconstructing the chroma image sample array ccAlfPicture[xCtbC+x][yCtbC+y]

[0728] The derivation is as follows:

[0729] ccAlfPicture[xCtbC+x][yCtbC+y]=Clip3(0,(1< <BitDepth C (8-1291)

[0730] Table x-xx – Specification of yM1, yP1, and yP2 based on vertical luminance sample point positions yL, clipTopPos, and clipBottomPos

[0731] Delete the following table:

[0732] condition yM1 yP1 yP2 yL==clipTopPos+1 -1 1 1 yL==clipTopPos 0 0 1 yL==clipBottomPos-1 0 0 1 yL==clipBottomPos-2 -1 1 1 otherwise -1 1 2

[0733] Add the following table:

[0734] condition yM1 yP1 yP2 yL==clipTopPos+1 -1 1 1 yL==clipTopPos 0 0 0 yL==clipBottomPos-1 0 0 0 yL==clipBottomPos-2 -1 1 1 otherwise -1 1 2

[0735] Table y-yy – Specification of xM1 and xP1 based on the horizontal luminance sample point positions xL, clipLeftPos, and clipRightPos

[0736] condition xM1 xP1 xL==clipLeftPos 0 0 xL==clipRightPos-1 0 0 xL==clipRightPos–2 –1 1 otherwise –1 1

[0737] Figure 32 This is a block diagram illustrating an example video processing system 1900 in which various techniques disclosed herein may be implemented. Various implementations may include some or all of the components of system 1900. System 1900 may include input 1902 for receiving video content. The video content may be received in a raw or uncompressed format (e.g., 8 or 10-bit multi-component pixel values), or in a compressed or encoded format. Input 1902 may represent a network interface, a peripheral bus interface, or a storage interface. Examples of network interfaces include wired interfaces (such as Ethernet, Passive Optical Network (PON), etc.) and wireless interfaces such as Wi-Fi or cellular interfaces.

[0738] System 1900 may include codec component 1904, which can implement the various codec or encoding methods described in this document. Codec component 1904 can reduce the average bit rate of the video from input 1902 to the output of codec component 1904 to produce a codec representation of the video. Codec techniques are therefore sometimes referred to as video compression or video transcoding techniques. The output of codec component 1904 can be stored or transmitted via a connected communication, as represented by component 1906. The bitstream (or codec) representation of the stored or communicated video received at input 1902 can be used by component 1908 to generate pixel values ​​or displayable video, which is sent to display interface 1910. The process of generating a user-viewable video from the bitstream representation is sometimes referred to as video decompression. Furthermore, although some video processing operations are referred to as “codec” operations or tools, it should be understood that codec tools or operations are used at the encoder, and the corresponding decoding tools or operations that reverse the results of codec are performed by the decoder.

[0739] Examples of peripheral bus interfaces or display interfaces may include Universal Serial Bus (USB), High Resolution Multimedia Interface (HDMI), or DisplayPort, etc. Examples of storage interfaces include SATA (Serial Advanced Technology Accessory), PCI, IDE, etc. The technologies described in this document can be implemented in a variety of electronic devices, such as mobile phones, laptops, smartphones, or other devices capable of digital data processing and / or video display.

[0740] Figure 33 This is a block diagram of a video processing device 3600. Device 3600 can be used to implement one or more methods described herein. Device 3600 can be implemented in smartphones, tablets, computers, Internet of Things (IoT) receivers, etc. Device 3600 may include one or more processors 3602, one or more memories 3604, and video processing hardware 3606. The processors (multiple) 3602 can be configured to implement one or more methods described herein. The memories (multiple) 3604 can be used to store data and code used to implement the methods and techniques described herein. The video processing hardware 3606 can be used to implement some of the techniques described herein in hardware circuitry.

[0741] Figure 35 This is a block diagram illustrating an example video codec system 100 that can employ the techniques disclosed herein.

[0742] like Figure 35 As shown, the video encoding / decoding system 100 may include a source device 110 and a destination device 120. The source device 110 generates encoded video data, which can be referred to as a video encoding device. The destination device 120 can decode the encoded video data generated by the source device 110, which can be referred to as a video decoding device.

[0743] The source device 110 may include a video source 112, a video encoder 114, and an input / output (I / O) interface 116.

[0744] Video source 112 may include sources such as a video capture device, an interface for receiving video data from a video content provider, and / or a computer graphics system for generating video data, or a combination of such sources. Video data may include one or more pictures. Video encoder 114 encodes the video data from video source 112 to generate a bitstream. The bitstream may contain sequences of bits that form a coded and decoded representation of the video data. The bitstream may contain coded and decoded pictures and associated data. A coded and decoded picture is a coded and decoded representation of a picture. The associated data may include a sequence parameter set, a picture parameter set, and other syntax structures. I / O interface 116 may include a modulator / demodulator (modem) and / or a transmitter. Encoded video data may be directly transmitted to destination device 120 via network 130a through I / O interface 116. Encoded video data may also be stored on storage medium / server 130b for access by destination device 120.

[0745] The destination device 120 may include an I / O interface 126, a video decoder 124, and a display device 122.

[0746] I / O interface 126 may include a receiver and / or a modem. I / O interface 126 may acquire encoded video data from source device 110 or storage medium / server 130b. Video decoder 124 may decode the encoded video data. Display device 122 may display the decoded video data to a user. Display device 122 may be integrated with destination device 120 or may be external to destination device 120, configured to connect to an external display device.

[0747] The video encoder 114 and the video decoder 124 can operate according to video compression standards, such as the High Efficiency Video Codec (HEVC) standard, the Multi-Functional Video Codec (VVM) standard, and other current and / or future standards.

[0748] Figure 36 This is a block diagram illustrating an example of a video encoder 200, which may be... Figure 35 The video encoder 114 in the system 100 shown.

[0749] The video encoder 200 can be configured to perform any or all of the techniques disclosed herein. Figure 36 In the example, the video encoder 200 includes multiple functional components. The techniques described in this disclosure can be shared among the various components of the video encoder 200. In some examples, the processor can be configured to perform any or all of the techniques described in this disclosure.

[0750] The video encoder 200 may include a segmentation unit 201, a prediction unit 202 including a mode selection unit 203, a motion estimation unit 204, a motion compensation unit 205 and an intra-frame prediction unit 206, a residual generation unit 207, a transform unit 208, a quantization unit 209, an inverse quantization unit 210, an inverse transform unit 211, a reconstruction unit 212, a buffer 213, and an entropy coding unit 214.

[0751] In other examples, the video encoder 200 may contain more, fewer, or different functional components. In one example, the prediction unit 202 may include an intra-block copy (IBC) unit. The IBC unit can perform prediction in IBC mode, where at least one reference picture is the picture in which the current video block is located.

[0752] Additionally, some components (such as motion estimation unit 204 and motion compensation unit 205) can be highly integrated, but for illustrative purposes... Figure 36 The examples are shown separately.

[0753] The segmentation unit 201 can segment an image into one or more video blocks. The video encoder 200 and the video decoder 300 can support various video block sizes.

[0754] The mode selection unit 203 can select one of the encoding / decoding modes, intra-frame or inter-frame, for example, based on the error result, and provide the resulting intra-frame or inter-frame encoded / decoded block to the residual generation unit 207 to generate residual block data, which is then provided to the reconstruction unit 212 to reconstruct the coded block for use as a reference picture. In some examples, the mode selection unit 203 can select a combination of intra-frame and inter-frame prediction (CIIP) modes, where prediction is based on inter-frame prediction signals and intra-frame prediction signals. In the case of inter-frame prediction, the mode selection unit 203 can also select the resolution of the block's motion vector (e.g., sub-pixel or integer pixel precision).

[0755] To perform inter-frame prediction on the current video block, motion estimation unit 204 can generate motion information for the current video block by comparing one or more reference frames from buffer 213 with the current video block. Motion compensation unit 205 can determine the predicted video block for the current video block based on motion information from images other than those associated with the current video block from buffer 213 and decoded samples.

[0756] The motion estimation unit 204 and the motion compensation unit 205 can perform different operations on the current video block, for example, depending on whether the current video block is in an I-band, P-band, or B-band.

[0757] In some examples, motion estimation unit 204 can perform unidirectional prediction of the current video block, and can search for reference images in list 0 or list 1 of reference video blocks for the current video block. Motion estimation unit 204 can then generate a reference index indicating the reference images in list 0 or list 1 containing the reference video blocks, and a motion vector indicating the spatial displacement between the current video block and the reference video blocks. Motion estimation unit 204 can output the reference index, prediction direction indicator, and motion vector as motion information for the current video block. Motion compensation unit 205 can generate a predicted video block for the current block based on the reference video blocks indicated by the motion information of the current video block.

[0758] In other examples, motion estimation unit 204 can perform bidirectional prediction of the current video block. Motion estimation unit 204 can search for reference images in list 0 for the current video block's reference video blocks, and can also search for reference images in list 1 for another reference video block of the current video block. Motion estimation unit 204 can then generate reference indices indicating the reference images containing reference video blocks in lists 0 and 1, and motion vectors indicating the spatial displacement between the reference video blocks and the current video block. Motion estimation unit 204 can output the reference indices and the motion vector of the current video block as the motion information of the current video block. Motion compensation unit 205 can generate a predicted video block for the current video block based on the reference video blocks indicated by the motion information of the current video block.

[0759] In some examples, the motion estimation unit 204 can output motion information for the entire decoding process of the decoder.

[0760] In some examples, motion estimation unit 204 may not output motion information for the entire current video block. Instead, motion estimation unit 204 may refer to motion information signaling from another video block to inform the motion information of the current video block. For example, motion estimation unit 204 may determine that the motion information of the current video block is sufficiently similar to the motion information of adjacent video blocks.

[0761] In one example, the motion estimation unit 204 may indicate a value in the syntax structure associated with the current video block that indicates to the video decoder 300 that the current video block has the same motion information as another video block.

[0762] In another example, motion estimation unit 204 can identify another video block and motion vector difference (MVD) in the syntax structure associated with the current video block. The motion vector difference indicates the difference between the motion vectors of the current video block and the motion vectors of the indicated video block. Video decoder 300 can use the motion vectors of the indicated video block and the motion vector difference to determine the motion vectors of the current video block.

[0763] As discussed above, the video encoder 200 can predictively signal motion vectors. Two examples of predictive signaling notification techniques that can be implemented by the video encoder 200 include Advanced Motion Vector Prediction (AMVP) and Merged Mode Signaling Notification.

[0764] Intra-prediction unit 206 can perform intra-prediction on the current video block. When intra-prediction unit 206 performs intra-prediction on the current video block, it can generate prediction data for the current video block based on the decoded samples of other video blocks in the same frame. The prediction data for the current video block can include the predicted video block and various syntax elements.

[0765] The residual generation unit 207 can generate residual data for the current video block by subtracting (e.g., indicated by a minus sign) multiple predicted video blocks from the current video block. The residual data for the current video block can contain residual video blocks corresponding to different sample components in the current video block.

[0766] In other examples (such as in skip mode), residual data for the current video block may not exist, and residual generation unit 207 may not perform subtraction operations.

[0767] The transform processing unit 208 can generate one or more transform coefficient video blocks of the current video block by applying one or more transforms to the residual video block associated with the current video block.

[0768] After the transform processing unit 208 generates a transform coefficient video block associated with the current video block, the quantization unit 209 can quantize the transform coefficient video block associated with the current video block based on one or more quantization parameter (QP) values ​​associated with the current video block.

[0769] Inverse quantization unit 210 and inverse transform unit 211 can apply inverse quantization and inverse transform to the transform coefficient video blocks respectively to reconstruct the residual video blocks from the transform coefficient video blocks. Reconstruction unit 212 can add the reconstructed residual video blocks to the corresponding samples from one or more predicted video blocks generated by prediction unit 202 to generate a reconstructed video block associated with the current block for storage in buffer 213.

[0770] After the video block is reconstructed by the reconstruction unit 212, a loop filtering operation can be performed to reduce video block artifacts in the video block.

[0771] Entropy encoding unit 214 can receive data from other functional components of video encoder 200. When entropy encoding unit 214 receives data, it can perform one or more entropy encoding operations to generate entropy-encoded data and output a bitstream containing the entropy-encoded data.

[0772] Figure 37 This is a block diagram illustrating an example of a video decoder 300, which may be... Figure 35 The video decoder 114 in the system 100 shown.

[0773] The video decoder 300 can be configured to perform any or all of the techniques disclosed herein. Figure 37 In the example, the video decoder 300 includes multiple functional components. The techniques described in this disclosure can be shared among the various components of the video decoder 300. In some examples, the processor can be configured to perform any or all of the techniques described in this disclosure.

[0774] exist Figure 37 In the example, video decoder 300 includes an entropy decoding unit 301, a motion compensation unit 302, an intra-frame prediction unit 303, an inverse quantization unit 304, an inverse transform unit 305, a reconstruction unit 306, and a buffer 307. In some examples, video decoder 300 can perform decoding pass, which is generally the opposite of that with respect to video encoder 200 ( Figure 36 The encoding described is passed.

[0775] The entropy decoding unit 301 can retrieve the encoded bitstream. The encoded bitstream may contain entropy-coded video data (e.g., encoded blocks of video data). The entropy decoding unit 301 can decode the entropy-coded video data, and from the entropy-coded video data, the motion compensation unit 302 can determine motion information, which includes motion vectors, motion vector precision, reference image list indexes, and other motion information. The motion compensation unit 302 can determine such information, for example, by performing AMVP and merging modes.

[0776] The motion compensation unit 302 can generate motion-compensated blocks, performing interpolation based on interpolation filters as possible. Identifiers of the interpolation filters required for use with sub-pixel precision can be included in the syntax elements.

[0777] The motion compensation unit 302 can use interpolation filters, such as those used by the video encoder 200 during the encoding of a video block, to calculate the interpolated values ​​of sub-integer pixels of the reference block. The motion compensation unit 302 can determine the interpolation filter used by the video encoder 200 based on the received syntax information, and use the interpolation filter to generate predictive blocks.

[0778] Motion compensation unit 302 may use some of the syntax information to determine the size of the blocks used to encode (multiple) frames and / or (multiple) stripes of the encoded video sequence, segmentation information describing how the pictures of the encoded video sequence are divided, a mode indicating how each segment is encoded, one or more reference frames (and a list of reference frames) for each inter-frame coded block, and other information for decoding the encoded video sequence.

[0779] Intra-prediction unit 303 can use, for example, an intra-prediction mode received in the bitstream to form prediction blocks from spatially adjacent blocks. Inverse quantization unit 303 performs inverse quantization (i.e., dequantization) on the quantized video block coefficients provided in the bitstream and decoded by entropy decoding unit 301. Inverse transform unit 303 applies an inverse transform.

[0780] The reconstruction unit 306 can sum the residual block with the corresponding prediction block generated by the motion compensation unit 202 or the intra-frame prediction unit 303 to form a decoded block. If desired, a deblocking filter can also be applied to filter the decoded block to remove block artifacts. The decoded video block is then stored in the buffer 307, which provides a reference block for subsequent motion compensation / intra-frame prediction, and the decoded video is generated for display on the display device.

[0781] Some embodiments may be described using the following clause-based format. The first set of clauses illustrates example embodiments of the techniques discussed in previous sections (e.g., item 1 of the example embodiments).

[0782] 1. A method for video processing (e.g., Figure 34 Method 3400 includes: for the conversion between video units and the encoded / decoded representation of a video, determining (3402) to use a cross-component adaptive loop filtering operation based on criteria, wherein the cross-component adaptive loop filtering uses a mirror fill technique for unavailable luminance samples; and performing a conversion (3404) based on the determination. This document discloses various embodiments of cross-component adaptive loop filtering and its operation, as well as the mirror fill technique, and their relationship to virtual buffer boundaries.

[0783] 2. The method according to Clause 1, wherein the mirror fill technique is also used to derive one or more corresponding luminance samples of the unavailable luminance samples.

[0784] 3. The method according to Clause 2, wherein one or more corresponding luminance samples are determined based on the distance of one or more corresponding luminance samples from the representative luminance sample or the distance of unavailable samples from the representative luminance sample.

[0785] 4. The method according to Clause 3, wherein the representative luminance sample corresponds to the location of the chrominance sample using cross-component adaptive loop filtering.

[0786] 5. The method described in Clause 3, wherein the location of the representative luminance sample depends on the color format of the video.

[0787] 6. The method according to any one of clauses 3-4, wherein the distance corresponds to the distance between a pixel line containing one or more luminance samples along a second direction in a first direction and a row containing representative samples.

[0788] 7. The method according to clause 6, where C represents the center line where the representative luminance sample is located along the second direction, M represents the line where the unavailable sample is located along the second direction, and N represents the line where one or more luminance samples are located along the second direction, where C, M, and N are positive integers and M is not equal to N, then the mirror filling technique is applied based on the size and shape of the cross-component adaptive loop filter.

[0789] 8. The method according to clause 1, where the cross-component adaptive loop filter has a K×L filter shape, where K is an even number and L is a positive integer, and where the mirror filling technique includes filling the unavailable samples located on the line M or N along the second direction at a distance from the virtual boundary of the cross-component adaptive loop filter, and the unavailable samples are filled from the closest sample line close to the virtual boundary.

[0790] 9. The method according to clause 3, where, in the case where the virtual boundary of the cross-component adaptive loop filter is below the representative luminance sample in the second direction, then the closest line above the virtual boundary in the second direction is used to fill the unavailable samples.

[0791] 10. The method according to clause 3, where, in the case where the unavailable sample is located in row M, where M is an integer less than C, where C is an integer indicating the center line of the representative luminance sample along the second direction, then when d(C, M) = d(N, C) – offset, the sample located in line N in the second direction is determined as the corresponding sample, where offset is an integer value, or d(C, M) < d(N, C), where d() is a distance function.

[0792] The following clauses illustrate example embodiments of the techniques discussed in the previous sections (e.g., item 2).

[0793] 11. The method according to any one of clauses 1-10, where the first direction is the vertical direction and the second direction is the horizontal direction.

[0794] 12. The method according to any one of clauses 1-10, where the first direction is the horizontal direction and the second direction is the vertical direction. <00​​​​​​​​​

[0798] 15. The method according to any one of Clauses 1-14, wherein performing the conversion includes encoding the video to generate a codec representation.

[0799] 16. The method according to any one of Clauses 1-14, wherein performing the conversion includes parsing and decoding the codec representation to generate video.

[0800] In the terms disclosed above, the orientation can be horizontal or vertical, and correspondingly the first and second directions can be vertical or horizontal directions referred to as pixel columns and pixel rows.

[0801] 17. A video decoding device including a processor configured to implement one or more of the methods described in clauses 1 to 16.

[0802] 18. A video encoding apparatus including a processor configured to implement one or more of the methods described in clauses 1 to 16.

[0803] 19. A computer program product having computer code stored thereon, which, when executed by a processor, causes the processor to perform any one of the methods described in any one of clauses 1 to 16.

[0804] 20. The methods, devices, or systems described in this document.

[0805] The second set of clauses illustrates example embodiments of the techniques discussed in the previous chapters (e.g., items 1-3 of the example embodiments).

[0806] 1. A method for video processing (e.g., Figure 38A The method 3810 includes: for the conversion between video units and bitstream representations of video, determining 3812 whether a mirror fill process for filling unavailable luminance samples is enabled for the video units during the application of a loop filter tool; and performing 3814 a conversion based on the determination.

[0807] 2. The method according to Clause 1, wherein the loop filtering tool includes the cross-component adaptive loop filtering (CC-ALF) tool.

[0808] 3. The method according to Clause 1, wherein the loop filtering tool includes an adaptive loop filtering (ALF) tool.

[0809] 4. The method according to Clause 1, wherein the mirror filling process includes filling a second sample point, which is the corresponding sample point of the first sample point in the filter support area of ​​the loop filter tool, even when the second sample point is available, and wherein the first sample point is unavailable and needs to be filled.

[0810] 5. The method according to Clause 1, wherein the mirror fill process is further used to derive unavailable luminance samples and one or more corresponding luminance samples of the unavailable luminance samples.

[0811] 6. The method according to Clause 5, wherein the corresponding luminance sample is determined based on the distance of the corresponding luminance sample from the representative luminance sample and / or the distance of the unavailable sample from the representative luminance sample.

[0812] 7. The method according to Clause 6, wherein a representative luminance sample is defined as a co-occurring luminance sample of the chrominance samples to be filtered.

[0813] 8. The method according to Clause 7, wherein the position of the co-bit luminance sample of the chroma sample depends on the color format of the video.

[0814] 9. The method according to Clause 8, wherein the co-occurring luminance sample of the chrominance sample at (x, y) is defined in color format 4:2:0 as the sample at (2x, 2y).

[0815] 10. The method according to Clause 8, wherein the co-occurring luminance sample of the chrominance sample located at (x, y) is defined in color format 4:2:2 as the sample located at (2x, y).

[0816] 11. The method according to Clause 8, wherein the co-occurring luminance sample of the chrominance sample located at (x, y) is defined in color format 4:4:4 as the sample located at (x, y).

[0817] 12. The method according to Clause 6, wherein distance refers to the distance along a first direction between a first row containing corresponding luminance samples and a second row containing representative luminance samples.

[0818] 13. The method according to Clause 6, wherein the distance is calculated as the difference between a pixel line containing corresponding luminance samples along a first direction and a row containing representative luminance samples along a second direction.

[0819] 14. The method according to Clause 13, wherein C represents the center line where a representative luminance sample point is located along the second direction, M represents the line where an unavailable sample point is located along the second direction, and N represents the line where the corresponding luminance sample point is located along the second direction, wherein C, M, and N are positive integers and M is not equal to N.

[0820] 15. The method according to Clause 5, wherein one or more corresponding luminance samples to be filled during the mirror fill process are based on the number of rows of samples adopted by the shape of the filter used by the CC-ALF tool.

[0821] 16. The method according to Clause 5, wherein unavailable luminance samples are located in row M, and samples located in row N are determined as one or more corresponding luminance samples to be filled, and wherein d(C, M) = d(N, C), where d(x, y) refers to the distance between row x and row y, and M, C, and N are positive integers.

[0822] 17. The method according to Clause 6, wherein in the case that 1) the center luminance sample is selected as the representative luminance sample and 2) the CC-ALF tool has a K×L filter shape, the mirror fill process corresponds to the mirror fill process used during the application of the Adaptive Loop Filter (ALF) tool, where K is an odd number and L is a positive integer.

[0823] 18. The method according to Clause 6, wherein, in the case where the CC-ALF tool has a K×L filter shape and unavailable luminance samples are located above row M or below row N of the virtual boundary, the mirror filling process comprises filling one or more corresponding luminance samples from the nearest sample row above row N or below row M located at a distance of row N or M from the virtual boundary, where K is an odd number and L, M, and N are positive integers.

[0824] 19. The method according to Clause 18, wherein when the virtual boundary is above a representative luminance sample, unavailable luminance samples are filled with the nearest row below the virtual boundary, and wherein when the virtual boundary is above a representative luminance sample, one or more corresponding luminance samples are filled with the nearest row above the row containing one or more corresponding luminance samples.

[0825] 20. The method according to Clause 18, wherein when the virtual boundary is below the representative luminance sample, unavailable luminance samples are filled with the nearest row above the virtual boundary, and wherein when the virtual boundary is below the representative luminance sample, one or more corresponding luminance samples are filled with the nearest row below the row containing one or more corresponding luminance samples.

[0826] 21. The method according to Clause 19 or 20, wherein K = 2, yL = 0, yP1 = 1, and the virtual boundary is equal to CtbSizeY - 4, where yL and yP1 are the y coordinates of the two sample rows, and CtbSizeY represents the size of the codec tree unit (CTU).

[0827] 22. The method according to Clause 21, wherein, when yL equals CtbSizeY–4 and the row above yL is unavailable, the sample point (x, yP1) in the row at yP1 is filled with the sample point (x, yL) in the row at yL.

[0828] 23. The method according to Clause 19 or 20, wherein K = 4, yM1 = -1, yL = 0, yP1 = 1, yP2 = 2, and the virtual boundary is equal to CtbSizeY - 4, where yM1, yL, yP1, and yP2 are the y-coordinates of the four sample rows, and CtbSizeY represents the size of the codec tree unit (CTU).

[0829] 24. The method according to Clause 23, wherein, when yL equals CtbSizeY–3 and the row above yM1 is unavailable, the sample point (x, yP2) in the row at yP2 is filled with the sample point (x, yP1) in the row at yP1.

[0830] 25. The method according to Clause 23, wherein, when yL equals CtbSizeY–4 and the row above yM1 is unavailable, the sample point (x, yM1) in the row at yM1 is filled with the sample point (x, yL) in the row at yL, and the sample points (x, yP2) and (x, yP1) in the rows at yP2 and yP1 are filled with the sample point (x, yL) in the row at yL.

[0831] 26. The method according to Clause 19 or 20, wherein K = 6, yM2 = -2, yM1 = -1, yL = 0, yP1 = 1, yP2 = 2, yP3 = 3, and the virtual boundary is equal to CtbSizeY - 4, where yM2, yM1, yL, yP1, yP2, and yP3 are the y-coordinates of the six sample rows, and CtbSizeY represents the size of the codec tree unit (CTU).

[0832] 27. The method according to Clause 26, wherein, when yL equals CtbSizeY–2 and the row above yM2 is unavailable, the sample point (x, yP3) in the row at yP3 is filled with the sample point (x, yP2) in the row at yP2.

[0833] 28. The method according to Clause 26, wherein, when yL equals CtbSizeY–3 and the rows above yM2 and at yM2 are unavailable, the sample (x, yM2) in the row at yM2 is filled with the sample (x, yM1) in the row at yM1, and the samples (x, yP3) and (x, yP2) in the rows at yP3 and yP2 are filled with the sample (x, yP1) in the row at yP1.

[0834] 29. The method according to Clause 26, wherein, when yL equals CtbSizeY–4 and the rows above yM2, the rows at yM2 and yM1 are unavailable, the samples (x, yM2) and (x, yM1) in the rows at yM2 and yM1 are filled with the sample (x, yL) in the row at yL, and the samples (x, yP3), (x, yP2) and (x, yP1) in rows yP3, yP2 and yP1 are filled with the sample (x, yL) in the row at yL.

[0835] 30. The method according to Clause 21, wherein, in the case that yL equals CtbSizeY–5 and the row at yP1 is unavailable, the sample point (x, yP1) in the row at yP1 is filled with the sample point (x, yL) in the row at yL.

[0836] 31. The method according to Clause 23, wherein, in the case that yL equals CtbSizeY–6 and the row at yP2 is unavailable, the sample point (x, yP2) in the row at yP2 is filled with the sample point (x, yP1) in the row at yP1.

[0837] 32. The method according to Clause 23, wherein, when yL equals CtbSizeY–5 and rows at yP2 and yP1 are unavailable, the sample point (x, yP1) in the row at yP1 and the sample point (x, yP2) in the row at yP2 are filled with the sample point (x, yL) in the row at yL, and the sample point (x, yM1) in the row at yM1 is filled with the sample point (x, yL) in the row at yL.

[0838] 33. The method according to Clause 26, wherein, in the case that yL equals CtbSizeY–7 and the row at yP3 is unavailable, the sample (x, yP3) in the row at yP3 is filled with the sample (x, yP2) in the row at yP2.

[0839] 34. The method according to Clause 26, wherein, when yL equals CtbSizeY–6 and the rows at yP3 and yP2 are unavailable, the sample point (x, yP3) in the row at yP3 and the sample point (x, yP2) in the row at yP2 are filled with the sample point (x, yP1) in the row at yP1, and the sample point (x, yM2) is filled with the sample point (x, yM1) in the row at yM1.

[0840] 35. The method according to clause 26, wherein, when yL is equal to CtbSizeY–5 and the lines at yP3, yP2, and yP1 are unavailable, the samples (x, yP3) in the line at yP3, the samples (x, yP2) in the line at yP2, and the samples (x, yP1) in the line at yP1 are filled with the samples (x, yL) in the line at yL, and the samples (x, yM2) in the line at yM2 and the samples (x, yM1) in the line at yM1 are filled with the samples (x, yL) in the line at yL.

[0841] 36. The method according to clause 5, wherein the unavailable luminance samples are located in row M, and the samples in row N are determined as one or more corresponding luminance samples to be filled, and wherein d(C, M) = d(N, C)-offset or d(C, M) < d(N, C), where d(x, y) refers to the distance between row x and row y, offset is an integer, and M, C, N are positive integers.

[0842] 37. The method according to clause 5, wherein the unavailable luminance samples are located in row M, and the samples in row N are determined as one or more corresponding luminance samples to be filled, and wherein d(M, C) = d(C, N)–offset or d(C, M) < d(N, C), where d(x, y) refers to the distance between row x and row y, offset is an integer, and M, C, N are positive integers.

[0843] 38. The method according to clause 36 or 37, wherein offset is equal to 1.

[0844] 39. The method according to clause 36, wherein, when the unavailable luminance samples are located in a row above row M or below row N from the virtual boundary, the mirror filling process includes filling one or more corresponding luminance samples from the closest sample row above row N or below row M to the virtual boundary below row N or above row M, where M and N are positive integers.

[0845] 40. The method according to clause 39, wherein, when the virtual boundary is below the representative luminance sample, the unavailable luminance samples are filled with the closest row above the virtual boundary, and one or more corresponding luminance samples are filled with the closest row below the row containing one or more corresponding luminance samples.

[0846] 41. The method according to clause 39, wherein, when the virtual boundary is above the representative luminance sample, the unavailable luminance samples are filled with the closest row below the virtual boundary, and one or more corresponding samples are filled with the closest row above the row containing one or more corresponding luminance samples.

[0847] 42. The method according to clause 40 or 41, wherein K = 2, yL = 0, yP1 = 1, and the virtual boundary is equal to CtbSizeY - 4, where yL and yP1 are the y coordinates of the two sample rows, and CtbSizeY represents the size of the codec tree unit (CTU).

[0848] 43. The method according to Clause 42, wherein, when yL equals CtbSizeY–5 and the row at yP1 is unavailable, the sample point (x, yP1) in the row at yP1 is filled with the sample point (x, yL) in the row at yL.

[0849] 44. The method according to clause 40 or 41, wherein K = 4, yM1 = -1, yL = 0, yP1 = 1, yP2 = 2, and the virtual boundary is equal to CtbSizeY - 4, where yM1, yL, yP1, yP2 are the y coordinates of the four sample rows, and CtbSizeY represents the size of the codec tree unit (CTU).

[0850] 45. The method according to Clause 44, wherein, when yL equals CtbSizeY–4 and the rows above yM1 and at yM1 are unavailable, the sample point (x, yM1) in the row at yM1 is filled with the sample point (x, yL) in the row at yL, and the sample point (x, yP2) in the row at yP2 is filled with the sample point (x, yP1) in the row at yP1.

[0851] 46. ​​The method according to Clause 44, wherein, in the case that yL equals CtbSizeY–6 and the row at yP2 is unavailable, the sample point (x, yP2) in the row at yP2 is filled with the sample point (x, yP1) in the row at yP1.

[0852] 47. The method according to Clause 44, wherein, when yL equals CtbSizeY–5 and rows at yP2 and yP1 are unavailable, the sample point (x, yP2) at row yP2 is filled with the sample point (x, yL) in the row at yL, and the sample point (x, yM1) in the row at yM1 is filled with the sample point (x, yL) in the row at yL.

[0853] 48. The method according to clause 40 or 41, wherein K = 6, yM2 = -2, yM1 = -1, yL = 0, yP1 = 1, yP2 = 2, yP3 = 3, and the virtual boundary is equal to CtbSizeY - 4, where yM2, yM1, yL, yP1, yP2, and yP3 are the y-coordinates of the six sample rows, and CtbSizeY represents the size of the codec tree unit (CTU).

[0854] 49. The method according to Clause 48, wherein, when yL equals CtbSizeY–3 and the rows above yM2 and at yM2 are unavailable, the sample point (x, yM2) in the row at yM2 is filled with the sample point (x, yM1) in the row at yM1, and the sample point (x, yP3) in the row at yP3 is filled with the sample point (x, yP2) in the row at yP2.

[0855] 50. The method according to Clause 48, wherein, when yL equals CtbSizeY–4 and the rows above yM2, the rows at yM2 and yM1 are unavailable, the samples (x, yM2) and (x, yM1) in the rows at yM2 and yM1 are filled with the sample (x, yL) in the row at yL, and the samples (x, yP3) and (x, yP2) in rows yP3 and yP2 are filled with the sample (x, yP1) in the row at yP1.

[0856] 51. The method according to Clause 48, wherein, when yL equals CtbSizeY–7 and the row at yP3 is unavailable, the sample point (x, yP3) in the row at yP3 is filled with the sample point (x, yP2) in the row at yP2, and the sample point (x, yM2) in the row at yM2 is filled with the sample point (x, yM1) in the row at yM1.

[0857] 52. The method according to Clause 48, wherein, when yL equals CtbSizeY–6 and the rows at yP3 and yP2 are unavailable, the sample points (x, yP3) in the row at yP3 and (x, yP2) in the row at yP2 are filled with the sample points (x, yP1) in the row at yP1, and the sample points (x, yM2) in the row at yM2 and (x, yM1) in the row at yM1 are filled with the sample points (x, yL) in the row at yL.

[0858] 53. The method according to Clause 48, wherein, when yL equals CtbSizeY–5 and rows at yP3, yP2 and yP1 are unavailable, the sample point (x, yP3) in the row at yP3, the sample point (x, yP2) in the row at yP2 and the sample point (x, yP1) in the row at yP1 are filled with the sample point (x, yL) in the row at yL, and the sample point (x, yM2) in the row at yM2 and the sample point (x, yM1) in the row at yM1 are filled with the sample point (x, yL) in the row at yL.

[0859] 54. The method according to Clause 1, wherein the determined result is included in the bitstream representation at the sequence level, picture level, strip level, or slice level.

[0860] 55. The method according to any one of the preceding clauses, wherein the first direction is a vertical direction and the second direction is a horizontal direction.

[0861] 56. The method according to any one of the preceding clauses, wherein the first direction is horizontal and the second direction is vertical.

[0862] 57. The method according to clause 55 or 56, wherein the orientation of the first direction and the second direction depends on the orientation of the boundary of the virtual buffer.

[0863] 58. A method for video processing (e.g., Figure 38B The method 3820 includes: for the conversion between video units and bitstream representations of a video, determining 3822 whether to apply a repeat padding process and / or a mirror padding process to pad samples located at virtual boundaries based on encoding and decoding information of the video units; and performing 3824 conversion based on the determination.

[0864] 59. The method according to Clause 58, wherein the encoding / decoding information includes the size of a video unit as a codec tree unit (CTU) or codec tree block (CTB).

[0865] 60. The method according to Clause 59, wherein a mirror fill process is applied when the CTU or CTB size is greater than or equal to T, where T is a positive integer.

[0866] 61. The method according to Clause 59, wherein a repeated filling process is applied when the CTU or CTB size is less than or equal to T, where T is a positive integer.

[0867] 62. The method according to any one of the preceding clauses, wherein the video unit includes a picture, sub-picture, strip, slice, or 360-degree boundary of the video.

[0868] 63. The method according to any one of the preceding clauses, wherein, in the CC-ALF tool, the sample value of a video unit of a video component is predicted from the sample value of a video unit of another video component.

[0869] 64. The method according to any one of clauses 1 to 63, wherein the conversion comprises encoding the video into a bitstream representation.

[0870] 65. The method according to any one of clauses 1 to 63, wherein the conversion comprises decoding video from a bitstream representation.

[0871] 66. A video processing apparatus including a processor configured to implement the method according to any one or more of clauses 1 to 65.

[0872] 67. A computer-readable medium storing program code that, when executed, causes a processor to perform the method according to any one or more of clauses 1 to 65.

[0873] 68. A computer-readable medium storing an encoded or decoded representation or a bitstream representation generated according to any of the methods described above.

[0874] In this document, the term "video processing" can refer to video encoding, video decoding, video compression, or video decompression. For example, a video compression algorithm can be applied during the conversion from a pixel representation of a video to its corresponding bitstream representation, and vice versa. The bitstream representation of the current video block can, for example, correspond to co-occurring or distributed bits within the bitstream, as defined in the syntax. For example, a macroblock can be encoded with transformed and encoded error residuals, and also uses bits from the header and other fields in the bitstream.

[0875] The disclosed and other solutions, examples, embodiments, modules, and functional operations described in this document can be implemented in digital electronic circuits or computer software, firmware, or hardware, including the structures disclosed in this document and their structural equivalents, or combinations thereof. The disclosed and other embodiments can be implemented as one or more computer program products, i.e., one or more computer program instruction modules encoded on a computer-readable medium for execution by or control of the operation of a data processing device. The computer-readable medium can be a machine-readable storage device, a machine-readable storage substrate, a memory device, a composition of substances influencing machine-readable propagation signals, or combinations thereof. The term "data processing apparatus" encompasses all means, devices, and machines for processing data, including, for example, a programmable processor, a computer, or multiple processors or computers. In addition to hardware, the apparatus may also include code that creates an execution environment for the computer program in question, such as code constituting processor firmware, a protocol stack, a database management system, an operating system, or a combination thereof. Propagation signals are artificially generated signals, such as machine-generated electrical, optical, or electromagnetic signals, generated to encode information for transmission to a suitable receiver device.

[0876] A computer program (also called a program, software, software application, script, or code) can be written in any programming language, including compiled or interpreted languages, and can be deployed in any form, including as a standalone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. A program can be stored as a portion of a file containing other programs or data (e.g., one or more scripts stored in a markup language document), a single file dedicated to the program in question, or multiple coordinating files (e.g., a file storing one or more modules, subroutines, or portions of code). Computer programs can be deployed to execute on a single computer or on multiple computers located at a site or distributed across multiple sites and interconnected via a communication network.

[0877] The processes and logic flows described herein can be executed by one or more programmable processors, which execute one or more computer programs to perform functions by manipulating input data and generating output. The processes and logic flows can also be executed by dedicated logic circuits, and the devices can be implemented as dedicated logic circuits, such as FPGAs (Field-Programmable Gate Arrays) or ASICs (Application-Specific Integrated Circuits).

[0878] Processors suitable for executing computer programs include, for example, general-purpose and special-purpose microprocessors, and any one or more processors of any type of digital computer. Typically, a processor receives instructions and data from read-only memory or random access memory, or both. The basic components of a computer are a processor for executing instructions and one or more storage devices for storing instructions and data. Typically, a computer will also include, or be operatively coupled to, receiving data from or transferring data to one or more mass storage devices (e.g., magneto-optical, magneto-optical, or optical discs) for storing data, or both. However, a computer does not need to have such devices. Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, such as semiconductor memory devices like EPROM, EEPROM, and flash memory devices; magnetic disks, such as internal hard disks or removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks. The processor and memory may be supplemented by or integrated into special-purpose logic circuitry.

[0879] While this patent document contains numerous details, these should not be construed as limiting the scope of any subject matter or potentially claimed content, but rather as descriptions of features specific to particular embodiments that may be specific to particular technologies. Certain features described in the context of individual embodiments in this patent document may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments. Furthermore, although features may be described as functioning in some combinations, and even initially claimed in this way, in certain circumstances, one or more features from a claimed combination may be removed from the combination, and a claimed combination may be for a sub-combination or a sub-combination of variations.

[0880] Similarly, although operations are described in a specific order in the accompanying drawings, this should not be construed as requiring such operations to be performed in the specific order shown or sequentially, or to perform all of the shown operations to achieve the desired result. Furthermore, the separation of various system components in the embodiments described in this patent document should not be construed as requiring such separation in all embodiments.

[0881] Only some implementation methods and examples are described, and other implementation methods, improvements and variations can be made based on the content described and explained in this patent document.

Claims

1. A method for processing video data, comprising: For the conversion between video units and the bitstream of the video, during the application of the cross-component adaptive loop filter CC-ALF tool to the video units, it is determined that a mirror fill process is used to fill unusable luminance samples that are unavailable due to the virtual boundaries when applying virtual boundaries to the video units. as well as The conversion is performed based on the determination. The mirror filling process includes filling the corresponding samples of the unavailable luminance samples and treating the corresponding samples as unavailable even if they are available for the virtual boundary, so that the corresponding samples are located in the filter support region of the CC-ALF tool. Wherein, the unavailable luminance sample is represented as row M in the video unit, and the corresponding sample that needs to be filled is represented as row N in the video unit, where d(C,M)=d(N,C), where d(x,y) represents the distance between row x and row y, where C represents the row in the video unit where the representative luminance sample is located, where C, M and N are integers, M is not equal to N, and where the representative luminance sample is the co-occurring luminance sample of the chrominance sample that needs to be filtered in the video unit.

2. The method according to claim 1, wherein, The position of the co-occurring luminance sample of the chroma sample depends on the color format of the video.

3. The method according to claim 2, wherein, The luminance sample of the chrominance sample at (x, y) is defined as the sample at (2x, 2y) when the color format is 4:2:0, or as the sample at (2x, y) when the color format is 4:2:2, or as the sample at (x, y) when the color format is 4:4:

4.

4. The method according to claim 1, wherein, The CC-ALF tool has a K×L filter shape, where K = 4 and L = 3; yM1, yL, yP1, and yP2 are the y-coordinates of four sample rows in the K×L filter shape, where yM1 = -1, yL = 0, yP1 = 1, and yP2 = 2, and the representative luminance sample is located in row yL = 0; and the virtual boundary is located in row CtbSizeY-4 in the video unit, where CtbSizeY represents the size of the video unit as a codec tree unit (CTU).

5. The method according to claim 4, wherein, When the virtual boundary is above the representative brightness sample point, the unavailable brightness sample point is filled with the nearest available row below the virtual boundary, and the corresponding sample point is filled with the nearest available row above the row containing the corresponding sample point.

6. The method according to claim 5, wherein, When yL equals CtbSizeY–3 and the row above yM1 is unavailable, the corresponding sample (x, yP2) in the row at yP2 is filled with the sample (x, yP1) in the row at yP1.

7. The method according to claim 5, wherein, When yL equals CtbSizeY–4 and the rows above yM1 and at yM1 are unavailable, the unavailable luminance sample (x, yM1) in the row at yM1 is filled with the sample (x, yL) in the row at yL, the corresponding sample (x, yP1) in the row at yP1 is filled with the sample (x, yL) in the row at yL, and the corresponding sample (x, yP2) in the row at yP2 is filled with the sample (x, yL) in the row at yL.

8. The method according to claim 4, wherein, When the virtual boundary is below the representative brightness sample point, the unavailable brightness sample point is filled with the nearest available row above the virtual boundary, and the corresponding sample point is filled with the nearest available row below the row containing the corresponding sample point.

9. The method according to claim 8, wherein, When yL equals CtbSizeY–6 and the row at yP2 is unavailable, the unavailable luminance sample (x, yP2) in the row at yP2 is filled with the sample (x, yP1) in the row at yP1.

10. The method according to claim 8, wherein, When yL equals CtbSizeY–5 and the rows at yP2 and yP1 are unavailable, the unavailable luminance sample (x, yP2) in the row at yP2 is filled with the sample (x, yL) in the row at yL, the unavailable luminance sample (x, yP1) in the row at yP1 is filled with the sample (x, yL) in the row at yL, and the corresponding sample (x, yM1) in the row at yM1 is filled with the sample (x, yL) in the row at yL.

11. The method according to claim 1, wherein, The conversion includes encoding the video into the bitstream.

12. The method according to claim 1, wherein, The conversion includes decoding the video from the bitstream.

13. An apparatus for processing video data, comprising a processor and a non-transitory memory having instructions thereon, wherein, When the instruction is executed by the processor, the processor: For the conversion between video units and the bitstream of the video, during the application of the cross-component adaptive loop filter CC-ALF tool to the video units, it is determined that a mirror fill process is used to fill unusable luminance samples that are unavailable due to the virtual boundaries when applying virtual boundaries to the video units. as well as The conversion is performed based on the determination. The mirror filling process includes filling the corresponding samples of the unavailable luminance samples and treating the corresponding samples as unavailable even if they are available for the virtual boundary, so that the corresponding samples are located in the filter support region of the CC-ALF tool. Wherein, the unavailable luminance sample is represented as row M in the video unit, and the corresponding sample that needs to be filled is represented as row N in the video unit, where d(C,M)=d(N,C), where d(x,y) represents the distance between row x and row y, where C represents the row in the video unit where the representative luminance sample is located, where C, M and N are integers, M is not equal to N, and where the representative luminance sample is the co-occurring luminance sample of the chrominance sample that needs to be filtered in the video unit.

14. The apparatus according to claim 13, wherein, The position of the co-occurring luminance sample of the chroma sample depends on the color format of the video; Wherein, the luminance sample of the co-occurring chrominance sample at (x, y) is defined as the sample at (2x, 2y) when the color format is 4:2:0, or as the sample at (2x, y) when the color format is 4:2:2, or as the sample at (x, y) when the color format is 4:4:

4.

15. The apparatus according to claim 13, wherein, The CC-ALF tool has a K×L filter shape, where K = 4 and L = 3; yM1, yL, yP1, and yP2 are the y-coordinates of four sample rows in the K×L filter shape, where yM1 = -1, yL = 0, yP1 = 1, and yP2 = 2, and the representative luminance sample is located in row yL = 0; and the virtual boundary is located in row CtbSizeY-4 in the video unit, where CtbSizeY represents the size of the video unit as a codec tree unit (CTU). Wherein, when the virtual boundary is above the representative brightness sample point, the unavailable brightness sample point is filled with the nearest available row below the virtual boundary, and wherein the corresponding sample point is filled with the nearest available row above the row containing the corresponding sample point, wherein, when yL equals CtbSizeY–3 and the row above yM1 is unavailable, the corresponding sample point (x, yP2) in the row at yP2 is filled with the sample point (x, yP1) in the row at yP1; or wherein, when yL equals CtbSizeY–4 and the row above yM1 and the row at yM1 are unavailable, the unavailable brightness sample point (x, yM1) in the row at yM1 is filled with the sample point (x, yL) in the row at yL, the corresponding sample point (x, yP1) in the row at yP1 is filled with the sample point (x, yL) in the row at yL, and the corresponding sample point (x, yP2) in the row at yP2 is filled with the sample point (x, yL) in the row at yL; or Wherein, when the virtual boundary is below the representative brightness sample, the unavailable brightness sample is filled with the nearest available row above the virtual boundary, and the corresponding sample is filled with the nearest available row below the row containing the corresponding sample, wherein, when yL equals CtbSizeY–6 and the row at yP2 is unavailable, the unavailable brightness sample (x, yP2) in the row at yP2 is filled with the sample (x, yP1) in the row at yP1; or wherein, when yL equals CtbSizeY–5 and the rows at yP2 and yP1 are unavailable, the unavailable brightness sample (x, yP2) in the row at yP2 is filled with the sample (x, yL) in the row at yL, the unavailable brightness sample (x, yP1) in the row at yP1 is filled with the sample (x, yL) in the row at yL, and the corresponding sample (x, yM1) in the row at yM1 is filled with the sample (x, yL) in the row at yL.

16. A non-transitory computer-readable storage medium for storing instructions, said instructions causing a processor to: For the conversion between video units and the video bitstream, during the application of the Cross-Component Adaptive Loop Filtering (CC-ALF) tool to the video units, it is determined that a mirror fill process should be used when applying virtual boundaries to the video units to fill unusable luminance samples that are unavailable due to the virtual boundaries; and The conversion is performed based on the determination. in, The mirror filling process includes filling the corresponding samples of the unavailable luminance samples and treating the corresponding samples as unavailable even if they are available for the virtual boundary, so that the corresponding samples are located in the filter support region of the CC-ALF tool. Wherein, the unavailable luminance sample is represented as row M in the video unit, and the corresponding sample that needs to be filled is represented as row N in the video unit, where d(C,M)=d(N,C), where d(x,y) represents the distance between row x and row y, where C represents the row in the video unit where the representative luminance sample is located, where C, M and N are integers, M is not equal to N, and where the representative luminance sample is the co-occurring luminance sample of the chrominance sample that needs to be filtered in the video unit.

17. The non-transitory computer-readable storage medium according to claim 16, wherein, The position of the co-occurring luminance sample of the chroma sample depends on the color format of the video; Wherein, the luminance sample of the co-occurring chrominance sample at (x, y) is defined as the sample at (2x, 2y) when the color format is 4:2:0, or as the sample at (2x, y) when the color format is 4:2:2, or as the sample at (x, y) when the color format is 4:4:

4.

18. The non-transitory computer-readable storage medium according to claim 16, wherein, The CC-ALF tool has a K×L filter shape, where K = 4 and L = 3; yM1, yL, yP1, and yP2 are the y-coordinates of four sample rows in the K×L filter shape, where yM1 = -1, yL = 0, yP1 = 1, and yP2 = 2, and the representative luminance sample is located in row yL = 0; and the virtual boundary is located in row CtbSizeY-4 in the video unit, where CtbSizeY represents the size of the video unit as a codec tree unit (CTU). Wherein, when the virtual boundary is above the representative brightness sample point, the unavailable brightness sample point is filled with the nearest available row below the virtual boundary, and wherein the corresponding sample point is filled with the nearest available row above the row containing the corresponding sample point, wherein, when yL equals CtbSizeY–3 and the row above yM1 is unavailable, the corresponding sample point (x, yP2) in the row at yP2 is filled with the sample point (x, yP1) in the row at yP1; or wherein, when yL equals CtbSizeY–4 and the row above yM1 and the row at yM1 are unavailable, the unavailable brightness sample point (x, yM1) in the row at yM1 is filled with the sample point (x, yL) in the row at yL, the corresponding sample point (x, yP1) in the row at yP1 is filled with the sample point (x, yL) in the row at yL, and the corresponding sample point (x, yP2) in the row at yP2 is filled with the sample point (x, yL) in the row at yL; or Wherein, when the virtual boundary is below the representative brightness sample, the unavailable brightness sample is filled with the nearest available row above the virtual boundary, and the corresponding sample is filled with the nearest available row below the row containing the corresponding sample, wherein, when yL equals CtbSizeY–6 and the row at yP2 is unavailable, the unavailable brightness sample (x, yP2) in the row at yP2 is filled with the sample (x, yP1) in the row at yP1; or wherein, when yL equals CtbSizeY–5 and the rows at yP2 and yP1 are unavailable, the unavailable brightness sample (x, yP2) in the row at yP2 is filled with the sample (x, yL) in the row at yL, the unavailable brightness sample (x, yP1) in the row at yP1 is filled with the sample (x, yL) in the row at yL, and the corresponding sample (x, yM1) in the row at yM1 is filled with the sample (x, yL) in the row at yL.

19. A non-transitory computer-readable recording medium for storing a bitstream of video generated by a method performed by a video processing apparatus, wherein, The method includes: During the application of the Cross-Component Adaptive Loop Filtering (CC-ALF) tool to the video units of the video, it is determined that a mirror fill process should be used when applying virtual boundaries to the video units to fill unavailable luminance samples that are unavailable due to the virtual boundaries; and The bit stream is generated based on the determination. The mirror filling process includes filling the corresponding samples of the unavailable luminance samples and treating the corresponding samples as unavailable even if they are available for the virtual boundary, so that the corresponding samples are located in the filter support region of the CC-ALF tool. Wherein, the unavailable luminance sample is represented as row M in the video unit, and the corresponding sample that needs to be filled is represented as row N in the video unit, where d(C,M)=d(N,C), where d(x,y) represents the distance between row x and row y, where C represents the row in the video unit where the representative luminance sample is located, where C, M and N are integers, M is not equal to N, and where the representative luminance sample is the co-occurring luminance sample of the chrominance sample that needs to be filtered in the video unit.

20. The non-transitory computer-readable recording medium according to claim 19, wherein, The position of the co-occurring luminance sample of the chroma sample depends on the color format of the video; Wherein, the luminance sample of the co-occurring chrominance sample at (x, y) is defined as the sample at (2x, 2y) when the color format is 4:2:0, or as the sample at (2x, y) when the color format is 4:2:2, or as the sample at (x, y) when the color format is 4:4:

4. The CC-ALF tool has a K×L filter shape, where K = 4 and L = 3; yM1, yL, yP1, and yP2 are the y-coordinates of four sample rows in the K×L filter shape, where yM1 = -1, yL = 0, yP1 = 1, and yP2 = 2, and the representative luminance sample is located in row yL = 0; and the virtual boundary is located in row CtbSizeY-4 in the video unit, where CtbSizeY represents the size of the video unit as a codec tree unit (CTU). Wherein, when the virtual boundary is above the representative brightness sample point, the unavailable brightness sample point is filled with the nearest available row below the virtual boundary, and wherein the corresponding sample point is filled with the nearest available row above the row containing the corresponding sample point, wherein, when yL equals CtbSizeY–3 and the row above yM1 is unavailable, the corresponding sample point (x, yP2) in the row at yP2 is filled with the sample point (x, yP1) in the row at yP1; or wherein, when yL equals CtbSizeY–4 and the row above yM1 and the row at yM1 are unavailable, the unavailable brightness sample point (x, yM1) in the row at yM1 is filled with the sample point (x, yL) in the row at yL, the corresponding sample point (x, yP1) in the row at yP1 is filled with the sample point (x, yL) in the row at yL, and the corresponding sample point (x, yP2) in the row at yP2 is filled with the sample point (x, yL) in the row at yL; or Wherein, when the virtual boundary is below the representative brightness sample, the unavailable brightness sample is filled with the nearest available row above the virtual boundary, and the corresponding sample is filled with the nearest available row below the row containing the corresponding sample, wherein, when yL equals CtbSizeY–6 and the row at yP2 is unavailable, the unavailable brightness sample (x, yP2) in the row at yP2 is filled with the sample (x, yP1) in the row at yP1; or wherein, when yL equals CtbSizeY–5 and the rows at yP2 and yP1 are unavailable, the unavailable brightness sample (x, yP2) in the row at yP2 is filled with the sample (x, yL) in the row at yL, the unavailable brightness sample (x, yP1) in the row at yP1 is filled with the sample (x, yL) in the row at yL, and the corresponding sample (x, yM1) in the row at yM1 is filled with the sample (x, yL) in the row at yL.

21. A method for storing a bitstream of video, comprising: During the application of the Cross-Component Adaptive Loop Filtering (CC-ALF) tool to the video units of the video, it is determined that a mirror fill process is used when applying virtual boundaries to the video units to fill unavailable luminance samples that are unavailable due to the virtual boundaries. The bit stream is generated based on the determination; as well as The bitstream is stored in a non-transitory computer-readable recording medium. The mirror filling process includes filling the corresponding samples of the unavailable luminance samples and treating the corresponding samples as unavailable even if they are available for the virtual boundary, so that the corresponding samples are located in the filter support region of the CC-ALF tool. Wherein, the unavailable luminance sample is represented as row M in the video unit, and the corresponding sample that needs to be filled is represented as row N in the video unit, where d(C,M)=d(N,C), where d(x,y) represents the distance between row x and row y, where C represents the row in the video unit where the representative luminance sample is located, where C, M and N are integers, M is not equal to N, and where the representative luminance sample is the co-occurring luminance sample of the chrominance sample that needs to be filtered in the video unit.

22. The method according to any one of claims 2-10, wherein, The conversion includes encoding the video into the bitstream.

23. The method according to any one of claims 2-10, wherein, The conversion includes decoding the video from the bitstream.

Images