Method, device and storage medium for processing video data

Abstract

A video processing method is described. The method includes: for a conversion between a current video unit of a video including one or more video blocks and a bitstream representation of the video, determining a filling process according to a rule, the filling process is used to fill unavailable samples during the application of a cross-component adaptive loop filtering (CC-ALF) tool to at least some of the video blocks of the current video unit; and performing the conversion based on the determination, and wherein the rule specifies that the filling process is also used to fill the unavailable samples during the application of the adaptive loop filtering (ALF) tool to the one or more video blocks of the current video unit.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application promptly claims priority and benefits under applicable patent law and / or the rules of the Paris Convention to International Patent Application No. PCT / CN2019 / 115321 filed on November 4, 2019. The entire disclosure of the above application is incorporated by reference as part of the disclosure of this application for all purposes prescribed by law. Technical Field

[0003] This patent document relates to video encoding and decoding technology, devices and systems. Background Art

[0004] Currently, efforts are being made to improve the performance of current video codec technologies to provide better compression ratios or to provide methods that allow for lower complexity video encoding and decoding schemes or parallel implementations. Some new video codec tools have recently been proposed by industry experts and are currently being tested to determine their effectiveness. Summary of the invention

[0005] Devices, systems and methods are described that relate to digital video coding and, in particular, to the management of motion vectors. The described methods can be applied to existing video coding standards (e.g., High Efficiency Video Codec (HEVC) or Universal Video Codec) and future video coding standards or video codecs.

[0006] In one representative aspect, the disclosed technology can be used to provide a method of video processing. The method includes: for a conversion between a current video unit of a video including one or more video blocks and a bitstream representation of the video, determining a filling process according to a rule, the filling process is used to fill unavailable samples during the application of a cross-component adaptive loop filtering (CC-ALF) tool to at least some video blocks of the current video unit; and performing the conversion based on the determination, wherein the rule specifies that the filling process is also used to fill unavailable samples during the application of the adaptive loop filtering (ALF) tool to one or more video blocks of the current video unit.

[0007] In another representative aspect, the disclosed technology can be used to provide another video processing method. The method includes performing conversion between a video unit of a video and a bitstream representation of the video, wherein during the conversion, unavailable samples of the video unit are filled in a predefined filling order according to rules in the application of an adaptive loop filtering (ALF) process or a cross-component adaptive loop filtering (CC-ALF) process.

[0008] In yet another representative aspect, the disclosed technology can be used to provide another video processing method. The method includes: for a video region of a video with an adaptive loop filter (ALF) application enabled, determining that the video region is spanned by a boundary of a video unit; and performing a conversion between the video and a bitstream representation of the video, wherein for the conversion, the video region is split into a plurality of partitions according to a rule because the video region is spanned by a boundary of the video unit.

[0009] In yet another representative aspect, the disclosed technology can be used to provide another video processing method. The method includes performing conversion between a video unit of a video and a bitstream representation of the video according to a rule, wherein the rule specifies that adaptive loop filtering (ALF) and / or cross-component adaptive loop filtering (CC-ALF) is not allowed to be applied to samples located at the boundary if filtering processes across the boundary of the video unit are not allowed.

[0010] In yet another representative aspect, the disclosed technology can be used to provide another video processing method. The method includes performing conversion between a video unit of a video and a bitstream representation of the video, wherein the bitstream representation conforms to a format rule; wherein a video region is different from a codec tree block; wherein the format rule specifies whether a syntax element indicating the applicability of an adaptive loop filtering (ALF) tool and / or a cross-component adaptive loop filtering (CC-ALF) tool to the video region is included in the bitstream representation.

[0011] In yet another representative aspect, the disclosed technology can be used to provide another video processing method. The method includes: for conversion between a video unit of a video and a bitstream representation of the video, determining the applicability of a cross-component adaptive loop filtering (CC-ALF) tool to samples of the video unit according to a rule; and performing the conversion according to the determination; wherein the bitstream representation includes an indication that CC-ALF is applicable to the video unit, and wherein the rule specifies one or more conditions that override the indication.

[0012] In yet another representative aspect, the disclosed technology can be used to provide another video processing method. The method includes performing conversion between a video unit of a video and a bitstream representation of the video according to a rule, wherein the rule specifies that an algorithm used during the conversion omits at least one of three cropping operations, the three cropping operations including a first cropping operation corresponding to chroma adaptive loop filtering (ALF) filtering, a second cropping operation corresponding to cross-component adaptive loop filtering (CC-ALF) offset derivation, and a third cropping operation corresponding to refinement of chroma filter samples to derive final chroma sample values.

[0013] In yet another representative aspect, the disclosed technology can be used to provide another video processing method. The method includes determining a cross-component adaptive loop filter (CC-ALF) offset according to a rule for conversion between a first video unit of a video and a bitstream representation of the video; and performing the conversion based on the determination, and wherein the rule specifies that the CC-ALF offset is clipped to a first range different from a second range, the second range being represented as [–(1<<(BitDepthC–1)), (1<<(BitDepthC–1))–1], where BitDepthC is a bit depth value.

[0014] In yet another representative aspect, the disclosed technology can be used to provide another video processing method. The method includes: for a conversion between a first video unit of a video and a bitstream representation of the video, deriving a cross-component adaptive loop filter (CC-ALF) offset according to a rule; and performing the conversion using the CC-ALF offset, wherein the rule specifies rounding the CC-ALF offset using a rounding offset based on a bit depth value rather than a fixed value.

[0015] In yet another representative aspect, the disclosed technology can be used to provide another video processing method. The method includes performing conversion between a video unit of a video and a bitstream representation of the video according to a rule, wherein the rule specifies that one or more processing steps used during a chroma adaptive loop filtering (ALF) process and / or a cross-component adaptive loop filtering (CC-ALF) process applied to samples of the video unit are the same.

[0016] In yet another representative aspect, the disclosed technology can be used to provide another video processing method. The method includes performing conversion between a video unit of a video and a bitstream representation of the video according to a rule, wherein the rule specifies that during the conversion, sample values ​​of a first color component of the video unit are modified by applying a modification using information of a second color component of the video unit, wherein the modification is based on one or more parameters used in an adaptive loop filtering (ALF) process of the video unit.

[0017] In another representative aspect, the disclosed technology can be used to provide another video processing method. The method includes: for a conversion between a first sub-picture of a video and a bitstream representation of the video, determining based on a rule whether cross-component loop filtering (CC-ALF) is applicable to a sample of the first sub-picture; and performing the conversion based on the determination; wherein the CC-ALF for the sample uses a sample from a second sub-picture; and wherein the rule is based on whether loop filtering across a sub-picture boundary is allowed for the first sub-picture and / or the second sub-picture.

[0018] In addition, in a representative aspect, an apparatus in a video system is disclosed, comprising a processor and a non-transitory memory having instructions thereon. The instructions, when executed by the processor, cause the processor to implement any one or more of the disclosed methods.

[0019] Furthermore, a computer program product stored on a non-transitory computer-readable medium is disclosed, the computer program product comprising a program code for performing any one or more of the disclosed methods.

[0020] The above and other aspects and features of the disclosed technology are described in more detail in the drawings, the description and the claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 An example of an encoder block diagram is shown.

[0022] Figure 2 Examples of geometric transform based adaptive loop filter (GALF) filter shapes are shown.

[0023] Figure 3 An example of a loop filter line buffer associated with a luma component is shown.

[0024] Figure 4 An example of a loop filter line buffer associated with a chroma component is shown.

[0025] Figure 5A and Figure 5B An example of modified ALF block classification at a virtual boundary when N=4 is shown.

[0026] Fig. 6A , Figure 6B and Figure 6C Examples of 1-line, 2-line, and 3-line near a virtual boundary (VB) combined with modified luma ALF filtering are shown.

[0027] Fig. 7A and Figure 7B Examples of 1-line and 2-line near a virtual boundary (VB) combined with modified chroma ALF filtering are shown.

[0028] Figure 8 An example of an ALF based on modified coefficients (MALF) is shown.

[0029] 9A to 9D An example of a subsampled Laplace calculation is shown.

[0030] Fig. 10A An example of the placement of the CC-ALF relative to other loop filters is shown.

[0031] Fig. 10BAn example of a diamond filter is shown.

[0032] Fig.11 An example of a 3x4 diamond filter is shown.

[0033] Fig.12 An example of raster scan striping of a picture is shown.

[0034] Fig.13 An example of rectangular strip division of a picture is shown.

[0035] Fig.14 Another example of rectangular striping of a picture is shown.

[0036] Fig.15 An example of sub-picture partitioning of a picture is shown.

[0037] Fig.16 An example of an ALF processing unit and an example of a narrow ALF processing unit are shown.

[0038] Fig.17 An example of applying repetition padding to an ALF processing unit is shown.

[0039] Fig.18 An example of filling unavailable samples is shown.

[0040] Fig.19 is a block diagram of an example of a hardware platform for implementing the visual media decoding or visual media encoding techniques described in this document.

[0041] Fig. 20 A flow chart of an example method for video processing is shown.

[0042] Fig.21 is a block diagram of an example video processing system in which the disclosed techniques may be implemented.

[0043] Fig. 22 is a block diagram illustrating an example video encoding and decoding system.

[0044] Fig.23 is a block diagram illustrating an encoder according to some embodiments of the disclosed technology.

[0045] Fig.24 is a block diagram illustrating a decoder according to some embodiments of the disclosed technology.

[0046] FIG. 25A to FIG. 25G A flow chart illustrating an example method of video processing based on some implementations of the disclosed technology is shown. DETAILED DESCRIPTION

[0047] 1. Video Codec in HEVC / H.265

[0048] Video codec standards have evolved primarily through the development of the well-known ITU-T and ISO / IEC standards. ITU-T produced H.261 and H.263, ISO / IEC produced MPEG-1 and MPEG-4 Vision, and the two organizations jointly produced the H.262 / MPEG-2 Video and H.264 / MPEG-4 Advanced Video Coding (AVC) and H.265 / HEVC standards. Since H.262, video codec standards have been based on a hybrid video codec structure, in which temporal prediction and transform codecs are utilized. In order to explore future video codec technologies beyond HEVC, VCEG and MPEG jointly established the Joint Video Exploration Team (JVET) in 2015. Since then, JVET has adopted many new methods and introduced them into a reference software named Joint Exploration Model (JEM). In April 2018, the Joint Video Experts Team (JVET) between VCEG (Q6 / 16) and ISO / IEC JTC1 SC29 / WG11 (MPEG) was created to work on the VVC standard, which targets a 50% bitrate reduction compared to HEVC.

[0049] 2.1. Color Space and Chroma Subsampling

[0050] A color space, also called a color model (or color system), is an abstract mathematical model that describes the range of colors simply as a tuple of numbers, usually 3 or 4 values ​​or color components (e.g. RGB). Basically, a color space is a refinement of coordinate systems and subspaces.

[0051] For video compression, the most commonly used color spaces are YCbCr and RGB.

[0052] YCbCr, Y′CbCr or Y Pb / Cb Pr / Cr, also written as YCBCR or Y′CBCR, is a family of color spaces used as part of the color picture pipeline in video and digital photography systems. Y′ is the luminance component, CB and CR are the blue difference and red difference chrominance components. Y′ (with superscript ') is different from Y luminance, which means that the light intensity is non-linearly encoded based on the gamma-corrected RGB primaries.

[0053] Chroma subsampling is the practice of encoding an image at a lower resolution for chroma information than for luminance information, taking advantage of the fact that the human visual system is less sensitive to color differences than to luminance. 2.1.1. 4:4:4

[0055] Each of the three Y'CbCr components has the same sampling rate, so there is no chroma subsampling. This scheme is sometimes used in high-end film scanners and in film post-production. 2.1.2. 4:2:2

[0057] The two chroma components are sampled at half the rate of luma: the horizontal chroma resolution is halved. This reduces the bandwidth of the uncompressed video signal by one third with little visual difference. 2.1.3. 4:2:0

[0059] In 4:2:0, horizontal sampling is doubled compared to 4:1:1, but the vertical resolution is halved because in this scheme, the Cb and Cr channels are sampled only on every alternate line. Therefore, the data rate is the same. Cb and Cr are each subsampled by a factor of 2 in both the horizontal and vertical directions. There are three variations of the 4:2:0 scheme with different horizontal and vertical addressing.

[0060] In MPEG-2, Cb and Cr are co-located horizontally. Cb and Cr are addressed between pixels in the vertical direction (intermittently addressed).

[0061] In JPEG / JFIF, H.261 and MPEG-1, Cb and Cr are addressed intermittently, intermediately between alternating luminance samples.

[0062] In 4:2:0 DV, Cb and Cr are co-located horizontally. Vertically, they are co-located on alternate lines.

[0063] 2.1.4. Encoding and decoding of different color components

[0064] Depending on the value of separate_colour_plane_flag, the value of the variable ChromaArrayType is assigned as follows:

[0065] – If separate_colour_plane_flag is equal to 0, ChromaArrayType is set equal to chroma_format_idc.

[0066] – Otherwise (separate_colour_plane_flag is equal to 1), ChromaArrayType is set equal to 0.

[0067] 2.2 Encoding and decoding process of typical video codecs

[0068] Figure 1An example of an encoder block diagram for VVC is shown, which includes three in-loop filtering blocks: deblocking filter (DF), sample adaptive offset (SAO) and ALF. Unlike DF, which uses a predefined filter, SAO and ALF use the original samples of the current picture to reduce the mean square error between the original samples and the reconstructed samples by adding an offset and applying a finite impulse response (FIR) filter, respectively, where the codec side information signals the offset and filter coefficients. ALF is located at the last processing stage of each picture and can be seen as a tool that attempts to capture and repair artifacts caused by previous stages.

[0069] 2.3 Adaptive loop filter based on geometric transformation in JEM

[0070] In JEM, a geometric transform based adaptive loop filter (GALF) with block-based filter adaptation is applied. For the luminance component, one of 25 filters is selected for each 2×2 block based on the direction and activity of the local gradient.

[0071] Filter shape

[0072] In JEM, up to three diamond filter shapes can be selected for the luminance component (such as Figure 2 shown). Figure 2 Three GALF filter shapes are shown, 5x5 diamond, 7x7 diamond, 9x9 diamond (from left to right). An index is signaled at the picture level to indicate the filter shape used for the luma component.

[0073] 2.3.1.1. Block classification

[0074] Each 2×2 block is classified into one of 25 categories. The classification index C is based on its directionality D and activity The quantized value of is derived as follows:

[0075]

[0076] To calculate D and First, use the following one-dimensional Laplace formula to calculate the gradient in the horizontal, vertical and two diagonal directions:

[0077]

[0078]

[0079]

[0080]

[0081] The indices i and j refer to the coordinates of the top left sample point in the 2×2 block, and R(i,j) represents the reconstructed sample point at coordinates (i,j).

[0082] Then the maximum and minimum values ​​of the horizontal and vertical gradients of D are set to:

[0083]

[0084] And the maximum and minimum values ​​of the gradients in the two diagonal directions are set to:

[0085]

[0086] To derive the value of the directivity D, these values ​​are compared with each other and with two thresholds t1 and t2:

[0087] Step 1. If is true, D is set to 0.

[0088] Step 2. If Then continue from step 3; otherwise continue from step 4.

[0089] Step 3. If Then D is set to 2; otherwise D is set to 1.

[0090] Step 4. If Then D is set to 4; otherwise D is set to 3.

[0091] The activity value A is calculated as follows:

[0092]

[0093] A is further quantized to the range of 0 to 4 (inclusive), and the quantized value is expressed as

[0094] For the two chrominance components in a picture, no classification method is applied, eg, a set of ALF coefficients is applied for each chrominance component.

[0095] 2.3.1.2. Geometric transformation of filter coefficients

[0096] Before filtering each 2×2 block, geometric transformations (such as rotations or diagonal and vertical flips) are applied to the filter coefficients f(k,l) according to the gradient values ​​calculated for that block. This is equivalent to applying these transformations to the samples in the filter support region. The idea is to make different blocks with ALF applied more similar by aligning their directionality.

[0097] Three geometric transformations are introduced, including diagonal, vertical flip and rotation:

[0098] Diagonal: f D (k,l)=f(l,k),

[0099] Vertical flip: ,f V (k,l)=f(k,Kl-1) (9)

[0100] Rotation: f R (k,l)=f(Kl-1,k).

[0101] Where K is the size of the filter and 0≤k,l≤K-1 are the coefficient coordinates such that position (0,0) is at the upper left corner and position (K-1,K-1) is at the lower right corner. The transform is applied to the filter coefficients f(k,l) according to the gradient value calculated for the block. Table 1 summarizes the relationship between the transform and the four gradients in the four directions.

[0102] Table 1: Mapping of gradients computed for a block to transformations

[0103] Gradient Value Transform <![CDATA[g d2 <g d1 And g o <g ave ]]> No transformation <![CDATA[g d2 <g d1 And g ive <g h ]]> diagonal <![CDATA[g d1 <g d2 And g h <g v ]]> Flip Vertically <![CDATA[g d1 <g d2 And g v <g h ]]> Rotation

[0104] 2.3.1.3. Filter parameter signaling

[0105] In JEM, GALF filter parameters are signaled for the first CTU, for example, after the slice header and before the SAO parameters of the first CTU. Up to 25 sets of luminance filter coefficients can be signaled. In order to reduce bit overhead, filter coefficients of different classifications can be merged. In addition, the GALF coefficients of the reference picture are stored and allowed to be reused as GALF coefficients for the current picture. The current picture can choose to use the GALF coefficients stored for the reference picture and bypass the GALF coefficient signaling. In this case, only the index of one of the reference pictures is signaled, and the stored GALF coefficients of the indicated reference picture are inherited for the current picture.

[0106] To support GALF time domain prediction, a candidate list of GALF filter sets is maintained. At the beginning of a new sequence of decoding, the candidate list is empty. After decoding a picture, the corresponding filter set can be added to the candidate list. Once the size of the candidate list reaches the maximum allowed value (for example, 6 in the current JEM), the set of new filters will overwrite the oldest set in the decoding order, that is, the first-in-first-out (FIFO) rule is applied to update the candidate list. In order to avoid duplication, the set can only be added to the list when the corresponding picture does not use GALF time domain prediction. In order to support time domain scalability, there are multiple candidate lists of filter sets, and each candidate list is associated with a time domain layer. More specifically, each array allocated by the time domain layer index (TempIdx) can constitute a filter set of a previously decoded picture with a lower TempIdx. For example, the kth array is allocated to be associated with a TempIdx equal to k, and it only includes filter sets from pictures with TempIdx less than or equal to k. After encoding or decoding a picture, the filter set associated with the picture will be used to update those arrays associated with equal or higher TempIdx.

[0107] Temporal prediction of the GALF coefficients is used for inter-coded frames to minimize signaling overhead. For intra frames, temporal prediction is not available and a set of 16 fixed filters is assigned to each class. To indicate the use of fixed filters, a flag for each class is signaled and, if necessary, the index of the selected fixed filter. Even when a fixed filter is selected for a given class, the coefficients f(k,l) of the adaptive filter may still be sent for that class, in which case the coefficients of the filter that will be applied to the reconstructed picture are the sum of the two sets of coefficients.

[0108] The filtering process of the luma component can be controlled at the CU level. A signaling flag is used to indicate whether GALF is applied to the luma component of the CU. For chroma components, whether GALF is applied is indicated only at the picture level.

[0109] 2.3.1.4. Filtering process

[0110] At the decoder side, when GALF is enabled for a block, each sample R(i,j) within the block is filtered to obtain the sample value R′(i,j) as shown below, where L indicates the filter length and f m,n represents the filter coefficient, and f(k,l) indicates the decoded filter coefficient.

[0111]

[0112] Alternatively, the filtering process of the adaptive loop filter can be expressed as follows:

[0113] O(x,y)=∑ (i,j) w(i,j).I(x+i,y+j), (11)

[0114] Wherein the sample point I (x + i, y + j) is the input sample point, O (x, y) is the filtered output sample point (ie, the filter result), which indicates the filter coefficient. In fact, in VTM4.0, it uses integer operations to implement fixed-point precision calculations:

[0115]

[0116] where L indicates the filter length, and where w(i,j) are the filter coefficients expressed in fixed-point precision.

[0117] 2.4. Nonlinear ALF

[0118] 2.4.1. Filter reformulation

[0119] Without affecting the coding efficiency, equation (11) can be reformulated as follows:

[0120] O(x,y)=I(x,y)+∑ (i ,j)≠(0 ,0) w(i,j).(I(x+i,y+j)-I(x,y)),(13) where w(i,j) is the same filter coefficient as w(i,j) in equation () [except that w(0,0) is equal to 1 in equation (13), whereas it is equal to 1-∑ in equation (11) (i ,j)≠(0 ,0) w(i,j)].

[0121] 2.4.2. Modified filter

[0122] Using the above filter formula of (13), we can easily introduce nonlinearity to make the ALF more effective by using a simple clipping function to reduce the impact of neighboring sample values ​​I(x+i,y+j) when they differ too much from the current sample value I(x,y) being filtered.

[0123] In this proposal, the ALF filter is modified as follows:

[0124] O′(x,y)=I(x,y)+∑ (i ,j)≠(0 ,0) w(i,j).K(I(x+i,y+j)-I(x,y),k(i,j)), (14)

[0125] where K(d,b)=min(b,max(-b,d)) is the clipping function, and k(i,j) is the clipping parameter depending on the (i,j)th filter coefficient. The encoder performs an optimization to find the best k(i,j).

[0126] For ease of implementation, the filter coefficients w(i,j) are stored and used with integer precision. The above equation can be rewritten as follows:

[0127] O′(i,j)=I(i,j)+((∑ k≠0 ∑ l≠0 w(k,l)×K(I(i+k,j+l)-I(i,j),c(k,l))+64)>>7) (15)

[0128] Where w(k, l) indicates the decoded filter coefficients, K(x, y) is the clipping function, and c(k, l) indicates the decoded clipping parameters. The variables k and L are in and where L indicates the filter length. The clipping function K(x,y)=min(y,max(-y,x)) corresponding to the function Clip3(-y,y,x)

[0129] In the JVET-N0242 implementation, a clipping parameter k(i,j) is specified for each ALF filter, and one clipping value is signaled per filter coefficient. This means that up to 12 clipping values ​​can be signaled in the bitstream for each luma filter and up to 6 clipping values ​​for chroma filters.

[0130] To limit the signaling cost and encoder complexity, we restrict the evaluation of the cropping value to a small set of possible values. In the proposal, we only use 4 fixed values ​​that are the same for inter and intra slice groups.

[0131] Because the variance of local differences for luma is usually higher than for chroma, we use two different sets for luma and chroma filters. We also include the maximum sample value in each set (here 1024 for 10-bit depth) so that clipping can be disabled if not necessary.

[0132] Table 2 provides the set of clipped values ​​used in the JVET-N0242 test. In the logarithmic domain, the 4 values ​​are selected by splitting approximately equally the full range of sample values ​​for luma (coded as 10 bits) and the range of 4 to 1024 for chroma.

[0133] More precisely, the brightness table of the clipped values ​​is obtained by the following formula:

[0134] Where M = 2 10and N = 4 (16)

[0135] Similarly, the chromaticity table for the clipped value is obtained according to the following formula:

[0136] Where M = 2 10 , N = 4 and A = 4 (17)

[0137] Table 2: Authorized clipping values

[0138]

[0139]

[0140] The selected clipping value is encoded in the "alf_data" syntax element by using the Golomb coding scheme corresponding to the index of the clipping value in the above Table 2. This coding scheme is the same as that of the filter index.

[0141] 2.5. Virtual Boundary

[0142] In hardware and embedded software, picture-based processing is actually unacceptable because it requires a very high picture buffer. Using an on-chip picture buffer is very expensive, and using an off-chip picture buffer significantly increases external memory access, power consumption, and data access latency. Therefore, in actual products, DF, SAO, and ALF will be changed from picture-based decoding to LCU-based decoding. When LCU-based processing is used for DF, SAO, and ALF, the entire decoding process can be completed by the LCU in a raster scan in an LCU pipeline manner for parallel processing of multiple LCUs. In this case, DF, SAO, and ALF require line buffers because processing one LCU row requires pixels from the upper LCU row. If an off-chip line buffer (such as DRAM) is used, the external memory bandwidth and power consumption will be increased; if an on-chip line buffer (such as SRAM) is used, the chip area will increase. Therefore, although the line buffer is already much smaller than the picture buffer, it is still desirable to reduce the line buffer.

[0143] In VTM-4.0, if Figure 3As shown, the total number of line buffers required for the luma component is 11.25 lines. The line buffer requirement can be explained as follows: deblocking of horizontal edges overlapping CTU edges cannot be performed because the decision and filtering require lines K, L, M, M from the first CTU and lines O, P from the bottom CTU. Therefore, deblocking of horizontal edges overlapping CTU boundaries is postponed until the CTU below arrives. Therefore, the reconstructed luma samples for lines K, L, M, N must be stored in the line buffer (4 lines). Then, SAO filtering can be performed on lines A to J. Line J can be SAO filtered because deblocking does not change the samples in line K. For SAO filtering of line K, the edge offset classification decision is only stored in the line buffer (this is 0.25 luma lines). ALF filtering can only be performed on line AF. As shown Figure 3 As shown, ALF classification is performed for each 4×4 block. Each 4×4 block classification requires an active window of size 8×8, which in turn requires a 9×9 window to compute a 1D Laplacian operator to determine the gradient.

[0144] Therefore, for block classification of 4×4 blocks overlapping lines G, H, I, J, the SAO filtered samples below the virtual boundary are required. In addition, the SAO filtered samples of lines D, E, F are necessary for ALF classification. Furthermore, ALF filtering of line G requires the three SAO filtered lines D, E, F from the above lines. Therefore, the bus buffer requirements are as follows:

[0145] – Line KN (horizontal DF pixels): 4 lines

[0146] – Line DJ (SAO filtered pixels): 7 lines

[0147] – SAO edge offset classifier value between line J and line K: 0.25 lines

[0148] Therefore, the total number of brightness lines required is 7+4+0.25=11.25.

[0149] Similarly, the line buffer requirements for chrominance components are as follows Figure 4 The line buffer requirement for the chrominance components is estimated to be 6.25 lines.

[0150] In order to eliminate the line buffer requirements of SAO and ALF, the concept of virtual boundary (VB) is introduced in the latest VVC. Figure 3 As shown in FIG. 1 , VB is the horizontal LCU boundary moved up by N pixels. For each LCU, SAO and ALF can process pixels above VB before the LCU below arrives, but cannot process pixels below VB until the LCU below arrives, which is caused by DF. Considering the hardware implementation cost, the space between the proposed VB and the horizontal LCU boundary is set to four pixels for brightness (i.e., Figure 3 9 ), and for chrominance it is set to two pixels (ie, N=2 in FIG. 9 ).

[0151] 2.5.1. Modified ALF block classification when VB size N is 4

[0152] Figure 5A and Figure 5B Depicted is the modified block classification for the case when the virtual boundary is 4 lines above the CTU boundary (N=4). Figure 5A shows the classification of a 4×4 block starting from line G, Figure 5B The classification of 4×4 blocks starting from line K is shown. Figure 5A As shown, for a 4×4 block starting from line G, block classification uses only lines E to J. However, the Laplace gradient calculation for the samples belonging to line J requires one more line below (line K). Therefore, line K is filled with line J.

[0153] Similarly, if Figure 5B As shown, for a 4×4 block starting from line K, block classification uses only lines K to P. However, the Laplace gradient calculation for samples belonging to line K requires one more line (line J) above. Therefore, line K is used to fill line J.

[0154] 2.5.2. Filling on both sides of the sample points across the virtual boundary

[0155] like FIG. 6A to FIG. 6C As shown, a truncated version of the filter is used to filter the luma samples belonging to lines close to the virtual boundary. Fig. 6A shows that one required line above / below the virtual boundary (VB) needs to be filled (on each side), Figure 6B shows that the two required lines above / below VB need to be filled (on each side), Figure 6C Shown are the three required lines above / below VB that need to be filled (on each side). Figures 6A to 6C In the figure, VB is represented by the gray line. Fig. 6A For example, Figure 3 When filtering line M as shown, i.e. the center sample of the 7×7 diamond support is in line M, a line above VB needs to be accessed. In this case, the samples above VB are copied from the samples just below VB, e.g. the P0 sample in the solid line is copied to the dashed position above. Symmetrically, the P3 sample in the solid line is also copied to the position just below the dashed position, even though a sample at that position is available. The copied samples are only used for the luma filtering process.

[0156] The padding method for the ALF virtual boundary can be expressed as "double-sided padding", where if a sample point located at (i, j) (e.g., Figure 6BIf the P0A with the dashed line in it is filled, then even if the corresponding sample point located at (m, n) sharing the same filter coefficients (e.g., Figure 6C the P3B with the dashed line in it is available, this sample point is also filled, as FIG. 6A to FIG. 6C and Fig. 7A shown in FIGS. 7C.

[0157] Similarly, as Fig. 7A shown in FIGS. 7C, the bilateral filling method is also used for chroma ALF filtering. Fig. 7A It shows that one required line above / below the virtual boundary (VB) needs to be filled (on each side), Figure 7B It shows that two required lines above / below the VB need to be filled (on each side). In Fig. 7A and Figure 7B the VB is represented by the gray line.

[0158] 2.5.3. Alternative Methods for Implementing Bilateral Filling When Nonlinear ALF Is Disabled

[0159] When the nonlinear ALF is disabled for a CTB, for example, the clipping parameter k(i, j) in Equation (14) is equal to (1 << Bitdepth), the filling process can be replaced by modifying the filter coefficients (also known as ALF based on modified coefficients, MALF). For example, when filtering the sample points in line L / I, the filter coefficient c5 is modified to c5', in this case, it is not necessary to copy the luminance sample points from Figure 8 the solid P0A in it to the dashed P0A and from the solid P3B to the dashed P3B. In this case, assuming the current sample point to be filtered is located at (x, y), then the bilateral filling and MALF will generate the same result.

[0160] c5.K(I(x - 1, y - 1) - I(x, y), k(-1, -1)) + c1.K(I(x - 1, y - 2) - I(x, y), k(-1, -2))

[0161] =(c5 + c1).K(I(x - 1, y - 1) - I(x, y), k(-1, -1)) (18)

[0162] Because K(d, b) = d and due to filling, I(x - 1, y - 1) = I(x - 1, y - 2).

[0163] However, when the nonlinear ALF is enabled, MALF and bilateral filling may generate different filtering results because the nonlinear parameters are associated with each coefficient. For example, for the filter coefficients c5 and c1, the clipping parameters are different. Therefore,

[0164] c5.K(I(x-1,y-1)-I(x,y),k(-1,-1))+c1.k(I(x-1,y-2)-I(x,y),k(-1,-2))

[0165] ! = (c5+c1).K(I(x-1,y-1)-I(x,y),k(-1,-1)) (19)

[0166] Because K(d,b)!=d, even though I(x-1,y-1)=I(x-1,y-2) due to padding.

[0167] 2.6. Geometric Transformation-Based Adaptive Loop Filter in VVC

[0168] The current design of VVC GALF has the following major changes compared to JEM:

[0169] 1) Adaptive filter shapes are removed. Only 7×7 filter shapes are allowed for luma components and 5×5 filter shapes are allowed for chroma components.

[0170] 2) The ALF filter coefficients are signaled in the ALF Adaptation Parameter Set (APS).

[0171] 3) Non-linear ALF can be applied.

[0172] 4) For each CTU, whether ALF is enabled or disabled is signaled with a one-bit flag for each color component.

[0173] 5) The class index is calculated at the 4×4 level instead of the 2×2 level. In addition, as proposed in JVET-L0147, a sub-sampled Laplacian calculation method for ALF classification is utilized. More specifically, the horizontal / vertical / 45 diagonal / 135 degree gradient does not need to be calculated for each sample point within a block. Instead, 1:2 sub-sampling is utilized.

[0174] 2.7. Signaling of ALF parameters in the adaptive parameter set

[0175] In the latest version of the VVC draft, ALF parameters can be signaled in an adaptive parameter set (APS) and can be adaptively selected by each CTU. In one APS, up to 25 sets of luminance filter coefficients and clipping value indices, and up to 8 sets of chrominance filter coefficients and clipping value indices can be signaled. In order to reduce bit overhead, filter coefficients of different classifications of luminance components can be merged. In the slice header, the index of the APS used for the current slice is signaled.

[0176] The filter coefficients are quantized with a norm equal to 128. To limit the multiplication complexity, bitstream consistency is applied so that coefficient values ​​in non-central positions should be between -27 and 2 7 The center position coefficient is not signaled in the bitstream and is considered equal to 128.

[0177] (In JVET-P2001-v9) The detailed signaling of ALF is as follows.

[0178] 7.3.2.5 Adaptation parameter set syntax

[0179]

[0180]

[0181] 7.3.2.16 Adaptive Loop Filter Data Syntax

[0182]

[0183]

[0184] 7.4.3.5. Adaptation Parameter Set Semantics

[0185] Each APS RBSP shall be available to the decoding process before being referenced, included in at least one access unit whose TemporalId is less than or equal to the TemporalId of the codec slice NAL unit that references it or is provided by external components.

[0186] Let aspLayerId be the nuh_layer_id of the APS NAL unit. If the layer with numh_layer_id equal to aspLayerId is an independent layer (i.e., VPS_independent_layer_flag[GeneralLayerIdx[aspLayerId]] is equal to 1), then the numh_layer_id of the APS NAL unit that includes the APS RBSP shall be equal to the numh_layer_id of the codec slice NAL unit that references it. Otherwise, the nuh_layer_id of the APS NAL unit that includes the APS RBSP shall be equal to the nuh_layer_id of the codec slice NAL unit that references it, or to the nuh_layer_id of a directly dependent layer of the layer that includes the codec slice NAL unit that references it.

[0187] All APS NAL units with a specific adaptation_parameter_set_id value and a specific aps_params_type value within an access unit shall have the same content.

[0188] The adaptation_parameter_set_id provides an identifier for the AP to be referenced by other syntax elements.

[0189] When aps_params_type is equal to ALF_APS or SCALING_APS, the value of adaptation_parameter_set_id shall be in the range of 0 to 7 (inclusive).

[0190] When aps_params_type is equal to LMCS_APS, the value of adaptation_parameter_set_id shall be in the range of 0 to 3 (inclusive).

[0191] aps_params_type specifies the type of APS parameters carried in the APS, as specified in Table 3. When aps_params_type is equal to 1 (LMCS_APS), the value of adaptation_parameter_set_id shall be in the range of 0 to 3 (inclusive).

[0192] Table 3 – APS parameter type codes and APS parameter types

[0193]

[0194] NOTE 1 - Each type of APS uses a separate value space for adaptation_parameter_set_id.

[0195] NOTE 2 – APS NAL units (with a specific adaptation_parameter_set_id value and a specific aps_params_type value) can be shared across pictures, and different slices within a picture can reference different ALF APSs.

[0196] aps_extension_flag equal to 0 specifies that the aps_extension_data_flag syntax element is not present in the APS RBSP syntax structure. aps_extension_flag equal to 1 specifies that the aps_extension_data_flag syntax element is present in the APS RBSP syntax structure.

[0197] aps_extension_data_flag can have any value. Its presence and value do not affect decoder conformance to the profile specified in this version of this specification. Decoders conforming to this version of this specification should ignore all aps_extension_data_flag syntax elements.

[0198] 7.4.3.14 Adaptive loop filter data semantics

[0199] alf_luma_filter_signal_flag equal to 1 specifies that the luma filter set is signaled. alf_luma_filter_signal_flag equal to 0 specifies that the luma filter set is not signaled.

[0200] alf_chroma_filter_signal_flag equal to 1 specifies that the chroma filter is signaled. alf_chroma_filter_signal_flag equal to 0 specifies that the chroma filter is not signaled. When ChromaArrayType is equal to 0, alf_chroma_filter_signal_flag shall be equal to 0.

[0201] The variable NumAlfFilters, which specifies the number of different adaptive loop filters, is set equal to 25.

[0202] alf_luma_clip_flag equal to 0 specifies that linear adaptive loop filtering is applied to the luma component. alf_luma_clip_flag equal to 1 specifies that non-linear adaptive loop filtering may be applied to the luma component.

[0203] alf_luma_num_filters_signaled_minus1 plus 1 specifies the number of adaptive loop filter classes that can signal luma coefficients. The value of alf_luma_num_filters_signaled_minus1 shall be in the range of 0 to NumAlfFilters-1, inclusive.

[0204] alf_luma_coeff_delta_idx[filtIdx] specifies the index of the adaptive loop filter luma coefficient delta signaled for the filter type indicated by filtIdx, in the range 0 to NumAlfFilters - 1. When alf_luma_coeff_delta_idx[filtIdx] is not present, it is inferred to be equal to 0. The length of alf_luma_coeff_delta_idx[filtidx] is Ceil(Log2(alf_luma_num_filters_signaled_minus1+1)) bits.

[0205] alf_luma_coeff_signalled_flag equal to 1 indicates that alf_luma_coeff_flag[sfIdx] is signaled. alf_luma_coeff_signalled_flag equal to 0 indicates that alf_luma_coeff_flag[sfIdx] is not signaled.

[0206] alf_luma_coeff_flag[sfIdx] equal to 1 specifies that the luma filter coefficients indicated by sfIdx are signaled. alf_luma_coeff_flag[sfIdx] equal to 0 specifies that all filter coefficients of the luma filter indicated by sfIdx are set equal to 0. When not present, alf_luma_coeff_flag[sfIdx] is set equal to 1.

[0207] alf_luma_coeff_abs[sfIdx][j] specifies the absolute value of the j-th coefficient of the signaled luma filter indicated by sfIdx. When alf_luma_coeff_abs[sfIdx][j] is not present, it is inferred to be equal to 0.

[0208] The order k of the exponential Golomb binarization uek(v) is set equal to 3.

[0209] alf_luma_coeff_sign[sfIdx][j] specifies the sign of the j-th luma coefficient of the filter indicated by sfIdx as follows:

[0210] – If alf_luma_coeff_sign[sfIdx][j] is equal to 0, the corresponding luma filter coefficient is positive.

[0211] – Otherwise (alf_luma_coeff_sign[sfIdx][j] is equal to 1), the corresponding luminance filter coefficient is negative.

[0212] When alf_luma_coeff_sign[sfIdx][j] is not present, it is inferred to be equal to 0.

[0213] The variable filtCoeff[sfIdx][j] where sfIdx=0..alf_luma_num_filters_signaled_minus1, j=0..1 is initialized as follows:

[0214] filtCoeff[sfIdx][j]=alf_luma_coeff_ABS[sfIdx][j]* (7-47)

[0215] (1-2*alf_luma_coeff_sign[sfIdx][j])

[0216] With element AlfCoeff L [adaptation_parameter_set_id][sfIdx][j] Luma filter coefficient AlfCoeff L [adaptation_parameter_set_id], where filtIdx=0..NumAlfFilters-1 and j=0..11 are derived as follows:

[0217] Alf Coeff L [adaptation_parameter_set_id][filtIdx][j]=filtCoeff[alf_luma_coeff_delta_idx[filtIdx]][j] (7-48)

[0218] The fixed filter coefficients AlfFixFiltCoeff[i][j] with i=0..64, j=0..11 and the class-to-filter map AlfClassToFiltMap[m][n] with m=0..15 and n=0..24 are derived as follows:

[0219]

[0220]

[0221]

[0222] AlfClassToFiltMap = (7-50)

[0223] {

[0224] {8,2,2,2,3,4,53,9,9,52,4,4,5,9,2,8,10,9,1,3,39,39,10,9,52}

[0225] {11,12,13,14,15,30,11,17,18,19,16,20,20,4,53,21,22,23,14,25,26,26,27,28,10}

[0226] {16,12,31,32,14,16,30,33,53,34,35,16,20,4,7,16,21,36,18,19,21,26,37,38,39}

[0227] {35,11,13,14,43,35,16,4,34,62,35,35,30,56,7,35,21,38,24,40,16,21,48,57,39}

[0228] {11,31,32,43,44,16,4,17,34,45,30,20,20,7,5,21,22,46,40,47,26,48,63,58,10}

[0229] {12,13,50,51,52,11,17,53,45,9,30,4,53,19,0,22,23,25,43,44,37,27,28,10,55}

[0230] {30,33,62,51,44,20,41,56,34,45,20,41,41,56,5,30,56,38,40,47,11,37,42,57,8}

[0231] {35,11,23,32,14,35,20,4,17,18,21,20,20,20,4,16,21,36,46,25,41,26,48,49,58}

[0232] {12,31,59,59,3,33,33,59,59,52,4,33,17,59,55,22,36,59,59,60,22,36,59,25,55}

[0233] {31,25,15,60,60,22,17,19,55,55,20,20,53,19,55,22,46,25,43,60,37,28,10,55,52}

[0234] {12,31,32,50,51,11,33,53,19,45,16,4,4,53,5,22,36,18,25,43,26,27,27,28,10}

[0235] {5,2,44,52,3,4,53,45,9,3,4,56,5,0,2,5,10,47,52,3,63,39,10,9,52}

[0236] {12,34,44,44,3,56,56,62,45,9,56,56,7,5,0,22,38,40,47,52,48,57,39,10,9}

[0237] {35,11,23,14,51,35,20,41,56,62,16,20,41,56,7,16,21,38,24,40,26,26,42,57,39}

[0238] {33,34,51,51,52,41,41,34,62,0,41,41,56,7,5,56,38,38,40,44,37,42,57,39,10}

[0239] {16,31,32,15,60,30,4,17,19,25,22,20,4,53,19,21,22,46,25,55,26,48,63,58,55}

[0240] },

[0241] The bitstream conformance requirement is that AlfCoeff where filtIdx = 0..NumAlfFilters-1, j = 0..11 L The value of [adaptation_parameter_set_id][filtIdx][j] should be between -2 7 to 2 7 -1 (including the endpoints).

[0242] alf_luma_clip_idx[sfIdx][j] specifies the clip index of the clip value to be used before multiplying the jth coefficient of the signaled filter indicated by sfIdx. A bitstream conformance requirement is that the value of alf_luma_clip_idx[sfIdx][j] with sfIdx=0..alf_luma_num_filters_signalled_minus1 and j=0..11 shall be in the range of 0 to 3 (inclusive).

[0243] AlfClip with element where filtIdx=0..NumAlfFilters-1 and j=0..11 L [adaptation_parameter_set_id][filtIdx][j] Luma filter clipping value AlfClip L [adaptation_parameter_set_id] is derived as specified in Table 4, depending on bitDepth being set to BitDepth Y , clipIdx is set to alf_luma_clip_idx[alf_luma_coeff_delta_idx[filtIdx]][j].

[0244] alf_chroma_num_alt_filters_minus1 plus 1 specifies the number of alternative filters for chroma components.

[0245] alf_chroma_clip_flag[altIdx] equal to 0 specifies that when the chroma filter with index altIdx is used, linear adaptive loop filtering is applied to the chroma components; alf_chroma_clip_flag[altIdx] equal to 1 specifies that when the chroma filter with index altIdx is used, non-linear adaptive loop filtering is applied to the chroma components. When not present, alf_chroma_clip_flag[altIdx] is inferred to be equal to 0.

[0246] alf_chroma_coeff_abs[altIdx][j] specifies the absolute value of the j-th chroma filter coefficient of the alternative chroma filter with index altIdx. When alf_chroma_coeff_abs[altIdx][j] is not present, it is inferred to be equal to 0. A bitstream conformance requirement is that the value of alf_chroma_coeff_abs[altIdx][j] shall be between 0 and 2. 7 -1 (including the endpoints).

[0247] The order k of the exponential Golomb binarization uek(v) is set equal to 3.

[0248] alf_chroma_coeff_sign[altIdx][j] specifies the sign of the j-th chroma filter coefficient of the alternative chroma filter with index altIdx as follows:

[0249] – If alf_chroma_coeff_sign[altIdx][j] is equal to 0, the corresponding chroma filter coefficient has a positive value.

[0250] – Otherwise (alf_chroma_coeff_sign[altIdx][j] is equal to 1), the corresponding chroma filter coefficient has a negative value.

[0251] When alf_chroma_coeff_sign[altIdx][j] is not present, it is inferred to be equal to 0.

[0252] AlfCoeff with elements altIdx=0..alf_chroma_num_alt_filters_minus1, j=0..5 C [adaptation_parameter_set_id][altIdx][j] chroma filter coefficient AlfCoeff C The derivation of [adaptation_parameter_set_id][altIdx] is as follows:

[0253] Alf Coeff C [adaptation_parameter_set_id][altIdx][j]=alf_chroma_coeff_abs[altIdx][j]* (7-51)

[0254] (1-2*alf_chroma_coeff_sign[altIdx][j])

[0255] The bitstream conformance requirement is that AlfCoeff where altIdx = 0..alf_chroma_num_alt_filters_minus1, j = 0..5 C The value of [adaptation_parameter_set_id][altIdx][j] should be between -2 7 -1 to 2 7 -1 range (including the endpoints).

[0256] alf_chroma_clip_idx[altIdx][j] specifies the clip index of the clip value to be used before multiplying the j-th coefficient of the alternative chroma filter with index altIdx. A bitstream conformance requirement is that the value of alf_chroma_clip_idx[altIdx][j] with altIdx=0..alf_chroma_num_alt_filters_minus1, j=0..5 shall be in the range 0 to 3 (inclusive).

[0257] AlfClip with element altIdx=0..alf_chroma_num_alt_filters_minus1,j=0..5 C [adaptation_parameter_set_id][altIdx][j] Chroma filter clipping value AlfClip C The derivation of [adaptation_parameter_set_id][altIdx] is shown in Table 4, depending on whether bitDepth is set to BitDepth. C , clipIdx is set to alf_chroma_clip_idx[altIdx][j].

[0258] Table 4 – Normative AlfClip depending on bitDepth and clipIdx

[0259]

[0260] 2.8. Signaling of ALF parameters of CTU

[0261] In VTM6, ALF filter parameters are signaled in adaptive parameter sets (APS). In one APS, up to 25 sets of luma filter coefficients and clipping value indices and up to 8 sets of chroma filter coefficients and clipping value indices can be signaled. In order to reduce bit overhead, filter coefficients for different classifications of luma components can be merged. In the slice header, the index of the APS for the current slice is signaled.

[0262] The clipping value index decoded from the AP allows the clipping value to be determined using the clipping value luma table and the clipping value chroma table. These clipping values ​​depend on the internal bit depth.

[0263] In the slice header, up to 7 APS indices can be signaled to specify the luma filter set for the current slice. The filtering process can be further controlled at the CTB level. A flag is always signaled to indicate whether the ALF is applied to the luma CTB. The luma CTB can select a filter set from 16 fixed filter sets and a filter set from the APS. The filter set index is signaled for the luma CTB to indicate which filter set is applied. The 16 fixed filter sets are predefined and hard-coded in both the encoder and decoder.

[0264] For chroma components, the APS index is signaled in the slice header to indicate the chroma filter set used for the current slice. At the CTB level, if there is more than one chroma filter set in the APS, the filter index is signaled for each chroma CTB.

[0265] More specifically, the following apply:

[0266] The slice on / off control flag is first encoded and decoded to indicate whether at least one CTU in the slice applies ALF. If true, for each CTU, the following will be checked and signaled in turn:

[0267] Underlined bold text indicates proposed changes to existing standards.

[0268] Related to the brightness part:

[0269] 1. Is ALF applied to the luma CTB? If yes, go to step 2. Otherwise, no further signaling is required.

[0270] 2. Check the number of ALF APS used for the previous stripe, denoted by numALFAPS.

[0271] 3. If numALFAPS is equal to 0, the index of the fixed filter is signaled (e.g., alf_luma_fixed_filter_idx). Otherwise, the following applies:

[0272] – Signaling flag to indicate whether it is predicted from the first ALF APS.

[0273] – If no, go to step 4. Otherwise, stop signaling the ALF parameters for the luma CTB.

[0274] 4. If numALFAPS is greater than 1, a flag is signaled to indicate whether it is predicted from ALFAPS.

[0275] – if not, signaling the index of the fixed filter;

[0276] – If yes, and numALFAPS is greater than 2, the index of the ALF APS is signaled minus 1 using truncated unary signaling.

[0277] Related to the chroma part:

[0278] 1. Whether ALF is applied to Cb / Cr CTB. If yes, go to step 2. Otherwise, no further signaling is required.

[0279] 2. Signaling the index of the filter associated with the i-th ALF APS, where the APS index is signaled in the slice header.

[0280] 7.3.8.2 Codec Tree Unit Syntax

[0281]

[0282]

[0283] 2.9. Cross-component adaptive loop filter

[0284] The Cross Component Adaptive Loop Filter (CC-ALF) uses the luma sample values ​​to refine each chroma component. Basically, if CC-ALF is applied, CC-ALF generates a correction for each chroma sample by filtering the luma samples. It is used as a loop filter step. The tool is controlled by information in the bitstream, and the information includes both (a) filter coefficients for each chroma component and (b) a mask that controls the application of the filter for a block of samples.

[0285] The following Fig. 10A Figure 2 shows the placement of CC-ALF relative to other loop filters. CC-ALF works by applying a linear diamond filter (e.g. Fig. 10B As shown), the chrominance component is expressed as

[0286]

[0287] in

[0288] (x,y) is the position of the chroma component i being refined

[0289] (x C ,y C ) is the brightness position based on (x,y)

[0290] S i is the filter support in luma for chroma component i

[0291] c i (x0,y0) represents the filter coefficients

[0292] The CC-ALF process is further described in JVET-O0636. Key features include:

[0293] The support area is centered around its luminance position (x C ,y C ) is calculated based on the spatial scaling factor between the luma and chroma planes.

[0294] • All filter coefficients are transmitted in ASP and have 8-bit dynamic range.

[0295] APS can be referenced in the slice header.

[0296] • The CC-ALF coefficients for each chroma component of a slice are also stored in the buffer corresponding to the temporal sub-layer. The use of a slice level flag facilitates the reuse of these temporal sub-layer filter coefficient sets.

[0297] • The application of CC-ALF filter is controlled on variable block size and is signaled by the context codec flag received for each sample block. For each chroma component, the block size is received at slice level along with the CC-ALF enable flag.

[0298] The border filling of the horizontal virtual borders utilizes repetition. For the remaining borders, the same type of filling as the regular ALF is used.

[0299] 2.9.1. Further simplification of CC-ALF in JVET-P1008

[0300] Compared with JVET-P0080, the following new aspects are intended to simplify the design of CC-ALF.

[0301] – Reduced complexity

[0302] o Such as Fig.11 As shown, the number of multiplications in the filter operation is reduced by changing the filter shape to a 3×4 diamond.

[0303] o Limit the dynamic range of CC-ALF coefficients to 6 bits

[0304] o Allows sharing of multipliers with chroma ALF

[0305] – Aligned with ALF

[0306] oLimit filter selection to CTU-level signaling

[0307] oRemove the time domain layer coefficient buffer

[0308] oUse symmetry line selection at ALF virtual border

[0309] Furthermore, since simplification reduces codec efficiency, the number of filters that can be applied to each chroma component is limited to a maximum of 4.

[0310] 2.9.1.1.CC-ALF Syntax and Semantics

[0311] Underlined bold text indicates proposed changes to existing standards.

[0312] 7.3.6 Strip Header Syntax

[0313] 7.3.6.1 Generic Strip Header Syntax

[0314]

[0315]

[0316]

[0317]

[0318]

[0319] ALF Data Syntax

[0320]

[0321]

[0322]

[0323] 7.3.8.2 Codec Tree Unit Syntax

[0324]

[0325]

[0326] alf_ctb_cross_component_cb_idc[xCtb>>CtbLog2SizeY][yCtb>>CtbLog2SizeY] is equal to 0 to indicate that the cross-component Cb filter is not applied to the Cb color component sample block at the luma position (xCtb, yCtb). alf_cross_component_cb_idc[xCtb>>CtbLog2SizeY][yCtb>>CtbLog2SizeY] is not equal to 0 to indicate that the alf_cross_component_cb_idc[xCtb>>CtbLog2SizeY][yCtb>>CtbLog2SizeY]th cross-component Cb filter is applied to the Cb color component sample block at the luma position (xCtb, yCtb)

[0327] alf_ctb_cross_component_cr_idc[xCtb>>CtbLog2SizeY][yCtb>>CtbLog2SizeY] is equal to 0 to indicate that the cross-component Cr filter is not applied to the Cr color component sample block at the luma position (xCtb, yCtb). alf_cross_component_cr_idc[xCtb>>CtbLog2SizeY][yCtb>>CtbLog2SizeY] is not equal to 0 to indicate that the alf_cross_component_cr_idc[xCtb>>CtbLog2SizeY][yCtb>>CtbLog2SizeY]th cross-component Cr filter is applied to the Cr color component sample block at the luma position (xCtb, yCtb)

[0328] 2.10. Partitioning of pictures, sub-pictures, slices, slices, and CTUs

[0329] 2.10.1. Segmentation of images into segments, strips, and slices

[0330] This subclause specifies how to partition a picture into sub-pictures, slices, and tiles.

[0331] A picture is divided into one or more slice rows and one or more slice columns. A slice is a series of CTUs covering a rectangular area of ​​a picture. The CTUs in a slice are scanned in a raster scan order within the slice.

[0332] A slice consists of an integer number of complete picture blocks or an integer number of consecutive complete CTU rows in a slice of a picture.

[0333] Two stripe modes are supported, namely raster scan stripe mode and rectangular stripe mode. In raster scan stripe mode, a stripe includes a complete sequence of slices in a slice raster scan of a picture. In rectangular stripe mode, a stripe includes multiple complete slices that together form a rectangular area of ​​a picture, or multiple consecutive complete CTU rows that together form a slice of a rectangular area of ​​a picture. The slices within a rectangular stripe are scanned in a slice raster scan order within the rectangular area corresponding to the stripe.

[0334] A sub-picture consists of one or more strips that together cover a rectangular area of ​​the picture.

[0335] Fig.12 An example of raster scan stripe partitioning of a picture is shown, where the picture is divided into 12 slices and 3 raster scan stripes.

[0336] Fig.13An example of rectangular strip partitioning of a picture is shown, where the picture is divided into 24 slices (6 slice columns and 4 slice rows) and 9 rectangular strips.

[0337] Fig.14 An example of a picture partitioned into slices and rectangular strips is shown, where the picture is divided into 4 slices (2 slice columns and 2 slice rows) and 4 rectangular strips.

[0338] Fig.15 An example of sub-picture segmentation of a picture is shown, where the picture is segmented into 28 sub-pictures of different dimensions. [Ed.(RS): Fig.15 Integrate into Fig.14 helps illustrate how these concepts fit together.]

[0339] When a picture is coded using three separate colour planes (separate_colour_plane_flag equal to 1), a slice consists of only CTUs of one colour component identified by the corresponding value of colour_plane_id, and each colour component array of the picture consists of slices with the same colour_plane_id value. Codec slices with different colour_plane_id values ​​within a picture may be interleaved with each other under the following constraint: For each colour_plane_id value, the codec slice NAL units with that colour_plane_id value shall be arranged in increasing order of CTU addresses in the slice scan order of the first CTU of each codec slice NAL unit.

[0340] When separate_colour_plane_flag is equal to 0, each CTU of the picture is included in exactly one slice. When separate_colour_plane_flag is equal to 1, each CTU of the color component is included in exactly one slice (ie, information of each CTU of the picture exists in exactly three slices, and the three slices have different colour_plane_id values).

[0341] 3. Disadvantages of existing implementations

[0342] The padding method for the ALF virtual boundary can be expressed as "double-sided padding" or "mirror padding", where if a sample at (i, j) is padded, the corresponding sample at (m, n) sharing the same filter coefficient is also padded, even if the sample is available, as shown in Figures 6 and 7.

[0343] The padding method for picture borders / 360-degree video virtual borders, normal borders (e.g., top and bottom borders) can be expressed as "single-sided padding" or "repeated padding", where if a sample to be used is outside the border, it is copied from an available sample inside the picture (e.g., the nearest available sample).

[0344] The current CC-ALF design has the following problems:

[0345] 1. The padding method used in CC-ALF is not fully aligned with the padding method used in nonlinear ALF.

[0346] 2. The on / off control of CC-ALF is in the CTB level. That is, if enabled, all chroma samples within a CTB will be modified by adding an offset derived from the filtered luma sample corresponding to one chroma sample.

[0347] 3. For each chroma sample to which both CC-ALF and chroma ALF are applied, at least 3 clipping operations from three steps are required (chroma ALF filtering; CC-ALF offset derivation; and refinement of chroma filter samples to derive final chroma samples). More specifically, the three stages are described as follows:

[0348] a. Clipping the filtered chroma samples during the chroma ALF process, i.e.,

[0349] sum chromaALF =curr+(sum chromaALF +64)>>7) (8 1290)

[0350] alfPicture[xCtbC+x][yCtbC+y]=Clip3(0,(1< <BitDepthC)–1,sum chromaALF ) (8-1291)

[0351] Where "curr" represents the chrominance sample before filtering; "sum chromaALF " represents the sum of the chroma ALF filter coefficients multiplied by the clipped chroma sample differences in the chroma ALF filter support

[0352] b. Clip the derived offset (represented by sum) to a range according to the chroma bit depth, i.e.,

[0353] sum=Clip3(-(1<<(BitDepthC–1)),(1<<(BitDepthC-1))–1,sum) (8-1292)

[0354] Where "sum" represents the sum of the CC-ALF filter coefficients multiplied by the luma samples in the CC-ALF filter support

[0355] c. After adding the derived offset to the filtered chroma samples, the final refined chroma samples are clipped:

[0356] sum=alfPicture[xCtbC+x][yCtbC+y]+(sum+64)>>(7+(BitDepthY–BitDepthC)) (8-1293)

[0357] ccAlfPicture[xCtbC+x][yCtbC+y]=Clip3(0,(1< <BitDepth C )–1,sum) (8-1294)

[0358] Where "alfPicture[xCtbC+x][yCtbC+y]" represents the chroma samples after filtering with the chroma ALF (output of stage a); "sum" represents the CC-ALF filter coefficients multiplied by the sum of the luma samples in the CC-ALF filter support (output of stage b); ccAlfPicture represents the final refined chroma samples (output of stage c, the final stage).

[0359] 4. As shown in equation (8-1293), the output of stage b (i.e., variable sum) is rounded with an offset equal to 64, which does not take into account (BitDepthY-BitDepthC). When (BitDepthY-BitDepthC) is not equal to 0, the rounding offset becomes suboptimal.

[0360] 4. Example Techniques and Implementations

[0361] The detailed embodiments described below should be considered as examples for explaining the general concept. These embodiments should not be interpreted narrowly. In addition, these embodiments can be combined in any way.

[0362] In the present invention, an ALF filter may refer to a filter applied to a given color component using information of a given color component (e.g., a luma ALF filter (linear or nonlinear) is applied to luma using luma information; a chroma ALF filter is applied to chroma using chroma information, e.g., a Cb chroma ALF filter is used to filter Cb samples; and a Cr chroma ALF filter is used to filter Cr samples); and a CC-ALF filter may refer to a filter applied to a first color component using information of at least a second color component different from the first color component information (e.g., the first color component may be Cb or Cr; the second color component may be luma).

[0363] In CC-ALF, a "correction of the sample" can be derived, which will be added to the sample before being processed by the ALF to generate a "refined sample"; or the "refined sample" can be derived directly.

[0364] A video unit may be a slice / tile / sub-picture / picture / 360-degree virtual picture (defined by a 360-degree virtual border) or other kinds of video regions containing multiple samples / pixels.

[0365] In the following discussion, a sample "at the boundary of a video unit" may mean that the distance between the sample and the boundary of the video unit is less than or not greater than a threshold value. A "line" may refer to samples that may be included at a same horizontal position or samples at a same vertical position (i.e., samples in the same row and / or samples in the same column). In one example, the function Abs(x) may be defined as follows:

[0366]

[0367] An "ALF processing unit" refers to a unit defined by two horizontal boundaries and two vertical boundaries. The two horizontal boundaries may include two ALF virtual boundaries or one ALF virtual boundary and one picture boundary (or slice boundary / tile boundary / sub-picture boundary / 360-degree virtual boundary). The two vertical boundaries may include two vertical CTU boundaries or one vertical CTU boundary and one picture boundary (or slice boundary / tile boundary / sub-picture boundary / 360-degree virtual boundary). Fig.16 An example is shown in .

[0368] A "narrow ALF processing unit" refers to a unit bounded by two horizontal boundaries and two vertical boundaries. One horizontal boundary may include an ALF virtual boundary or a 360-degree virtual boundary, and the other horizontal boundary may include a slice / brick / slice / sub-picture boundary or a 360-degree virtual boundary or a picture boundary. The vertical boundary may be a CTU boundary or a picture boundary or a 360-degree virtual boundary or a slice / brick / slice / sub-picture boundary. Fig.16 An example is shown in .

[0369] In the present disclosure, if a neighboring sample is outside the current video unit including the current sample (e.g., outside the current picture / sub-picture / slice / strip / brick / CTU) and filtering is not allowed across such video unit boundaries, or the neighboring sample and the current sample are on different sides of a 360-degree virtual boundary and filtering is not allowed across the 360-degree virtual boundary, then the neighboring sample may be marked as "unavailable". In yet another example, if a neighboring sample is outside a current processing unit (e.g., an ALF processing unit or a narrow ALF processing unit) in a different video unit and filtering is not allowed across the video unit, then the neighboring sample may be marked as "unavailable".

[0370] Similarly, for those samples to be filtered in CC-ALF (e.g., luma samples in the current CC-ALF design), they may also be marked as "unavailable" if the samples are located in a different video unit than the video unit that includes the current chroma samples.

[0371] 1. How to fill "unavailable" samples in CC-ALF may follow the same rules as those used for ALF.

[0372] a. In one example, “mirror padding” can be applied to fill the “unusable” samples at the ALF virtual boundary in CC-ALF.

[0373] b. In one example, "repeated filling" can be applied to fill in the "unavailable" samples at the strip / slice / brick / picture / sub-picture / 360 degree virtual boundaries in CC-ALF.

[0374] c. In one example, “repeat padding” can be applied to all boundaries in CC-ALF except the ALF virtual boundary.

[0375] i. Alternatively, "Repeat Fill" can be applied to all borders.

[0376] ii. Alternatively, "Repeat Fill" can be applied to all horizontal borders.

[0377] iii. Alternatively, "Repeat Fill" can be applied to all vertical borders.

[0378] iv. Alternatively, "Mirror Fill" can be applied to all horizontal borders.

[0379] v. Alternatively, "Mirror Fill" can be applied to all vertical borders.

[0380] vi. Alternatively, "Mirror Fill" can be applied to all borders.

[0381] d. In one example, "repetition padding" may be applied to an "ALF processing unit" or CTU or "narrow ALF processing unit" or "basic ALF processing unit" (defined below) or a video region other than all of these processing units.

[0382] e. How to apply repetitive padding may depend on whether adjacent samples are in different adjacent strips, where the current strip may be a raster scan strip and / or the adjacent strip may be a raster scan strip.

[0383] i. In one example, for samples in a raster scan strip, "repeated filling" can be applied to an "ALF processing unit". For example, in the current "ALF processing unit", "unavailable" adjacent samples are filled with their nearest samples. Fig.17 An example is shown in FIG. 4 , where the left adjacent samples of the current “ALF processing unit” are “unavailable”, and each “unavailable” sample is filled from its nearest sample in the current “ALF processing unit”.

[0384] 2. Unavailable neighboring samples of a processing unit (eg, ALF processing unit or / and narrow ALF processing unit or / and CTU) may be filled in a predefined order in ALF or / and CC-ALF as follows.

[0385] a. If the upper neighboring samples of a processing unit are not available, they can be filled with the top row(s) of the processing unit.

[0386] i. Alternatively, in addition, the upper left sample point(s) of the processing unit may be filled with the upper left sample point.

[0387] ii. Alternatively, in addition, the upper right sample point(s) of the processing unit may be filled with the upper right sample point.

[0388] b. If the lower neighboring samples of the processing unit are not available, they can be filled with the bottom row(s) of the processing unit.

[0389] i. Alternatively, in addition, the lower left sample point of the processing unit may be used to fill in the lower left adjacent sample point(s).

[0390] ii. Alternatively, in addition, the lower right sample point(s) of the processing unit may be filled with the lower right sample point.

[0391] c. If the left neighboring samples of a processing unit are not available, they can be filled with the left column(s) of the processing unit.

[0392] d. If the right neighboring samples of the processing unit are not available, they can be filled with the right column(s) of the processing unit.

[0393] e. If the left neighboring sample(s) and the top neighboring sample(s) of the processing unit are available, but the top left neighboring sample(s) of the processing unit are not available, then the top neighboring sample(s) may be used to fill the top left neighboring sample(s) of the processing unit.

[0394] i. Alternatively, the left-side neighboring sample(s) may be used to fill the (top-left) top-left neighboring sample of the processing unit.

[0395] f. If the upper left and above neighboring sample(s) of the processing unit are not available, and the upper left neighboring sample(s) of the processing unit are available, then the upper left neighboring sample(s) may be used to fill the upper left neighboring sample(s) of the processing unit.

[0396] g. If the top left and left neighboring sample(s) of the processing unit are not available, and the top neighboring sample(s) of the processing unit are available, then the top neighboring sample(s) may be used to fill the top left neighboring sample(s) of the processing unit.

[0397] h. If the right neighboring sample(s) and the bottom neighboring sample(s) of the processing unit are available, and the bottom right neighboring sample(s) of the processing unit are not available, then the bottom neighboring sample(s) may be used to fill the bottom right neighboring sample(s) of the processing unit.

[0398] i. Alternatively, the right-side neighboring sample(s) may be used to fill the bottom-right neighboring sample(s) of the processing unit.

[0399] i. If the lower right and lower neighboring sample(s) of the processing unit are not available, and the right neighboring sample(s) of the processing unit are available, then the right neighboring sample(s) may be used to fill the lower right neighboring sample(s) of the processing unit.

[0400] j. If the (multiple) lower right and right neighboring sample points of the processing unit are not available, and the (multiple) lower neighboring sample points of the processing unit are available, then the (multiple) lower neighboring sample points may be used to fill the (multiple) lower right neighboring sample points of the processing unit.

[0401] k. If the (multiple) top right and right neighboring sample points of the processing unit are not available, and the (multiple) top neighboring sample points of the processing unit are available, then the (multiple) top neighboring sample points may be used to fill the (multiple) top right neighboring sample points of the processing unit.

[0402] 1. If the top right and upper neighboring sample(s) of the processing unit are not available, and the top right neighboring sample(s) of the processing unit are available, then the top right neighboring sample(s) may be used to fill the top right neighboring sample(s) of the processing unit.

[0403] m. If the (multiple) lower left and left neighboring sample points of the processing unit are not available, and the (multiple) lower neighboring sample points of the processing unit are available, then the (multiple) lower neighboring sample points may be used to fill the (multiple) lower left neighboring sample points of the processing unit.

[0404] n. If the lower left and lower neighboring sample(s) of the processing unit are not available, and the left neighboring sample(s) of the processing unit are available, then the left neighboring sample(s) may be used to fill the lower left neighboring sample(s) of the processing unit.

[0405] o. A processing unit may include N rows of CTU represented by ctuUp (N is an integer, e.g., N=4) and CTB size-M rows of CTU represented by ctuDown below ctuUp (M is an integer, e.g., M=N). When checking whether neighboring samples of a processing unit are available, the relationship between neighboring samples and ctuDown may be considered.

[0406] i. In one example, if a neighboring sample is in a different video unit than the ctuDown (e.g., the neighboring sample and the ctuDown belong to different bricks / slices / strips / sub-pictures, or they are on different sides of the 360-degree virtual boundary), and filtering across such video units is not allowed, it is considered "unavailable".

[0407] ii. Alternatively, the availability of neighboring samples may be checked by considering the relationship between the neighboring samples and ctuUp.

[0408] 3. The "unavailable" samples in the raster scan slice can be filled in the CC-ALF or / and ALF in a predetermined order. The current CTU is in the current slice.

[0409] a. In one example, when the upper CTU and the left CTU are in the same slice as the current slice, and the upper-left CTU is in a slice different from the current slice, the "unavailable" neighboring samples in the upper-left CTU are filled with their nearest neighboring samples in the upper CTU of the current CTU. Fig.18 An example is shown in .

[0410] i. Alternatively, fill the "unavailable" neighboring samples in the upper left CTU with their nearest neighboring samples in the left CTU of the current CTU.

[0411] b. In one example, when the lower CTU and the right CTU are in the same slice as the current slice, and the lower right CTU is in a slice different from the current slice, the "unavailable" neighboring samples in the lower right CTU are filled with their nearest neighboring samples in the lower CTU of the current CTU. Fig.18 An example is shown in .

[0412] i. Alternatively, the "unavailable" neighboring samples in the lower right CTU are filled with their nearest neighboring samples in the right CTU of the current CTU.

[0413] 4. When a processing unit (e.g., an ALF processing unit or / and a narrow ALF processing unit or / and a CTU) is spanned by one or more brick / slice / slice / sub-picture boundaries or / and a 360-degree virtual boundary and filtering across such boundaries is not allowed, the processing unit may be split (horizontally or / and vertically) into multiple processing units.

[0414] a. Alternatively, further, the splitting process may be recursively performed until no processing unit is crossed by any brick / strip / slice / sub-picture boundary or / and 360 degree virtual boundary or / and ALF virtual boundary, wherein filtering processes crossing such boundaries are not allowed, e.g., such boundaries can only be boundaries of processing units. Such processing units are hereinafter referred to as "basic ALF processing units". If an "ALF processing unit" is not crossed by any such boundary, then the "basic ALF processing unit" may be an "ALF processing unit".

[0415] b. Alternatively, in addition, the ALF process or / and CC-ALF process is performed after such splitting process is completed, for example, the ALF process or / and CC-ALF is performed on a "basic ALF processing unit".

[0416] c. Alternatively, furthermore, the above-mentioned filling method may be performed on a "basic ALF processing unit".

[0417] d. When generating a "basic ALF processing unit", different horizontal boundaries may be checked in a predefined order.

[0418] i. In one example, the checking order is ALF virtual boundary, 360 degree virtual boundary and CTU boundary which is brick / slice / slice / sub-picture boundary.

[0419] ii. In one example, the checking order is 360 degree virtual boundary, ALF virtual boundary and CTU boundary which is brick / slice / slice / sub-picture boundary.

[0420] iii. In one example, when generating a "basic ALF processing unit", different checking orders may be used for top and bottom boundaries.

[0421] a) For example, for the top boundary, the checking order is ALF virtual boundary, 360 degree virtual boundary and CTU boundary which is brick / slice / slice / sub-picture boundary.

[0422] b) For example, for the bottom boundary, the checking order is 360 degree virtual boundary, ALF virtual boundary and CTU boundary which is brick / slice / slice / sub-picture boundary.

[0423] e. When generating a "basic ALF processing unit", different vertical boundaries may be checked in a predefined order.

[0424] i. In one example, the checking order is 360 degree virtual boundaries as CTU boundaries of brick / slice / slice / sub-picture boundaries.

[0425] ii. In one example, the checking order is CTU boundaries and 360 degree virtual boundaries as brick / slice / slice / sub-picture boundaries.

[0426] iii. In one example, when generating a "basic ALF processing unit", different checking orders may be used for the left and right boundaries.

[0427] 5. It is recommended that if filtering processes across video unit boundaries are not allowed, then the ALF and / or CC-ALF processes may be disabled for samples at such boundaries.

[0428] a. In one example, if at least N (N is an integer) "unavailable" samples are involved in the ALF or / and CC-ALF process of a sample, the ALF or / and CC-ALF process is skipped for the sample. For example, N=1 or 2 or 3, etc.

[0429] b. In one example, if a sample is at a horizontal video unit boundary and filtering across such a boundary is not allowed, and the vertical distance between the sample and the video unit boundary is less than or equal to M samples, then ALF and / or CC-ALF are skipped for the sample. For example, M=0 or 1 or 2 or 3.

[0430] c. In one example, if a sample is at a vertical video unit boundary and filtering across such a boundary is not allowed, and the horizontal distance between the sample and the video unit boundary is less than or equal to M samples, then ALF or / and CC-ALF is skipped for the sample. For example, M=0 or 1 or 2 or 3.

[0431] 6. An on / off control that can determine ALF and / or CC-ALF for video areas other than CTB is proposed.

[0432] a. In one example, a video region is an "ALF processing unit" or a "narrow ALF processing unit" or a "basic ALF processing unit".

[0433] b. In one example, on / off control may be explicitly signaled for the video region.

[0434] c. In one example, on / off control can be derived implicitly for a video region.

[0435] 7. The signaled CC-ALF on / off control flag may be associated with the first video unit (eg, CTB), however, for samples within the first video unit, CC-ALF may be disabled even if the signaled flag tells that CC-ALF is enabled.

[0436] a. In one example, whether CC-ALF is enabled or disabled for samples within the first video unit may depend on the location of the samples.

[0437] b. In one example, whether CC-ALF is enabled or disabled for samples within the first video unit may depend on luma samples involved in the CC-ALF process.

[0438] c. In one example, whether CC-ALF is enabled or disabled for a sample within the first video unit may depend on the current sample and its neighboring samples, such as the difference between them.

[0439] d. In one example, whether CC-ALF is enabled or disabled for samples within the first video unit may depend on the derived offset.

[0440] i. In one example, if the derived offset is greater than (or not less than) a threshold, the offset may be discarded, ie CC-ALF is disabled (equivalent to the case where the offset is reset to 0).

[0441] ii. In one example, if the derived offset is less than (or not greater than) a threshold, the offset may be discarded, ie, CC-ALF is disabled (equivalent to the case where the offset is reset to 0).

[0442] e. In one example, whether CC-ALF is enabled or disabled for samples within the first video unit may depend on coded information of a block containing chroma samples and / or neighboring blocks.

[0443] 8. It is recommended to reduce the number of stages of the tailoring process from 3 (as described in Section 3, point 3) to 1 or 2.

[0444] a. In one example, if CC-ALF is enabled, the clipping process in the chroma ALF filtering process is omitted.

[0445] b. In one example, the clipping process of the derived offset in the CC-ALF process is omitted.

[0446] c. In one example, both of the above methods are applied.

[0447] 9. The clipping range of the offset derived in the CC-ALF process can be set to a range other than [-(1<<(BitDepthC-1)),(1<<(BitDepthC-1))–1] (inclusive of the endpoints).

[0448] a. In one example, a smaller range may be utilized.

[0449] b. In one example, the range may be independent of the bit depth.

[0450] c. In one example, the range may adaptively change from a first video region to another video region.

[0451] d. In one example, this range can be derived from the non-linear clipping range used in the chroma ALF process (eg, k(i,j) in equation (14)).

[0452] 10. The derived offset (eg, sum in bullet c of section 3) may be rounded with a rounding offset that depends on the bit depth information instead of a fixed value (ie, 64).

[0453] a. In one example, the rounding offset may depend on the bit depth difference between a first color component (eg, Y) and a second color component (eg, Cbr or Cr).

[0454] i. Alternatively, furthermore, it may also depend on the bit depth of the CC-ALF filter coefficients.

[0455] ii. In one example, the rounding offset may be set to (1<<(K–1+(BitDepthY–BitDepthC))), where K represents the bit depth of the CC-ALF filter coefficients; BitDepthY and BitDepthC represent the bit depths of the luma and chroma components, respectively.

[0456] b. In one example, the following equation can be used to replace equation (8-1293). Deleted text is marked with double brackets (e.g., [[a]] indicates that the character "a" is deleted). sum = alfPicture[xCtbC+x][yCtbC+y]+(sum+Off[

[64] ])>>(7+(BitDepthY-BitDepthC)) (8-1293)ccAlfPicture[xCtbC+x][yCtbC+y]=Clip3(0,(1< <BitDepth C )-1,sum') (8-1294)

[0457] Where Off is set equal to (1<<(6+(BitDepthY–BitDepthC))).

[0458] 11. It is suggested that chroma ALF and CC-ALF can be optimized jointly.

[0459] a. In one example, when both chroma ALF and CC-ALF are enabled for a CTB or video region, if CC-ALF is applied after chroma ALF, the clipping operation may be applied only in the CC-ALF process (e.g., the chroma samples generated by the chroma ALF or / and CC-ALF may be clipped to a predefined range that may depend on the chroma bit depth).

[0460] b. In one example, when both chroma ALF and CC-ALF are enabled for a CTB or video region, if CC-ALF is applied before chroma ALF, the clipping operation may be applied only during the chroma ALF process (e.g., clipping the chroma samples generated by the chroma ALF or / and CC-ALF to within a predetermined range that may depend on the chroma bit depth).

[0461] 12. Assuming that the sample S in the first color component is modified to S=S+D, where D is an offset derived from an adaptive loop filtering method using information of the second color component, such as CC-ALF, then D can be adaptively clipped. For example, S can be modified to S=S+Clip3(Dlow, Dhigh, D), where Dlow and / or Dhigh can be different for different samples.

[0462] a. For example, Dlow and / or Dhigh may be signaled from the codec to the decoder.

[0463] b. For example, Dlow and / or Dhigh can be derived based on the value of S.

[0464] c. For example, Dlow and / or Dhigh can be derived based on the position of S.

[0465] d. For example, several candidates for Dlow and / or Dhigh may be signaled or predefined. The selected candidate may be signaled from the codec, or derived at the decoder.

[0466] 13. Assuming that the CC-ALF on the first sample S in the first sub-picture can depend on the sample(s) in the second sub-picture, determine whether S is CC-ALF filtered according to whether loop filtering across sub-pictures is allowed for the first sub-picture and / or the second sub-picture. (e.g., loop_filter_across_subpic_enabled_flag of the first sub-picture and / or the second sub-picture.

[0467] a. In one example, if cross-sub-picture loop filtering is not allowed for the first sub-picture, S is not filtered by CC-ALF.

[0468] b. In one example, if cross-sub-picture loop filtering is not allowed for the second sub-picture, S is not filtered by CC-ALF.

[0469] c. In one example, if cross-sub-picture loop filtering is not allowed for the first sub-picture and the second sub-picture, S is not filtered by CC-ALF.

[0470] d. In one example, if cross-sub-picture loop filtering is not allowed for the first sub-picture or the second sub-picture, S is not filtered by CC-ALF.

[0471] 14. Whether and / or how the above methods are applied may be based on one or more of the following conditions:

[0472] a. Video content (e.g. screen content or natural content)

[0473] b. Messages signaled in DPS / SPS / VPS / PPS / APS / Picture Header / Slice Header / Slice Group Header / Largest Codec Unit (LCU) / Codec Unit (CU) / LCU Line / LCU Group / TU / PU Block / Video Codec Unit

[0474] c. Location of CU / PU / TU / block / video codec unit

[0475] d. Decoded information of the current block and / or its neighboring blocks

[0476] i. Block latitude / block shape of the current block and / or its neighboring blocks

[0477] e. Indication of the color format (such as 4:2:0, 4:4:4, RGB, or YUV)

[0478] f. Codec tree structure (such as dual tree or single tree)

[0479] g. Stripe / stripe group type and / or picture type

[0480] h. Color components (e.g., may only apply to luma components and / or chroma components)

[0481] i. Temporal layer ID

[0482] j. Standard Profiles / Levels / Tiers

[0483] 5. Additional Embodiments

[0484] In this document, textual changes in the VVC draft are shown in bold underlined text.

[0485] 6. Example Implementations of the Disclosed Technology

[0486] Fig.19 A is a block diagram of a video processing device 1900. Device 1900 can be used to implement one or more methods described herein. Device 1900 can be embodied in a smartphone, a tablet computer, a computer, an Internet of Things (IoT) receiver, etc. Device 1900 may include one or more processors 1902, one or more memories 1904, and video processing hardware 1906. (Multiple) processors 1902 can be configured to implement one or more methods described in this document. Memory 1904 can be used to store data and code for implementing the methods and techniques described herein. (Multiple) video processing hardware 1906 can be used to implement some of the techniques described in this document in hardware circuits, and can be partially or completely part of a processor 1902 (e.g., a graphics processor core GPU or other signal processing circuit).

[0487] Fig.21 2100 is a block diagram illustrating an example video processing system 2100 in which various techniques disclosed herein may be implemented. Various implementations may include some or all of the components of the system 2100. The system 2100 may include an input 2102 for receiving video content. The video content may be received in a raw or uncompressed format, such as 8 or 10 bit multi-component pixel values, or may be in a compressed or encoded format. The input 2102 may represent a network interface, a peripheral bus interface, or a storage interface. Examples of network interfaces include wired interfaces such as Ethernet, a passive optical network (PON), and wireless interfaces such as Wi-Fi or a cellular interface.

[0488] System 2100 may include codec component 2104, which can implement various codecs or coding methods described in this document. Codec component 2104 can reduce the average bit rate of the video from input 2102 to the output of codec component 2104 to generate the codec representation of the video. Therefore, codec technology is sometimes referred to as video compression or video transcoding technology. As represented by component 2106, the output of codec component 2104 can be stored, or sent via connected communication. Component 2108 can use the storage or transmitted bitstream (or codec) representation of the video received at input 2102 to generate pixel values ​​or displayable video sent to display interface 2110. The process of generating user-visible video from bitstream representation is sometimes referred to as video decompression. In addition, although some video processing operations are referred to as "codec" operations or tools, it should be understood that codec tools or operations are used at encoders, and corresponding decoding tools or operations opposite to codec results will be performed by decoders.

[0489] Examples of peripheral bus interfaces or display interfaces may include Universal Serial Bus (USB) or High Definition Multimedia Interface (HDMI) or Display Port, etc. Examples of storage interfaces include SATA (Serial Advanced Technology Attachment), PCI, IDE interfaces, etc. The techniques described in this document may be implemented in various electronic devices, such as mobile phones, laptops, smartphones, or other devices capable of performing digital data processing and / or video display.

[0490] Fig. 22 is a block diagram illustrating an example video coding system 100 that may utilize the techniques of this disclosure.

[0491] like Fig. 22 As shown, the video codec system 100 may include a source device 110 and a destination device 120. The source device 110 generates encoded video data, which may be referred to as a video encoding device. The destination device 120 may decode the encoded video data generated by the source device 110, which may be referred to as a video decoding device.

[0492] Source device 110 may include a video source 112 , a video encoder 114 , and an input / output (I / O) interface 116 .

[0493] The video source 112 may include a source such as a video capture device, an interface for receiving video data from a video content provider, and / or a computer graphics system for generating video data, or a combination of these sources. The video data may include one or more pictures. The video encoder 114 encodes the video data from the video source 112 to generate a bit stream. The bit stream may include a bit sequence that forms a coded representation of the video data. The bit stream may include coded pictures and associated data. The coded picture is a coded representation of the picture. The associated data may include a sequence parameter set, a picture parameter set, and other syntax structures. The I / O interface 116 may include a modulator / demodulator (modem) and / or a transmitter. The coded video data may be directly transmitted to the destination device 120 via the I / O interface 116 through the network 130a. The coded video data may also be stored on a storage medium / server 130b for access by the destination device 120.

[0494] Destination device 120 may include an I / O interface 126 , a video decoder 124 , and a display device 122 .

[0495] The I / O interface 126 may include a receiver and / or a modem. The I / O interface 126 may obtain the encoded video data from the source device 110 or the storage medium / server 130b. The video decoder 124 may decode the encoded video data. The display device 122 may display the decoded video data to the user. The display device 122 may be integrated with the destination device 120, or may be external to the destination device 120, and the destination device 120 is configured to interface with an external display device.

[0496] The video encoder 114 and the video decoder 124 may operate according to a video compression standard, such as the High Efficiency Video Codec (HEVC) standard, the Versatile Video Codec (VVC) standard, and other current and / or future standards.

[0497] Fig.23 is a block diagram showing an example of a video encoder 200, which may be Fig. 22 The video encoder 114 in the system 100 is shown in FIG.

[0498] Video encoder 200 may be configured to perform any or all of the techniques of this disclosure. Fig.23 In the example of , video encoder 200 includes multiple functional components. The techniques described in this disclosure may be shared between various components of video encoder 200. In some examples, a processor may be configured to perform any or all of the techniques described in this disclosure.

[0499] The functional components of the video encoder 200 may include a segmentation unit 201, a prediction unit 202, a residual generation unit 207, a transform unit 208, a quantization unit 209, an inverse quantization unit 210, an inverse transform unit 211, a reconstruction unit 212, a buffer 213 and an entropy coding unit 214, wherein the prediction unit 202 may include a mode selection unit 203, a motion estimation unit 204, a motion compensation unit 205 and an intra-frame prediction unit 206.

[0500] In other examples, the video encoder 200 may include more, fewer, or different functional components. In one example, the prediction unit 202 may include an intra block copy (IBC) unit. The IBC unit may perform prediction in an IBC mode in which at least one reference picture is a picture in which the current video block is located.

[0501] Furthermore, some components (eg, motion estimation unit 204 and motion compensation unit 205) may be highly integrated but are not shown for purposes of explanation. Fig.23 are represented separately in the examples.

[0502] The partitioning unit 201 may partition a picture into one or more video blocks. The video encoder 200 and the video decoder 300 may support various video block sizes.

[0503] The mode selection unit 203 may select one of the coding modes (intra or inter), for example, based on the error result, and provide the resulting intra or inter coded block to the residual generation unit 207 to generate residual block data, and to the reconstruction unit 212 to reconstruct the coded block for use as a reference picture. In some examples, the mode selection unit 203 may select a mode of combination of intra and inter prediction (CIIP), in which the prediction is based on an inter prediction signal and an intra prediction signal. In the case of inter prediction, the mode selection unit 203 may also select a resolution of motion vectors for the block (e.g., sub-pixel or integer pixel precision).

[0504] In order to perform inter-frame prediction on the current video block, the motion estimation unit 204 may generate motion information of the current video block by comparing the current video block with one or more reference frames from the buffer 213. The motion compensation unit 205 may determine a predicted video block of the current video block based on the motion information of pictures other than the picture associated with the current video block from the buffer 213 and decoded samples.

[0505] Motion estimation unit 204 and motion compensation unit 205 may perform different operations on the current video block, eg, depending on whether the current video block is in an I slice, a P slice, or a B slice.

[0506] In some examples, the motion estimation unit 204 may perform unidirectional prediction on the current video block, and the motion estimation unit 204 may search for a reference video block of the current video block in the reference picture of list 0 or list 1. The motion estimation unit 204 may then generate a reference index indicating the reference picture containing the reference video block in list 0 or list 1 and a motion vector indicating a motion vector of a spatial displacement between the current video block and the reference video block. The motion estimation unit 204 may output the reference index, the prediction direction indicator, and the motion vector as the motion information of the current video block. The motion compensation unit 205 may generate a predicted video block of the current block based on the reference video block indicated by the motion information of the current video block.

[0507] In other examples, the motion estimation unit 204 may perform bidirectional prediction on the current video block, the motion estimation unit 204 may search for a reference video block of the current video block in the reference pictures in list 0, and may also search for another reference video block of the current video block in the reference pictures in list 1. The motion estimation unit 204 may then generate a reference index indicating the reference pictures in list 0 and list 1 that include the reference video block and a motion vector indicating a spatial displacement between the reference video block and the current video block. The motion estimation unit 204 may output the reference index and the motion vector of the current video block as the motion information of the current video block. The motion compensation unit 205 may generate a predicted video block of the current video block based on the reference video block indicated by the motion information of the current video block.

[0508] In some examples, motion estimation unit 204 may output a complete set of motion information for use in the decoding process of a decoder.

[0509] In some examples, motion estimation unit 204 may not output a complete set of motion information for the current video. Instead, motion estimation unit 204 may reference motion information of another video block to signal motion information for the current video block. For example, motion estimation unit 204 may determine that the motion information of the current video block is sufficiently similar to the motion information of a neighboring video block.

[0510] In one example, motion estimation unit 204 may indicate a value in a syntax structure associated with the current video block that indicates to video decoder 300 that the current video block has the same motion information as another video block.

[0511] In another example, motion estimation unit 204 may identify another video block and a motion vector difference (MVD) in a syntax structure associated with the current video block. The motion vector difference indicates the difference between the motion vector of the current video block and the motion vector of the indicated video block. Video decoder 300 may use the motion vector of the indicated video block and the motion vector difference to determine the motion vector of the current video block.

[0512] As described above, the video encoder 200 may predictively signal motion vectors. Two examples of predictive signaling techniques that may be implemented by the video encoder 200 include advanced motion vector prediction (AMVP) and merge mode signaling.

[0513] The intra prediction unit 206 may perform intra prediction on the current video block. When the intra prediction unit 206 performs intra prediction on the current video block, the intra prediction unit 206 may generate prediction data for the current video block based on decoded samples of other video blocks in the same picture. The prediction data for the current video block may include a prediction video block and various syntax elements.

[0514] The residual generation unit 207 may generate residual data for the current video block by subtracting (eg, indicated by a minus sign) the predicted video block of the current video block from the current video block. The residual data for the current video block may include residual video blocks corresponding to different sample components of samples in the current video block.

[0515] In other examples, there may be no residual data for the current video block, such as in skip mode, and the residual generation unit 207 may not perform a subtraction operation.

[0516] Transform processing unit 208 may generate one or more transform coefficient video blocks for the current video block by applying one or more transforms to the residual video block associated with the current video block.

[0517] After transform processing unit 208 generates a transform coefficient video block associated with the current video block, quantization unit 209 may quantize the transform coefficient video block associated with the current video block based on one or more quantization parameter (QP) values ​​associated with the current video block.

[0518] The inverse quantization unit 210 and the inverse transform unit 211 may apply inverse quantization and inverse transform to the transform coefficient video block, respectively, to reconstruct a residual video block from the transform coefficient video block. The reconstruction unit 212 may add the reconstructed residual video block to corresponding samples from one or more prediction video blocks generated by the prediction unit 202 to generate a reconstructed video block associated with the current block for storage in the buffer 213.

[0519] After reconstruction unit 212 reconstructs the video block, a loop filtering operation may be performed to reduce video blockiness artifacts in the video block.

[0520] The entropy coding unit 214 may receive data from other functional components of the video encoder 200. When the entropy coding unit 214 receives the data, the entropy coding unit 214 may perform one or more entropy coding operations to generate entropy-coded data and output a bitstream including the entropy-coded data.

[0521] Fig.24 is a block diagram illustrating an example of a video decoder 300, which may be Fig. 22 The video decoder 114 in the system 100 is shown.

[0522] Video decoder 300 may be configured to perform any or all of the techniques of this invention. Fig.24 In the example of , video decoder 300 includes multiple functional components. The techniques described in this disclosure can be shared between various components of video decoder 300. In some examples, a processor can be configured to perform any or all of the techniques described in this disclosure.

[0523] exist Fig.24 In the example of FIG. 3 , the video decoder 300 includes an entropy decoding unit 301, a motion compensation unit 302, an intra-frame prediction unit 303, an inverse quantization unit 304, an inverse transform unit 305, a reconstruction unit 306, and a buffer 307. In some examples, the video decoder 300 may perform operations that are generally similar to those for the video encoder 200 (e.g., Fig.23 ) is the reverse of the encoding pass described in .

[0524] The entropy decoding unit 301 may retrieve an encoded bitstream. The encoded bitstream may include entropy-encoded video data (e.g., encoded video data blocks). The entropy decoding unit 301 may decode the entropy-encoded video data, and based on the entropy-encoded video data, the motion compensation unit 302 may determine motion information, including motion vectors, motion vector precision, reference picture list indexes, and other motion information. The motion compensation unit 302 may determine such information, for example, by performing AMVP and merge modes.

[0525] Motion compensation unit 302 may generate a motion compensated block, possibly performing interpolation based on an interpolation filter. An identifier of an interpolation filter to be used with sub-pixel precision may be included in the syntax element.

[0526] Motion compensation unit 302 may calculate interpolation of sub-integer pixels of a reference block using interpolation filters as used by video encoder 200 during encoding of the video block. Motion compensation unit 302 may determine the interpolation filters used by video encoder 200 based on received syntax information and use the interpolation filters to generate a prediction block.

[0527] The motion compensation unit 302 may use some syntax information to determine the sizes of blocks used to encode (multiple) frames and / or (multiple) slices of the encoded video sequence, partitioning information describing how each macroblock of a picture of the encoded video sequence is partitioned, a mode indicating how each partition is encoded, one or more reference frames (and reference frame lists) for each inter-frame coded block, and other information for decoding the encoded video sequence.

[0528] The intra prediction unit 303 may form a prediction block from spatially adjacent blocks using, for example, an intra prediction mode received in the bitstream. The inverse quantization unit 303 inversely quantizes (i.e., dequantizes) the quantized video block coefficients provided in the bitstream and decoded by the entropy decoding unit 301. The inverse transform unit 303 applies an inverse transform.

[0529] The reconstruction unit 306 may add the residual block to the corresponding prediction block generated by the motion compensation unit 202 or the intra prediction unit 303 to form a decoded block. If necessary, a deblocking filter may also be applied to filter the decoded block to remove blocking artifacts. The decoded video block is then stored in a buffer 307, which provides a reference block for subsequent motion compensation.

[0530] Some embodiments of the disclosed technology include making a decision or determination to enable a video processing tool or mode. In one example, when a video processing tool or mode is enabled, the codec will use or implement the tool or mode in the processing of a video block, but may not necessarily modify the resulting bitstream based on the use of the tool or mode. That is, when a video processing tool or mode is enabled based on a decision or determination, the conversion from a video block to a bitstream representation of a video will use the video processing tool or mode. In another example, when a video processing tool or mode is enabled, the decoder will process the bitstream knowing that the bitstream has been modified based on the video processing tool or mode. That is, the conversion from the bitstream representation of a video to a video block will be performed using the video processing tool or mode enabled based on the decision or determination.

[0531] In this document, the term "video processing" may refer to video encoding, video decoding, video compression, or video decompression. For example, during the conversion from a pixel representation of a video to a corresponding bitstream representation, a video compression algorithm may be applied, or vice versa. The bitstream representation of the current video block may correspond, for example, to bits that are co-located or dispersed at different locations within the bitstream, as defined by the syntax. For example, a macroblock may be encoded based on a transformed and encoded error residual value, and also using bits in the header and other fields in the bitstream.

[0532] It will be appreciated that the disclosed methods and techniques will benefit video encoder and / or decoder embodiments incorporated into video processing devices such as smartphones, laptops, desktop computers, and similar devices by enabling use of the techniques disclosed in this document.

[0533] Fig. 20 2000. The method 2000 includes, at 2010, performing a conversion between a video unit of a visual media and a codec representation of the visual media, wherein, during the conversion, one or more rules for using an adaptive loop filter (ALF) are applied to fill in unavailable samples associated with the use of a cross-component adaptive loop filter (CC-ALF).

[0534] Some embodiments may be described using the following clause-based format. The first set of clauses illustrates example embodiments of the techniques discussed in the previous sections.

[0535] 1. A video processing method, comprising: performing conversion between a video unit of a visual media and a codec representation of the visual media, wherein during the conversion, one or more rules for using an adaptive loop filter (ALF) are applied to fill in unavailable samples associated with the use of a cross-component adaptive loop filter (CC-ALF).

[0536] 2. The method of clause 1, wherein the ALF filters applied to the color components of the video unit are defined using information provided by the color components of the video unit.

[0537] 3. A method as described in any one or more of clauses 1 to 2, wherein a CC-ALF filter applied to a first color component of a video unit is defined using information provided by at least a second color component of the video unit.

[0538] 4. A method according to any one or more of clauses 1 to 3, wherein the video unit corresponds to a slice, a tile, a sub-picture, a picture, or a 360 degree virtual picture defined by a 360 degree virtual boundary.

[0539] 5. A method according to any one or more of clauses 1 to 4, wherein rules for mirror padding are applied to unavailable samples at one or more ALF virtual boundaries associated with the use of CC-ALF.

[0540] 6. A method as described in any one or more of clauses 1 to 5, wherein rules for repeated filling are applied to unavailable samples at one of the slices, slices, tiles, pictures, sub-pictures or 360 degree virtual boundaries associated with the use of CC-ALF.

[0541] 7. A method according to any of clauses 1 to 6, wherein rules for repetition padding are applied at one or more boundaries except at least one or more ALF virtual boundaries associated with the use of CC-ALF.

[0542] 8. The method of clause 7, wherein the one or more boundaries include one of: all boundaries, horizontal boundaries, or vertical boundaries.

[0543] 9. A method according to any of clauses 1 to 6, wherein rules for mirror padding are applied at one or more boundaries except at one or more ALF virtual boundaries associated with the use of CC-ALF.

[0544] 10. The method of clause 9, wherein the one or more boundaries include one of: all boundaries, horizontal boundaries, or vertical boundaries.

[0545] 11. A method according to any one or more of clauses 1 to 10, wherein the video unit corresponds to an ALF processing unit, a codec tree unit (CTU) or a basic ALF processing unit.

[0546] 12. The method of clause 11, wherein the ALF processing unit defines a unit bounded by two horizontal boundaries and two vertical boundaries.

[0547] 13. The method of clause 11, wherein the two horizontal boundaries comprise one of: two ALF virtual boundaries, or an ALF virtual boundary and a boundary of a picture / slice / tile / brick / sub-picture, or a 360 degree virtual boundary.

[0548] 14. A method according to clause 11, wherein a basic ALF processing unit is generated based on recursively partitioning the ALF processing unit, CTU or narrow ALF processing unit, and wherein the boundaries of the basic ALF processing unit are not crossed by brick / strip / slice / sub-picture boundaries, and / or 360-degree virtual boundaries, and / or ALF virtual boundaries that do not allow cross-boundary filtering.

[0549] 15. A method as described in any one or more of clauses 1 to 14, wherein the rules for repetition padding are based on whether adjacent samples of a video unit are in different adjacent slices.

[0550] 16. The method of clause 15, wherein a current slice of the video unit is a raster scan slice, and wherein a neighboring slice is also a raster scan slice.

[0551] 17. A method according to any one or more of clauses 15 to 16, wherein the current video unit is an ALF processing unit, and wherein unavailable neighbouring samples are filled in the ALF processing unit using its nearest samples.

[0552] 18. A video processing method, comprising:

[0553] Conversion between video units of visual media and codec representations of the visual media is performed, wherein during the conversion, one or more rules for filling unavailable samples of the video units specify a predefined filling order associated with use of an ALF process or a CC-ALF process.

[0554] 19. The method of clause 18, wherein the predefined filling order comprises using at least one top row of the video unit to fill unavailable samples.

[0555] 20. The method of clause 18, wherein the predefined filling order comprises using at least one bottom row of the video unit to fill unavailable samples.

[0556] 21. The method of clause 18, wherein predefining the filling order comprises using at least one left column of the video unit to fill unavailable samples.

[0557] 22. The method of clause 18, wherein predefining the filling order comprises using at least one right column of the video unit to fill unavailable samples.

[0558] 23. A video processing method, comprising:

[0559] Conversion between a video unit of visual media and a codec representation of the visual media is performed, wherein during the conversion, one or more rules for determining whether one or more neighboring samples of the video unit are available include a relationship between the one or more neighboring samples and a parameter associated with the video unit.

[0560] 24. A method according to clause 23, wherein a parameter associated with a video unit (denoted ctuDown) is calculated as CTU down = CTB size - M, where M is an integer.

[0561] 25. The method of clause 24, wherein if the neighboring sample is in a second video unit having a different value than ctuDown, the one or more rules specify that filtering across the second video unit is not allowed.

[0562] 26. A method according to clause 23, wherein the one or more rules additionally comprise a relationship between one or more adjacent samples and a ctuUp parameter based on a number of lines of the video unit.

[0563] 27. A method according to any one or more of clauses 18 to 26, wherein the unusable sample points are adjacent sample points.

[0564] 28. A method according to any one or more of clauses 18 to 26, wherein unusable samples are included in a raster scan strip.

[0565] 29. The method of clause 28, wherein unavailable neighboring samples spatially located in a top-left CTU relative to the video unit are filled with their nearest neighboring samples spatially located in an upper CTU relative to the video unit.

[0566] 30. The method of clause 28, wherein unavailable neighboring samples spatially located in a lower right CTU relative to the video unit are filled with their nearest neighboring samples spatially located in a lower CTU relative to the video unit.

[0567] 31. A video processing method, comprising:

[0568] Conversion between video units of visual media and codec representations of the visual media is performed, wherein during the conversion, one or more rules for applying ALF and / or CC-ALF specify that application of ALF and / or CC-ALF is prohibited for samples located at a boundary of the video unit if filtering across the boundary is not allowed.

[0569] 32. A method as described in clause 31, wherein applying ALF and / or CC-ALF is disabled upon determining that at least N neighboring samples associated with ALF and / or CC-ALF are not available.

[0570] 33. The method of clause 31, wherein the boundary of the video unit is a horizontal boundary, and wherein applying ALF and / or CC-ALF is disabled upon determining that the vertical distance between the sample and the horizontal boundary of the video unit is less than or equal to M samples.

[0571] 34. The method of clause 31, wherein the boundary of the video unit is a vertical boundary, and wherein applying ALF and / or CC-ALF is disabled upon determining that the horizontal distance between the sample and the vertical boundary of the video unit is less than or equal to M samples.

[0572] 35. A video processing method, comprising:

[0573] A conversion between a first video unit of visual media and a coded representation of the visual media is performed, wherein during the conversion, one or more rules for ALF and / or CC-ALF are applied to specify selectively turning ALF and / or CC-ALF on or off for a second video unit different from the first video unit.

[0574] 36. A method as described in clause 35, wherein whether ALF and / or CC-ALF is turned on or off for the second video unit is explicitly indicated in the codec representation of the second video unit.

[0575] 37. The method of clause 35, wherein whether ALF and / or CC-ALF is turned on or off for the second video unit is derived from a codec representation of the second video unit.

[0576] 38. A method according to any one or more of clauses 35 to 37, wherein a flag associated with the first video unit indicates enabling of ALF and / or CC-ALF, and wherein during conversion, based on one or more conditions, applying ALF and / or CC-ALF is selectively disabled for samples located in the first video unit.

[0577] 39. The method of clause 38, wherein the one or more conditions include a spatial position of the sample relative to the first video unit.

[0578] 40. The method of clause 38, wherein the one or more conditions include luma sample values ​​used in CC-ALF.

[0579] 41. The method of clause 38, wherein the one or more conditions include a sample point value difference between the sample point and its neighboring sample points.

[0580] 42. The method of clause 38, wherein the one or more conditions comprise a relationship between an offset value derived using filtered luma samples corresponding to chroma samples and a threshold value.

[0581] 43. A video processing method, comprising:

[0582] A conversion between a first video unit of a visual media and a codec representation of the visual media is performed, wherein during the conversion, one or more rules of ALF and / or CC-ALF are selectively applied to specify the omission of at least one of three cropping operations, wherein a first cropping operation corresponds to chroma ALF filtering, a second cropping operation corresponds to CC-ALF offset derivation, and a third cropping operation corresponds to refinement of chroma filter samples to derive final chroma sample values.

[0583] 44. The method of clause 43, wherein the one or more rules for applying ALF and / or CC-ALF specify that the first cropping operation is omitted if applying CC-ALF is enabled.

[0584] 45. The method of clause 43, wherein the one or more rules for applying ALF and / or CC-ALF specify that the second cropping operation is omitted if applying CC-ALF is disabled.

[0585] 46. ​​The method of clause 43, wherein applying one or more rules of ALF and / or CC-ALF specifies that the first clipping operation and the second clipping operation are omitted.

[0586] 47. A method according to clause 43, wherein, during the second cropping operation, the derived CC-ALF offset is defined to be within a first range different from a second range, the second range being represented as [-(1<<(BitDepthC-1)),(1<<(BitDepthC-1))–1], wherein the endpoints of the second range are included in the second range, and wherein BitDepthC is a bit depth value.

[0587] 48. A method according to clause 47, wherein the first range is independent of the bit depth value.

[0588] 49. The method of clause 47, wherein the third range associated with the second video unit is defined to be different from the first range associated with the first video unit.

[0589] 50. The method according to clause 47, wherein the first range is derived from a non-linear clipping range used in a first clipping operation corresponding to chroma ALF filtering.

[0590] 51. The method according to clause 47, wherein during the second clipping operation, the derived CC-ALF offset is rounded using the bit depth value instead of a fixed value.

[0591] 52. The method according to clause 47, wherein during the second clipping operation, the derived CC-ALF offset is rounded using the following expression:

[0592] sum = alfPicture[xCtbC + x][yCtbC + y] + (sum + Off) >> (7+(BitDepthY - BitDepthC)) ccAlfPicture[xCtbC + x][yCtbC + y] = Clip3(0, (1 << BitDepthC)-1, sum), where Off is set to be equal to (1 << (6+(BitDepthY – BitDepthC))), where alfPicture[xCtbC + x][yCtbC + y] represents the chroma sample after applying the first clipping operation, where sum represents the sum of the CC-ALF filter coefficients multiplied by the luma samples in the CC-ALF filter support after applying the second clipping operation, and where ccAlfPicture represents the final chroma sample value after applying the third clipping operation.

[0593] 53. A video processing method, comprising:

[0594] performing a conversion between a video unit of a visual media and a codec representation of the visual media, wherein during the conversion, the chroma ALF process and / or the CC-ALF process are jointly optimized for the video unit.

[0595] 54. The method according to clause 53, wherein the conversion includes a clipping operation for clipping chroma samples generated by the chroma ALF process and / or the CC-ALF process, which results in the chroma samples being within a predefined range.

[0596] 55. The method according to clause 54, wherein the predefined range is at least partially based on the chroma bit depth value.

[0597] 56. The method according to clause 54, wherein the joint optimization includes applying the clipping operation only during the CC-ALF process if the CC-ALF process is applied after the chroma ALF process.

[0598] 57. A method as described in clause 54, wherein jointly optimizing includes applying a clipping operation only during the chroma ALF process if the CC-ALF process is applied before the chroma ALF process.

[0599] 58. A video processing method, comprising:

[0600] A conversion between a video unit of visual media and a codec representation of the visual media is performed, wherein during the conversion, information of a second color component of the video unit is used to modify sample values ​​of a first color component of the video unit, wherein the information of the second color component of the video unit is based on one or more parameters used in an adaptive loop filtering process.

[0601] 59. A method according to clause 58, wherein one or more parameters used in the adaptive loop filtering process are cropping parameters, wherein the cropping parameters used on a first sampling value are different from the cropping parameters used on a second sampling value, wherein the first sampling value and the second sampling value are included in a video unit.

[0602] 60. The method of clause 59, wherein the clipping parameter is defined to be within a range having a minimum clipping value and a maximum clipping value.

[0603] 61. A method according to clause 60, wherein the minimum cropping value and / or the maximum cropping value are explicitly signalled in the codec representation.

[0604] 62. A method as recited in clause 60, wherein the minimum clipping value and / or the maximum clipping value is based in part on the sample value.

[0605] 63. The method of clause 60, wherein the minimum crop value and / or the maximum crop value is based in part on a spatial position of the sample value relative to the video unit.

[0606] 64. A method according to clause 60, wherein the minimum clipping value and / or the maximum clipping value are predefined values.

[0607] 65. A method according to clause 60, wherein the minimum clipping value and / or the maximum clipping value are derived from other parameters.

[0608] 66. A video processing method, comprising:

[0609] A conversion is performed between a first video unit of visual media and a codec representation of the visual media, wherein during the conversion, application of a CC-ALF process to a first sample of the first video unit is selectively based on a second sample of a second video unit of the visual media, wherein the selective based includes determining whether a loop filter across the video units is enabled or disabled.

[0610] 67. The method of clause 66, wherein based on a determination to disable a loop filter across video units for the first video unit and / or the second video unit, application of a CC-ALF process to a first sample of the first video unit is skipped.

[0611] 68. The method of clause 66, wherein based on a determination to disable a loop filter across video units for the first video unit, application of a CC-ALF process to a first sample of the first video unit is skipped.

[0612] 69. The method of clause 66, wherein based on a determination to disable a loop filter across video units for a second video unit, application of a CC-ALF process to a first sample of a first video unit is skipped.

[0613] 70. A method according to one or more of clauses 66 to 69, wherein determining whether a loop filter across video units is enabled or disabled is based on reading a flag associated with the first video unit and / or the second video unit.

[0614] 71. A method according to any one or more of clauses 1 to 70, wherein the CC-ALF process and / or the ALF process is associated with at least one of: i. the content of the video unit, ii. a message signaled in a DPS / SPS / VPS / PPS / APS / picture header / slice header / slice group header / largest codec unit (LCU) / codec unit (CU) / LCU row / LCU group / TU / PU block / video codec unit, iii. the location of the CU / PU / TU / block / video codec unit, iv. the shape or dimensions of the video unit and / or the shape or dimensions of neighboring video units, vii. an indication of the color format, viii. the codec tree structure. Slice / slice group type and / or picture type, x. the color components of the video unit, xi. the temporal layer ID or xii. the profile / level / layer of the standard.

[0615] 72. A video decoding apparatus comprising a processor configured to implement one or more of the methods described in clauses 1 to 71.

[0616] 73. A video encoding and decoding apparatus comprising a processor configured to implement the method of one or more of clauses 1 to 71.

[0617] 74. A computer program product having computer code stored thereon which, when executed by a processor, causes the processor to implement the method of any one of clauses 1 to 71.

[0618] 75. The method, apparatus or system described in this document.

[0619] The second set of clauses illustrates example embodiments of the techniques discussed in the previous sections.

[0620] 1. A video processing method (eg, Fig.25A 2510), comprising: for a conversion between a current video unit of video including one or more video blocks and a bitstream representation of the video, determining 2512 a filling process according to a rule, the filling process being used to fill unavailable samples during application of a cross-component adaptive loop filtering (CC-ALF) tool to at least some video blocks of the current video unit; and performing 2514 the conversion based on the determination; wherein the rule specifies that the filling process is also used to fill unavailable samples during application of an adaptive loop filtering (ALF) tool to one or more video blocks of the current video unit.

[0621] 2. The method of clause 1, wherein the ALF tool is applied to color components of the current video unit, and the ALF tool comprises correcting sample values ​​of the color components using information provided by the color components of the current video unit.

[0622] 3. A method according to any one or more of clauses 1 to 2, wherein the CC-ALF tool is applied to a color component of a current video unit, and the CC-ALF tool includes using the sample values ​​of the second component of the current video unit to correct the sample values ​​of the first component of the current video unit.

[0623] 4. A method according to any one or more of clauses 1 to 3, wherein the current video unit corresponds to a slice, a tile, a sub-picture, a picture, or a 360 degree virtual picture defined by a 360 degree virtual boundary.

[0624] 5. A method according to any one or more of clauses 1 to 4, wherein the padding process corresponds to mirror padding applied on unusable samples at one or more ALF virtual boundaries associated with the use of the CC-ALF tool.

[0625] 6. A method according to any one or more of clauses 1 to 4, wherein the padding process corresponds to repeated padding applied on unusable samples at one of the stripes, slices, tiles, pictures, sub-pictures or 360 degree virtual boundaries associated with the use of CC-ALF.

[0626] 7. A method according to any of clauses 1 to 4, wherein the padding process corresponds to applying repetitive padding on one or more boundaries other than one or more ALF virtual boundaries associated with the use of CC-ALF.

[0627] 8. The method of clause 7, wherein the one or more boundaries include one of: all boundaries, horizontal boundaries, or vertical boundaries.

[0628] 9. A method as described in any one or more of clauses 1 to 4, wherein the padding process corresponds to applying mirror padding on one or more boundaries other than one or more ALF virtual boundaries associated with the use of CC-ALF.

[0629] 10. The method of clause 9, wherein the one or more boundaries include one of: all boundaries, horizontal boundaries, or vertical boundaries.

[0630] 11. A method as described in any one or more of clauses 1 to 10, wherein the current video unit corresponds to an ALF processing unit, a codec tree unit (CTU), or a basic ALF processing unit.

[0631] 12. The method of clause 11, wherein the ALF processing unit defines a unit bounded by two horizontal boundaries and two vertical boundaries.

[0632] 13. The method of clause 11, wherein the two horizontal boundaries comprise one of: two ALF virtual boundaries, or an ALF virtual boundary and a boundary of a picture / slice / tile / brick / sub-picture or a 360 degree virtual boundary.

[0633] 14. A method according to clause 11, wherein a basic ALF processing unit is generated based on recursively partitioning the ALF processing unit, CTU or narrow ALF processing unit, and wherein the boundaries of the basic ALF processing unit are not crossed by brick / strip / slice / sub-picture boundaries, and / or 360-degree virtual boundaries, and / or ALF virtual boundaries that do not allow cross-boundary filtering.

[0634] 15. A method according to any one or more of clauses 1 to 4, wherein the filling process corresponds to repeated filling based on whether neighboring samples of the current video unit are in different neighboring slices.

[0635] 16. The method of clause 1, wherein a current slice of a current video unit is a raster scan slice, and wherein a neighboring slice of the current slice is also a raster scan slice.

[0636] 17. A method according to any one or more of clauses 15 to 16, wherein the current video unit is an ALF processing unit, and wherein unavailable samples are filled in the ALF processing unit using its most recent samples.

[0637] 18. A video processing method (e.g., Fig.25BThe method 2520 shown includes: performing 2522 conversion between a video unit of a video and a bitstream representation of the video, wherein, during the conversion, unavailable samples of the video unit are filled in a predefined filling order according to rules in an adaptive loop filtering (ALF) process or an application of a cross-component adaptive loop filtering (CC-ALF).

[0638] 19. The method of clause 18, wherein the video unit corresponds to a processing unit, and the rule specifies a predefined filling order for filling unavailable samples of the video unit.

[0639] 20. The method of clause 18, wherein the predefined filling order comprises filling unavailable samples with at least one row or at least one column of the video unit, and wherein a position of the at least one row or at least one column is determined based on a position of the unavailable samples.

[0640] 21. The method of clause 18, wherein predefining the filling order comprises filling unavailable samples with at least one sample of the video unit, and wherein a position of the at least one sample is determined based on a position of the unavailable samples.

[0641] 22. A method according to clause 18, wherein the availability of at least one neighboring sample of the video unit is determined based on a relationship between one or more neighboring samples and a parameter (ctuDown or ctuUp) associated with a video region, and wherein the video region corresponds to a processing unit including ctuUP corresponding to N rows of a codec tree unit (CTU) and ctuDown corresponding to 'CTBsize - M' rows of the CTU, where M is an integer.

[0642] 23. The method of clause 22, wherein if the neighboring samples are in a second video unit different from ctuDown, the rule specifies that filtering is not allowed across the second video unit.

[0643] 24. The method of clause 18, wherein the video unit corresponds to a current codec tree unit included in a raster scan slice, and the rule specifies a predefined filling order for filling unavailable samples of the video unit.

[0644] 25. The method of clause 18, wherein the predefined filling order comprises filling unavailable samples in an upper left CTU located in a stripe different from the raster scan stripe with its nearest neighboring samples in an upper CTU located in a raster scan stripe.

[0645] 26. The method of clause 18, wherein the predefined filling order comprises filling unavailable samples in an upper left CTU located in a stripe different from the raster scan stripe with its nearest neighboring samples in a left CTU located in a raster scan stripe.

[0646] 27. The method of clause 18, wherein the predefined filling order comprises filling unavailable samples in a bottom right CTU located in a stripe different from the raster scan stripe with its nearest neighboring samples in a lower CTU located in a raster scan stripe.

[0647] 28. The method of clause 18, wherein the predefined filling order comprises filling unavailable samples in a bottom right CTU located in a stripe different from the raster scan stripe with its nearest neighboring samples in a right CTU located in a raster scan stripe.

[0648] 29. A video processing method (e.g., Fig.25C The method 2530 shown in FIG. 2 comprises: for a video region of a video with an adaptive loop filter (ALF) application enabled, determining 2532 that the video region is spanned by a boundary of a video unit; and performing 2534 a conversion between the video and a bitstream representation of the video, wherein, for the conversion, the video region is split into a plurality of partitions according to a rule because the video region is spanned by a boundary of a video unit.

[0649] 30. The method of clause 29, wherein filtering processes that cross boundaries are not allowed.

[0650] 31. The method of clause 29, wherein the video region corresponds to an ALF processing unit, a narrow ALF processing unit, or a codec tree unit.

[0651] 32. The method of clause 29, wherein the video unit corresponds to a tile, a strip, a slice, a sub-picture, or a 360-degree virtual picture.

[0652] 33. The method of clause 29, wherein the rule further specifies recursively splitting the video region until the video region is not spanned by a boundary of a video unit.

[0653] 34. A method according to clause 29 or 30, wherein the filtering process corresponds to an ALF process or a CC-ALF process, and wherein the rules further specify that the filtering process is performed after the splitting process of splitting the video region is completed.

[0654] 35. The method of clause 29, wherein the padding process for padding at least one sample is performed on a basic ALF processing unit obtained after splitting the video region until the video region is not crossed by a boundary of a video unit.

[0655] 36. The method of clause 29, wherein the rules specify that different horizontal boundaries are checked in a predefined order to generate basic ALF processing units, the basic ALF processing units being obtained after splitting the video region until the video region is not spanned by a boundary of a video unit.

[0656] 37. The method of clause 36, wherein the predefined order is an ALF virtual boundary, a 360 degree virtual boundary, and a codec tree unit (CTU) boundary as a video unit boundary.

[0657] 38. The method of clause 36, wherein the predefined order is a 360 degree virtual boundary, an ALF virtual boundary, and a codec tree unit (CTU) boundary as a video unit boundary.

[0658] 39. The method of clause 36, wherein the rule specifies different predefined orders for top and bottom borders of the video unit.

[0659] 40. The method of clause 29, wherein the rule specifies that different vertical boundaries are checked in a predefined order to generate basic ALF processing units, the basic ALF processing units being obtained after splitting the video region until the video region is not spanned by a boundary of a video unit.

[0660] 41. The method of clause 40, wherein the predefined order is a 360 degree virtual boundary and a codec tree unit (CTU) boundary that is a boundary of the video unit.

[0661] 42. The method of clause 40, wherein the predefined order is a codec tree unit (CTU) boundary and a 360 degree virtual boundary of a video unit.

[0662] 43. The method of clause 40, wherein the rule specifies different predefined orders for left and right borders of the video unit.

[0663] 44. A video processing method, comprising: performing conversion between a video unit of a video and a bitstream representation of the video according to a rule, wherein the rule specifies that in the case where a filtering process crossing a boundary of the video unit is not allowed, adaptive loop filtering (ALF) and / or cross-component adaptive loop filtering (CC-ALF) is not allowed to be applied to samples located at the boundary.

[0664] 45. A method as described in clause 44, wherein applying ALF and / or CC-ALF is disabled upon determining that at least N neighboring samples associated with ALF and / or CC-ALF are not available, wherein N is an integer.

[0665] 46. ​​A method as described in clause 44, wherein the boundary of the video unit is a horizontal boundary, and wherein applying ALF and / or CC-ALF is disabled when it is determined that the vertical distance between the sample and the horizontal boundary of the video unit is less than or equal to M samples, where M is an integer.

[0666] 47. A method as described in clause 44, wherein the boundary of the video unit is a vertical boundary, and wherein applying ALF and / or CC-ALF is disabled when it is determined that the horizontal distance between the sample and the vertical boundary of the video unit is less than or equal to M samples, where M is an integer.

[0667] 48. A video processing method, comprising: performing conversion between a video unit of a video and a bitstream representation of the video, wherein the bitstream representation conforms to a format rule, wherein a video region is different from a codec tree block, wherein the format rule specifies whether a syntax element indicating the applicability of an adaptive loop filter (ALF) tool and / or a cross-component adaptive loop filter (CC-ALF) tool to the video region is included in the bitstream representation.

[0668] 49. The method of clause 48, wherein the video region corresponds to an ALF processing unit.

[0669] 50. A method according to clause 48, wherein the applicability of ALF tools and / or CC-ALF tools is explicitly indicated in the bitstream representation.

[0670] 51. A method according to clause 48, wherein the suitability of the ALF tool and / or the CC-ALF tool is derived.

[0671] 52. A video processing method (e.g., Fig.25D The method 2540 shown includes: for conversion between a video unit of a video and a bitstream representation of the video, determining 2542 the applicability of a cross-component adaptive loop filtering (CC-ALF) tool to samples of the video unit based on a rule; and performing 2544 the conversion based on the determination; wherein the bitstream representation includes an indication that CC-ALF is applicable to the video unit; and wherein the rule specifies one or more conditions that override the indication.

[0672] 53. The method of clause 52, wherein the one or more conditions include a spatial location of the sample relative to the video unit.

[0673] 54. The method of clause 52, wherein the one or more conditions include luma sample values ​​used in CC-ALF.

[0674] 55. The method of clause 52, wherein the one or more conditions include a sample point value difference between the sample point and its neighboring sample points.

[0675] 56. The method of clause 52, wherein the one or more conditions comprise a relationship between an offset value derived using filtered luma samples corresponding to chroma samples and a threshold value.

[0676] 57. The method of clause 52, wherein the one or more conditions include coded information of a block comprising chroma samples and / or neighboring blocks of the block.

[0677] 58. A video processing method, comprising: performing conversion between video units of a video and a bitstream representation of the video according to a rule, wherein the rule specifies that an algorithm used during the conversion omits at least one of three cropping operations, the three cropping operations including a first cropping operation corresponding to chroma adaptive loop filtering (ALF) filtering, a second cropping operation corresponding to cross-component adaptive loop filtering (CC-ALF) offset derivation, and a third cropping operation corresponding to refinement of chroma filter samples to derive final chroma sample values.

[0678] 59. The method of clause 58, wherein the rule specifies that the first cropping operation is omitted if application CC-ALF is enabled.

[0679] 60. The method of clause 58, wherein the rule specifies that the second cropping operation is omitted if application CC-ALF is disabled.

[0680] 61. The method of clause 58, wherein the rule specifies that the first cropping operation and the second cropping operation are to be omitted.

[0681] 62. A video processing method (e.g., Fig.25E The method 2550 shown in FIG. 2 comprises: for a conversion between a first video unit of a video and a bitstream representation of the video, determining 2552 a cross-component adaptive loop filter (CC-ALF) offset according to a rule; and performing 2554 the conversion based on the determination, and wherein the rule specifies that the CC-ALF offset is clipped to a first range different from a second range, the second range being represented as [-(1<<(BitDepthC-1)),(1<<(BitDepthC-1))–1], where BitDepthC is a bit depth value.

[0682] 63. A method according to clause 62, wherein the first range is independent of the bit depth value.

[0683] 64. The method of clause 62, wherein a third range associated with the second video unit is defined to be different than a first range associated with the first video unit.

[0684] 65. The method according to clause 62, wherein the first range is derived from a non-linear clipping range used in another clipping operation corresponding to chroma ALF filtering.

[0685] 66. A video processing method (e.g., method 2560 as Fig.25F shown), comprising: for the conversion between a first video unit of a video and a bitstream representation of the video, deriving 2562 a cross-component adaptive loop filter (CC-ALF) offset according to a rule; and performing 2564 the conversion using the CC-ALF offset, and wherein the rule specifies rounding the CC-ALF offset using a rounding offset based on a bit depth value rather than a fixed value.

[0686] 67. The method according to clause 66, wherein the rounding offset depends on a bit depth difference between a first color component and a second color component.

[0687] 68. The method according to clause 66, wherein the CC-ALF offset is rounded using the following expressions: sum = ALF picture[xCtbC+x][yCtbC+y] + (sum + Off)>>(7+(BitDepthY BitDepthC)) and ccalfpicture[xCtbC+x][yCtbC+y] = clip 3(0, (1<<BitDepthC)-1, sum), where Off is set to be equal to (1<<<(6+(BitDepthY-BitDepthC)), where alfPicture[xCtbC+x][yCtbC+y] represents a chroma sample after applying a first clipping operation, where sum represents the sum of the CC-ALF filter coefficients multiplied by the luminance samples in the CC-ALF filter support after applying a second clipping operation, where ccAlfPicture represents the final chroma sample value after applying a third clipping operation, and wherein BitDepthC and BitDepthY represent the bit depths of the luminance and chroma components, respectively.

[0688] 69. A video processing method, comprising: performing a conversion between a video unit of a video and a bitstream representation of the video according to a rule, wherein the rule specifies that one or more processing steps used during a chroma adaptive loop filtering (ALF) process and / or a cross-component adaptive loop filtering (CC-ALF) process applied to samples of the video unit are the same.

[0689] 70. The method according to clause 69, wherein the conversion includes a clipping operation for clipping chroma samples generated by the chroma ALF process and / or the CC-ALF process, which results in the chroma samples being within a predefined range.

[0690] 71. A method according to clause 70, wherein the predefined range is based at least in part on a chroma bit depth value.

[0691] 72. A method according to clause 70, wherein if a CC-ALF process is applied after a chroma ALF process, the clipping operation is only applied during the CC-ALF process.

[0692] 73. A method according to clause 70, wherein if the CC-ALF process is applied before the chroma ALF process, the clipping operation is only applied during the chroma ALF process.

[0693] 74. A method of video processing, comprising: performing conversion between a video unit of a video and a bitstream representation of the video according to a rule, wherein the rule specifies that during the conversion, a sample value of a first color component of the video unit is modified by modification using information of a second color component of the video unit, wherein the modification is based on one or more parameters used in an adaptive loop filtering (ALF) process of the video unit.

[0694] 75. A method according to clause 74, wherein one or more parameters used in the adaptive loop filtering process are cropping parameters, wherein the cropping parameters used on a first sample value are different from the cropping parameters used on a second sample value, wherein the first sample value and the second sample value are included in a video unit.

[0695] 76. A method according to clause 75, wherein the clipping parameter is defined as being within a range having a minimum clipping value and a maximum clipping value.

[0696] 77. A method according to clause 76, wherein the minimum clipping value and / or the maximum clipping value are explicitly signalled in the bitstream representation.

[0697] 78. A method as described in clause 76, wherein the minimum clipping value and / or the maximum clipping value are based in part on the sample point value.

[0698] 79. The method of clause 76, wherein the minimum crop value and / or the maximum crop value is based in part on a spatial position of the sample value relative to the video unit.

[0699] 80. A method according to clause 76, wherein the minimum clipping value and / or the maximum clipping value is signalled or predefined.

[0700] 81. A method according to clause 76, wherein the minimum clipping value and / or the maximum clipping value are derived from other parameters.

[0701] 82. A video processing method (e.g., Figure 25GThe method 2570 shown includes: for a conversion between a first sub-picture of a video and a bitstream representation of the video, determining 2572 based on a rule whether cross-component loop filtering (CC-ALF) is applicable to a sample of the first sub-picture; and performing 2574 conversion based on the determination, wherein the CC-ALF for the sample uses a sample from a second sub-picture; and wherein the rule is based on whether loop filtering across sub-picture boundaries is allowed for the first sub-picture and / or the second sub-picture.

[0702] 83. A method as recited in clause 82, wherein CC-ALF is skipped on a sample point in case loop filtering across sub-picture boundaries is not allowed for the first sub-picture.

[0703] 84. A method as recited in clause 82, wherein CC-ALF is skipped on the first sub-picture if loop filtering across sub-picture boundaries is not allowed for the second sub-picture.

[0704] 85. A method as recited in clause 82, wherein CC-ALF is skipped on a first sample point if loop filtering across sub-picture boundaries is not allowed for the first sub-picture and / or the second sub-picture.

[0705] 86. A method according to one or more of clauses 82 to 84, wherein whether to allow loop filtering across a sub-picture boundary is based on a flag associated with the first sub-picture and / or the second sub-picture.

[0706] 87. A method according to any of the preceding clauses, wherein the method is also based on at least one of the following: 1) the type of video content, 2) in a sequence parameter set (SPS), a video parameter set (VPS), a picture parameter set (PPS), a dependency parameter set (DPS), an adaptation parameter set (APS), a picture header, a slice header, a slice group header, a maximum codec unit (LCU), a codec unit (CU), an LCU row, an LCU group, a transform unit (TU), a prediction unit (PU) block or a video codec unit, 3) the position of a CU, a PU, a TU block or a video codec unit, 4) decoded information of the current block and / or neighboring blocks, 5) the latitude or shape of the current block and / or neighboring blocks, 6) an indication of a color format, 7) a codec tree structure, 8) a slice type, a slice group type and / or a picture type, 9) the type of color component, 10) a temporal layer identifier, 11) a profile, level or hierarchy of a standard.

[0707] 88. A method according to any one of clauses 1 to 87, wherein converting comprises encoding the video into a bitstream representation.

[0708] 89. A method according to any of clauses 1 to 87, wherein converting comprises decoding video from a bitstream representation.

[0709] 90. A video processing apparatus comprising a processor configured to implement the method of any one or more of clauses 1 to 89.

[0710] 91. A computer readable medium storing program code which, when executed, causes a processor to implement the method of any one or more of clauses 1 to 89.

[0711] 92. A computer readable medium storing a coded representation or a bitstream representation generated according to any of the above methods.

[0712] The disclosed and other solutions, examples, embodiments, modules and functional operations described herein may be implemented in digital electronic circuits, or in computer software, firmware or hardware, including the structures disclosed herein and their structural equivalents, or in a combination of one or more of them. The disclosed and other embodiments may be implemented as one or more computer program products, i.e., one or more modules of computer program instructions encoded on a computer-readable medium, for execution by a data processing device or for controlling the operation of the data processing device. The computer-readable medium may be a machine-readable storage device, a machine-readable storage substrate, a storage device, a material combination that implements a machine-readable propagation signal, or a combination of one or more of them. The term "data processing device" includes all devices, equipment and machines for processing data, including, for example, a programmable processor, a computer or multiple processors or computers. In addition to hardware, the device may include code that creates an execution environment for the computer program in question, for example, code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them. A propagation signal is an artificially generated signal, for example, a machine-generated electrical, optical or electromagnetic signal, which is generated to encode information for transmission to a suitable receiver device.

[0713] A computer program (also referred to as a program, software, software application, script, or code) may be written in any form of programming language, including compiled or interpreted languages, and it may be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for a computing environment. A computer program does not necessarily correspond to a file in a file system. A program may be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language file), in a single file dedicated to the program in question, or in multiple cooperating files (e.g., files storing one or more modules, subroutines, or portions of code). A computer program may be deployed to execute on a single computer or on multiple computers located in one location or distributed in multiple locations and interconnected by a communications network.

[0714] The processes and logic flows described herein may be performed by one or more programmable processors executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows may also be performed by, and apparatus may be implemented as, special purpose logic circuitry, such as an FPGA (field programmable gate array) or an ASIC (application specific integrated circuit).

[0715] For example, processors suitable for executing computer programs include general and special purpose microprocessors, and any one or more processors of any kind of digital computer. Typically, the processor will receive instructions and data from a read-only memory or a random access memory or both. The basic elements of a computer are a processor for executing instructions and one or more storage devices for storing instructions and data. Typically, the computer will also include or be operably coupled to one or more large-capacity storage devices for storing data, such as magnetic disks, magneto-optical disks, or optical disks, to receive data from the large-capacity storage device or to transfer data to it, or both. However, the computer does not need to have such a device. Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and storage devices, including, for example, semiconductor storage devices, such as EPROM, EEPROM, and flash memory devices; magnetic disks, such as internal hard disks or removable disks; magneto-optical disks; and CD ROM and DVD-ROM disks. The processor and memory can be supplemented or incorporated by dedicated logic circuits.

[0716] Although this patent document contains many details, these details should not be interpreted as limitations on the scope of any subject matter or the content claimed, but rather as descriptions of features unique to specific embodiments of specific technologies. Certain features described in the context of separate embodiments in this patent document may also be implemented in combination in a single embodiment. Conversely, the various features described in the context of a single embodiment may also be implemented in multiple embodiments alone or in any suitable sub-combination. In addition, although features may be described above as working in certain combinations and even initially claimed as such, one or more features from the claimed combination may be deleted from the combination in some cases, and the claimed combination may be directed to a sub-combination or a variation of a sub-combination.

[0717] Similarly, although operations are described in a particular order in the drawings, this should not be understood as requiring that these operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, in order to achieve the desired results. In addition, the separation of various system components in the embodiments described in this patent document should not be understood as requiring such separation in all embodiments.

[0718] Only a few implementations and examples are described, and other implementations, enhancements, and variations may be made based on what is described and shown in this patent document.

Claims

1. A method for processing video data, comprising: For a conversion between a current video unit of a video including one or more samples and a bitstream of the video, determining a filling process for filling unavailable samples during application of a cross-component adaptive loop filter (CC-ALF) to at least some samples of the current video unit, wherein the current video unit corresponds to a current codec tree unit; and performing said converting based on said determining; wherein the CC-ALF tool is applied to a first color component of the current video unit and includes correcting a sample value of the first color component of the current video unit using a sample value of a luminance component of the current video unit; and Among them, when the first unavailable sample point is located in the lower right codec tree unit of the current video unit, the lower right codec tree unit is located in a second stripe different from the first stripe where the current video unit is located, the lower codec tree unit of the current video unit and the right codec tree unit of the current video unit are both located in the first stripe, and the lower codec tree unit of the current video unit and the right codec tree unit of the current video unit are both available, the first unavailable sample point is filled with the adjacent sample point located in the lower codec tree unit and the closest to the first unavailable sample point.

2. The method according to claim 1, wherein: When the second unavailable sample is located in an upper-left codec tree unit of the current video unit, the upper-left codec tree unit is located in a second slice different from a first slice in which the current video unit is located, both the upper codec tree unit of the current video unit and the left codec tree unit of the current video unit are located in the first slice, and both the upper codec tree unit of the current video unit and the left codec tree unit of the current video unit are available, the second unavailable sample is filled with a neighboring sample located in the upper codec tree unit that is closest to the second unavailable sample.

3. The method according to claim 1, wherein: The padding process corresponds to a mirror padding applied on unavailable samples at one or more ALF virtual boundaries associated with the use of the CC-ALF tool.

4. The method according to claim 1, wherein: The padding process corresponds to a one-sided padding applied on unavailable samples at one of a slice, a slice, a picture, a sub-picture, or a 360-degree virtual boundary associated with the application of the CC-ALF.

5. The method according to claim 1, wherein: The padding process corresponds to a one-sided padding that is applied on one or more boundaries other than at one or more ALF virtual boundaries associated with application of the CC-ALF.

6. The method according to any one of claims 1 to 5, wherein: The converting includes encoding the video into the bitstream.

7. The method according to any one of claims 1 to 5, wherein: The converting includes decoding the video from the bitstream.

8. An apparatus for processing video data, comprising a processor and a non-transitory memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to: For a conversion between a current video unit of a video including one or more samples and a bitstream of the video, a filling process is determined, the filling process being used to fill unavailable samples during application of a cross-component adaptive loop filter CC-ALF to at least some samples of the current video unit, wherein The current video unit corresponds to a current codec tree unit; as well as performing said converting based on said determining; wherein the CC-ALF tool is applied to a first color component of the current video unit and includes correcting a sample value of the first color component of the current video unit using a sample value of a luminance component of the current video unit; and Among them, when the first unavailable sample point is located in the lower right codec tree unit of the current video unit, the lower right codec tree unit is located in a second stripe different from the first stripe where the current video unit is located, the lower codec tree unit of the current video unit and the right codec tree unit of the current video unit are both located in the first stripe, and the lower codec tree unit of the current video unit and the right codec tree unit of the current video unit are both available, the first unavailable sample point is filled with the adjacent sample point located in the lower codec tree unit and the closest to the first unavailable sample point.

9. The device according to claim 8, wherein: When the second unavailable sample is located in an upper-left codec tree unit of the current video unit, the upper-left codec tree unit is located in a second slice different from a first slice in which the current video unit is located, both the upper codec tree unit of the current video unit and the left codec tree unit of the current video unit are located in the first slice, and both the upper codec tree unit of the current video unit and the left codec tree unit of the current video unit are available, the second unavailable sample is filled with a neighboring sample located in the upper codec tree unit that is closest to the second unavailable sample.

10. The device according to claim 8, wherein: The padding process corresponds to a mirror padding, the mirror padding being applied on unavailable samples at one or more ALF virtual boundaries associated with the use of the CC-ALF tool; or The padding process corresponds to a one-sided padding, and the one-sided padding is applied to unavailable samples at one of a slice, a slice, a picture, a sub-picture, or a 360-degree virtual boundary associated with the application of the CC-ALF; or The padding process corresponds to a one-sided padding, and the one-sided padding is applied on one or more boundaries other than one or more ALF virtual boundaries associated with the application of the CC-ALF.

11. A non-transitory computer-readable storage medium storing instructions that cause a processor to: For a conversion between a current video unit of a video including one or more samples and a bitstream of the video, a filling process is determined, the filling process being used to fill unavailable samples during application of a cross-component adaptive loop filter CC-ALF to at least some samples of the current video unit, wherein The current video unit corresponds to a current codec tree unit; as well as performing said converting based on said determining; wherein the CC-ALF tool is applied to a first color component of the current video unit and includes correcting a sample value of the first color component of the current video unit using a sample value of a luma component of the current video unit; and Among them, when the first unavailable sample point is located in the lower right codec tree unit of the current video unit, the lower right codec tree unit is located in a second stripe different from the first stripe where the current video unit is located, the lower codec tree unit of the current video unit and the right codec tree unit of the current video unit are both located in the first stripe, and the lower codec tree unit of the current video unit and the right codec tree unit of the current video unit are both available, the first unavailable sample point is filled with the adjacent sample point located in the lower codec tree unit and the closest to the first unavailable sample point.

12. The non-transitory computer-readable storage medium of claim 11, wherein: When the second unavailable sample is located in an upper-left codec tree unit of the current video unit, the upper-left codec tree unit is located in a second slice different from a first slice in which the current video unit is located, both the upper codec tree unit of the current video unit and the left codec tree unit of the current video unit are located in the first slice, and both the upper codec tree unit of the current video unit and the left codec tree unit of the current video unit are available, the second unavailable sample is filled with a neighboring sample located in the upper codec tree unit that is closest to the second unavailable sample.

13. The non-transitory computer-readable storage medium of claim 11, wherein: The padding process corresponds to a mirror padding, the mirror padding being applied on unavailable samples at one or more ALF virtual boundaries associated with the use of the CC-ALF tool; or The padding process corresponds to a one-sided padding, and the one-sided padding is applied to unavailable samples at one of a slice, a slice, a picture, a sub-picture, or a 360-degree virtual boundary associated with the application of the CC-ALF; or The padding process corresponds to a one-sided padding, and the one-sided padding is applied on one or more boundaries other than one or more ALF virtual boundaries associated with the application of the CC-ALF.

14. A non-transitory computer-readable recording medium storing a bit stream of a video generated by a method performed by a video processing device, wherein the method comprises: For a current video unit of a video including one or more samples, determining a filling process for filling unavailable samples during application of a cross-component adaptive loop filter (CC-ALF) to at least some samples of the current video unit, wherein the current video unit corresponds to a current codec tree unit; and generating the bitstream based on the determination; wherein the CC-ALF tool is applied to a first color component of the current video unit and includes correcting a sample value of the first color component of the current video unit using a sample value of a luminance component of the current video unit; and Among them, when the first unavailable sample point is located in the lower right codec tree unit of the current video unit, the lower right codec tree unit is located in a second stripe different from the first stripe where the current video unit is located, the lower codec tree unit of the current video unit and the right codec tree unit of the current video unit are both located in the first stripe, and the lower codec tree unit of the current video unit and the right codec tree unit of the current video unit are both available, the first unavailable sample point is filled with the adjacent sample point located in the lower codec tree unit and the closest to the first unavailable sample point.

15. The non-transitory computer-readable recording medium according to claim 14, in, When the second unavailable sample is located in an upper-left codec tree unit of the current video unit, the upper-left codec tree unit is located in a second slice different from a first slice in which the current video unit is located, both the upper codec tree unit of the current video unit and the left codec tree unit of the current video unit are located in the first slice, and both the upper codec tree unit of the current video unit and the left codec tree unit of the current video unit are available, the second unavailable sample is filled with a neighboring sample located in the upper codec tree unit that is closest to the second unavailable sample.

16. The non-transitory computer-readable recording medium according to claim 14, wherein: The padding process corresponds to a mirror padding, the mirror padding being applied on unavailable samples at one or more ALF virtual boundaries associated with the use of the CC-ALF tool; or The padding process corresponds to a one-sided padding, and the one-sided padding is applied to unavailable samples at one of a slice, a slice, a picture, a sub-picture, or a 360-degree virtual boundary associated with the application of the CC-ALF; or The padding process corresponds to a one-sided padding, and the one-sided padding is applied on one or more boundaries other than one or more ALF virtual boundaries associated with the application of the CC-ALF.

17. A method for storing a bitstream of a video, comprising: Determining, for a current video unit of a video including one or more samples, a filling process for filling unavailable samples during application of a cross-component adaptive loop filter (CC-ALF) to at least some samples of the current video unit, wherein the current video unit corresponds to a current codec tree unit; generating the bitstream based on the determination; and storing the bitstream in a non-transitory computer-readable storage medium, wherein the CC-ALF tool is applied to a first color component of the current video unit and includes correcting a sample value of the first color component of the current video unit using a sample value of a luminance component of the current video unit; and Among them, when the first unavailable sample point is located in the lower right codec tree unit of the current video unit, the lower right codec tree unit is located in a second stripe different from the first stripe where the current video unit is located, the lower codec tree unit of the current video unit and the right codec tree unit of the current video unit are both located in the first stripe, and the lower codec tree unit of the current video unit and the right codec tree unit of the current video unit are both available, the first unavailable sample point is filled with the adjacent sample point located in the lower codec tree unit and the closest to the first unavailable sample point.

18. An apparatus for processing video data, comprising a processor and a non-transitory memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to implement the method of any one of claims 6 to 7.

19. A computer readable medium storing program code, which, when executed, causes a processor to implement the method of any one of claims 6 to 7.

Images