Filter deblocking for video coding

Abstract

In an exemplary aspect, a video media processing method comprises the steps of: identifying boundaries at the vertical and / or horizontal edges of two video blocks; calculating boundary strengths of a filter based on one or more of the two video blocks coded using a combined intra-inter prediction (CIIP) mode across the vertical or horizontal edges; determining whether to turn the filter on or off; selecting the strength of the filter if the filter is turned on; and performing a deblocking filter (DB) process on the video blocks based on the selection step.

Description

Technology Field

[0001] This patent document relates to video coding / decoding technology, devices, and systems.

[0002] This application, claiming priority and benefit of International Patent Application No. PCT / CN2019 / 086488 filed on May 11, 2019, is the national phase of International Patent Application PCT / CN2020 / 089557 filed on May 11, 2020. For all purposes in accordance with the law, the full disclosure of the aforementioned application is incorporated by reference as part of the disclosure of this application. Background Technology

[0003] Despite advancements in video compression, digital video still accounts for the largest share of bandwidth usage on the Internet and other digital communication networks. As the number of connected user devices capable of receiving and displaying video increases, the demand for bandwidth for digital video usage is expected to continue rising. means of solving the problem

[0004] Devices, systems, and methods related to video coding / decoding technology, in particular deblocking filters and methods for image / video coding, are described. The described methods may be applied to standards under development (e.g., Versatile Video Coding and future video coding standards or video codecs).

[0005] In one aspect, the disclosed technology may be used to provide a method for visual media processing. The method comprises the steps of: identifying a boundary at a vertical edge and / or horizontal edge of two video blocks; calculating a boundary strength of a filter based on one or more of the two video blocks coded using a combined intra-inter prediction (CIIP) mode across the vertical edge or horizontal edge; deciding whether to turn the filter on or off; selecting a strength of the filter if the filter is turned on; and performing a deblocking filter (DB) process on the video blocks based on the selection step.

[0006] In another aspect, the disclosed technology may be used to provide a method for processing video media. The method comprises the steps of: determining a coding scheme applicable to a conversion between a video block of video media data and a bitstream representation of the video block; and, based on the determination of the coding scheme, selectively enabling or disabling the application of an in-loop filter to the video block during the conversion of the video block.

[0007] In another aspect, the disclosed technology may be used to provide a method for processing video media. The method comprises the steps of: determining a type of transform matrix applicable to a transformation between a video block of video media data and a bitstream representation of the video block; and, based on the determination of the type of transform matrix, selectively enabling or disabling the application of a deblocking (DB) filter process to the video block while transforming the video block.

[0008] In another aspect, the disclosed technology may be used to provide a method for processing video media. The method comprises: a step of determining the application of an in-loop filtering step to a video block during a conversion between a video block of video media data and a bitstream representation of the video block, wherein the in-loop filtering step includes a clipping operation applied to samples within the video block to be filtered; and a step of selectively enabling or disabling the clipping operation in response to a step of determining one or more conditions associated with the video block.

[0009] In another aspect, the aforementioned method is implemented in the form of processor executable code and stored on a computer-readable program medium.

[0010] In another aspect, a device configured or capable of performing the aforementioned method is disclosed. The device may include a processor programmed to implement this method.

[0011] In another aspect, the video decoder device can implement the method described herein.

[0012] In another aspect, the video encoder device can implement the method described herein.

[0013] Other aspects and features of the above and disclosed technology are described in more detail in the drawings, description, and claims. Brief explanation of the drawing

[0014] Figure 1 shows an example of an encoder block diagram of Universal Video Coding (VVC). Figure 2 shows an example processing flowchart of a deblocking filter process. Figure 3 shows how the BS value is calculated based on the intra-coding mode, non-zero transform coefficient and motion information, reference picture, number of motion vectors and the existence of motion vector difference. Figure 4 shows an example of referenced information for Bs calculation at the CTU boundary. Figure 5 shows an example of pixels included in the filter on / off decision. Figure 6a shows an example of the division of 4x8 and 8x4 blocks. Figure 6b shows an example of the division of all blocks except 4x8, 8x4, and 4x4. Figure 7a shows an example of affine linear weighted intra prediction (ALWIP) for a 4x4 block. Figure 7b shows an example of ALWIP for an 8x8 block. Figure 7c shows an example of ALWIP for an 8x4 block. Figure 7d shows an example of ALWIP for a 16x16 block. Figure 8 shows an example of a second transformation in JEM. FIG. 9 shows an example of a reduced second transformation (RST) based on an embodiment of the disclosed technology. Figure 10 shows examples of subblock conversion modes SBT-V and SBT-H. FIG. 25 shows the subblock MV VSB and pixel Δv(i,j) Shows an example of. FIG. 12 shows a flowchart of an example method for video processing based on some embodiments of the disclosed technology. FIG. 13 shows a flowchart of an example method for video processing based on some embodiments of the disclosed technology. FIG. 14 shows a flowchart of an example method for video processing based on some embodiments of the disclosed technology. FIG. 15 shows a flowchart of an example method for video processing based on some embodiments of the disclosed technology. FIG. 16 shows a flowchart of an example method for video processing based on some embodiments of the disclosed technology. FIG. 17 shows a flowchart of an example method for video processing based on some embodiments of the disclosed technology. FIG. 18 shows a flowchart of an example method for video processing based on some embodiments of the disclosed technology. FIG. 19 shows a flowchart of an example method for video processing based on some embodiments of the disclosed technology. FIG. 12 is a block diagram of an example of a hardware platform for implementing the video media decoding or video media encoding technology described in this document. FIG. 13 is a block diagram of an exemplary video processing system in which the disclosed technology can be implemented. Figure 11 shows a flowchart of an example method for video media processing. Figure 11 shows a flowchart of an example method for video media processing. Figure 11 shows a flowchart of an example method for video media processing. Figure 11 shows a flowchart of an example method for video media processing. Specific details for implementing the invention

[0015] With the increasing demand for high-resolution video, video coding methods and technologies have become ubiquitous in modern technology. Video codecs typically involve electronic circuits or software that compress or decompress digital video and are continuously being improved to increase coding efficiency. Video codecs convert uncompressed video into a compressed format or vice versa. There is a complex relationship between video quality, video (determined by the bitrate), the complexity of encoding and decoding algorithms, sensitivity to data loss and errors, ease of editing, random access, and end-to-end latency. Compression formats generally comply with standard video compression specifications (e.g., HEVC standards, also known as H.265 or MPEG-H Part 2), Universal Video Coding (VVC) standards, or other current and / or future video coding standards.

[0016] In some embodiments, future video coding techniques are explored using reference software known as the Joint Research Model (JEM). In the JEM, subblock-based prediction is adopted in various coding tools such as affine prediction, alternative temporal motion vector prediction (ATMVP), spatial-temporal motion vector prediction (STMVP), bi-directional optical flow (BIO), Frame-Rate Up Conversion (FRUC), Locally Adaptive Motion Vector Resolution (LAMVR), Overlapped Block Motion Compensation (OBMC), Local Illumination Compensation, and Decoder-side Motion Vector Refinement (DMVR).

[0017] Embodiments of the disclosed technology may be applied to existing video coding standards (e.g., HEVC, H.265) and future standards to improve runtime performance. This document provides various techniques that may be used by decoders of image or video bitstreams to improve the quality of decompressed or decoded digital video or images. For brevity, the term "video" is used to include both continuous pictures (traditionally referred to as video) and individual images. Additionally, video encoders may implement these techniques during the encoding process to reconstruct decoded frames used for further encoding.

[0018] Section titles are used in this document to improve the readability of the description and do not limit the discussion or examples (and / or implementations) to each section only.

[0019] 1 Example of a typical video codec coding flow

[0020] Figure 1 shows an example of an encoder block diagram of a VVC containing three in-loop filtering blocks: a deblocking filter (DF), a sample adaptive offset (SAO), and an ALF. Unlike the DF, which uses a predefined filter, the SAO and ALF utilize the original sample of the current picture to reduce the mean squared error between the original sample and the reconstructed sample by adding an offset and applying a finite impulse response (FIR) filter, respectively, along with coded additional information that signals the offset and filter coefficients. The ALF is located at the final processing step of each picture and can be considered a tool intended to capture and correct artifacts created in the previous step.

[0021] 2 Examples of deblocking schemes in HEVC

[0022] The deblocking filter process is performed for each CU in the same order as the decoding process. First, vertical edges are filtered (horizontal filtering), and then horizontal edges are filtered (vertical filtering). Filtering is applied to 8x8 block boundaries determined to be filtered for both the Luma and Chroma components. To reduce complexity, 4x4 block boundaries are not processed.

[0023] Figure 2 shows the overall processing flowchart of the deblocking filter process. There can be three filtering states: no filtering, weak filtering, and strong filtering. Each filtering decision involves a boundary strength, Bs, and a threshold value, and t C It is based on.

[0024] 2.1 Boundary determination (decision)

[0025] Three types of boundaries can be included in the filtering process: CU boundaries, TU boundaries, and PU boundaries. Since a CU boundary is always either a TU boundary or a PU boundary, the CU boundary, which is the outer edge of the CU boundary, is always involved in the filtering. When the PU geometry is 2NxN (N > 4) and the RQT depth is equal to 1, the TU boundaries of the 8x8 block grid and each PU boundary inside the CU are included in the filtering. One exception is that the boundary is not filtered when the PU boundary is inside the TU.

[0026] 2.2 Boundary strength calculation

[0027] Figure 3 shows a flowchart for the calculation of Bs. Generally speaking, the boundary strength (Bs) reflects how strong filtering is required at the boundary. If Bs is large, strong filtering should be considered. P and Q can be defined as blocks involved in the filtering, where P represents a block on the left (vertical edge case) or above (horizontal edge case) of the boundary, and Q represents a block on the right (vertical edge case) or above (horizontal edge case) of the boundary. Figure 3 illustrates how the Bs value is calculated based on the intra-coding mode, non-zero transformation coefficients, motion information, reference picture, number of motion vectors, and the presence of motion vector differences.

[0028] Bs is calculated based on a 4x4 block but is mapped back to an 8x8 grid. The maximum value of Bs corresponding to 8 pixels consisting of lines of a 4x4 grid is selected as the Bs for the boundary of the 8x8 grid.

[0029] Figure 4 shows an example of referenced information for Bs calculation at the CTU boundary. To reduce line buffer memory requirements, information from all second blocks (4x4 grid) to the left or top is reused only for the CTU boundary, as depicted in Figure 4.

[0030] 2.3 and t C decision (decision)

[0031] Decide whether to turn the filter on or off Threshold included in (decision) and t C The threshold, strong and weak filter selection, and weak filtering process are derived based on the luminance quantization parameters of the P and Q blocks, QPP, and QPQ, respectively. and t C Q used to derive is calculated as follows.

[0032] Q = ( ( QP P + QP Q + 1 ) >> 1 ).

[0033] variable It is derived based on Q as shown in Table 1. When Bs is greater than 1, the variable t C It specifies Table 1 with Clip3(0, 55, Q + 2) as input. Otherwise (if BS is equal or less than 1, variable t C Set Q as input to Table 1.

[0034] In input Q, threshold variables β and t C induction of Q 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 6 7 8 t C 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 Q 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 9 10 11 12 13 14 15 16 17 18 20 22 24 26 28 30 32 34 36 t C 1 1 1 1 1 1 1 1 2 2 2 2 3 3 3 3 4 4 4 Q 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 38 40 42 44 46 48 50 52 54 56 58 60 62 64 64 64 64 64 t C 5 5 6 6 7 8 9 9 10 10 11 11 12 12 13 13 14 14

[0035] 1.2.4 Turn on / off decision for each of the 4 lines (decision) Filtering

[0036] Figure 5 shows an example of pixels included in a filter on / off decision. In the following description, note that pNM represents the Nth sample from the left of the Mth row relative to the vertical edge, or the Nth sample from the top of the Mth column relative to the horizontal edge, and qNM represents the Nth sample from the right of the Mth row relative to the vertical edge or the Nth sample from the bottom of the Mth column relative to the horizontal edge. Examples of pNM and qNM are depicted in Figure 5.

[0037] Note that in the following description, pN represents the Nth sample from the left of the row based on the vertical edge or the Nth sample from the top of the column based on the horizontal edge, or qN represents the Nth sample from the right of the row based on the vertical edge or the Nth sample from the bottom of the column based on the horizontal edge.

[0038] The filter on / off decision is performed for four lines as a unit. Figure 5 shows the pixels involved in the filter on / off decision. The 6 pixels in the two red boxes for the first four lines are used to determine the filter on / off for the 4 lines. The 6 pixels in the two red boxes for the second four lines are used to determine the filter on / off for the second four lines.

[0039] In the case where dp0 + dq0 + dp3 + dq3 < β, filtering is enabled for the first four lines and a strong / weak filter selection process is applied. Each variable is derived as follows.

[0040] dp0 = | p20 - 2*p10 + p00 |, dp3 = | p23 - 2*p13 + p03 |, dp4 = | p24 - 2*p14 + p04 |, dp7 = | p27 - 2*p17 + p07 |

[0041] dq0 = | q20 - 2*q10 + q00 |, dq3 = | q23 - 2*q13 + q03 |, dq4 = | q24 - 2*q14 + q04 |, dq7 = | q27 - 2*q17 + q07 |

[0042] If the condition is not met, filtering is not performed on the first 4 lines. Also, if the condition is met, dE, dEp1, and dEp2 are derived for the weak filtering process. The variable dE is set to equal 1. If dp0 + dp3 < (β + (β >> 1)) >> 3, the variable dEp1 is set to equal 1. If dq0 + dq3 < (β + (β >> 1)) >> 3, the variable dEq1 is set to equal 1. For the second 4 lines, a decision is made in the same manner as above.

[0043]

[0044] 2.5 Select strong / weak filter for 4 lines

[0045] After the first four lines are examined to determine whether the filter is activated, if the following two conditions are met, a strong filter is used to filter the first four lines. Otherwise, a weak filter is used. The relevant pixels are the same as the pixels used to filter the on / off decision as depicted in Fig. 5.

[0046] 2*(dp0+dq0) < ( β >> 2 ), | p30 - p00 | + | q00 - q30 | < ( β >> 3 ) and | p00 - q00 | < ( 5* tC + 1 ) >> 1

[0047] 2*(dp3+dq3) < ( β >> 2 ), | p33 - p03 | + | q03 - q33 | < ( β >> 3 ) and | p03 - q03 | < ( 5* tC + 1 ) >> 1

[0048] Likewise, if both conditions are met, a strong filter is used to filter the second 4 lines. Otherwise, a weak filter is used.

[0049] 2*(dp4+dq4) < ( β >> 2 ), | p34 - p04 | + | q04 - q34 | < ( β >> 3 ) and | p04 - q04 | < ( 5* tC + 1 ) >> 1

[0050] 2*(dp7+dq7) < ( β >> 2 ), | p37 - p07 | + | q07 - q37 | < (β >> 3 ) and | p07 - q07 | < ( 5* tC + 1 ) >> 1

[0051] 2.5.1 Strong filtering

[0052] For strong filtering, the filtered pixel value is obtained by the following equation. 3 pixels are modified using 4 pixels each as input for each P and Q block.

[0053] p0'= ( p2 + 2*p1 + 2*p0 + 2*q0 + q1 + 4 ) >> 3

[0054] q0'= ( p1 + 2*p0 + 2*q0 + 2*q1 + q2 + 4 ) >> 3

[0055] p1'= ( p2 + p1 + p0 + q0 + 2 ) >> 2

[0056] q1'= ( p0 + q0 + q1 + q2 + 2 ) >> 2

[0057] p2'= ( 2*p3 + 3*p2 + p1 + p0 + q0 + 4 ) >> 3

[0058] q2'= ( p0 + q0 + q1 + 3*q2 + 2*q3 + 4 ) >> 3

[0059] 2.5.2 Weak filtering

[0060] Let's define △ as follows.

[0061] △ = ( 9 * ( q0 - p0 ) - 3 * ( q1 - p1 ) + 8 ) >> 4

[0062] abs(△) is t C If less than *10,

[0063] △ = Clip3( - t C , t C , △ )

[0064] p0' = Clip1 Y ( p0 + △ )

[0065] q 0' = Clip1 Y ( q0 - △ )

[0066] If dEp1 is equal to 1

[0067] △p = Clip3 (t C >> 1), t C >> 1, (((p2 + p0 + 1) >> 1 ) - p1 + ) △ >>1)

[0068] p1'=lip1 Y ( p1 + △p )

[0069] If dEq1 is equal to 1,

[0070] △q = Clip3 (-(t C >> 1), t C >> 1, (((q2 + q0 + 1) >> 1) - q1 - ) △ >>1)

[0071] q1' = Clip1 Y ( q1 + Dq )

[0072] It is necessary to modify up to 2 pixels as input for each P and Q block.

[0073] 2.5.3 Chroma filtering

[0074] Chroma filtering B is inherited from luminance. Chroma filtering is performed in the case where Bs > 1 or the coded chroma coefficient exists. There are no other filtering decisions. And only one filter is applied to chroma. The filter selection process for chroma is not used. The filtered sample values ​​p0' and q0' are derived as follows.

[0075] △ = Clip3 ( -t C , t C , ( ( ( ( q0 - p0 ) << 2 ) + p1 - q1 + 4 ) >> 3 ) )

[0076] p0'= Clip1 C ( p0 + △ )

[0077] q0'= Clip1 C ( q0 - △ )

[0078] 3 Example of deblocking configuration in VTM-4.0

[0079] The deblocking method described in VTM-4.0, JVET-M0471, is currently used in VTM. The vertical edges of the picture are filtered first. Then, the horizontal edges of the picture are filtered into samples modified as input by the vertical edge filtering process. The vertical and horizontal edges of the CTB of each CTU are processed separately based on the coding unit. The vertical edges of the coding blocks of the coding unit are filtered starting from the left edge of the coding block, passing through the edge toward the right of the coding block in geometric order. The horizontal edges of the coding blocks of the coding unit are filtered starting from the top edge of the coding block, passing through the edge toward the bottom of the coding block in geometric order.

[0080] 3.1 Boundary determination (decision)

[0081] Filtering is applied to 8x8 block boundaries. Additionally, it must be a transform block boundary or a coding subblock boundary (e.g., due to affine motion prediction or ATMVP usage). If it is not one of these boundaries, the filter is disabled.

[0082] 3.2 Boundary strength calculation

[0083] For transform block boundaries / coding subblock boundaries, if they are on an 8x8 grid, they can be filtered, and the setting of bS[ xDi ][yDj ] for these edges is defined as follows.

[0084] - If sample p0 or q0 is in a coding block of a coding unit coded in intra-prediction mode, bS[xDi][[yDj]] is set to 2.

[0085] - Otherwise, if the block edge is a transformation block edge and the sample p0 or q0 is in a transformation block containing one or more non-zero transformation coefficient levels, bS[xDi][yDj] is set to 1.

[0086] - Otherwise, if the prediction mode of the coding subblock containing sample p0 is different from the prediction mode of the coding subblock containing sample q0, bS[xDi][[yDj]] is set to 1.

[0087] - Otherwise, if one or more of the following conditions are true, bS[ xDi][yDj] is set to 1:

[0088] -- Both the coding subblock containing sample p0 and the coding subblock containing sample q0 are coded in IBC prediction mode, and the absolute difference between the horizontal or vertical components of the motion vectors used for prediction in the two coding subblocks is greater than or equal to 4 in the branching luminous sample units.

[0089] -- Different numbers of different reference pictures or motion vectors are used for the prediction of a coding subblock containing sample p0, and for the prediction of a coding subblock containing sample q0.

[0090] --- NOTE 1 - The determination of whether the reference pictures used in two coding subblocks are the same or different is based solely on the referenced picture, regardless of whether the prediction is formed using the index for picture list 0 or the index for reference picture list 1, and also regardless of whether the index positions within the reference picture list are different.

[0091] --- NOTE 2 - The number of motion vectors used for prediction of coding subblocks using the top-left sample covering (xSb, ySb) is equal to PredFlagL0[xSb] [ ySb] + PredFlagL1[ xSb][ ySb].

[0092] -- One motion vector is used to predict a coding subblock containing sample p0, and one motion vector is used to predict a coding subblock containing sample q0, and the absolute difference between the horizontal or vertical components of the motion vectors used is greater than or equal to 4 in quarter luma sample units.

[0093] -- Two motion vectors and two different reference pictures are used to predict a coding subblock containing sample p0, two motion vectors for the same two reference pictures are used to predict a coding subblock containing sample q0, and the absolute difference between the horizontal or vertical components of the two motion vectors used to predict the two coding subblocks for the same reference pictures is greater than or equal to 4 in 1 / 4 luma sample unit.

[0094] -- Two motion vectors for the same reference picture are used to predict the coding subblock containing sample p0, two motion vectors for the same reference picture are used to predict the coding subblock containing sample q0, and all of the following conditions are true:

[0095] The absolute difference between the horizontal or vertical components of the List 0 motion vector used for the prediction of the two coding subblocks is greater than or equal to 4 in 1 / 4 luma sample, or the absolute difference between the horizontal or vertical components of the List 1 motion vector used for the prediction of the two coding subblocks is greater than or equal to 4 in 1 / 4 luma sample unit.

[0096] The absolute difference between the horizontal or vertical components of the List 0 motion vector used for prediction of the coding subblock containing sample p0 and the List 1 motion vector used for prediction of the coding subblock containing sample q0 is greater than or equal to 4 in 1 / 4 luminous sample unit, or the absolute difference between the horizontal or vertical components of the List 1 motion vector used for prediction of the coding subblock containing sample p0 and the List 0 motion vector used for prediction of the coding subblock containing sample q0 is greater than or equal to 4 in 1 / 4 luminous sample unit.

[0097] - Otherwise, variable bS [xD i ][yDj] is set to be equal to 0.

[0098] Tables 2 and 3 summarize the BS calculation rules.

[0099] Boundary strength (when SPS IBC is disabled) priority condition Y U V 5 At least one neighboring block is intra. 2 2 2 4 At least one of the blocks adjacent to the TU boundary has a non-zero transform coefficient. 1 1 1 3 The reference picture or MV (single prediction 1, double prediction 2) of neighboring blocks is different. 1 N / A N / A 2 The absolute difference between motion vectors of the same reference picture belonging to an adjacent block is greater than or equal to one integer luminance sample. 1 N / A N / A 1 otherwise 0 0 0

[0100] Boundary strength (when SPS IBC is enabled) priority condition Y U V 8 At least one neighboring block is intra. 2 2 2 7 At least one of the blocks adjacent to the TU boundary has a non-zero transform coefficient. 1 1 1 6 The prediction modes of neighboring blocks are different (e.g., one is IBC, the other is Inter). 1 5 The IBC and absolute difference between motion vectors belonging to adjacent blocks is greater than or equal to one integer luminous sample. 1 N / A N / A 4 The reference picture or MV (single prediction 1, double prediction 2) of neighboring blocks is different. 1 N / A N / A 3 The absolute difference between motion vectors of the same reference picture belonging to an adjacent block is greater than or equal to one integer luminance sample. 1 N / A N / A 1 otherwise 0 0 0

[0101] 3.3 Deblocking decision for Luma components

[0102] The deblocking decision process is described in this subsection. A wider-stronger luma filter is all Condition 1, Condition 2 and Condition 3 The filter is used only when it is true.

[0103] Condition 1 is the "Large Block Condition." This condition detects whether samples on the P-side and Q-side belong to a large block, denoted by the variable bSidePisLargeBlk and bSideQisLargeBlk, respectively. bSidePisLargeBlk and bSideQisLargeBlk are defined as follows.

[0104] bSidePisLargeBlk = ((Edge type is vertical and p0 belongs to CU with width >= 32| | (Edge type is horizontal and p0 belongs to CU with height >= 32)? True : False

[0105] bSideQisLargeBlk = ((Edge type is vertical and q0 belongs to CU with width >= 32) | | (Edge type is horizontal and q0 belongs to CU with height >= 32)? True : False

[0106] Based on bSidePisLargeBlk and bSideQisLargeBlk, condition 1 is defined as follows.

[0107] Condition 1 = (bSidePisLargeBlk || bSidePisLargeBlk) ? True : False

[0108] Next condition If 1 is true, condition 2 is further checked. First, the following variable is derived:

[0109] dp0, dp3, dq0, and dq3 are derived first, as in HEVC.

[0110] (If p is greater than or equal to 32)

[0111] dp0 = ( dp0 + Abs(p50-2*p40+p30) + 1 ) >> 1

[0112] dp3 = ( dp3 + Abs(p53-2*p43+p33) + 1 ) >> 1

[0113] (If q is greater than or equal to 32)

[0114] dq0 = ( dq0 + Abs(q50-2*q40+q30) + 1 ) >> 1

[0115] dq3 = ( dq3 + Abs(q53-2*q43+q33) + 1 ) >> 1

[0116] condition 2 = (d < β) ? True : False

[0117] Here, as seen in Section 2.4, d = dp0 + dq0 + dp3 + dq3.

[0118] If conditions 1 and 2 are valid, additionally check whether there is a block among the blocks using the subblock:

[0119] (bSidePisLargeBlk)

[0120] If(mode block P == SUBBLOCKMODE)

[0121] Sp = 5

[0122] else

[0123] Sp =7

[0124] else

[0125] Sp = 3

[0126] (bSideQisLargeBlk)

[0127] (mode block P == SUBBLOCKMODE)

[0128] Sq = 5

[0129] else

[0130] Sq =7

[0131] else

[0132] Sq = 3

[0133] Finally, if both Condition 1 and Condition 2 are valid, the proposed deblocking method checks Condition 3 (large block strong filter condition) defined as follows, which is defined as follows.

[0134] Condition 3 The following variables are derived from the StrongFilter condition:

[0135] dpq is derived as in HEVC.

[0136] SP3= Abs(p3-p0) derived from HEVC,

[0137] (If p is greater than or equal to 32)

[0138] if(Sp==5)

[0139] sp3= (sp3+ Abs(p5-p3) + 1) >> 1

[0140] else

[0141] sp3= (sp3+ Abs(p7-p3) + 1) >> 1

[0142] 3 squared = Abs(q0 - q3), derived from HEVC

[0143] (If q is greater than or equal to 32)

[0144] If(Sq==5)

[0145] sq3= ( sq3+ Abs(q5-q3) + 1) >> 1

[0146] else

[0147] sq3= ( sq3+ Abs(q7-q3) + 1) >> 1

[0148] As in HEVC, StrongFilterCondition = (dpq is less than (β >> 2)), sp3 + sq3 is less than (3 * β >> 5), and Abs(p0 - q0) is less than (5 * tC + 1) >> 1) ? True: False

[0149] 3.4 Stronger deblocking filter for Luma (designed for larger blocks)

[0150] A bilinear filter is used when samples on both sides of a boundary belong to a larger block. Samples belonging to a larger block are defined as having a width >= 32 for the vertical edge and a height >= 32 for the horizontal edge. Bilinear filters are listed below.

[0151] The block boundary samples pi for i = 0 ~ Sp-1 and qi for j = 0 ~ Sq-1 (where pi and qi are the i-th samples within a row for filtering vertical edges or horizontal edges in the HEVC deblocking described above) are replaced by linear interpolation as follows.

[0152] p i ´=(f i *Middle s,t +(64-f i )*P s +32)≫6),clipped to p i ±tcPD i

[0153] q j ´=(g j *Middle s,t +(64-g j )*Q s +32)≫6),clipped to q j ±tcPD j

[0154] The terms {tcPD}_i and {tcPD}_j are as follows when the position-dependent clipping described in Section 3.6 and g_j f_i, {Middle}_{s,t}, P_s, Q_s is as follows.

[0155] Sp, Sq7, 7(p side: 7, q side: 7) 7, 3 (p side: 7q side: 3) 3, 7 (p side: 3q side: 7) 7, 5 (p side: 7q side: 5) 5, 7 (p side: 5q side: 7) 5, 5 (p side: 5q side: 5) 5, 3 (p side: 5q side: 3) 3, 5 (p side: 3q side: 5)

[0156] 3.5 Deblocking control for Chroma

[0157] A chroma strong filter is used on both sides of the block boundary. Here, the chroma filter is selected when both sides of the chroma edge are greater than or equal to 8 (chroma position), and the following decision is satisfied when three conditions are met: The first is for the decision of large blocks as well as boundary intensity. The proposed filter can be applied when the block width or height orthogonally traversing the block edge is greater than or equal to 8 in the chroma sample domain. The second and third are essentially the same as the HEVC luminance deblocking decision, which are the on / off decision and the strong filter decision, respectively.

[0158] In the first decision, the boundary intensity (bS) is modified for chroma filtering as shown in Table 2. The conditions in Table 2 are checked sequentially. If a condition is met, the remaining conditions with lower priority are skipped.

[0159] Chroma deblocking is performed when bS is equal to 2 or when bS is equal to 1 when a large block boundary is detected.

[0160] The second and third conditions are basically the same as the HEVC luminance strong filter decision. In the second condition:

[0161] d is derived as in HEVC lumens deblocking. When d is less than β, the second condition is It becomes true.

[0162] Under the third condition, StrongFilterCondition is derived as follows:

[0163] dpq is derived as in HEVC.

[0164] sp3 = Abs (p3 - p 0),

[0165] sq3 = Abs (q0 - q3) derived from HEVC

[0166] As in the HEVC design, StrongFilterCondition = (dpq is less than (β >> 2), sp3 + sq3 is less than (β >> 3), and Abs(p0 - q0) is less than (5 * tC + 1) >> 1).

[0167] 3.6 Strong deblocking filter for chroma

[0168] The following strong deblocking filter for chroma is defined.

[0169] p2'= (3*p3+2*p2+p1+p0+q0+4)>>3

[0170] p1'= (2*p3+p2+2*p1+p0+q0+q1+4)>>3

[0171] p0'= (p3+p2+p1+2*p0+q0+q1+q2+4)>>3

[0172] The proposed chroma filter performs deblocking on a 4x4 chroma sample grid.

[0173] 3.7 Position-dependent clipping

[0174] Position-dependent clipping (tcPD) is applied to the output samples of a luminance filtering process that includes strong, long filters that correct samples 7, 5, and 3 at the boundaries. Assuming a quantification error distribution, increasing the clipping value for samples expected to have higher quantification noise is expected to result in a higher deviation of the sample values ​​reconstructed from the actual sample values.

[0175] For each P or Q boundary filtered by an asymmetric filter, a position-dependent threshold table is selected from two tables (e.g., Tc7 and Tc3 shown in the table below) provided to the decoder as side information according to the result of the decision process of Section 3.2.

[0176] Tc7 = { 6, 5, 4, 3, 2, 1, 1};

[0177] Tc3 = { 6, 4, 2};

[0178] tcPD = (Sp == 3) ? Tc3 : Tc7;

[0179] tcQD = (Sq == 3) ? Tc3 : Tc7;

[0180] For P or Q boundaries filtered by a short symmetric filter, a low-magnitude position-dependent threshold is applied.

[0181] Tc3 = { 3, 2, 1};

[0182] Define the threshold and then filter p' i and q' i Sample values ​​are truncated based on tcP and tcQ clipping values.

[0183] P" i = Clip3(p' i + tcP i , p' i - tcP i , p' i );

[0184] Q" j = Clip3(q' j + tcQ j , q' j - tcQ j , q' j );

[0185] Here, p' i wa q' i is the filtered sample value, and p' iand q' j is the output sample value after clipping, and tcPi and tcQi are VVC t C It is the clipping threshold derived from the parameters and tcPD and tcQD. The function Clip3 is the clipping function specified in VVC.

[0186] 3.8 Subblock deblocking adjustment

[0187] To enable parallel-friendly deblocking using both a long filter and subblock deblocking, the long filter is limited to modifying at most 5 samples on the side using subblock deblocking (affine, ATMVP, or DMVR) as indicated in the luma control for the long filter. Additionally, subblock deblocking is adjusted to limit the subblock boundaries of an 8x8 grid near the CU or implicit TU boundary to modifying at most 2 samples on each side.

[0188] The following applies to CU boundaries and sub-block boundaries that are not aligned.

[0189] (mode block Q == SUBBLOCKMODE && edge !=0) {

[0190] if (!(implicitTU && (edge ​​== (64 / 4))))

[0191] if (edge ​​== 2 || edge == (orthogonalLength - 2) || edge == (56 / 4) || edge == (72 / 4))

[0192] Sp = Sq = 2;

[0193] else

[0194] Sp = Sq = 3;

[0195] else

[0196] Sp = Sq = bSideQisLargeBlk ? 5:3

[0197] }

[0198] If an edge equal to 0 corresponds to a CU boundary, then an edge equal to or equal to 2 or orthogonalLength-2 corresponds to 8 samples of subblock boundaries in the CU boundary, etc. If an implicit split of TU is used, then implicit TU is true.

[0199] 3.9 Limitations on 4CTU / 2CTU line buffers for Luma / Chroma

[0200] Filtering of horizontal edges restricts Sp = 3 for Luma and Sp = 1 and Sq = 1 for Chroma when the horizontal edge is aligned with the CTU boundary.

[0201] 4 Example of Intra Subblock Partitioning (ISP)

[0202] Figure 6a shows an example of the division of 4x8 and 8x4 blocks. Figure 6b shows an example of the division of all blocks except 4x8, 8x4, and 4x4.

[0203] In JVET-M0102, an ISP is proposed that divides the luma intra prediction block into 2 or 4 sub-partitions vertically or horizontally depending on the block size dimensions, as shown in Table 3. Figures 6a and 6b show examples of the two possibilities. All sub-partitions satisfy the condition of having at least 16 samples. Regarding block size, 4 × N or N × 4 (including N > 8) or, if permitted, 1 × N or N × 1 sub-partitions may exist.

[0204] Division direction Block size Number of subpartitions N / A Minimum conversion size Not divided 4x8: Horizontal 8x4: Vertical 4x8 and 8x4 2 signal The case where it is neither 4x8 nor 8x4, and W <= maxTBSize and H <= maxTBSize 4 horizontality In the above case and H > maxTBSize 4 perpendicular In the above case and H > maxTBSize 4

[0205] For each of these subpartitions, the coefficients transmitted by the encoder are entropy-decoded, followed by inverse quantization and inverse transformation to generate a residual signal. Subsequently, the subpartition is intra-predicted, and finally, the corresponding reconstructed sample is obtained by adding the residual signal to the predicted signal. Thus, the reconstructed value of each subpartition can be used to generate the prediction for the next subpartition, and this value is used to repeat the process. All subpartitions share the same intra mode.

[0206] predModeIntra trTypeHor trTypeVer INTRA_PLANAR, INTRA_ANGULAR31, INTRA_ANGULAR32, INTRA_ANGULAR34, INTRA_ANGULAR36, INTRA_ANGULAR37 ( nTbW >= 4 && nTbW <= 16 ) ? DST-VII : DCT-II ( nTbH >= 4 && nTbH <= 16 ) ? DST-VII : DCT-II INTRA_ANGULAR33, INTRA_ANGULAR35 DCT-II DCT-II INTRA_ANGULAR2, INTRA_ANGULAR4, INTRA_ANGULAR28, INTRA_ANGULAR30, INTRA_ANGULAR39, INTRA_ANGULAR41, INTRA_ANGULAR63, INTRA_ANGULAR65 ( nTbW >= 4 && nTbW <= 16 ) ? DST-VII : DCT-II DCT-II INTRA_ANGULAR3, INTRA_ANGULAR5, , INTRA_ANGULAR27, INTRA_ANGULAR29, INTRA_ANGULAR38, INTRA_ANGULAR40, , INTRA_ANGULAR64, INTRA_ANGULAR66 DCT-II ( nTbH >= 4 && nTbH <= 16 ) ? DST-VII : DCT-II

[0207] 5 Example of Affine Linear Weighted Intra Prediction (ALWIP, also known as Matrix-Based Intra Prediction).

[0208] Affine linear weighted intra-prediction (ALWIP, also known as Matrix-based Intra-prediction (MIP)) is proposed in JVET-N0217.

[0209] 5.1 Generation of reduced prediction signals by matrix vector multiplication

[0210] Neighbor reference samples reduce the reference signal bdry red To generate it, it is first downsampled through the mean. Then, the reduced predicted pred red The signal is calculated by calculating the matrix vector product and adding an offset.

[0211] pred red =A·bdry red +b.

[0212] Here, if A is W = H = 4, then W red ·H red It is a matrix with rows and 4 columns, and in all other cases, 8 columns. b is W red ·H red It is a vector of size.

[0213] 5.2 City of the entire ALWIP process

[0214] The entire process of averaging, matrix vector multiplication, and linear interpolation is illustrated for the various shapes in Figures 7a-7d. The remaining shapes are treated as one of the depicted cases.

[0215] Figure 7a shows an example of affine linear weighted intra prediction (ALWIP) for a 4x4 block. Given a 4x4 block, ALWIP takes the average of two values ​​along each axis of the boundary. The four resulting input samples are fed into a matrix vector product. The matrices are taken from set S0. After adding an offset, 16 final prediction samples are generated. Linear interpolation does not need to generate a prediction signal. Therefore, per sample (4 16) / (4 4) = 4 multiplications are performed.

[0216] Figure 7b shows an example of ALWIP for an 8x8 block. Given an 8x8 block, ALWIP takes the average of 4 values ​​along each axis of the boundary. The 8 resulting input samples are put into matrix vector multiplication. The matrix is ​​taken from set S1. This generates 16 samples at the odd positions of the prediction block. Therefore, a total of (8 per sample 16) / (8 8) = 2 multiplications are performed. After adding the offset, these samples are vertically interpolated using the reduced top boundary. Horizontal interpolation is followed by using the original left boundary.

[0217] Figure 7c shows an example of ALWIP for an 8x4 block. Given an 8x4 block, ALWIP takes the average of 4 values ​​along the horizontal axis of the boundary and 4 original boundary values ​​from the left boundary. The resulting 8 input samples enter matrix vector multiplication. The matrix is ​​taken from set S1. This generates 16 samples for the horizontal odds of the prediction block and for each vertical position. Therefore, a total of (8 per sample 16) / (8 4) = 4 multiplications are performed. After adding an offset, this sample is horizontally interpolated using the original left boundary. The transposed case is processed accordingly.

[0218] Figure 7d shows an example of ALWIP for a 16x16 block. Given a 16x16 block, ALWIP takes the average of 4 along each axis of the boundary. The 8 resulting input samples enter matrix vector multiplication. The matrix is ​​taken from set S2. This generates 64 samples at the odd positions of the prediction block. Therefore, a total of (8 per sample 64) / (16 16) = 2 multiplications are performed. After adding the offset, this sample is vertically interpolated using the average of the top boundary 8. Horizontal interpolation is followed by using the original left boundary. In this case, the interpolation process does not add a multiplication. Therefore, a total of 2 multiplications per sample are required to calculate the ALWIP prediction.

[0219] For large shapes, the procedure is essentially the same, and it is easy to check if the number of multiplications per sample is less than 4.

[0220] For Wx8 blocks where W > 8, only horizontal interpolation is required because samples are given at odd horizontal and vertical positions.

[0221] Finally, for a Wx4 block where W > 8, the matrix A is obtained by omitting all rows corresponding to odd entries along the horizontal axis of the downsampled block. k It is set to . Therefore, the output size is 32, and again, only horizontal interpolation is performed. The transposed case is handled accordingly.

[0222] 6 Examples of multiple transform sets (MTS) in VVC

[0223] 6.1 Explicit Multiple Transform Set (MTS)

[0224] In VTM4, large block size transforms up to 64x64 are enabled, which is useful for high-resolution videos such as 1080p and 4K sequences. High frequency transform coefficients are zeroed out for transform blocks with a size (width, height, or both) equal to 64, ensuring that only lower frequency coefficients are retained. For example, for an MxN transform block, where M is the block width and N is the block height, only the left 32 columns of transform coefficients are retained when M is equal to 64. Similarly, when N is equal to 64, only the top 32 columns of transform coefficients are retained. If transform skip mode is used for large blocks, the entire block is used without zeroing out its values.

[0225] In addition to DCT-II used in HEVC, the MTS (Multiple Transform Selection) scheme is used for residual coding in both inter- and intra-coded blocks. It utilizes multiple transformations selected from DCT8 / DST7. The newly introduced transformation matrices are DST-VII and DCT-VIII. Table 7 shows the base functions of the selected DST / DCT.

[0226] To maintain the orthogonality of the transform matrix, the transform matrix is ​​quantized more accurately than the HEVC transform matrix. To keep the intermediate values ​​of the transformed coefficients within a 16-bit range, all coefficients must be 10-bit after horizontal and vertical transformations.

[0227] To control MTS configuration, separate usage flags are assigned to Intra and Inter, respectively, at the SPS level. When MTS is enabled at the SPS, a CU level flag signals whether MTS is applied. Here, MTS is applied only to Luma. The MTS CU level flag signals when the following conditions are met.

[0228] Width and height are less than or equal to 32

[0229] The CBF flag is identical to one.

[0230] Conversion Type basis function T i (j ), i , j = 0, 1,..., N -1 DCT-II Here, DCT-VIII DST-VII

[0231] If the MTS CU flag is equal to 0, DCT2 is applied bidirectionally. However, if the MTS CU flag is equal to 1, two additional flags are signaled to indicate the transformation type for the horizontal and vertical directions, respectively. Table 8 is the transformation and signaling mapping table. An 8-bit first-order transformation core is used when transforming matrix precision. Therefore, all transformation cores used in HEVC remain the same, including 4-point DCT-2 and DST-7, and 8-point, 16-point, and 32-point DCT-2. Additionally, other transformation cores, including 64-point DCT-2, 4-point DCT-8, and 8-point, 16-point, and 32-point DST-7 and DCT-8, use the 8-bit first-order transformation core.

[0232] Mapping of the decoded values ​​of tu_mts_idx with the corresponding transformation matrices for the horizontal and vertical directions tu_mts_idx empty string tu_mts_idx Intra / Inter horizontality perpendicular 0 0 DCT2 1 0 1 DST7 DST7 1 1 0 2 DCT8 DST7 1 1 1 0 3 DST7 DCT8 1 1 1 1 4 DCT8 DCT8

[0233] To reduce the complexity of the large DST-7 and DCT-8, high-frequency conversion factors are set to zero for DST-7 and DCT-8 blocks with the same size (width or height, or both) as 32. Only factors within the 16x16 low-frequency region are retained. In addition to cases where other conversions are applied, VVC also supports a mode called Transform Skip (TS), which is similar to the concept of TS in HEVC. TS is treated as a special case of MTS.6.2 Implicit Multiple Transform Set (MTS)

[0234] ISP, SBT, and MTS are configured to be used, but all implicit signals are processed as implicit MTS.

[0235] In this patent document, implicitMtsEnabled is used to define whether implicit MTS is enabled.

[0236] 8.7.4 Transformation process for scaled transformation coefficients

[0237] 8.7.4.1 common

[0238] The variable implicitMtsEnabled is derived as follows:

[0239] - If sps_mts_enabled_flag is equal to 1 and one of the following conditions is true, implicitMtsEnabled is set to the same as 1:

[0240] - IntraSubPartitionsSplitType is not the same as ISP_NO_SPLIT.

[0241] - cu_sbt_flag is equal to 1 and Max(nTbW, nTbH) is less than or equal to 32.

[0242] - sps_explicit_mts_intra_enabled_flag and sps_explicit_mts_inter_enabled_flag are both equal to 0, and cuPredMode[xTbY][yTbY] is the same as MODE_INTRA.

[0243] - Otherwise, implicitMtsEnabled is set to 0.

[0244] The variable trTypeHor, which specifies the horizontal and vertical conversion kernels, is derived as follows:

[0245] - If cIdx is greater than 0, trTypeHor and trTypeVer are set to 0.

[0246] - Otherwise, if implicitMtsEnabled is equal to 1, the following applies:

[0247] - If IntraSubPartitionsSplitType is not equal to ISP_NO_SPLIT, trTypeHor and trTypeVer are specified in Table 8-14, which depends on intraPredMode.

[0248] - Otherwise, if cu_sbt_flag is equal to 1, trTypeHor and trTypeVer are specified in Table 8-15, depending on cu_sbt_horizontal_flag and cu_sbt_pos_flag.

[0249] - Otherwise (sps_explicit_mts_intra_enabled_flag and sps_explicit_mts_inter_enabled_flag are equal to 0), trTypeHor and trTypeVer are derived as follows:

[0250] trTypeHor = ( nTbW >= 4 && nTbW <= 16 && nTbW <= nTbH ) ? 1:0 (8-1030)

[0251] trTypeVer = ( nTbH >= 4 && nTbH <= 16 && nTbH <= nTbW ) ? 1:0 (8-1031)

[0252] - Otherwise, trTypeHor and trTypeVer are specified in Table 8-13, depending on tu_mts_idx[xTbY][yTbY].

[0253] Table 8-13 - Specifications of trTypeHor and trTypeVer depending on tu_mts_idx[x][y]

[0254]

[0255] Table 8-14 - Specifications of trTypeHor and trTypeVer depending on cu_sbt_horizontal_flag and cu_sbt_pos_flag

[0256]

[0257] 7 Example of Reduced Second Transformation (RST) proposed in JVET-N0193

[0258] 7.1 Inseparable second transformation (NSST) in JEM

[0259] Figure 8 shows an example of a second-order transformation in JEM. In JEM, the second-order transformation is applied between the forward primary transformation and quantization (at the encoder) and the dequantization and inverse primary transformation (at the decoder side). As illustrated in Figure 8, the 4x4 (or 8x8) second-order transformation is performed depending on the block size. For example, the 4x4 second-order transformation is applied to small blocks (i.e., minimum (width, height) < 8) and the 8x8 second-order transformation is applied to larger blocks (i.e., minimum (width, height) > 4) per 8x8 block.

[0260] The application of a non-separable transform is explained as follows using the input as an example. To apply the non-separable transform, the 4x4 input block X

[0261]

[0262] is first a vector It is expressed as:

[0263]

[0264] Inseparable transformation It is calculated as and here represents the transformation coefficient vector, and T is a 16x16 transformation matrix. 16x1 coefficient vector It is reconstructed into 4x4 blocks using the scanning order (horizontal, vertical, or diagonal) for that block. Counts with smaller indices are placed in the 4x4 count block along with smaller scanning indices. There are a total of 35 transformation sets, and 3 non-separable transformation matrices (kernels) are used per transformation set. In intra-prediction mode, mapping to transformation sets is predefined. For each transformation set, selected NSST (non-separable quadratic transformation) candidates are further specified by explicitly signaled quadratic transformation indices. The indices are signaled in the bitstream once per intra-CU after the transformation counts.

[0265] 7.2 Reduced Secondary Transform (RST) of JVET-N0193

[0266] FIG. 9 shows an example of a reduced second-order transformation (RST) based on an embodiment of the disclosed technology. The RST (Low-Frequency Non-Separable Transformation (LFNST)) was introduced in JVET-K0099, and a mapping of 4 transformation sets (instead of 35 transformation sets) was introduced in JVET-L0133. In this JVET-N0193, 16x48 and 16x16 matrices are used in 8x8 and 4x4 blocks, respectively. For convenience of notation, the 16x48 transformation is denoted as RST8x8 and the 16x16 transformation as RST4x4.

[0267] 7.3 Application of RST

[0268] The selection of RST is defined as follows:

[0269] - For 4*N or N*4 blocks (N<=8), apply RST4x4 to the top-left 4×4 CG (e.g., 4x4, 4x8, 8x4).

[0270] - For 4*N or N*4 blocks (N>8), apply RST4x4 twice to two adjacent top-left 4*4 CGs (e.g., 4x16, 16x4, etc.).

[0271] - In all other cases, a quadratic transformation is applied to the top-left 3 4x4 CGs to generate the top-left 4x4 CG and two other CGs with all coefficients being 0.

[0272] 8 Example of subblock transformation

[0273] For mutually predicted CUs identical to cu_cbf 1, cu_sbt_flag can be signaled to indicate whether the entire residual block or a sub-part of the residual block is decoded. In the former case, information between MTS is further parsed to determine the transform type of the CU. In the latter case, part of the residual block is coded with the inferred adaptive transform, and other parts of the residual block are zeroed out. SBT is not applied to combined intertra modes.

[0274] Figure 10 shows examples of subblock transformation modes SBT-V and SBT-H. In subblock transformation, position-dependent transformations are applied to the luminance transformation blocks of SBT-V and SBT-H (chroma TB, always using DCT-2). The two positions of SBT-H and SBT-V are associated with different core transformations. More specifically, the horizontal and vertical transformations for each SBT position are specified in Figure 10. For example, the horizontal and vertical transformations for SBT-V position 0 are DCT-8 and DST-7, respectively. If one side of the residual TU is greater than 32, the corresponding transformation is set to DCT-2. Thus, the subblock transformation specifies the TU tiling, cbf, and horizontal and vertical transformations jointly for the residual block, which can be considered a syntax shortcut when the block's principal residual is on one side of the block.

[0275] 9 Example of Joint Chroma Residual Coding in JVET-N0054

[0276] Since the Cb and Cr residuals appear to be inversely proportional to each other, it is proposed to utilize this phenomenon and introduce a joint coding mode for the Crodo residuals. In this mode, there is a single residual for the two Crodo blocks of the transform device. The residual is added to the prediction block of the first channel (typically representing Cb) and subtracted from the prediction block of the second channel (typically representing Cr).

[0277] The joint residual mode is flagged in the bitstream when the coded block flag (cbf) for both Cb and Cr is true. When the mode is enabled, a single residual block is decoded. The bitstream syntax and decoding process for the joint residual block follow the Cb residuals of the VTM-4. The residuals for the Cr block are generated by invalidating the decoded joint residuals. Since a single residual is used to represent the residuals of two blocks, it may be desirable to use a lower QP for this mode than is used for separate coding of chroma residuals. In CE testing, a chroma QP offset of -1 was used for the joint mode and +1 was used for separate chroma coding (as specified in the JVET Common Test Conditions).

[0278] 10 Example of JVET-N0236

[0279] This contribution proposes a method for refining subblock-based affine motion compensation prediction using optical flow. After subblock-based affine motion compensation is performed, prediction samples are refined using optical flow (PROF) by adding differences derived from the optical flow equation, referred to as prediction refinement. The proposed method can achieve inter-prediction with pixel-level granularity without increasing memory access bandwidth.

[0280] To achieve granularity in motion compensation, this contribution proposes a method for refining subblock-based affine motion that compensates predictions with optical flow. After subblock-based affine motion compensation is performed, luminance prediction samples are refined by adding the difference derived by the optical flow equation. The proposed PROF (prediction refinement with optical flow) is described in the following four steps.

[0281] Step 1) Subblock-based affine motion reward is subblock prediction I(i,j) It is performed to generate.

[0282] Step 2) Spatial Gradient of Subblock Prediction g x (i,j) and g y (i,j) It is calculated at each sample location using a 3-tab filter [-1, 0, 1].

[0283] g x (i,j)=I(i+1,j)-I(i-1,j)

[0284] g y (i,j)=I(i,j+1)-I(i,j-1)

[0285] Fig. 11 shows the subblock MV VSB and pixel This shows an example where the subblock prediction is extended by 1 pixel on each side for gradient calculation. To reduce memory bandwidth and complexity, the pixels of the extended border are copied from the nearest integer pixel location in the reference picture. Thus, additional interpolation for the padding region can be avoided.

[0286] Step 3) Luma Prediction Refinement ( (indicated by) is calculated by the optical flow equation.

[0287]

[0288] Here, delta MV( (indicated by) is the difference between the pixel MV calculated for the sample location (i,j) denoted by v(i,j) as shown in FIG. 11 and the MV of the subblock to which the pixel (i,j) belongs.

[0289] Since the affine model parameters and pixel positions relative to the subblock center do not change from subblock to subblock, can be calculated for the first subblock and can be reused for other subblocks of the same CU. If x and y are the horizontal and vertical offsets from a pixel position to the center of the subblock, It can be derived by the following equation,

[0290]

[0291] In the case of a 4-parameter affine model

[0292]

[0293] For a 6-parameter affine model

[0294]

[0295] Here, (v 0x ,v 0y ), (v 1x ,v 1y ), (v 2x ,v 2y ) are the top-left, top-right, and bottom-left control point motion vectors, and w and h are the width and height of the CU.

[0296] Step 4) Finally, the luma prediction refinement is added to the sub-block prediction I(i,j). The final prediction I' is generated by the following equation:

[0297]

[0298] 11 Disadvantages of the existing implementation

[0299] The following problems occur in some existing implementations (e.g., current VVC / VTM deblocking designs).

[0300] (1) BDOF refines each prediction sample. Therefore, it may not be necessary to filter it again during the deblocking process.

[0301] (2) CIIP-coded blocks are processed in an inter-prediction mode in the deblocking filter process while still performing an intra-prediction process.

[0302] 12 Best practices for improving deblocking filtering (DB)

[0303] Embodiments of the disclosed technology overcome the shortcomings of existing implementations and thus provide video coding with higher coding efficiency. Improvements to deblocking filtering (DB) based on the disclosed technology can enhance both existing and future video coding standards, as described in the following examples regarding various implementations. The examples of the disclosed technology provided below describe general concepts and are not to be interpreted as limitations. For instance, various features described in these examples may be combined unless explicitly indicated otherwise.

[0304] In some embodiments of the disclosed technology, it is proposed to modify the deblocking filter (DB) process in various ways. However, it should be noted that the method presented in this document may be applied to other types of in-loop filtering methods (e.g., SAO, ALF) and post-reconstruction filters (e.g., bilayer filters, diffusion filters, Hadamard transform domain filters applied to the prediction or reconstruction signal prior to the deblocking filter).

[0305] It should be noted that the chroma component can represent Cb / Cr or B / R color components in RGB format. The following explanation uses 'Cb / Cr' as an example.

[0306] yes1. The method of applying filtering in the DB and / or DB process may vary depending on the use of other coding tools that use BDOF and / or optical flow to refine motion / predicted samples / reconstructed samples.

[0307] (a) In one example, for a sample within a BDOF-coded block, there is no need to apply DB.

[0308] (b) Alternatively, boundary strength / filter strength may vary depending on the use of BDOF.

[0309] (i) A weak filter may be applied to samples located within a BDOF coding block.

[0310] yes 2. The method of applying filtering in the DB and / or DB process may vary depending on the type of transformation matrix (e.g., DCT-II, DST-VII, DCT-VIII, transformation skip mode (including identification matrix), coding method that does not apply transformation such as RDPCM / QR-BDPCM mode).

[0311] (a) In one example, position-dependent clipping may be disabled so that samples can be filtered into a block coded with a specific transformation type (e.g., transformation code block).

[0312] (i) Also, all samples to be filtered use the same clipping parameter.

[0313] (b) In one example, the position-dependent clipping parameter may vary based on the transformation matrix applied to the block.

[0314] (c) In one example, DB may not be executed in a conversion skip mode or a block of coding method that does not apply conversion, such as RDPCM mode.

[0315] yes 3. Clipping operations and / or position-dependent clipping may be disabled based on the following:

[0316] (a) Video content (e.g., screen content or natural content)

[0317] (b) Messages signaled in SPS / VPS / PPS / Picture Header / Slice Header / Tile Group Header / Maximum Coding Unit (LCU) / LCU Row / LCU Group

[0318] yes 4. Whether to enable DB or apply filtering in the DB process may vary depending on the use of Combined Intra-Inter Prediction (CIIP).

[0319] (a) In one example, a CIIP-coded block can be treated similarly to an intra-coded block.

[0320] (b) In one example, if one or more of the two blocks crossing the edge are coded in CIIP mode, the boundary strength is set to 2.

[0321] (c) or if only one of the two blocks crossing the edge is coded in CIIP mode and the other block is intercoded, the boundary strength is set to 1.

[0322] (d) or if only one of the two blocks crossing the edge is coded in CIIP mode and the other block is inter or IBC coded, the boundary strength is set to 1.

[0323] yes 5. The boundary strength (BS) of an IBC-coded block can be independent of the difference in motion vectors.

[0324] (a) In one example, when adjacent blocks are all IBC coded, the boundary strength is set to M (e.g., M=0 / 1 / 2) regardless of the MV difference and regardless of whether there is a non-zero coefficient.

[0325] (b) Alternatively, BS depends on whether the horizontal and / or vertical MV difference is greater than M0 / M1-pels (M>1).

[0326] (i) In one example, M0 and M1 are both set to 4.

[0327] (ii) In one example, M0 is set as the minimum CU width for the MV difference of the horizontal component, and M1 is set as the minimum CU height for the MV difference of the horizontal component.

[0328] (iii) In one example, M0 is set to the minimum IBC coding CU width allowed for the MV difference of the horizontal component, and M1 is set to the minimum IBC coding CU height allowed for the MV difference of the horizontal component.

[0329] (c) Alternatively, DB may not be applied to the boundary between two IBC-coded blocks.

[0330] (d) Alternatively, DB may not be applied to samples of IBC coding blocks.

[0331] yes 6. The boundary strength (BS) values ​​on the P side or Q side may differ.

[0332] (a) In one example, the block on the P side or the Q side may have its own boundary strength. Deblocking performed on the P side and the Q side may be based on the BS values ​​of the P side and the Q side, respectively.

[0333] (i) In one example, if the block on the P side is an intra block and the block on the Q side is an RDPCM code block, the deblocking condition can be checked on the P side and the deblocking condition can be performed only on the P side.

[0334] (ii) In one example, the BS for the P side and the Q side may depend on the QP of both sides.

[0335] (iii) In one example, the BS for the P side and the Q side may depend on the LMCS coefficients used on both sides.

[0336] yes7. RDPCM coding blocks can be processed as intra-blocks for deblocking.

[0337] (a) In one example, if the RDPCM coding block is on the P side and / or Q side, the boundary strength (BS) value may be set to 1 or 2.

[0338] yes 8. The diagonal edges of a triangle mode coding block can be deblocked.

[0339] (a) In one example, the deblocking filter and / or deblocking control logic of the current VVC can also be applied to the diagonal edges of the triangle mode coding block.

[0340] yes 9.

[0341] (a) In one example, if a palette coding block is on the Q side and a block not coded in palette mode is on the P side, deblocking can be applied to the P side.

[0342] (b) In one example, if a palette coding block is on the P side and a block not coded in palette mode is on the Q side, deblocking can be applied to the Q side.

[0343] (c) In one example, if both the P-side and the Q-side are palette-coded blocks, deblocking may not be applied to both sides.

[0344] Example 10 Sub-regions within a single block coded with DMVR or other decoder side motion enhancement coding tools can be treated as sub-block boundaries and filtered.

[0345] (a) In one example, a sub-region is defined as a sub-unit to which motion vector materialization is applied.

[0346] (i) In one example, the sub-region is defined as an MxN sub-block within the DMVR coding block, where M and N can be set to 64 or 16.

[0347] Example 11 The method of applying filtering in the DB and / or DB process may vary depending on whether a second transformation (also known as NSST, RST, or LFNST) is applied.

[0348] (a) In one example, the boundary of the sub-region to which RST is applied can be treated as a transformation block boundary or a coding sub-block boundary.

[0349] (i) Alternatively, samples located in the RST application sub-region can also be filtered.

[0350] (ii) Alternatively, position-dependent clipping can be disabled for the RST-applied sub-region.

[0351] Example 12 Whether deblocking is performed on subblock boundaries (e.g., internal subblock boundaries) within an affine mode block depends on whether interweave prediction and / or phase-shift affine subblock motion compensation is applied to the block.

[0352] In one example, deblocking can be disabled when inserting prediction and / or PROF and / or phase variant affine subblock motion compensation applied to the block.

[0353] In one example, deblocking may be enabled when the correlated prediction or / and PROF and / and phase variant affine subblock motion compensation is not applied to the block.

[0354] FIG. 12 illustrates a flowchart of an example method (1200) for video processing based on some embodiments of the disclosed technology. The video processing method (1200) comprises, in step (1210), a step of determining a coding scheme applicable to a conversion between a video block and a bitstream representation of the video block; in step (1220), a step of determining a mode of an in-loop filtering process including at least one of a deblocking filter (DB) process, a sample adaptive offset (SAO), and an adaptive loop filtering (ALF), based on the determination of the coding scheme; and in step (1230), a step of performing a conversion using the in-loop filtering mode. Additional features related to the video processing method (1200) can be found in the examples 1, 2, 3, and 11 described above.

[0355] FIG. 13 illustrates a flowchart of another example method (1300) for video processing based on some embodiments of the disclosed technology. The video processing method (1300) comprises, in step (1310), determining that combined intra-inter prediction (CIIP) coding is applicable to the current block; and, in step (1320), performing a conversion between the current block and the bitstream representation of the current block; and, based on the use of CIIP, Identify the boundaries between the block and the vertical and horizontal edges; calculate the boundary strength; decide whether to turn the filter on or off; and select the filter strength. It includes a step of performing a deblocking filter (DB) process. Additional features related to the method (1300) for video processing can be found in Example 4 described above.

[0356] FIG. 14 illustrates a flowchart of another example method (1400) for video processing based on some embodiments of the disclosed technology. The video processing method (1400) includes, in step (1410), a step of determining whether adjacent blocks are all coded intra-block copies (IBC); and in step (1420), a step of, based on the determination, identifying the boundaries between the blocks and vertical and horizontal edges, calculating the boundary intensity, deciding whether to turn a filter on or off, and performing a deblocking filter (DB) process by selecting the intensity of the filter. The boundary intensity is independent of the motion vector difference. Additional features related to the method (1400) for video processing can be found in Example 5 described above.

[0357] FIG. 15 illustrates a flowchart of another example method (1500) for video processing based on some embodiments of the disclosed technology. The video processing method (1500) includes, in step (1510), a step of determining boundary strengths on the P side and the Q side; and, in step (1520), a step of performing a deblocking filter (DB) process on the P side and the Q side by identifying boundaries between blocks and vertical and horizontal edges based on the determination, calculating boundary strengths, deciding whether to turn a filter on or off, and selecting the strength of the filter. Additional features related to the method (1500) for video processing can be found in Example 6 described above.

[0358] FIG. 16 illustrates a flowchart of another example method (1600) for video processing based on some embodiments of the disclosed technology. The video processing method (1600) includes, in step (1610), a step of determining a residual differential pulse-code modulation (RDPCM) coding block; and, in step (1620), a step of, based on the determination, identifying the boundary between the block and the vertical and horizontal edges, calculating the boundary strength, deciding whether to turn the filter on or off, and performing a deblocking filter (DB) process on the P side and Q side by selecting the strength of the filter. The RDPCM coding block is processed as an intra-block in the deblocking filter (DB) process. Additional features related to the method (1600) for video processing can be found in Example 7 described above.

[0359] FIG. 17 illustrates a flowchart of another example method (1700) for video processing based on some embodiments of the disclosed technology. The video processing method (1700) includes, in step (1710), a step of determining diagonal edges in a triangle mode coded block; and in step (1720), a step of, based on the determination, identifying boundaries between the block and vertical and horizontal edges, calculating boundary strengths, deciding whether to turn a filter on or off, and performing a deblocking filter (DB) process on the diagonal edges by selecting the strength of the filter. Additional features related to the method (1700) for video processing can be found in Example 8 described above.

[0360] FIG. 18 shows a flowchart of another example method (1800) for video processing based on some embodiments of the disclosed technology. The video processing method (1800) includes, in step (1810), a step of determining whether a palette coded block is on the P side or the Q side; and in step (1820), a step of, based on the determination, identifying the boundary between the block and the vertical and horizontal edges, calculating the boundary intensity, deciding whether to turn a filter on or off, and performing a deblocking filter (DB) process on the diagonal edge by selecting the intensity of the filter. The deblocking filter (DB) process is applied to one side when the palette coded block is on the P side or the Q side. Additional features related to the video processing method (1800) can be found in Example 9 described above.

[0361] FIG. 19 illustrates a flowchart of another example method (1900) for video processing based on some embodiments of the disclosed technology. The video processing method (1900) comprises, in step (1910), determining a sub-region within a block coded by decoder-side motion vector refinement (DMVR) or other decoder-side motion refinement coding tool; and, in step (1920), performing a deblocking filter (DB) process on the sub-region by, based on the determination, identifying the boundary between the block and vertical and horizontal edges, calculating the boundary intensity, deciding whether to turn a filter on or off, and selecting the intensity of the filter. Additional features related to the video processing method (1900) can be found in Example 10 described above.

[0362] FIG. 20 is a block diagram of a video processing apparatus (1400). The apparatus (2000) may be used to implement one or more methods described herein. The apparatus (2000) may be implemented as a smartphone, tablet, computer, Internet of Things (IoT) receiver, etc. The apparatus (2000) may include one or more processors (2002), one or more memories (2004), and video processing hardware (2006). The processor(s) (2002) may be configured to implement one or more methods described herein (including, but not limited to, methods (1200 to 1900)). The memory (memories) (2004) may be used to store data and code used to implement the methods and techniques described herein. The video processing circuitry (2006) may be used to implement some of the techniques described herein in hardware circuitry.

[0363] In some embodiments, the video coding method may be implemented using a device implemented on a hardware platform as described in relation to FIG. 20.

[0364] FIG. 21 is a block diagram illustrating an exemplary video processing system (2100) in which the various technologies disclosed herein may be implemented. Various implementations may include some or all of the components of the system (2100). The system (2100) may include an input (2102) for receiving video content. The video content may be received in a raw or uncompressed format (e.g., 8 or 10-bit multi-component pixel values) or in a compressed or encoded format. The input (2102) may represent a network interface, a peripheral bus interface, or a storage interface. Examples of network interfaces include wired interfaces such as Ethernet or Passive Optical Network (PON) and wireless interfaces such as Wi-Fi or a cellular interface.

[0365] The system (2100) may include a coding component (2104) capable of implementing various coding or encoding methods described in this document. The coding component (2104) may reduce the average bitrate of the video from the input (2102) to the output of the coding component (2104) to generate a coded representation of the video. Thus, the coding technique is also referred to as video compression or video transcoding technique. The output of the coding component (2104) may be stored or transmitted via connected communication, as represented by component (2106). The storage or communication bitstream (or coded) representation of the video received at the input (2102) may be used by component (2108) to generate pixel values ​​or a displayable video transmitted to a display interface (2110). The process of generating a video viewable by a user from the bitstream representation is also referred to as video decompression. In addition, you will understand that while specific video processing tasks are referred to as "coding" tasks or tools, coding tools or tasks are used in the encoder, and the corresponding decoding tools or tasks that return the coding results are performed by the decoder.

[0366] Examples of peripheral bus interfaces or display interfaces may include the Universal Serial Bus (USB), the High Definition Multimedia Interface (HDMI), or DisplayPort. Examples of storage interfaces include SATA (Serial Advanced Technology Attachment), PCI, and IDE interfaces. The technology described in this document may be implemented in various electronic devices, such as mobile phones, laptops, smartphones, or other devices capable of performing digital data processing and / or video display.

[0367] FIG. 22 shows a flowchart of an example method for video media processing. The steps of this flowchart are described in relation to Example 4 of Section 12 of this document. In step (2202), the process identifies boundaries at the vertical edges and / or horizontal edges of two video blocks. In step (2204), the process calculates the boundary intensity of a filter based on at least one of the two video blocks crossing the vertical edge, or the horizontal edge is coded using a combined intra-inter prediction (CIIP) mode. In step (2206), the process decides whether to turn the filter on or off. In step (2208), if the filter is turned on, the intensity of the filter is selected. In step (2210), the process performs a deblocking filter (DB) process for the video block based on the selected step.

[0368] FIG. 23 shows a flowchart of an example method for video media processing. The steps of this flowchart are described in relation to Example 1 of Section 12 of this document. In step (2302), a coding scheme applicable to the conversion between a video block of video media data and a bitstream representation of the video block is determined. In step (2304), based on the determination of the coding scheme, the process optionally enables or disables the application of an in-loop filter to the video block while converting the video block.

[0369] FIG. 24 shows a flowchart of an example method for video media processing. The steps of this flowchart are described in relation to Example 2 of Section 12 of this document. In step (2402), the process determines a type of transformation matrix applicable to the transformation between a video block of video media data and a bitstream representation of the video block. In step (2404), based on the determination of the type of transformation matrix, the process optionally enables or disables the application of a deblocking (DB) filter process to the video block during the transformation of the video block.

[0370] FIG. 25 shows a flowchart of an example method for video media processing. The steps of this flowchart are described in relation to Example 3 of Section 12 of this document. In step (2502), during the conversion between a video block of video media data and a bitstream representation of the video block, the process determines whether to apply an in-loop filtering step to the video block, wherein the in-loop filtering step includes a clipping operation applied to samples of the video block to be filtered. In step (2504), in response to determining one or more conditions associated with the video block, the process optionally enables or disables the clipping operation.

[0371] Some embodiments disclosed in this specification are now presented in a section-based format.

[0372] X1. In a method for processing video media,

[0373] A step of identifying boundaries at the vertical edges and / or horizontal edges of two video blocks;

[0374] A step of calculating the boundary strength of a filter based on one or more of two video blocks coded using a combined intra-inter prediction (CIIP) mode that crosses a vertical edge or a horizontal edge;

[0375] A step of deciding whether to turn the filter on or off;

[0376] A step of selecting the intensity of the filter when the filter is turned on; and

[0377] Based on the selection step, it includes a step of performing a deblocking filter (DB) process on the video block.

[0378] X2. In the method of X1, the CIIP-coded block is treated as an internally coded block to calculate the boundary strength.

[0379] X3. In the method of X1, the boundary strength is selected as 2 if one or more of the two video blocks crossing the vertical or horizontal edge are coded as CIIP.

[0380] X4. In the method of X1, the boundary strength is selected as 1 if only one of the two blocks crossing the vertical or horizontal edge is coded as CIIP and the other video block is intercoded.

[0381] X5. In the method of X1, the boundary strength is selected as 1 if only one of the two block blocks crossing the edge is coded as CIIP and the other video block is an coded inter or intra block copy (IBC).

[0382] A1. In a method for processing video media,

[0383] A step of determining a coding scheme applicable to the conversion between a video block of video media data and a bitstream representation of the video block; and

[0384] Based on the determination of the coding method, the method includes a step of selectively enabling or disabling the application of an in-loop filter to the video block during the conversion of the video block.

[0385] A2. In the method of A1, the in-loop filter includes one of: a deblocking filter (DB), a sample adaptive offset (SAO) filter, and / or an adaptive loop filter (ALF).

[0386] A3. In any one of the methods of A1 to A2, the coding method includes the use of an optical flow to refine a prediction sample and / or motion information and / or a reconstructed sample.

[0387] A4. In the method of A3, the coding method using optical flow to refine prediction samples includes bi-directional optical flow (BDOF).

[0388] A5. In the method of A4, the determination of the coding scheme includes the step of identifying whether the predicted sample is within the bidirectional optical flow (BDOF), and the application of an in-loop filter is disabled in response to the determination that the predicted sample is located within the bidirectional optical flow (BDOF).

[0389] A6. In the method of A5, at least one of the boundary strength and / or filter strength of the in-loop filter is determined based on the use of bidirectional optical flow (BDOF).

[0390] A7. In the method of A4, the determination of the coding scheme includes the step of identifying whether the predicted sample is within the bidirectional optical flow (BDOF), the application of an in-loop filter is activated in response to the determination that the predicted sample is located within the bidirectional optical flow (BDOF), and additionally, the in-loop filter has a weaker strength than another in-loop filter associated with a predicted sample that is not located within the BDOF.

[0391] B1. In a method for processing video media,

[0392] A step of determining the type of transform matrix applicable to the transformation between a video block of video media data and a bitstream representation of the video block; and

[0393] Based on the determination of the type of transformation matrix, it includes a step of selectively enabling or disabling the application of a deblocking (DB) filter process to the video block during transformation.

[0394] B2. In the method of B1, the types of transformation matrices include DCT-II, DST-VII, DCT-VIII, and / or transformation skip modes.

[0395] B3. In any one or more of the methods in B1 to B2, the application of the deblocking filtering process further includes a clipping operation.

[0396] B4. In the method of B3, the clipping operation is position-dependent based on the sample position within the video block in relation to the DB filter.

[0397] B5. In the method of B4, clipping operations are disabled for samples located in the converted skip-coding video block.

[0398] B6. In the method of B5, the deblocking filter (DB) process is disabled on the transform skip coding block, or is disabled based on the decision that video block coding is excluded from the transform-based coding technique.

[0399] B7. In the method of B6, the transform-based coding technique includes residual differential pulse-code modulation (RDPCM) or quantified residual differential pulse-code modulation (QR-BDPCM).

[0400] B8. In the method of B3, when the DB filter process is activated on the sample set, all samples included in the set receive the same parameters of the clipping operation.

[0401] B9. In the method of B4, the parameters of the position-dependent clipping operation are related to the basis functions of the transformation matrix applicable to the transformation.

[0402] C1. In a method for processing video media,

[0403] A step for determining the application of an in-loop filtering step to a video block during the conversion between a video block of video media data and a bitstream representation of the video block—the in-loop filtering step includes a clipping operation applied to samples within the video block to be filtered—; and

[0404] In response to a step of determining one or more conditions associated with a video block, the step includes selectively enabling or disabling a clipping operation.

[0405] C2. In the method of C1, one or more conditions associated with the video block are:

[0406] (a) the content of the video block, or

[0407] (b) Includes fields included in Sequence Parameter Set (SPS) / Video Parameter Set (VPS) / Picture Parameter Set (PPS) / Picture Header / Slice Header / Tile Group Header / Maximum Coding Unit (LCU) / LCU Row / LCU Row / LCU Group

[0408] C3. In any one or more of the methods of X1 to C2, the conversion includes the step of generating a bitstream representation from a video block.

[0409] C4. In any one or more of the methods of X1 to C2, the conversion includes the step of generating pixel values ​​of a video block in a bitstream representation.

[0410] C5. A video encoder device comprising a processor configured to implement the method cited in any one or more of X1 to C2.

[0411] C6. A video decoder device comprising a processor configured to implement the method cited in any one or more of X1 to C2.

[0412] C7. A computer-readable medium storing code, wherein the code implements processor-executable instructions for implementing the method cited in any one or more of X1 to C2.

[0413] In this document, the terms “video processing” or “video media processing” may refer to video encoding, video decoding, video compression, or video decompression. For example, a video compression algorithm may be applied during the conversion from a pixel representation of the video to a corresponding bitstream representation, or vice versa. The bitstream representation of the current video block may correspond to bits that are placed together or spread out at different locations within the bitstream, for example, as defined by the syntax. For example, a block may be encoded using bits in the header and other fields of the bitstream, along with transformed and coded error residual values. In this specification, a block may be a grouping of pixels according to operations performed, such as a coding unit, a prediction unit, or a transformation unit. Additionally, during the conversion, the decoder may parse the bitstream with the knowledge that some fields may be present or absent based on a decision as described in the solution above. Similarly, the encoder may determine whether a specific syntax field is included and, accordingly, generate a coded representation by including or excluding the syntax field from the coded representation.

[0414] From the foregoing, it will be understood that while specific embodiments of the technology currently disclosed are described herein for illustrative purposes, various modifications may be made without departing from the scope of the invention. Accordingly, the technology currently disclosed is not limited except as provided by additional claims.

[0415] The implementation and functional operation of the subject matter described in this patent document may be implemented in various systems, digital electronic circuits, or computer software, firmware, or hardware by this specification and structural equivalents or a combination of one or more of these. The implementation of the subject matter described in this specification may be implemented as one or more computer program products, namely, one or more modules of computer program instructions encoded on physical and non-transient computer-readable media to control the operation of a data processing device. The computer-readable media may be a machine-readable storage device, a machine-readable storage substrate, a memory device, a composition of a material that affects machine-readable radio signals, or one or more combinations thereof. The term “data processing device” or “data processing device” includes all devices, devices, and machines for processing data, including, for example, a programmable processor, a computer, or a multiprocessor or computer. This device may include code that creates an execution environment for a corresponding computer program (e.g., processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more thereof).

[0416] A computer program (also referred to as a program, software, software application, script, or code) may be written in any form of programming language, including compiled languages, and may be distributed in any form as a standalone program or as modules, components, subroutines, or other units suitable for use in a computing environment. A computer program is not necessarily a file in a file system. A program may be stored in other programs or data (e.g., one or more scripts stored in a markup language document), a single file dedicated to a corresponding program, or multiple coordinated files (e.g., files storing one or more modules, subprograms, or parts of code). A computer program may be distributed to run on a single computer or site, or on multiple computers distributed across multiple sites and interconnected by a communication network.

[0417] The processes and logic flows described in this specification may be performed by one or more programmable processors to operate on input data and generate outputs to perform functions. The processes and logic flows may also be performed, and the device may be implemented as a special-purpose logic circuit, for example, a Field Programmable Gate Array (FPGA) or an Application-Specific Integrated Circuit (ASIC).

[0418] Processors suitable for the execution of computer programs include, for example, general-purpose and special-purpose microprocessors and one or more processors of all types of digital computers. Generally, processors receive instructions and data from read-only memory or random access memory, or both. An essential element of a computer is a processor for executing one or more memory devices for storing instructions and data. Generally, a computer is operatively coupled or combined to receive or transmit data from one or more mass storage devices for storing data (e.g., magnetic, magneto-optical discs, or optical discs). However, computers do not require these devices. Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, including semiconductor memory devices (e.g., EPROM, EEPROM, and flash memory devices). Processors and memory may be supplemented or integrated by special-purpose logic circuits.

[0419] This specification is to be regarded as an exemplary example together with the drawings, and an exemplary example implies an example. As used herein, the singular forms "one (a)," "an," and "the" are intended to include the plural forms unless the context clearly indicates otherwise. Additionally, unless the context clearly indicates otherwise, the use of "or" is intended to include "and / or."

[0420] Although this patent document contains various specific details, they should not be interpreted as limiting the scope of the invention or what may be claimed, rather than as describing features that may be specific to specific embodiments of a particular invention. Specific features described in this patent document may be implemented in combination in a single embodiment in the context of separate embodiments. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments. Furthermore, the features may be described as operating in a specific combination, and even as initially claimed, one or more features from the claimed combination may, in some cases, be excised from the combination and may lead to a sub-combination or variation of the claimed combination.

[0421] Likewise, while operations are depicted in the drawings in a specific order, it should not be understood that such operations must be performed in the specific order or sequential order indicated, or that all illustrated operations must be performed to achieve a desired result. Furthermore, the separation of various system components in the embodiments described in this patent document should not be understood as requiring such separation in all embodiments.

[0422] Only a few implementations and examples are described, and other implementations, improvements, and variations may be made based on what is described and explained in this patent document.

Claims

Claim 1 For a deblocking filtering process for said two video blocks of said video during conversion between said two video blocks and a bitstream of said video, the method comprises: identifying boundaries at the vertical edges and / or horizontal edges of said two video blocks; determining the boundary intensity of said boundaries; determining whether to turn on or off a filter for said boundaries based on said boundary intensity; and selecting the intensity of said filter if said filter is turned on. A method comprising the step of performing a transformation based on the selected step, wherein, when at least one of two video blocks intersecting the vertical edge or the horizontal edge is coded using a combined inter-intra prediction mode, the boundary strength is equal to 2, and in the combined inter-intra prediction mode, the prediction signal of at least one of the two video blocks is generated based on an intra prediction signal and an inter prediction signal, and when both of the two video blocks are coded using an intra block copy (IBC) prediction mode, the boundary strength is determined based on whether the absolute difference between the horizontal or vertical components of the block vector used for prediction of the two video blocks is greater than or equal to a threshold, wherein the threshold is a non-integer M0 / M1, M0 is set as the minimum intra block copy coded unit width allowed for the difference in motion vectors of the horizontal components, and M1 is set as the minimum intra block copy coded unit height allowed for the difference in motion vectors of the horizontal components. Claim 2 A method according to claim 1, wherein the boundary is at least one of a transformation subblock boundary, a coding unit boundary, or a prediction subblock boundary. Claim 3 In paragraph 2, the method comprises the prediction subblock boundary including the prediction unit boundary introduced by the affine motion compensation mode. Claim 4 A method according to any one of claims 1 to 3, wherein the deblocking filtering process is applied to an 8x8 grid for the predicted subblock boundary. Claim 5 A method according to any one of claims 1 to 3, wherein the boundary strength is determined based on whether different reference pictures or different numbers of motion vectors are used in the prediction of the two video blocks. Claim 6 A method according to any one of claims 1 to 3, wherein a stronger deblocking filter is used in the deblocking filtering process when a sample on one side of the boundary belongs to a large block, wherein the sample belonging to the large block is defined as having a width of 32 or more for a vertical edge and a height of 32 or more for a horizontal edge. Claim 7 In paragraph 6, the method wherein the stronger deblocking filter is a bilinear filter. Claim 8 A method according to any one of claims 1 to 3, wherein the conversion comprises encoding the two video blocks into the bitstream. Claim 9 A method according to any one of claims 1 to 3, wherein the conversion comprises decoding the two video blocks from the bitstream. Claim 10 In an apparatus for processing video data comprising a processor and a non-transient memory containing instructions, wherein, when executed by the processor, the instructions cause the processor to: identify boundaries at the vertical edges and / or horizontal edges of the two video blocks for a deblocking filtering process for the two video blocks during a conversion between the two video blocks and the bitstream of the video; determine boundary strengths of the boundaries; determine whether to turn on or off a filter for the boundaries based on the boundary strengths; and if the filter is turned on, select the strength of the filter; A device that performs the transformation based on the selected step, wherein if at least one of the two video blocks intersecting the vertical edge or the horizontal edge is coded using a combined inter-intra prediction mode, the boundary strength is equal to 2, and in the combined inter-intra prediction mode, the prediction signal of at least one of the two video blocks is generated based on at least an intra prediction signal and an inter prediction signal, and if both of the two video blocks are coded using an intra block copy (IBC) prediction mode, the boundary strength is determined based on whether the absolute difference between the horizontal or vertical components of the block vector used for prediction of the two video blocks is greater than or equal to a threshold, wherein the threshold is a non-integer M0 / M1, M0 is set as the minimum intra block copy coded unit width allowed for the difference in motion vectors of the horizontal components, and M1 is set as the minimum intra block copy coded unit height allowed for the difference in motion vectors of the horizontal components. Claim 11 delete Claim 12 In a non-transient computer-readable storage medium storing instructions for causing a processor to execute, for a deblocking filtering process for two video blocks of a video during a conversion between two video blocks and a bitstream of the video, the method comprises: identifying boundaries at the vertical edges and / or horizontal edges of the two video blocks; determining boundary strengths of the boundaries; determining whether to turn on or off a filter for the boundaries based on the boundary strengths; and selecting the strength of the filter if the filter is turned on. A non-transient computer-readable storage medium, wherein the transformation is performed based on the selected step, and when at least one of the two video blocks intersecting the vertical edge or the horizontal edge is coded using a combined inter-intra prediction mode, the boundary strength is equal to 2, and in the combined inter-intra prediction mode, the prediction signal of at least one of the two video blocks is generated based on at least an intra prediction signal and an inter prediction signal, and when both of the two video blocks are coded using an intra block copy (IBC) prediction mode, the boundary strength is determined based on whether the absolute difference between the horizontal or vertical components of the block vector used for prediction of the two video blocks is greater than or equal to a threshold, and the threshold is a non-integer M0 / M1, M0 is set as the minimum intra block copy coded unit width allowed for the difference in motion vectors of the horizontal components, and M1 is set as the minimum intra block copy coded unit height allowed for the difference in motion vectors of the horizontal components. Claim 13 A computer-readable recording medium storing a bitstream of video generated by a method performed in a video processing device, wherein the method comprises: identifying a boundary at a vertical edge and / or horizontal edge of two video blocks for a deblocking filtering process for two video blocks of video; determining a boundary intensity of said boundary; determining whether to turn on or off a filter for said boundary based on said boundary intensity; and selecting an intensity of said filter when said filter is turned on. A computer-readable recording medium comprising the step of generating the bitstream based on a selection step, wherein if at least one of the two video blocks intersecting the vertical edge or the horizontal edge is coded using a combined inter-intra prediction mode, the boundary strength is equal to 2, and in the combined inter-intra prediction mode, the prediction signal of at least one of the two video blocks is generated based on at least an intra prediction signal and an inter prediction signal, and if both of the two video blocks are coded using an intra block copy (IBC) prediction mode, the boundary strength is determined based on whether the absolute difference between the horizontal or vertical components of the block vector used for prediction of the two video blocks is greater than or equal to a threshold, wherein the threshold is a non-integer M0 / M1, M0 is set as the minimum intra block copy coded unit width allowed for the difference in motion vectors of the horizontal components, and M1 is set as the minimum intra block copy coded unit height allowed for the difference in motion vectors of the horizontal components. Claim 14 delete Claim 15 delete Claim 16 delete Claim 17 delete Claim 18 delete Claim 19 delete Claim 20 delete Claim 21 delete Claim 22 delete Claim 23 delete Claim 24 delete Claim 25 delete Claim 26 delete Claim 27 delete Claim 28 delete

Images