Partition information in neural network-based video coding and decoding

Abstract

A method implemented by a video encoding and decoding apparatus. The method includes applying a neural network (NN) filter to unfiltered samples of a video unit to generate filtered samples. The NN filter includes an NN filter model generated based on segmentation information of the video unit. The use of the NN filter is indicated by one or more syntax elements in the bitstream. A conversion is performed between the video media file and the bitstream based on the filtered sample points.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This patent application claims the benefit of U.S. Provisional Patent Application No. 63 / 353,513, filed on June 17, 2022. All of the foregoing patent applications are incorporated herein by reference in their entirety. Technical Field

[0003] The present disclosure relates generally to image and video encoding and decoding. Background Art

[0004] Digital video accounts for the largest use of bandwidth on the Internet and other digital communications networks. As the number of connected user devices capable of receiving and displaying video increases, bandwidth demand for digital video usage is expected to continue to grow. Summary of the invention

[0005] A first aspect relates to a method implemented by a codec. The method includes applying a neural network (NN) filter to unfiltered samples of a video unit to generate filtered samples, wherein the NN filter includes an NN filter model generated based on segmentation information of the video unit. The use of the NN filter is indicated by one or more syntax elements in a bitstream. The method also includes performing conversion between a video media file and a bitstream based on the filtered samples.

[0006] A second aspect relates to a non-transitory computer-readable medium comprising a computer program product for use with a video codec device, the computer program product comprising computer-executable instructions, wherein the computer-executable instructions are stored on the non-transitory computer-readable medium so that when executed by a processor, the video codec device performs the method of any of the preceding aspects.

[0007] A third aspect relates to a non-transitory computer-readable recording medium storing a bitstream of a video generated by a method performed by a video processing device. The method includes: determining to apply a neural network (NN) filter to unfiltered samples of a video unit to generate filtered samples, wherein the NN filter includes an NN filter model generated based on segmentation information of the video unit, and wherein use of the NN filter is indicated by one or more syntax elements in the bitstream; and generating a bitstream based on the determination.

[0008] A fourth aspect relates to a method for storing a bitstream of a video, comprising: determining to apply a neural network (NN) filter to unfiltered samples of a video unit to generate filtered samples, wherein the NN filter includes an NN filter model generated based on segmentation information of the video unit, and wherein use of the NN filter is indicated by one or more syntax elements in the bitstream; generating a bitstream based on the determination; and storing the bitstream in a non-temporary computer-readable recording medium.

[0009] For purposes of clarity, any of the aforementioned embodiments may be combined with any one or more of the other aforementioned embodiments to create new embodiments within the scope of the present disclosure.

[0010]

[0011] These and other features will become more clearly understood from the following detailed description and claims taken in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] For a more complete understanding of the present disclosure, reference is now made to the following brief description, taken in conjunction with the accompanying drawings and detailed description, wherein like reference numerals represent like parts.

[0012] Figure 1 is an example of raster scan striping of an image.

[0013] Figure 2 is an example of rectangular strip segmentation of an image.

[0014] Figure 3 are examples of pictures segmented into slices, bricks, and rectangular strips.

[0015] Figure 4A is an example of a codec tree block (CTB) that crosses the bottom picture boundary.

[0016] Figure 4B is an example of a CTB that crosses the right picture boundary.

[0017] Figure 4C is an example of a CTB that crosses the lower right picture boundary.

[0018] Figure 5 is an example of an encoder block diagram.

[0019] Figure 6 is a diagram of the samples within an 8×8 sample block.

[0020] Figure 7 is an example of a pixel involved in the filter on / off decision and strong / weak filter selection.

[0021] Figure 8 Four one-dimensional (1-D) directional patterns for EO spot classification are shown.

[0022] Fig. 9 Examples of geometric transform based adaptive loop filter (GALF) filter shapes are shown.

[0023] Fig.10 An example of relative coordinates for a 5x5 diamond filter support is shown.

[0024] Fig.11 Another example of relative coordinates for a 5x5 diamond filter support is shown.

[0025] Fig. 12A is an example architecture of the proposed CNN filter.

[0026] Fig. 12B This is an example of the construction of a residual block (ResBlock).

[0027] Fig.13 is an example of a process for generating filtered samples based on a neural network filter model that receives codec parameters as input, according to various examples.

[0028] Fig.14 is an example of applying attention obtained using external information such as codec parameters to a feature map of a neural network filter model to provide a recalibrated feature map according to various examples.

[0029] Fig.15 is an example of a process for applying a neural network filter with configurable depth to unfiltered samples of a video unit to generate filtered samples.

[0030] Fig.16A is a schematic block diagram of the architecture of a neural network filtering method according to various examples, and Fig. 16B Based on various examples Fig.16A Schematic diagram of the attention residual block used in the architecture of .

[0031] Fig.17 is an example of a hierarchical codec structure for random access setting according to various examples.

[0032] Fig.18 A- Fig.18 C illustrates an example of the over-filtering phenomenon.

[0033] Fig.19 is an example of a hybrid video coding scheme including filters according to various examples.

[0034] Fig. 20 is an example of the proposed convolutional neural network for filtering the luminance component according to various examples.

[0035] Fig.21A is an example of normal or traditional training using a VTM encoder to compress training images according to various examples.

[0036] Fig. 21B is an example of iterative training based on various examples.

[0037] Fig.22A is an example of an encoder strategy for selecting a filter according to various examples.

[0038] Fig. 22B is an example of a decoder strategy for selecting a filter according to various examples.

[0039] Fig.23A is a schematic diagram of a conditional loop filter according to various examples.

[0040] Fig. 23B is an example of expanding the quantization parameter into a 1-dimensional vector according to various examples.

[0041] Fig.24A is a schematic block diagram of another architecture of a neural network filtering method according to various examples, and Fig. 24B Based on various examples Fig.24A Schematic diagram of the attention residual block used in the architecture of .

[0042] Fig.25 is a block diagram illustrating an exemplary video processing system.

[0043] Fig.26 is a block diagram of a video processing device.

[0044] Fig. 27 is a block diagram illustrating an exemplary video coding system.

[0045] Fig.28 is a block diagram illustrating an example of a video encoder.

[0046] Fig.29 is a block diagram illustrating an example of a video decoder.

[0047] Fig.30 is a flow chart of an example method 3000 of video processing. DETAILED DESCRIPTION

[0048] First, it should be understood that although illustrative implementations of one or more embodiments are provided below, the disclosed systems and / or methods can be implemented using any number of techniques, whether currently known or existing. The present disclosure should in no way be limited to the illustrative implementations, drawings, and techniques shown below, including the exemplary designs and implementations shown and described herein, but can be modified within the scope of the appended claims and their full scope of equivalents.

[0049] Versatile Video Codec (VVC), also known as H.266, is used in some descriptions for ease of understanding only and is not intended to limit the scope of the disclosed technology. Therefore, the technology described herein is also applicable to other video codec protocols and designs.

[0050] This patent document relates to video coding. In particular, the present specification relates to loop filters in image / video coding. The disclosed examples can be applied individually or in various combinations to video bitstreams encoded and decoded using existing video coding standards such as the VVC standard, the High Efficiency Video Codec (HEVC) standard, or a standard to be completed (e.g., the third generation audio video codec standard (AVS3)). The disclosed examples can also be applied to future video coding standards, future video codecs, or as a post-processing method outside the encoding / decoding process.

[0051] Video codec standards have evolved primarily through the development of the well-known International Telecommunication Union-Telecommunication (ITU-T) and International Organization for Standardization (ISO) / International Electrotechnical Commission (IEC) standards. ITU-T produced H.261 and H.263, ISO / IEC produced the Moving Picture Experts Group MPEG-1 and MPEG-4 Video, and the two organizations jointly produced the H.262 / MPEG-2 Video and H.264 / MPEG-4 Advanced Video Codec (AVC) and H.265 / High Efficiency Video Codec (HEVC) standards.

[0052] Since H.262, the video codec standard is based on a hybrid video codec structure, where temporal prediction plus transform coding is used. In order to explore future video codec technologies beyond HEVC, the Joint Video Exploration Team (JVET) was established in 2015 by the Video Codec Experts Group (VCEG) and MPEG. Since then, JVET has adopted many new methods and put them into a reference software called Joint Exploration Model (JEM).

[0053] In April 2018, the Joint Video Experts Group (JVET) between VCEG (Q6 / 16) and ISO / IEC JTC1 SC29 / WG11 (MPEG) was created to specify the VVC standard with the goal of reducing the bit rate by fifty percent (50%) compared to HEVC. VVC version 1 was completed in July 2020.

[0054] Color spaces and chroma subsampling are discussed. A color space, also called a color model (or color system), is an abstract mathematical model that simply describes the range of colors as a tuple of numbers, usually 3 or 4 values ​​or color components (e.g., red, green, and blue (RGB)). Basically, a color space is a detailed interpretation of coordinate systems and subspaces.

[0055] For video compression, the most commonly used color spaces are YCbCr and RGB. YCbCr, Y′CbCr, or Y Pb / Cb Pr / Cr (also written as YC B C R or Y′C B C R ) is a family of color spaces used as part of the color imaging pipeline in video and digital photography systems. Y' is the luma component, CB and CR are the blue-difference and red-difference chroma components. Y' (with a superscript notation) is distinguished from Y, which is luminance, meaning that light intensity is encoded nonlinearly based on the gamma-corrected RGB primaries.

[0056] Chroma subsampling is the practice of encoding an image by achieving a smaller resolution for chroma information than for luminance information, taking advantage of the human visual system's lower sensitivity to color differences than to contrast.

[0057] For 4:4:4 chroma subsampling, each of the three Y'CbCr components has the same sampling rate, so there is no chroma subsampling. Such a scheme is sometimes used in high-end film scanners and film post-production.

[0058] For 4:2:2 chroma subsampling, the two chroma components are sampled at half the sampling rate of luma: the horizontal chroma resolution is halved. This reduces the bandwidth of the uncompressed video signal by one third with almost no visual difference.

[0059] For 4:2:0 chroma subsampling, the horizontal sampling is doubled compared to 4:1:1, but the vertical resolution is halved since the Cb and Cr channels are sampled only on every alternate line in this scheme. Therefore the data rate is the same. Cb and Cr are subsampled by a factor of two both horizontally and vertically. There are three variants of the 4:2:0 scheme with different horizontal and vertical sitings.

[0060] In MPEG-2, Cb and Cr are co-sited horizontally. Cb and Cr are located between pixels in the vertical direction (interstitially located). In Joint Photographic Experts Group (JPEG) / JPEG File Interchange Format (JFIF), H.261, and MPEG-1, Cb and Cr are interstitially located in the middle between alternate luma samples. In 4:2:0DV, Cb and Cr are co-sited horizontally. Vertically, they are co-sited on alternate lines.

[0061] A definition of video units is provided. A picture is partitioned into one or more tile rows and one or more tile columns. A slice is a sequence of codec tree units (CTUs) covering a rectangular area of ​​a picture. A slice is partitioned into one or more bricks, each brick consisting of multiple CTU rows within a slice. Slices that are not split into multiple bricks are also called bricks. However, bricks that are true subsets of a slice are not called slices. A slice either contains multiple slices of a picture or multiple bricks of a slice.

[0062] Two modes of strips are supported, namely raster scan strip mode and rectangular strip mode. In raster scan strip mode, a strip contains a sequence of slices in the raster scan of the image. In rectangular strip mode, a strip contains multiple bricks of the image, which together form a rectangular area of ​​the image. The bricks within a rectangular strip are in the order of the raster scan of the strip's bricks. The bricks within a rectangular strip are in the order of the raster scan of the strip's bricks.

[0063] Figure 1 1 is an example of a raster scan stripe partitioning of a picture 100, wherein the picture is partitioned into twelve slices 102 and three raster scan stripes 104. As shown, each of the slices 102 and stripes 104 includes a plurality of CTUs 106.

[0064] Figure 2 2 is an example of a rectangular slice partition of a picture 200 according to the VVC specification, where the picture is partitioned into twenty-four slices 202 (six slice columns 203 and four slice rows 205) and nine rectangular slices 204. As shown, each of the slices 202 and slices 204 includes a plurality of CTUs 206.

[0065] Figure 3 is an example of an image 300 partitioned into slices, bricks, and rectangular strips according to the VVC specification, wherein the image is partitioned into four slices 302 (two slice columns 303 and two slice rows 305), eleven bricks 304 (the upper left slice contains one brick, the upper right slice contains five bricks, the lower left slice contains two bricks, and the lower right slice contains three bricks), and four rectangular strips 306.

[0066] The CTU and Codec Tree Block (CTB) sizes are discussed. In VVC, the Codec Tree Unit (CTU) size signaled in the Sequence Parameter Set (SPS) through the syntax element log2_ctu_size_minus2 can be as small as 4x4. The sequence parameter set raw byte sequence payload (RBSP) syntax is as follows.

[0067]

[0068]

[0069]

[0070] log2_ctu_size_minus2 plus 2 specifies the luma codec tree block size for each CTU.

[0071] log2_min_luma_coding_block_size_minus2 plus 2 specifies the minimum luma coding block size.

[0072] The variables CtbLog2SizeY, CtbSizeY, MinCbLog2SizeY, MinCbSizeY, MinTbLog2SizeY, MaxTbLog2SizeY, MinTbSizeY, MaxTbSizeY, PicWidthInCtbsY, PicHeightInCtbsY, PicSizeInCtbsY, PicWidthInMinCbsY, PicHeightInMinCbsY, PicSizeInMinCbsY, PicSizeInSamplesY, PicWidthInSamplesC, and PicHeightInSamplesC are exported as follows.

[0073] CtbLog2SizeY=log2_ctu_size_minus2+2 (7-9)

[0074] CtbSizeY=1< <CtbLog2SizeY (7-10)

[0075] MinCbLog2SizeY=log2_min_luma_coding_block_size_minus2+2 (7-11)

[0076] MinCbSizeY=1< <MinCbLog2SizeY (7-12)

[0077] MinTbLog2SizeY=2 (7-13)

[0078] MaxTbLog2SizeY=6 (7-14)

[0079] MinTbSizeY=1<<MinTbLog2SizeY(7-15)

[0080] MaxTbSizeY=1<<MaxTbLog2SizeY (7-16)

[0081] PicWidthInCtbsY=Ceil(pic_width_in_luma_samples÷CtbSizeY) (7-17)

[0082] PicHeightInCtbsY=Ceil(pic_height_in_luma_samples÷CtbSizeY) (7-18)

[0083] PicSizeInCtbsY=PicWidthInCtbsY*PicHeightInCtbsY (7-19)

[0084] PicWidthInMinCbsY=pic_width_in_luma_samples / MinCbSizeY (7-20)

[0085] PicHeightInMinCbsY=pic_height_in_luma_samples / MinCbSizeY (7-21)

[0086] PicSizeInMinCbsY=PicWidthInMinCbsY*PicHeightInMinCbsY (7-22)

[0087] PicSizeInSamplesY=pic_width_in_luma_samples*pic_height_in_luma_samples (7-23)

[0088] PicWidthInSamplesC=pic_width_in_luma_samples / SubWidthC (7-24)

[0089] PicHeightInSamplesC=pic_height_in_luma_samples / SubHeightC (7-25)

[0090] Figure 4A is an example of a CTB that crosses the bottom picture boundary. Figure 4B is an example of a CTB that crosses the right picture boundary. Figure 4C is an example of a CTB that crosses the lower right picture boundary. Figure 4A-4C In the equation, K=M, L <N;K<M,L=N;K<M,L<N。

[0091] Reference Figure 4A-4C Discuss the CTU in the picture 400. Assume that the size of the CTB / largest codec unit (LCU) is indicated by M×N (typically, M equals N, as defined in HEVC / VVC), and for a CTB located at a picture (or slice or strip or other type, picture boundary is taken as an example) boundary, K×L samples are within the picture boundary, where K<M or L<N. Figure 4A-4C As for those CTBs 402 depicted in , the size of the CTB is still equal to M×N, however, the bottom / right border of the CTB is outside the picture 400.

[0092] The encoding and decoding flows of a typical video encoder / decoder (also called codec) are discussed. Figure 5 is an example of an encoder block diagram for VVC, which contains three loop filter blocks: deblocking filter (DF), sample adaptive offset (SAO) and adaptive loop filter (ALF). Unlike DF, which uses a predetermined filter, SAO and ALF use the original samples of the current picture to reduce the mean square error between the original samples and the reconstructed samples by adding an offset and by applying a finite impulse response (FIR) filter, respectively, where the encoded side information signals the offset and filter coefficients. ALF is located at the last processing stage of each picture and can be considered a tool that tries to capture and repair artifacts produced by previous stages.

[0093] Figure 5 5 is a schematic diagram of an encoder 500. The encoder 500 is suitable for implementing the technology of VVC. The encoder 500 includes three loop filters, namely a deblocking filter (DF) 502, a sample adaptive offset (SAO) 504 and an ALF 506. Unlike the DF 502 using a predetermined filter, the SAO 504 and the ALF 506 utilize the original samples of the current picture to reduce the mean square error between the original samples and the reconstructed samples by adding an offset and by applying a FIR filter, respectively, wherein the offset and the filter coefficients are signaled by the encoded and decoded side information. The ALF 506 is located at the last processing stage of each picture and can be considered as a tool that attempts to capture and repair artifacts produced by previous stages.

[0094] The encoder 500 also includes an intra prediction component 508 and a motion estimation / compensation (ME / MC) component 510 configured to receive an input video. The intra prediction component 508 is configured to perform intra prediction, while the ME / MC component 510 is configured to perform inter prediction using a reference picture obtained from a reference picture buffer 512. The residual block from the inter prediction or intra prediction is fed into a transform component 514 and a quantization component 516 to generate a quantized residual transform coefficient, which is fed into an entropy coding component 518. The entropy coding component 518 entropy codes and decodes the prediction result and the quantized transform coefficient and transmits it toward a video decoder (not shown). The quantized component output from the quantization component 516 can be fed into an inverse quantization component 520, an inverse transform component 522, and a reconstruction (REC) component 524. The REC component 524 may output images to the DF 502 , SAO 504 , and ALF 506 for filtering before the images are stored in the reference picture buffer 512 .

[0095] The input to DF 502 is the reconstructed samples before the loop filter. First, the vertical edges in the picture are filtered. Then, the horizontal edges in the picture are filtered with the samples modified by the vertical edge filtering process as input. The vertical and horizontal edges in the CTB of each CTU are processed separately on a codec unit basis. Starting from the edges on the left hand side of the codec block, the vertical edges of the codec blocks in the codec unit are filtered through these edges in a geometric order to the right hand side of the codec block. Starting from the edges at the top of the codec block, the horizontal edges of the codec blocks in the codec unit are filtered through the edges in a geometric order to the bottom of the codec block.

[0096] Figure 6 is a diagram 600 of samples 602 within an 8x8 sample block 604. As shown, the diagram 600 includes horizontal and vertical block boundaries on an 8x8 grid 606, 608, respectively. Additionally, the diagram 600 depicts non-overlapping blocks of 8x8 samples 610 that may be deblocked in parallel.

[0097] Discussion of boundary decisions. The filter is applied on 8×8 block boundaries. In addition, it must be a transform block boundary or a codec subblock boundary (e.g., due to the use of affine motion prediction, optional temporal motion vector prediction (ATMVP)). For those that are not such boundaries, the filter is disabled.

[0098] Discussion of boundary strength calculation. For transform block boundary / codec subblock boundary, if it is located in an 8×8 grid, the transform block boundary / codec subblock boundary can be filtered, and the bS for the boundary is defined in Table 1 and Table 2, respectively (where [xD i ][yD j] indicates the setting of coordinates).

[0099] Table 1. Boundary Strength (When SPS IBC is Disabled)

[0100]

[0101]

[0102] Table 2. Boundary Strength (When SPS IBC is Enabled)

[0103]

[0104] Discuss the deblocking decision of the luminance component.

[0105] Figure 7 700 is an example of pixels involved in filter on / off decisions and strong / weak filter selection. A wider and stronger luma filter is used only when conditions 1, 2, and 3 are all true (TRUE). Condition 1 is the "large block condition". This condition detects whether the samples at the P side and the Q side belong to a large block, which is represented by the variables bSidePisLargeBlk and bSideQisLargeBlk, respectively.

[0106] bSidePisLargeBlk and bSideQisLargeBlk are defined as follows.

[0107] bSidePisLargeBlk=((edge ​​type is vertical and p 0 belongs to CU, width>=32)||(edge ​​type is horizontal and p 0 Belongs to CU, height>=32))? True: False:

[0108] bSideQisLargeBlk=((edge ​​type is vertical and q 0 belongs to CU, width>=32)||(edge ​​type is horizontal and q 0 Belongs to CU, height>=32))? True: False:

[0109] Based on bSidePisLargeBlk and bSideQisLargeBlk, condition 1 is defined as follows.

[0110] Condition 1 = (bSidePisLargeBlk || bSidePisLargeBlk)? True: False:

[0111] Next, if condition 1 is true, it will further check condition 2. First, export the following variables.

[0112] First, export dp0, dp3, dq0, and dq3 as in HEVC

[0113] If (p side is greater than or equal to 32)

[0114] dp0 = (dp0 + Abs(p5 0 - 2 * p4 0 + p3 0 )) + 1) >> 1

[0115] dp3 = (dp3 + Abs(p5 3 - 2 * p4 3 + p3 3 )) + 1) >> 1

[0116] If (p side is greater than or equal to 32)

[0117] dq0 = (dq0 + Abs(q5 0 - 2 * q4 0 + q3 0 )) + 1) >> 1

[0118] dq3 = (dq3 + Abs(q5 3 - 2 * q4 3 + q3 3 )) + 1) >> 1

[0119] Condition 2 = (d < D)? true : false

[0120] where d = dp0 + dq0 + dp3 + dq3

[0121] If both Condition 1 and Condition 2 are valid, further check if any block uses sub - blocks

[0122]

[0123] Finally, if both Condition 1 and Condition 2 are valid, the proposed de - blocking method will check Condition 3 (large - block strong filter condition), which is defined as follows

[0124] In Condition 3 StrongFilterCondition, export the following variables

[0125] Export dpq as in HEVC

[0126] Export sp as in HEVC 3 = Abs(p 3 - p 0 )

[0127] If (p side is greater than or equal to 32)

[0128] if(Sp==5)

[0129] sp 3 =(sp 3 +Abs(p 5 -p 3 )+1)>>1

[0130] else

[0131] sp 3 =(sp 3 +Abs(p 7 -p 3 )+1)>>1

[0132] Export sq as in HEVC 3 =Abs(q 0 -q 3 )

[0133] if (q side is greater than or equal to 32)

[0134] If (Sq=5)

[0135] sq 3 =(sq 3 +Abs(q 5 -q 3 )+1)>>1

[0136] else

[0137] sq3=(sq 3 +Abs(q 7 -q 3 )+1)>>1

[0138] As in HEVC, StrongFilterCondition = (dpq less than (β>>2), sp 3 +sq 3 Less than (3*β>>5), and Abs(p 0 -q 0 ) is less than (5*t C +1)>>1)? True: False:

[0139] A stronger deblocking filter for luma (designed for larger blocks) is discussed.

[0140] A bilinear filter is used when samples on either side of the boundary belong to a large block. Samples belonging to a large block are defined as when the width is >= 32 for vertical edges and when the height is >= 32 for horizontal edges.

[0141] The bilinear filters are listed below.

[0142] Then, in the above HEVC deblocking, for the block boundary samples pi from i = 0 to sp-1 and for the block boundary samples q from j = 0 to Sq-1 j (p i and q j is the i-th sample in the row for filtering vertical edges, or the j-th sample in the column for filtering horizontal edges) is replaced by linear interpolation as described below.

[0143] p i ′=(f i *Middle s,t +(64-f i )*P s +32)>>6), limit to p i ±tcPD i

[0144] q j ′=(g j *Middle s,t +(64-g j )*Q s +32)>>6), limit to q j ±tcPD j

[0145] Among them, tcPD i and tcPD j The term is a position-dependent clip described below and is given below g j 、f i 、Middle s，t , P s and Q s .

[0146] Chroma deblocking control is discussed.

[0147] A chroma strong filter is used on both sides of the block boundary. Here, the chroma filter is selected when both sides of the chroma edge are greater than or equal to 8 (chroma position), and the following decision with three conditions is met: The first is for decision boundary strength and large blocks. The proposed filter can be applied when the width or height of the block orthogonally across the block edge is equal to or greater than 8 in the chroma sample domain. The second and third conditions are basically the same as the HEVC luma deblocking decision, which are an on / off decision and a strong filtering decision, respectively.

[0148] In the first decision, the boundary strength (bS) is modified for chroma filtering, and the conditions are checked sequentially. If a condition is met, the remaining conditions with lower priority are skipped.

[0149] When bS is equal to 2, or when bS is equal to 1 when a large block boundary is detected, chroma deblocking is performed.

[0150] The second and third conditions are essentially the same as the HEVC luma strong filtering decision, as described below.

[0151] In the second case: d is then derived as in HEVC luma deblocking. The second condition will be true when d is less than β.

[0152] In the third condition, StrongFilterCondition is derived as follows.

[0153] Export dpq as in HEVC

[0154] Export sp as in HEVC 3 =Abs(p 3 -p 0 )

[0155] Export sq as in HEVC 3 =Abs(q 0 -q 3 )

[0156] As in the HEVC design, StrongFilterCondition = (dpq less than (β>>2), sp 3 +sq 3 Less than (β>>3), and Abs(p 0 -q 0 ) is less than (5*t C +1)>>1).

[0157] A strong deblocking filter for chroma is discussed. The following strong deblocking filter for chroma is defined.

[0158] p 2 ′=(3*p 3 +2*p 2 +p 1 +p 0 +q 0 +4)>>3

[0159] p 1 ′=(2*p 3 +p 2 +2*p 1 +p 0 +q 0 +q 1 +4)>>3

[0160] p 0 ′=(p 3 +p2 +p 1 +2*p 0 +q 0 +q 1 +q 2 +4)>>3

[0161] The proposed chroma filter performs deblocking on a 4×4 chroma sample grid.

[0162] Position dependent clipping (tcPD) is discussed. Position dependent clipping tcPD is applied to the output samples of a luminance filtering process involving strong and long filters modifying 7, 5 and 3 samples at the boundaries. Assuming a quantization error distribution, it is proposed to increase the clipping value for samples that are expected to have higher quantization noise and therefore higher deviation of the reconstructed sample value from the true sample value.

[0163] For each P or Q boundary filtered using an asymmetric filter, a position-dependent threshold table is selected from two tables (i.e., Tc7 and Tc3 in the table below) based on the results of the decision process in the boundary strength calculation and provided to the decoder as side information.

[0164] Tc7={6,5,4,3,2,1,1}; Tc3={6,4,2};

[0165] tcPD=(Sp==3)? Tc3:Tc7;

[0166] tcQD=(Sq==3)? Tc3:Tc7;

[0167] For P or Q boundaries filtered with a short symmetric filter, a lower magnitude position-dependent threshold is applied.

[0168] Tc3 = {3, 2, 1};

[0169] After defining the thresholds, the filtered p' is clipped according to the tcP and tcQ clipping values i and q' j Sample value.

[0170] p" i = Clip3(p' i +tcP i , p' i -tcP i , p' i );

[0171] q" j = Clip3(q' j +tcQ j , q' j -tcQ j , q'j );

[0172] Among them, p' i and q' j is the filtered sample value, p" i and q” j is the output sample value after limiting, and tcP i QTCE j is the clipping threshold derived from the VVC tc parameters and tcPD and tcQD. Function Clip3 is the clipping function as specified in VVC.

[0173] Subblock deblocking adjustments are now discussed. To achieve parallel friendly deblocking using both long filters and subblock deblocking, the long filter is restricted to modify at most 5 samples on one side using subblock deblocking (AFFINE or ATMVP or decoder-side motion vector refinement (DMVR)), as shown in the luminance control for the long filter. In addition, subblock deblocking is adjusted so that subblock boundaries close to codec unit (CU) or implicit TU boundaries on the 8×8 grid are restricted to modify at most two samples on each side.

[0174] The following applies to sub-block boundaries that are not aligned with CU boundaries.

[0175]

[0176] Where an edge equal to 0 corresponds to a CU boundary, an edge equal to 2 or equal to orthogonalLength-2 corresponds to a sub-block boundary 8 samples from a CU boundary, etc. Where implicit-TU is true if implicit partitioning of TUs is used.

[0177] Sample Adaptive Offset (SAO) is discussed. The input of SAO is the reconstructed samples after deblocking (DB). The concept of SAO is to reduce the average sample distortion of a region by first classifying the region samples into multiple categories with a selected classifier, obtaining an offset for each category, and then adding the offset to each sample of the category, where the classifier index and the offset of the region are encoded in the bitstream. In HEVC and VVC, the region (the unit for SAO parameter signaling) is defined as CTU.

[0178] Two SAO types are used in HEVC to meet the low complexity requirements. These two types are edge offset (EO) and band offset (BO), which will be discussed in more detail below. The index of the SAO type is encoded and decoded (it is in the range of [0, 2]). For EO, ​​the sample classification is based on the comparison between the current sample and the neighboring samples according to the 1-D directional pattern: horizontal, vertical, 135° diagonal and 45° diagonal.

[0179] Figure 8 Four one-dimensional (1-D) directional patterns 800 for EO spot classification are shown: horizontal (EO Class=0), vertical (EO Class=1), 135° diagonal (EO Class=2), and 45° diagonal (EO Class=3).

[0180] For a given EO class, each sample point within the CTB is classified into one of five categories. The current sample point value, labeled "c", is compared with its two neighbors along the selected 1-D pattern. The classification rules for each sample point are summarized in Table 3. Classes 1 and 4 are associated with local valleys and local peaks along the selected 1-D pattern, respectively. Classes 2 and 3 are associated with concave corners and convex corners along the selected 1-D pattern, respectively. If the current sample point does not belong to EO category 1-4, it is class 0 and SAO is not applied.

[0181] Table 3: Sample point classification rules for edge offset

[0182]

[0183] The geometric transformation based adaptive loop filter in the joint exploration model (JEM) is discussed. The input of DB is the reconstructed samples after DB and SAO. The sample classification and filtering process is based on the reconstructed samples after DB and SAO.

[0184] In JEM, a geometric transform based adaptive loop filter (GALF) with block-based filter adaptation is applied. For the luma component, one of twenty-five filters is selected for each 2x2 block based on the direction and activity of the local gradient.

[0185] Discuss filter shapes. Fig. 9 Examples of GALF filter shapes 900 are shown, including a 5×5 diamond on the left, a 7×7 diamond in the middle, and a 9×9 diamond on the right. In JEM, up to three diamond filter shapes can be selected for the luma component (e.g. Fig. 9 As shown in Figure 1. ...

[0186] Block classification is discussed. Each 2×2 block is classified into one of twenty-five classes. The classification index C is based on its directionality D and the quantized value of activity And export as described below.

[0187]

[0188] To calculate D and First, the 1-D Laplacian operator is used to calculate the gradients in the horizontal, vertical, and two diagonal directions.

[0189]

[0190]

[0191] The indices i and j refer to the coordinates of the upper left sample point in the 2×2 block, and R(i, j) indicates the reconstructed sample point at the coordinates (i, j).

[0192] Then, the maximum and minimum values ​​of the gradient D in the horizontal and vertical directions are set to:

[0193]

[0194] And the maximum and minimum values ​​of the gradients in the two diagonal directions are set to:

[0195]

[0196] To derive the value D of the directionality, these values ​​are compared with each other and with two threshold values ​​t 1 and t 2 Compare:

[0197] Step 1. If and If both are true, D is set to 0.

[0198] Step 2. If Then continue from step 3; otherwise continue from step 4.

[0199] Step 3. If Then set D to 2; otherwise set D to 1.

[0200] Step 4. If Then set D to 4; otherwise set D to 3.

[0201] The activity value A is calculated as follows:

[0202]

[0203] A is further quantized to the range of 0 to 4, inclusive, and the quantized value is represented as

[0204] For the two chroma components in a picture, no classification method is applied, ie, a single set of ALF coefficients is applied for each chroma component.

[0205] The geometric transformation of filter coefficients is discussed.

[0206] Fig.10 Relative coordinates 1000 are shown for a 5x5 diamond filter support - diagonal, vertical flip and rotation respectively (from left to right).

[0207] Before filtering each 2×2 block, geometric transformations such as rotation or diagonal and vertical flipping are applied to the filter coefficients f(k,l) associated with the coordinates (k,l) according to the gradient values ​​calculated for that block. This is equivalent to applying these transformations to the samples in the filter support region. The idea is to make different blocks to which the ALF is applied more similar by aligning the directionality of different blocks.

[0208] Three geometric transformations are introduced, including diagonal, vertical flip and rotation:

[0209] Diagonal: f D (k, l) = f(l, k),

[0210] Flip vertically: f V (k, l) = f(k, Kl-1), (9)

[0211] Rotation: f R (k, l) = f(Kl-1, k).

[0212] Where K is the size of the filter, and 0≤k, l≤K-1 are the coefficient coordinates, so that the position (0,0) is in the upper left corner, and the position (K-1,K-1) is in the lower right corner. The transform is applied to the filter coefficients f(k,l) according to the gradient value calculated for the block. The relationship between the transform and the four gradients in the four directions is summarized in Table 4.

[0213] Table 4: Mapping of gradients computed for a block to transformations

[0214]

[0215] Filter parameter signaling is discussed. In JEM, GALF filter parameters are signaled for the first CTU, i.e., after the slice header and before the SAO parameters of the first CTU. Up to 25 groups of luminance filter coefficients can be signaled. In order to reduce bit overhead, filter coefficients of different classifications can be merged. Moreover, GALF coefficients of reference pictures are stored and allowed to be reused as GALF coefficients for the current picture. The current picture can choose to use the GALF coefficients stored for the reference picture and bypass the GALF coefficient signaling. In this case, only the index of one of the reference pictures is signaled, and the stored GALF coefficients of the indicated reference picture are inherited for the current picture.

[0216] To support GALF temporal prediction, a candidate list of GALF filter sets is maintained. At the beginning of decoding a new sequence, the candidate list is empty. After decoding a picture, the corresponding filter set can be added to the candidate list. Once the size of the candidate list reaches the maximum allowed value (i.e., 6 in the current JEM), the new filter set is written to the oldest set in the decoding order, that is, the first-in-first-out (FIFO) rule is applied to update the candidate list. In order to avoid duplication, the set can be added to the list only when the corresponding picture does not use GALF temporal prediction. In order to support temporal scalability, there are multiple candidate lists of filter sets, and each candidate list is associated with a temporal layer. More specifically, each array allocated by the temporal layer index (TempIdx) can constitute a filter set of a previously decoded picture with a lower TempIdx. For example, the kth array is allocated to be associated with a TempIdx equal to k, and the kth array contains only filter sets from pictures with TempIdx less than or equal to k. After encoding and decoding a particular picture, the filter set associated with that picture will be used to update those arrays associated with equal or higher Templdx.

[0217] Temporal prediction of the GALF coefficients is used for inter-coded frames to minimize signaling overhead. For intra frames, temporal prediction is not available and a set of 16 fixed filters is assigned to each class. To indicate the use of fixed filters, a flag for each class is signaled and, if necessary, the index of the selected fixed filter is signaled. Even when a fixed filter is selected for a given class, the coefficients of the adaptive filter f(k, l) may still be sent for that class, in which case the coefficients of the filter to be applied to the reconstructed image are the sum of the two sets of coefficients.

[0218] The filtering process of the luma component can be controlled at the CU level. A signaling flag is used to indicate whether GALF is applied to the luma component of the CU. For chroma components, whether GALF is applied is only indicated at the picture level.

[0219] Let’s discuss the filtering process. At the decoder side, when GALF is enabled for a block, each sample point R(i, j) within the block is filtered to produce the sample value R′(i, j) shown below, where L represents the filter length and f m，n represents the filter coefficient, and F(k, l) represents the decoded filter coefficient.

[0220]

[0221] Fig.11An example of relative coordinates for a 5x5 diamond filter support is shown assuming that the coordinates (i, j) of the current sample point are (0, 0). Samples in different coordinates filled with the same color are multiplied by the same filter coefficient.

[0222] The geometric transformation based adaptive loop filter (GALF) in VVC is discussed. In the VVC Test Model 4.0 (VTM4.0), the filtering process of the adaptive loop filter is performed as follows:

[0223] O(x, y) = ∑ (i，j) w(i, j).I(x+i, y+j), (11)

[0224] Wherein, sample I(x+i, y+j) is the input sample, O(x, y) is the filtered output sample (i.e., the filter result), and w(i, j) represents the filter coefficient. In fact, in VTM4.0, integer operations are used to implement fixed-point precision calculations.

[0225]

[0226] where L represents the filter length and where w(i, j) are the filter coefficients in fixed-point precision.

[0227] The current design of GALF in VVC has the following major changes compared to the design of GALF in JEM:

[0228] 1) Remove adaptive filter shapes. Only 7×7 filter shapes are allowed for luma components and 5×5 filter shapes are allowed for chroma components.

[0229] 2) Remove the signaling notification of ALF parameters from the slice / picture level to the CTU level.

[0230] 3) The calculation of the class index is performed in the 4×4 level instead of the 2×2 level. In addition, as proposed in JVET-L0147, the sub-sampled Laplacian calculation method for ALF classification is utilized. More specifically, the horizontal / vertical / 45 degree diagonal / 135 degree gradient does not need to be calculated for each sample point within a block. Instead, 1:2 sub-sampling is utilized.

[0231] Regarding filter reformulation, the nonlinear ALF in current VVC is discussed.

[0232] Without affecting the coding efficiency, equation (11) can be re-expressed as follows:

[0233] O(x, y) = I(x, y) + ∑ (i，j)≠(0，0) w(i,j).(I(x+i,y+j)-I(x,y)), (13)

[0234] where w(i, jj) are the same filter coefficients as in equation (11). (Except w(0, 0), which is equal to 1 in equation (13) and 1-∑ in equation (11) (i，j)≠(0，0) w(i, j)

[0235] Using the filter formula of (13) above, VVC introduces nonlinearity to make the ALF more effective by using a simple clipping function to reduce the impact of neighboring sample values ​​(I(x+i, y+j)) when they differ too much from the current sample value being filtered (I(x, y)).

[0236] More specifically, the ALF filter is modified as follows:

[0237] O′(x, y)=I(x, y)+∑ (i，j)≠(0，0) w(i,j).K(I(x+i,y+j)-I(x,y),k(i,j)), (14)

[0238] Where K(d,b)=min(b,max(-b,d)) is a clipping function and k(i,j) is a clipping parameter that depends on the (i,j) filter coefficient. The encoder performs an optimization to find the best k(i,j).

[0239] In the JVET-N0242 implementation, a clipping parameter k(i, j) is specified for each ALF filter, and one clipping value is signaled for each filter coefficient. This means that up to 12 clipping values ​​can be signaled in the bitstream for each luminance filter, and up to 6 clipping values ​​can be signaled for chrominance filters.

[0240] To limit the signaling cost and encoder complexity, only 4 fixed values ​​that are the same for inter and intra slices are used.

[0241] Since the variance of local differences is usually higher for luma than for chroma, two different sets are applied for luma and chroma filters. A maximum sample value in each set is also introduced (here 1024 for 10-bit bit depth) so that clipping can be disabled if not needed.

[0242] The set of clipping values ​​used in the JVET-N0242 test is provided in Table 5. The 4 values ​​are chosen by dividing the entire range of sample values ​​for luma (encoded at 10 bits) and the range from 4 to 1024 for chroma approximately equally in the logarithmic domain.

[0243] More precisely, the brightness table of the clipping values ​​is obtained by the following formula:

[0244] Where M = 210 And N = 4. (15)

[0245] Similarly, the chromaticity table of the clipped value is obtained according to the following formula:

[0246] Where M = 2 10 , N = 4 and A = 4. (16)

[0247] Table 5: Authorized limit values

[0248] Intra / Inter slice group brightness {1024，181，32，6} Chroma {1024，161，25，4}

[0249] The selected clipping value is encoded and decoded in the "alf data" syntax element by using the Golomb coding scheme corresponding to the index of the clipping value in the above Table 5. This coding scheme is the same as the coding scheme used for the filter index.

[0250] Convolutional neural network based loop filters for video codecs are discussed.

[0251] In deep learning, convolutional neural networks (CNN or ConvNet) are a class of deep neural networks that are most commonly used to analyze visual images. They have very successful applications in image and video recognition / processing, recommendation systems, image classification, medical image analysis, and natural language processing.

[0252] CNNs are regularized versions of multilayer perceptrons. Multilayer perceptrons usually mean fully connected networks, i.e., each neuron in one layer is connected to all neurons in the next layer. The "full connectivity" of these networks makes them prone to overfitting the data. Typical ways of regularization involve adding some form of magnitude measure of the weights to the loss function. CNNs take a different approach to regularization: they exploit hierarchical patterns in the data and use smaller and simpler patterns to assemble more complex patterns. Therefore, on the scale of connectivity and complexity, CNNs are at the lower limit.

[0253] Compared to other image classification / processing algorithms, CNNs use relatively little preprocessing. This means that the network learns filters that are hand-designed in traditional algorithms. This independence from prior knowledge and human effort in feature design is a major advantage.

[0254] Deep learning-based image / video compression usually has two meanings: pure neural network-based end-to-end compression, and traditional frameworks enhanced by neural networks. Pure neural network-based end-to-end compression is discussed in: Johannes Balle, Valero Laparra, and Eero P. Simoncelli, "End-to-end optimization of nonlinear transform codes for perceptual quality", 2016 Picture Coding Summit (PCS), pages 1-5, Institute of Electrical and Electronics Engineers (IEEE) and Lucas Theis, Wenzhe Shi, Andrew Cunningham, and Ferenc Huszár, "Lossy image compression with compressive autoencoders", arXiv preprint arXiv:1703.00395 (2017).Traditional frameworks enhanced by neural networks are discussed in: Jiahao Li, Bin Li, Jizheng Xu, Ruiqin Xiong, and Wen Gao, “Fully Connected Network-Based Intra Prediction for Image Coding,” IEEE Transactions on Image Processing, July 27, 2018, pp. 3236–3247, Yuanying Dai, Dong Liu, and Feng Wu, “A convolutional neural network approach for post-processing in HEVC intracoding,” MMM. Springer, pp. 28–39, Rui Song, Dong Liu, Houqiang Li, and Feng Wu, “Neural network-based arithmetic coding of intra prediction modes in HEVC (Neural network based arithmetic coding for intra prediction modes in HEVC),” VCIP. IEEE, pp. 1–4, and J. Pfaff, P. Helle, D. Maniry, S. Kaltenstadler, W. Samek, H. Schwarz, D. Marpe, and T. Wiegand, “Neural network based intra prediction for video coding,” in Applications of Digital Image Processing XLI, vol. 10752. International Society for Optical Engineering, 1075213.

[0255] The first type usually adopts an autoencoder-like structure, which is implemented by a convolutional neural network or a recurrent neural network. Although relying solely on neural networks for image / video compression can avoid any manual optimization or hand-crafted design, the compression efficiency may not be satisfactory. Therefore, the work of the second type of distribution is supplemented by neural networks to enhance the traditional compression framework by replacing or enhancing some modules. In this way, they can inherit the advantages of highly optimized traditional frameworks. For example, as discussed below, a fully connected network for intra-frame prediction is proposed in HEVC: Jiahao Li, Bin Li, Jizheng Xu, Ruiqin Xiong, and Wen Gao, "Fully Connected Network-Based IntraPrediction for Image Coding", IEEE Transactions on Image Processing, July 27, 2018, pp. 3236-3247.

[0256] In addition to intra prediction, deep learning is also used to enhance other modules. For example, the loop filter of HEVC is replaced by a convolutional neural network, and good results are achieved in the following: Yuanying Dai, Dong Liu, and Feng Wu, "A convolutional neural network approach for post-processing in HEVC intracoding", MMM. Springer, 28-39. The following work applies neural networks to improve the arithmetic codec engine: Rui Song, Dong Liu, Houqiang Li, and Feng Wu, "Neural network-based arithmetic coding of intra prediction modes in HEVC", VCIP. IEEE, 1-4.

[0257] Convolutional neural network based loop filtering is discussed. In lossy image / video compression, the reconstructed frame is an approximation of the original frame because the quantization process is irreversible and thus results in distortion of the reconstructed frame. To mitigate such distortion, a convolutional neural network can be trained to learn the mapping from the distorted frame to the original frame. In practice, the training must be performed before deploying the CNN based loop filtering.

[0258] Discussion of Training. The goal of the training process is to find optimal values ​​for the parameters including weights and biases.

[0259] First, a codec (e.g., HM, JEM, VTM, etc.) is used to compress the training data set to generate distorted reconstructed frames. Then, the reconstructed frames are fed into the CNN, and the cost is calculated using the output of the CNN and the ground truth frame (original frame). Commonly used cost functions include the sum of absolute differences (SAD) and the mean square error (MSE). Next, the cost gradient for each parameter is derived by the back propagation algorithm. Using the gradient, the value of the parameter can be updated. Repeat the above process until the convergence criterion is met. After completing the training, the derived optimal parameters are saved for use in the inference stage.

[0260] Discuss the convolution process. During convolution, the filter is moved across the image from left to right and top to bottom, with a one-pixel-column change in horizontal movement and then a one-pixel-row change in vertical movement. The amount of movement between applying the filter to the input image is called the stride, and it is almost always symmetric across the height and width dimensions. The default stride or strides in two dimensions are (1, 1) for both height and width movement.

[0261] Fig. 12A is an exemplary architecture 1200 of the proposed CNN filter, and Fig. 12B is an example of a residual block (ResBlock) construction 1250. In most deep convolutional neural networks, residual blocks are used as basic modules and are stacked several times to construct the final network, where, in one example, the residual block is obtained by combining a convolutional layer, a ReLU / PReLU activation function, and a convolutional layer, such as Fig. 12B shown.

[0262] Discussion of Inference. During the inference phase, the distorted reconstructed frame is fed into the CNN and processed by the CNN model, whose parameters have been determined in the training phase. The input samples to the CNN can be the reconstructed samples before or after DB, or the reconstructed samples before or after SAO, or the reconstructed samples before or after ALF.

[0263] Current CNN-based loop filtering has certain problems. For example, the NN model does not use or otherwise exploit external information (e.g., including information from a video codec, such as codec parameters and / or codec syntax, which is information outside of the information generated by the NN model itself) as attention or as an attention mechanism. Therefore, information including various codec parameters such as prediction (e.g., prediction mode, motion vector, etc.), partitioning (e.g., segmentation information), etc., such as to recalibrate feature maps, may not be fully utilized.

[0264] Disclosed herein is a technology for solving one or more of the aforementioned problems. For example, the present disclosure provides one or more neural network (NN) filter models trained as codec tools to improve the efficiency of video codecs. Codec tools based on NN can be used to replace or otherwise enhance modules involved in video codecs. For example, the NN model can be used as an additional intra-frame prediction mode, inter-frame prediction mode, transform kernel or loop filter. The present disclosure also describes in detail how to design an NN model based on segmentation information of video units. For example, segmentation information (e.g., partitioning) can be used as attention or attention mechanism, which will be further described below. It should be noted that the NN model can be used as any codec tool, such as NN-based intra / inter-frame prediction, NN-based super-resolution, NN-based motion compensation, NN-based reference generation, NN-based fractional pixel interpolation, NN-based loop / post-filtering, etc.

[0265] The list of the following embodiments should be considered as examples to explain the general concept. These embodiments should not be interpreted in a narrow manner. In addition, these embodiments can be combined in any way.

[0266] In the present disclosure, the NN model may include any kind of NN architecture, such as a convolutional neural network (CNN) or a fully connected NN, or a combination of a CNN and a fully connected NN. In the following discussion, the NN model may also be referred to as a CNN model.

[0267] In the following discussion, a video unit may be a sequence, a picture, a slice, a tile, a sub-picture, a CTU / CTB, a CTU / CTB row, one or more CUs / codec blocks (CBs), one or more CTUs / CTBs, one or more virtual pipeline data units (VPDUs), a sub-region within a picture / slice / slice / tile. A parent video unit represents a unit that is larger than a video unit. Typically, a parent unit will contain several video units, for example, when the video unit is a CTU, the parent unit may be a slice, a CTU row, multiple CTUs, etc. In some embodiments, a video unit may be a sample / pixel.

[0268] Fig.13is an example of a process 1300 for generating filtered samples based on an NN filter model that receives codec parameters (e.g., external information to the NN filter model) as input. The NN filter model has an attention mechanism based on the codec parameter input. The attention mechanism is configured to generate or obtain attention, which is useful for simulating cognitive attention and actually enhances some portions of the data input to the NN filter while reducing other portions of the data input to the NN filter. For example, attention can be obtained by processing external information to the NN filter model, such as one or more codec parameters, to extract attention. Attention is configured to be applied to one or more feature maps, such as by weighting the feature maps according to the attention.

[0269] exist Fig.13 In the illustrated process 1300, at least some unfiltered samples are provided as input to the NN filter. In one example, the unfiltered samples are samples (e.g., pixels) of a video unit that have not been subjected to any filtering or have not been subjected to a sufficient amount of filtering. The output of the NN filter may therefore be filtered samples. The output of the NN filter is also based on a NN filter model generated using a codec parameter input. The codec parameter input may be related to a reconstruction of the video unit, a partitioning or segmentation scheme of the video unit, a prediction mode of the video unit, a quantization parameter QP associated with the video unit, and / or a boundary strength parameter of the video unit.

[0270] For example, the convolutional layer can be used to extract features from codec parameters (e.g., external information), or to extract features from both codec parameters and internal information, such as features extracted inside a NN filter model. At least one of the extracted features is used as the attention obtained in the NN filter model.

[0271] In another example, the codec parameter provided as input to the NN filter is more specifically segmentation information of the video unit. In some cases, the NN filter model on which the NN filter is based is configured to obtain attention based on the segmentation information.

[0272] The segmentation information may include an average pixel value for each of the one or more CUs of the video unit. In this example, each CU may be represented by its average pixel value (or sample value) so that the CUs can be distinguished from each other, because different CUs typically have different average pixel values ​​(or sample values). Therefore, the average pixel value of a CU (e.g., a CU of a CTU) may be used to indicate how the CTU is divided or partitioned into CUs. In a specific example, the average pixel value of a CU is the average value of the luma values ​​of the pixels in the CU, or the average value of the chroma values ​​of the pixels in the CU. For example, luma segmentation may be represented by the average value of the luma samples in the CU, and chroma segmentation may be represented by the average value of the chroma samples in the CU.

[0273] The segmentation information may also include pixel values ​​for each of the one or more CUs of the video unit. For example, the pixel value may be based on the proximity of the pixel to the boundary of the CU. Pixels closer to the boundary of the CU may have a first value, while pixels farther from the boundary of the CU may have a second value different from the first value.

[0274] In some examples, the segmentation information is represented by an M×N array, where M is the width of the video unit to be encoded and N is the height of the video unit to be encoded. In another example, M is the number of columns of another input to the NN filter model and N is the number of rows of another input to the NN filter model. For example, M may be the number of columns of the video unit to be encoded and N may be the number of rows of the video unit to be encoded.

[0275] The segmentation information may include sample boundary values, brightness component values, color component values, or a combination thereof.

[0276] The unfiltered samples may include a luma component and a chroma component. In some examples, the NN filter model is generated based on the luma component and the chroma component. For example, a first value is assigned to the luma component and a second value is assigned to the chroma component. The second value may be different from the first value, and the NN filter model is generated based on the first value and the second value.

[0277] In some examples, the NN filter model is additionally generated based on a quantization parameter (QP) of the video unit, a slice type of the video unit, a temporal layer identifier of the video unit, a boundary strength of the video unit, a motion vector of the video unit, a prediction mode of the video unit, an intra-prediction mode of the video unit, a scaling factor of the video unit, or a combination thereof.

[0278] The granularity of the NN filter model specifies the size of the video unit to which the NN filter model can be applied. In some examples, the NN filter model granularity can be derived. In other examples, the NN filter model granularity can be signaled in the bitstream, such as in a sequence header, a picture header, a slice header, a sequence parameter set (SPS), a picture parameter set (PPS), an adaptation parameter set (APS), or a combination thereof.

[0279] Fig.14is an example of a process 1400 of applying attention obtained using external information such as codec parameters or more specifically segmentation information to a feature map of a NN filter model to provide a recalibrated feature map according to some examples. The feature map of the NN filter model is the result of applying a filter to an input image (or as a feature map provided by the output of a previous layer). For example, at each layer of the NN filter model, the feature map is the output of the layer. Therefore, the feature map produced by the layer inside the NN filter model can be referred to as an intermediate feature map, and the feature map produced by the last layer of the NN filter model can be referred to as a final feature map. For example, the final output of the NN filter model can be the final feature map.

[0280] exist Fig.14 In the example, the operation performed by the convolutional layer of the NN filter model is represented as α, while the external information (e.g., codec parameters) is represented as E, and the internal information (e.g., feature maps extracted inside the NN filter model) is represented as I, the obtained attention is represented as A, and process 1400 applies the attention A to the feature map G to generate a recalibrated feature map φ.

[0281] Thus, operation α is applied to external information (e.g., codec parameters) or a concatenation of external information and internal information to extract or otherwise obtain attention A. The attention A is applied to feature map G to generate a recalibrated feature map φ. For example, applying attention A to feature map G effectively reweights feature map G (e.g., applying different weights to different elements of feature map G), and the recalibrated feature map φ is the reweighted feature map resulting from applying attention A to feature map G. In process 1400, N is the channel number, W is the channel width, and H is the channel height.

[0282] exist Fig.14 In the example, the intermediate feature map of the NN filter model is represented as G, where G∈R N×W×H The obtained attention is denoted as A, where A∈R W×H In this case, the symbol ∈ indicates that G is obtained by R N×W×H Given the elements of the set, and A is composed of R W×H The elements of the given collection.

[0283] For example, R is the domain to which a particular element in a feature map or attention belongs. R can be a domain that includes any real numbers, or a domain that includes any integer real numbers. In some examples, the feature map is a two-dimensional or three-dimensional array. Thus, a feature map G, such as an intermediate feature map G, where G∈R N×W×H, indicating that each element of the feature map G belongs to R, and there are N×W×H elements in the feature map G. In this example, the values ​​of N, W, and H indicate that the feature map G is a three-dimensional array with a dimension of N, W, and H values ​​along each dimension, respectively. In another example, such as attention A, where A∈R W×H , each element of attention A belongs to R, and there are W×H elements in attention A. In this example, having values ​​of W and H indicates that attention A is a two-dimensional array, with dimensions having values ​​of W and H along each dimension, respectively.

[0284] In one example, the recalibrated feature map is generated according to the following formula:

[0285] φ i，j，k =G i,j,k ×A j，k , 1≤i≤N, 1≤j≤W and 1≤k≤H.

[0286] In another example, the recalibrated feature map is generated according to:

[0287] φ i，j，k =G i，j,k ×f(A j，k ), 1≤i≤N, 1≤j≤W, 1≤k≤H, and f represents the mapping function applied to each attention element.

[0288] In yet another example, the recalibrated feature map is generated according to the following formula:

[0289] φ i，j，k =G i，j,k ×F(A j，k )+G i，j,k , 1≤i≤N, 1≤j≤W, 1≤k≤H, and f represents the mapping function applied to each attention element.

[0290] As described above, the residual block is used as a basic module of the NN model, and multiple residual blocks can be stacked to construct the final network. For example, a residual block can be obtained by combining a convolutional layer, a ReLU / PreLU activation function, and another convolutional layer, such as Fig. 12B Thus, in at least some examples, attention is applied only to the last layer of the residual block (e.g., on the feature map output of the last layer), rather than to every layer of the residual block. Fig. 16B An example of applying attention in this way is shown in and discussed further below.

[0291] In another example, attention is applied only to a specified layer of the NN filter model (e.g., on a feature map output by the specified layer), rather than to every layer of the NN filter model. The specified layer may be specified by the topology of the NN filter model.

[0292] Fig.15 is an example of a process 1500 for applying a neural network filter with a configurable depth to unfiltered samples of a video unit to generate filtered samples. Fig.15 In the illustrated process 1500, at least some unfiltered samples are provided as input to a filtering system 1502. In one example, unfiltered samples are samples (e.g., pixels) of a video unit that have not been subjected to any filtering or to a sufficient amount of filtering. Thus, the output of the filtering system 1502 may be filtered samples.

[0293] The filtering system 1502 includes an NN filter 1504 based on an NN filter model having a first depth, and an NN filter 1506 based on an NN filter model having a second depth. Thus, the depth of the filtering system 1502 or its NN filter model is configurable. The NN filters 1504, 1506 are similar to those described above, such as with reference to Fig. 12A and Fig. 12B As described above, the NN filters 1504, 1506 include multiple residual blocks, and the number of residual blocks of each NN filter 1504, 1506 is the depth of the NN filter 1504, 1506. Fig.15 The example shown is illustrative, and in other examples, the filtering system 1502 may include more than two filters (e.g., the example of two filters shown is not intended to be limiting unless expressly stated.) Thus, the applied NN filter(s) may achieve more than two depths.

[0294] As described above, in some examples, the NN filter model may have different depths, which may be selected in a predetermined manner or derived in an on-the-fly manner. In some cases, the filtering system 1502 includes an NN filter with a configurable depth, which is represented by an NN filter 1504 (based on a filtering model with a first depth) and an NN filter 1506 (based on a filtering model with a second depth). In other cases, the filtering system 1502 includes multiple NN filters 1504, 1506, each NN filter is based on an NN filter model with a different depth, and depending on various characteristics of the video unit or its unfiltered samples, one of the NN filters 1504, 1506 is selected (e.g., by filter selection logic 1508) to be used as the NN filter.

[0295] In various examples, the NN filter is applied to the unfiltered samples to generate filtered samples. Fig.15 , the depth of the NN filter model is selectable, so the NN filter can be a first NN filter 1504 (using a NN filter model with a first depth) or a second NN filter 1506 (using a NN filter model with a second depth). The filtering system 1502 includes filter selection logic 1508, which is configured to determine the depth to be applied to the unfiltered samples and select one of the NN filters 1504, 1506 (or the filter models used thereby) based on the depth to be applied. For example, the filter selection logic 1508 may determine to apply the NN filter 1504 with a first depth to the first video unit and to apply the NN filter 1506 with a second depth to the second video unit.

[0296] Regardless of the depth of the NN filter model selected by the filter selection logic 1508, applying the NN filter having a particular depth to the unfiltered samples generates filtered samples (e.g., the output of the filtering system 1502). Then, as described above, conversion between a video media file and a bitstream can be performed based on the generated filtered samples.

[0297] In one example, the filter selection logic 1508 is configured to determine the depth based on one or more of: the temporal layer of the video unit, the type of slice or picture containing the video unit (e.g., "i" for intra, "e" for inter, or "b" for bidirectional), a list of reference pictures that match the video unit, the color components of the video unit, and the color format of the video unit (e.g., luma-chroma or RGB).

[0298] In some examples, filter selection logic 1508 is configured to determine to apply NN filter models with different depths to different video units in a picture or slice. For example, NN filter 1504 is selected for a first video unit in a picture or slice, while NN filter 1506 is selected for a second video unit in the picture or slice.

[0299] In other examples, the filter selection logic 1508 is configured to determine that NN filter models with different depths are applied to different video units in different pictures or stripes. For example, NN filter 1504 is selected for a first video unit in a first picture or a first stripe, while NN filter 1506 is selected for a second video unit in a second picture or a second stripe.

[0300] In yet other examples, the filter selection logic 1508 is configured to determine, based on the temporal layer or slice type of the video unit, to apply NN filter models with different depths to the video unit. In some cases, the temporal layers are grouped into subgroups, and NN filter models with the same depth are applied to video units in temporal layers in a particular subgroup, while NN filter models with different depths are applied to video units in temporal layers in different subgroups.

[0301] In some examples, the filter selection logic 1508 is configured to determine to apply NN filter models with different depths based on whether the video unit is a codec tree unit CTU, a codec tree block CTB, a row of a CTU, a row of a CTB, a slice, a slice, a picture, a sequence, or a sub-picture.

[0302] The depth of a particular video unit may also be signaled to a decoder of the video unit, such as by an encoder of the video unit. For example, the depth may be signaled to a decoder using a supplemental enhancement information (SEI) message, a sequence parameter set (SPS), a picture parameter set (PPS), an adaptation parameter set (APS), a video parameter set (VPS), a picture header, or a slice header.

[0303] In some examples, different depths are signaled for different video units. In other examples, such as where the unfiltered samples include luma and chroma components, different depths may be signaled independently for each of the luma and chroma components.

[0304] A discussion of model selection is provided.

[0305] Example 1

[0306] 1. The NN filter model can use external information such as reconstruction parameters, partition or segmentation parameters, prediction parameters, boundary strength parameters, QP, etc. (usually, codec parameters related to the video unit being filtered) as input to obtain attention. For example, the NN filter model has an attention mechanism based on the codec parameter input.

[0307] a. In one example, the convolutional layer of the NN filter model is used to extract features from external information or both external and internal information, where internal information refers to features extracted inside the network, and external information refers to other information that cannot be obtained from features inside the network, such as codec parameters related to the filtered video unit. At least one of these extracted features will be used as attention. As described above, Fig.14 An illustration of an attention mechanism according to some examples is provided, where the operation performed by (one or more) convolutional layers is represented as α, external information is represented as E, internal information is represented as I, and the attention obtained is represented as A.

[0308] i. In one example, E is one of a reconstruction parameter, a partition or segmentation parameter, a prediction parameter, a boundary strength parameter, a QP, etc.

[0309] ii. In one example, E can be any combination of reconstruction parameters, partition or segmentation parameters, prediction parameters, boundary strength parameters, QP, etc.

[0310] iii. In one example, I is the intermediate feature map(s) of the NN model, which will be recalibrated by the obtained attention.

[0311] iv. In one example, A=α(E).

[0312] v. In one example, A = α(E, I), where E and I are first concatenated and then fed into a convolutional layer.

[0313] vi. In one example, A = α(E), where E is a cascade of a reconstructed image and a segmented image, α is a two-layer convolutional neural network, and A is a single-channel feature map with the same spatial resolution as the feature map to which A will be applied.

[0314] b. In one example, the obtained attention is used to recalibrate the intermediate feature map. The intermediate feature map of the NN model is denoted as G, where G∈R N×W×H , where N, W, and H are the channel number, width, and height, respectively. The obtained attention is denoted as A, where A∈R W×H In this case, the symbol ∈ indicates that G is obtained by R N×W×H Given the elements of the set, and A is composed of R W×H The elements of the given collection.

[0315] i. In one example, the process of applying attention can be written as:

[0316] φ i，j，k =G i,j,k ×A j，k , 1≤i≤N, 1≤j≤W, 1≤k≤H, where φ is the recalibrated feature map.

[0317] ii. In one example, the process of applying attention can be written as:

[0318] φ i，j，k =G i，j,k ×F(A j，k), 1≤i≤N, 1≤j≤W, 1≤k≤H, where φ is the recalibrated feature map and f represents the mapping function applied to each element of attention. The mapping function can be a sigmoid function, a hyperbolic tangent (e.g., tanh) function, etc. For example, a sigmoid function is a function having a characteristic "S" shape or an S-shaped curve. The hyperbolic tangent function is an example of a sigmoid function.

[0319] 1. In one example, different A and / or different f may be used for different channels of the feature map.

[0320] iii. In one example, the process of applying attention can be written as:

[0321] φ i，j，k =G i，j,k ×f(A j，k )+G i,j，k , 1≤i≤N, 1≤j≤W, 1≤k≤H, where φ is the recalibrated feature map and f represents the mapping function applied to each element of the attention. The mapping function can be a sigmoid function, a hyperbolic tangent (e.g., tanh) function, etc.

[0322] 1. In one example, different A and / or different f may be used for different channels of the feature map.

[0323] iv. In one example, the attention operation can be applied to a specified layer within the network.

[0324] 1. In one example, when the network contains residual blocks, the attention operation is applied only to the feature maps from the last layer of each residual block.

[0325] Example 2

[0326] 2. In the second embodiment, the external attention mechanism described herein can be applied in various ways. For example, the proposed external attention mechanism can be used in any NN model for video encoding and decoding, such as NN-based intra / inter prediction, NN-based super-resolution, NN-based motion compensation, NN-based reference generation, NN-based fractional pixel interpolation, NN-based loop / post-filtering, etc.

[0327] a. In one example, an external attention mechanism is used for NN-based loop filtering.

[0328] Example 3

[0329] 3. In the third embodiment, the NN model depth can be specified in various ways. For example, the depth of the NN model (e.g., the number of residual blocks) can be predefined or derived in real time.

[0330] a. In one example, the derivation may depend on codec information such as temporal layer, slice / picture type, reference picture list information, characteristics of the video unit in which the NN model is applied, color components, color format.

[0331] b. In one example, the depth can be adaptive to different video units within one picture / slice.

[0332] c. In one example, the depth may be adaptive to two video units in two different pictures / slices.

[0333] d. In one example, the depth of the NN model can be the same or different for video units in different temporal layers, different types of slices, different slices or different pictures, or any video unit.

[0334] e. In one example, the depth of the NN model may be the same for video units in different temporal layers or different types of slices.

[0335] f. In one example, a different network depth is used for each temporal layer or each type of slice.

[0336] i. In one example, the depth of the NN model is smaller for video units in higher temporal layers.

[0337] 1. In one example, the NN model for the higher time domain layer includes Fig. 12A The number of residual blocks is shown. For a higher temporal layer (eg, relative to a lower temporal layer), the number of residual blocks is 8.

[0338] ii. In one example, for video units in inter-frame slices, the depth of the NN model is smaller.

[0339] 1. In one example, the NN model for inter-frame strips includes Fig. 12A Multiple residual blocks are shown. The number of residual blocks is 8.

[0340] iii. In one example, for video units in intra slices, the depth of the NN model is larger.

[0341] 1. In one example, the NN model for intra-frame strips includes Fig. 12A Multiple residual blocks are shown. The number of residual blocks is 32.

[0342] g. In one example, the time domain layers are grouped into multiple subgroups. For each subgroup, a network depth is used.

[0343] Example 4

[0344] 4. In a fourth embodiment, an indication of the depth of the NN model of the video unit depth may be signaled to the decoder.

[0345] a. In one example, an indication of at least one depth of the NN model may be signaled in a SEI message / SPS / PPS / APS / VPS / picture header / slice header / CTU, etc.

[0346] b. In one example, multiple depths of the NN model may be signaled to the decoder.

[0347] c. In one example, the depth of the NN model for luma and the depth of the NN model for chroma may be signaled separately to the decoder.

[0348] Example 5

[0349] 5. In a fifth embodiment, the NN model may additionally take or receive segmentation information of video units as input.

[0350] a. In one example, the partition information is derived by filling each codec unit (CU) with its average value.

[0351] b. In one example, the segmentation information is derived by setting different pixel values ​​for samples near the CU boundary relative to samples within the CU.

[0352] c. In one example, the segmentation information may be represented by an M×N array.

[0353] i. In one example, M and N may represent the width and height of the video unit or image to be encoded.

[0354] ii. In one example, M and N may represent the number of columns and rows of another array, such as a video unit to be encoded and decoded that will be input to the NN model.

[0355] Example 6

[0356] 6. In the sixth embodiment, a shared NN model can be used to process both luma and chroma components. For example, the NN model can be generated based on the luma component and the chroma component of the unfiltered samples.

[0357] a. In one example, the shared NN model may additionally take color component information as input.

[0358] i. In one example, different values ​​are assigned to different color components.

[0359] ii. In one example, different values ​​are assigned to the luma component and the chroma components.

[0360] b. In one example, the component information may be represented by an M×N array.

[0361] i. In one example, M and N may represent the width and height of the video unit or image to be encoded.

[0362] ii. In one example, M and N may represent the number of columns and rows of another array, such as a video unit to be encoded and decoded that will be input to the NN model.

[0363] c. As input, the segmentation information can be represented by numbers.

[0364] i. For example, samples at a block boundary and samples not at a block boundary may be represented by different numbers (such as 0 and 1).

[0365] ii. As input, the color components can be represented as numbers. For example, brightness: 0, Cb: 1, Cr: 2.

[0366] Example 7

[0367] 7. In the seventh embodiment, the output of the NN model may be adjusted or generated based on any information, which may be a signaled syntax element or derived information.

[0368] a. In one example, the NN model may additionally take or receive a quantization parameter (QP) as input. For example, the output of the NN model may be conditioned on the QP value.

[0369] b. In one example, the NN model may additionally take or receive as input the slice type and / or temporal layer identifier (ID) of the video unit. For example, the output of the NN model may be conditioned on the slice type and / or temporal layer ID.

[0370] c. In one example, the NN model may additionally take as input the boundary strength of the video unit or a function based on the boundary strength of the video unit.

[0371] d. In one example, the NN model may additionally employ or receive motion information, such as motion vectors of video units, as input.

[0372] e. In one example, the NN model may additionally take or receive as input a prediction mode of a video unit.

[0373] f. In one example, the NN model may additionally take or receive as input an intra prediction mode of a video unit.

[0374] g. In one example, the NN model may take as input or receive as input the scaling factor of the video unit. For example, the output of the NN model may be conditioned on the scaling factor.

[0375] i. In one example, the scaling factor refers to a factor that scales the difference between the reconstruction and the output of the NN model.

[0376] h. In one example, the information may be represented by an M x N array.

[0377] i. In one example, M and N may represent the width and height of the video unit or image to be encoded.

[0378] ii. In one example, M and N may represent the number of columns and rows of another array, such as a video unit to be encoded and decoded that will be input to the NN model.

[0379] c. As input, information can be represented by numbers.

[0380] Example 8

[0381] 8. In an eighth embodiment, the indication of the NN model may be adaptively binarized.

[0382] a. In one example, the binarization of the NN model may depend on the slice type, and / or color component, and / or temporal layer ID.

[0383] b. In one example, binarization of the NN model can be signaled from the encoder to the decoder.

[0384] Example 9

[0385] 9. In a ninth embodiment, the NN model granularity (e.g., the NN model granularity at which the NN model is applied or the NN model selection is performed) may be signaled in the bitstream or derived in real time.

[0386] a. In one example, an indication of granularity is signaled in a bitstream within a sequence header, a picture header, a slice header, a sequence parameter set (SPS), a picture parameter set (PPS), an adaptation parameter set (APS), or a combination thereof, and the NN filter model granularity specifies the size of a video unit to which the NN model can be applied.

[0387] The first exemplary embodiment is discussed.

[0388] This first exemplary embodiment proposes a convolutional neural network based loop filtering method, in which adaptive model selection is introduced. The proposed deep loop filter (DAM) method with adaptive model selection is developed from the existing contribution JVET-V0100, introducing a new network structure into the code base of VTM-11.0+New MCTF. Compared with VTM-11.0+New MCTF, the proposed method demonstrates a reduced BD rate of {Y,Cb,Cr} under artificial intelligence AI, RA and LDB configurations.

[0389] In a first exemplary embodiment, a convolutional neural network (CNN) is used to construct a loop filter in order to reduce the distortion caused during compression. The network structure is modified from the one discussed in Yue Li, LiZhang, Kai Zhang, "Deep in-loop filter with adaptive model selection", JVET-V0100. Similar to JVET-V0100, residual blocks are used as basic modules and are stacked several times to construct the final network. As a further development from JVET-V0100, an external attention mechanism is introduced in this contribution, resulting in increased representation capabilities with similar model size. In addition, in order to handle different types of content, separate networks are trained for different types of stripes and quality levels.

[0390] The first exemplary embodiment Fig. 12A and Fig. 12B In order to improve the architecture, we now introduce Fig.16A and Fig. 16B , which includes an external attention mechanism. Fig.16A is a schematic block diagram of an architecture 1600 of a NN filtering method according to various examples, and Fig. 16B is a diagram showing various examples of Fig.16A Schematic block diagram of the construction of the attention residual block 1650 used in the architecture 1600.

[0391] Except for the attention residual block 1650, the other parts of the architecture 1600 are the same as those in JVET-V0100. The computation process in the attention module 1650 can be written as:

[0392] F_out=F_in×f(Rec,Pred)+F_in

[0393] Wherein, F_in and F_out represent the input and output of the attention module 1650, respectively. Rec and Pred represent reconstruction and prediction, respectively. In this example, f includes 2 convolutional layers, where an activation function is applied after the first convolutional layer. The purpose of f is to generate a spatial attention map from external information, which then recalibrates the feature map F_in.

[0394] In the proposed technology of the first exemplary embodiment, each slice or block can determine whether to apply a CNN-based filter. When a CNN-based filter is determined to be applied to a slice / block, it can be further decided which model from a candidate model list including three models. To this end, each model is trained with a QP in {17, 22, 27, 32, 37, 42}. Given a test QP for a sequence, denoted as q, the candidate model list includes models trained with a QP equal to {q, q-5, q-10}. The selection process is based on the rate-distortion cost on the encoder side. If necessary, an indication of the on / off control and the model index are signaled in the bitstream. Additional details about model selection are provided as follows.

[0395] The granularity of filter determination and model selection depends on the resolution and QP. In the proposed scheme, given a higher resolution and a larger QP, the determination and selection will be performed for a larger area.

[0396] The candidate model list is different on different temporal layers. For the low temporal layer, if the ratio of intra-frame codec samples is not less than the threshold, the third candidate is replaced by the intra-slice NN filter model. For the high temporal layer, the third candidate, the model corresponding to q-10, is replaced by the model corresponding to q+5.

[0397] For full intra-frame configurations, model selection is disabled while still maintaining on / off control.

[0398] Open source machine learning frameworks such as PyTorch are useful for performing inference of the proposed CNN filters in VTM. As suggested by S. Liu, A. Segall, E. Alshina, R.-L. Liao., "JVET common test conditions and evaluation procedures for neural network-based video coding technology", JVET-V2016, the network information in the inference phase is provided in Table 1-1.

[0399] Table 1-1. Network information used for NN-based video codec tool testing in the inference phase

[0400]

[0401] PyTorch can also be used as a training platform. DIV2K (from https: / / data.vision.ee.ethz.ch / cvl / DIV2K / , R.Timofte, E.Agustsson, S.Gu, J.Wu, A.Ignatov, LVGool) and BVI-DVC (from Ma, Di, Fan Zhang, and David R.Bull. "BVI-DVC: "ATraining Database for Deep Video Compression", arXiv preprint arXiv:2003.13552(2020)) datasets are adopted to train CNN filters for I and B bands respectively. Different CNN models are trained to adapt to different QP points. The network information in the training phase is provided in Table 1-2, as shown in "JVET common test conditions and evaluation procedures for neuralnetwork-based video coding" by S.Liu, A.Segall, E.Alshina, R.-L.Liao. technology (JVET Common Test Conditions and Evaluation Procedures for Neural Network-Based Video Codec Technology), as recommended by JVET-V2016.

[0402] Table 1-2. Network information used for NN-based video codec tool testing during the training phase

[0403]

[0404] The proposed CNN-based loop filtering method is tested on VTM-11.0 + new MCTF (from https: / / vcgit.hhi.fraunhofer.de / jvet / VVCSoftware_VTM / - / tags / VTM-11.0) according to the common test conditions recommended by S.Liu, A.Segall, E.Alshina, R.-L.Liao. in “JVET common test conditions and evaluation procedures for neuralnetwork-based video coding technology, JVET-V2016”. The new MCTF patches are from https: / / vcgit.hhi.fraunhofer.de / jvet-ahg-nnvc / nnvc-ctc / - / tree / master / Software%20Patches.

[0405] Deblocking filtering and SAO are disabled, while ALF (and CCALF) are placed after the proposed CNN-based filtering. The experimental results are shown in Tables 1-3 to 1-5. Under the AI ​​configuration, the proposed method can reduce the BD rates of Y, Cb, and Cr by an average of 9.12%, 22.39%, and 22.60%, respectively. Under the RA configuration, the proposed method can reduce the BD rates of Y, Cb, and Cr by an average of 12.32%, 27.48%, and 27.22%, respectively. Under the LDB configuration, the proposed method can bring an average %, % and % BD rate reduction for Y, Cb, and Cr, respectively.

[0406] Table 1-3. Execution of the proposed method on VTM11.0 + new MCTF (RA)

[0407]

[0408] Table 1-4. Execution of the proposed method on VTM11.0 + new MCTF (LDB)

[0409]

[0410] Table 1-5. Execution of the proposed method on VTM11.0 + new MCTF (AI)

[0411]

[0412] The first exemplary embodiment provides a CNN-based loop filtering method. The proposed CNN-based filtering method shows useful codec gain.

[0413] A second exemplary embodiment is discussed.

[0414] With the rapid development of NN-based machine learning algorithms, deep learning methods are experimentally used for a much wider range of artificial intelligence applications than well-known artificial intelligence applications such as facial recognition or autonomous driving. Recently, deep learning models have been intensively studied to improve the compression efficiency of video codecs, especially in the loop filtering stage. Although the loop filtering methods based on deep learning in the prior art have shown significant potential in video codecs, the content propagation problem is still not well understood and solved.

[0415] Content propagation is the fact that the content of reference frames is propagated to frames that reference them, which often leads to over-filtering problems. In some examples, an iteratively trained deep loop filter (iDAM) with adaptive model selection solves the content propagation problem. First, an iterative training scheme may be useful, which enables the network to gradually consider the impact of content propagation. Second, a filter selection mechanism may be useful, such as by allowing blocks to select from a set of candidate filters with different filtering strengths. In addition, one approach is to design a conditional loop filtering method that can handle multiple quality levels with a single model and provide the functionality of filter selection by modifying input parameters. Extensive experiments have been conducted to evaluate the proposed techniques on the latest video codec standards (e.g., Versatile Video Codec, VVC). Compared to the VVC reference software VTM-11.0, certain examples described herein implement new state-of-the-art techniques, resulting in average reductions in BD rates of {7.91%, 20.25%, 20.44%}, {11.64%, 26.40%, 26.50%} and {10.97%, 26.63%, 26.77%} for {Y, Cb, Cr} in all-intra, random access and low-latency configurations, respectively. The proposed iDAM scheme provides improved codec performance over existing solutions. In addition, the syntax elements of the proposed scheme were adopted at the 76th meeting of the Audio Video Codec Standard (AVS).

[0416] Various examples described include adaptive model selection, convolutional neural networks, conditional loop filtering, deep loop filtering, iterative training, and VVC.

[0417] Lossy video codecs cause distortion in the reconstructed image. This distortion is detrimental in two ways. First, it directly affects the compression efficiency by reducing the objective quality as well as the subjective quality of the reconstructed image. Second, it indirectly reduces the compression efficiency by weakening the correlation between the distorted frame and the frame that references the distorted frame. To reduce the distortion, loop filters are integrated to improve the compression efficiency as well as improve the subjective quality in video codecs.

[0418] The latest video codec standard (ie, VVC) includes three loop filters, called deblocking filter, sample adaptive offset (SAO) and adaptive loop filter (ALF). The deblocking filter is designed to reduce the blocking effect caused by block-based segmentation, while SAO and ALF aim to mitigate reconstruction artifacts due to quantization.

[0419] Although loop filters in VVC can reduce compression artifacts, those handcrafted filters are not sufficient to accommodate complex natural videos. Learning-based filtering has been considered, which can be classified into out-of-loop filtering (also called post-processing or post-filtering) and loop filtering, where in the former scheme, the filtered frame cannot be referenced by the next frame, while in the latter scheme, the filtered frame can be referenced by the next frame.

[0420] Post-filtering for image codecs has been considered, especially JPEG. For example, a 4-layer convolutional neural network (CNN) for compression artifact reduction has been proposed, which has achieved an improvement of more than 1dB in PSNR compared to JPEG. Post-filtering for video codecs has also been proposed, such as High Efficiency Video Codec (HEVC), which is a video codec standard before VVC. It is possible to improve the quality of a frame by utilizing its neighboring frames with higher quality frames. In addition, loop filtering has several advantages over post-filtering, including a guaranteed quality level from a content provider, potential savings in frame buffers, improved objective and subjective quality due to the availability of better references, and reduced decoding complexity due to fewer residuals. However, because the filtered frame will act as a reference frame and will affect the compression of subsequent frames, loop filtering based on learning filters integrated into the codec loop may be more challenging. It has been considered to incorporate a deep network into the loop filtering stage. For example, it is possible to train a 3-layer CNN as a loop filter for HEVC. To handle different quality levels during the test phase, two models are trained, corresponding to the low QP range 20-29 and the high QP range 30-39, and one model is selected for each frame based on the frame QP. During testing, CNN-based filtering is activated based on the picture order count (POC) or frame level cost. Attention-based CNN for loop filtering of VVC is possible. To prevent the situation where the CNN filter cannot generate improved results, a block level on / off control mechanism can be introduced.

[0421] While deep learning based loop filtering methods have improved the potential capabilities in video codecs, they still face a major obstacle - the content propagation problem. Content propagation is an inherent feature of video compression and usually means that the content of a reference frame will be propagated to the frames that refer to it. For example, when skip mode is used, the current block is predicted from the previously coded reference block without signaling the additional residual. If a deep filter is applied to both the reference block and the current block, it is equivalent to applying the filter twice, resulting in over-filtering of the current block. If the content propagation problem is not solved, it will hinder the performance of deep loop filters. For example, some existing methods achieve much lower significant gains under random access configurations.

[0422] In some examples described herein, an iteratively trained deep loop filter (iDAM) with adaptive model selection can be used to address the content propagation problem in video coding and decoding. First, iterative training can enable the network to gradually consider the impact of content propagation. By iteratively generating training samples, the filtering applied to the reference frame is implicitly considered during the training process. Second, the filter selection strategy enables the block to select a suitable filter from multiple NN-based filters with different filtering strengths. Considering that filters trained at low quality levels need to compensate for large distortions more, and vice versa, filters with different filtering strengths are obtained by reusing models trained at multiple quality levels. In addition, conditional loop filters can be trained by feeding auxiliary parameters based on QP into the NN. Conditional filters can handle multiple quality levels with a single model. In addition, conditional filters equipped with parameter selection can also provide the function of filter selection and can provide comparable performance compared to individual filters trained according to each quality level.

[0423] Experiments demonstrate the effectiveness of the proposed techniques. Specifically, VTM-11.0 is a platform for evaluating the potential of the proposed techniques. Experimental results show that some of the examples described herein achieve meaningful improvements, such as average reductions in BD rates of {7.91%, 20.25%, 20.44%}, {11.64%, 26.40%, 26.50%} and {10.97%, 26.63%, 26.77%} for {Y, Cb, Cr} in full-frame, random access and low-latency configurations, respectively. These represent relatively large codec gains achieved by a single deep tool on top of VVC. In addition, the effectiveness of iterative training and filter selection is verified in simulations. Through the various techniques proposed, these examples rank highly among NN-based techniques in all configurations of the 21st and 22nd meetings of the Joint Video Detection Team (JVET). In addition, the grammatical elements of the proposed scheme were adopted at the 76th meeting of AVS.

[0424] The development of filtering improvements is useful for video codec technology. Generally, filtering methods in video codecs can be classified into two categories, loop filtering, where the filtered picture can be used to predict subsequent pictures, and out-of-loop filtering (also called post-filtering), where the filtered picture cannot be used to predict subsequent pictures. Loop filtering is useful because it can be part of the video codec standard. Loop filters such as deblocking filters (DF), sample adaptive offset (SAO), and adaptive loop filters (ALF) have been developed for decades as video codec standards have evolved.

[0425] Filtering methods play an important role in video coding due to some inherent weaknesses of hybrid video coding schemes. Modern video coding standards such as AVC, HEVC, and VVC are built on a hybrid codec structure using block-based prediction and transform coding. Although the hybrid codec structure has been proven to be an efficient practical coding scheme, it leads to several unavoidable artifacts. On the one hand, the block-based transform used for intra- and inter-frame prediction error coding may lead to discontinuities at block boundaries due to the coarse quantization of transform coefficients.

[0426] In addition, block-based inter-frame prediction, i.e., copying interpolated samples from different locations of possibly different reference frames, may further worsen boundary discontinuities. On the other hand, quantization errors and low-pass fractional interpolation filters may introduce ringing artifacts. In order to attenuate boundary artifacts, DF has been introduced into AVC. DF can detect discontinuities based on quantization steps, codec modes, motion information, and sample values ​​along block boundaries. When discontinuities are found, samples along the boundaries are processed by nonlinear filtering with predetermined low-pass characteristics. In order to reduce ringing artifacts, HEVC and VVC have adopted SAO. SAO can reduce the average distortion of a region by first classifying the samples of the region into multiple categories with a selected classifier, obtaining an offset for each category, and then adding the offset to each sample of the category, wherein the classifier index and the offset of the region are signaled in the bitstream. ALF is also considered to be related to HEVC and was eventually incorporated into VVC. On DF and SAO, ALF attempts to capture and repair artifacts caused by the previous stage through a Wiener-based adaptive filter. The appropriate filter coefficients for different regions are determined by solving the Wiener-Hopf equation at the encoder and signaled to the decoder.

[0427] DF, SAO and ALF are designed from the perspective of signal processing under the assumption that the signal is stationary. However, the complex characteristics of natural videos may make those filters suboptimal. Note that although ALF can be considered as a learning-based filter, its one-layer structure may limit the efficiency. In order to take advantage of the powerful learning ability of deep neural networks and the rich external datasets that can be obtained relatively easily, deep learning-based out-of-loop filtering and loop filtering are proposed.

[0428] Traditionally, the focus of depth-based filtering has been on post-filtering for image codecs, especially JPEG. For example, a 4-layer CNN has been introduced for compression artifact reduction, namely Artifact Reduction CNN (ARCNN). ARCNN provides an improvement of more than 1dB in PSNR compared to JPEG. A 20-layer CNN trained with residual learning has been proposed, resulting in better performance than ARCNN. In order to better reduce compression artifacts, dual-domain processing has also been considered. For example, a method is proposed in which very deep convolutional networks are jointly learned in both the DCT (discrete cosine transform) and pixel domains. The dual-domain representation can make full use of the DCT domain prior knowledge of JPEG compression, which is usually ignored in traditional network-based methods. Advanced loss functions for achieving better visual quality are also considered. In addition, it is considered how to use one model to handle general image restoration tasks, such as JPEG post-filtering, denoising, super-resolution, etc.

[0429] Research has been conducted on out-of-loop filtering in video codecs, especially HEVC and VVC. For example, a 4-layer CNN (VRCNN) featuring variable filter size and residual connections has been proposed for post-filtering of HEVC intra frames. In full intra configuration, VRCNN can achieve an average of 5% bit rate reduction compared to HEVC. A 10-layer CNN for out-of-loop filtering may be useful. During training data generation, the transform unit size is taken into account for balancing the training samples. In some cases, this can provide more than 5% bit rate reduction in all configurations.

[0430] Training different CNN models for intra frames and inter frames can provide benefits. Unlike the examples that consider a single frame, some examples study how to collect information from multiple frames for quality enhancement. For example, a memory-based bidirectional long-term short-term detector may be developed to locate peak quality frames. Then, a CNN-based multi-frame enhancer takes the current frame and its two closest peak quality frames as input. In order to align the motion of multiple frames, a motion compensation network is integrated. Although the above examples only use decoded frames as input, some examples can feed auxiliary information such as segmentation, residuals, etc. In order to cover multiple quality levels using a single model, both decoded frames and quantization parameters can be input into the CNN. In addition, model quantization and fixed-point calculations can be performed to ensure consistency between different computing platforms.

[0431] Although using CNN as a post-filter can avoid the interaction with the compression process and thus simplify the design, it also loses several advantages of the above-mentioned loop filtering.

[0432] It is more challenging to integrate CNN-based filters into the codec loop because the filtered frames will act as references and will interact with other codec tools. It is possible to train a 3-layer CNN as a loop filter for HEVC. In order to handle different quality levels during the test phase, two models are trained, corresponding to the low QP range 20-29 and the high QP range 30-39, and one model is selected for each frame based on its QP. CNN replaces SAO as the loop filter. During testing, CNN-based filtering can be activated based on the picture order count (POC) or frame level cost. For low latency and random access configurations, such proposed filters can achieve bit rate savings over HEVC. Consider a network characterized by multi-channel and long-term-short-term correlation for loop filtering. Multi-channels are able to resolve the illumination difference between the average amplitude of the predicted and real images. The network takes decoded frames and segmentation information as input and is trained with mixed loss multi-scale structural similarity index measure (MS-SSIM). The proposed scheme can achieve more than 8% and 7% BD rate savings in low latency and random access configurations, respectively. Another network for loop filtering in HEVC can be built on the residual highway unit. In order to maximize performance, different models are trained for different types of frames. In order to consider the impact of quality level, a separate model is trained for each QP range. Regarding the test results, the proposed method saves about 4% of BD rate over HEVC under random access configuration. However, it should also be noted that the advantage of the proposed deep filter over ALF is not obvious. A dense residual network including multiple dense residual units for loop filtering in HEVC can be considered. Different models are trained for different QPs. A deep CNN for loop filtering in HEVC can be considered. The impact of analyzing network depth and connection units is performed to design an efficient network. Multiple models can be trained in an iterative manner based on the distortion of the filtered samples. In order to avoid the overhead for signaling model indexes, a classification network is further trained to predict the best model for each block. The test results report an average BD rate reduction of 4.1%, 6.0%, and 6.0% under full-frame, low-latency, and random access configurations, respectively.

[0433] Although the above-mentioned deep filters are designed for HEVC, some examples include loop filtering for VVC. Since VVC achieves higher quality than HEVC at the same bit rate and additionally includes an adaptive loop filter, designing deep filters for VVC is more challenging. CNN can be built on the attention block for loop filtering of VVC. The block-level on / off control mechanism avoids situations where the CNN filter may generate bad or unexpected outputs. Experimental results show that the proposed filter can save 6.54% and 2.81% of BD rate in full-frame and random access configurations, respectively.

[0434] Some examples include a CNN architecture featuring multi-level feature review and residual dense blocks for post- and loop filtering of HEVC and VVC. The proposed loop filtering and out-of-loop filtering reportedly show 4.6% and 6.7% BD rate savings, respectively, over VVC for random access testing.

[0435] Some examples also include multiple dense attention networks for VVC loop filtering. Multi-scale features are extracted using multi-resolution convolutional flows, sample correlations are learned using a single attention branch, and information from multiple branches is finally fused. The proposed network achieves a 6.57% BD rate reduction under random access configuration. The recent JVET conference focused on depth-based loop filtering for VVC. For example, a deep filter with a kernel that includes boundary strength as input, residual scaling applied to the network output, and a wide convolutional layer is used. The deep filter achieves 8.09% and 8.28% BD rate savings under low latency and random access configurations.

[0436] Depth-based loop filtering has attracted attention, however most of the focus is on designing advanced network architectures or handling the changing characteristics of video samples. At the same time, the content propagation problem is rarely studied or considered an important issue, resulting in limited performance of depth-based loop filtering. The examples described in this paper take advantage of a careful treatment of this problem, which enables the potential of improving depth-based loop filtering.

[0437] Fig.17 is an example 1700 of a hierarchical codec structure for a random access setting, which is adopted by most video codec standards. The POC or picture order count determines the display (output) order of the decoded frames, while the codec order determines the order in which those frames are encoded. In such a codec structure, except for pictures POC 0 and POC 8 at temporal layer 0, each picture will have both forward and backward reference pictures. As this prediction process progresses, the content is propagated from the reference frame to the frames that reference it, such as Fig.17 As shown by the arrows in , such a reference structure can provide more efficient encoding and decoding performance. Fig.17 The temporal layer identifier (ID) of each frame is also shown.

[0438] Frames at lower temporal layers are encoded earlier (typically at higher quality) and used as references for the remaining frames. Therefore, frames at higher temporal layers can be well predicted and are more likely to choose skip mode. It is worth noting that in a random access setting, the encoded / decoded frames after a random access point (RAP) are independent of any pictures that appeared earlier, so the content is spread between two adjacent random access points.

[0439] Unlike random access settings, in low latency configurations, pictures are encoded and decoded in the order in which they are displayed. In addition, the temporal prediction process can continue until all frames are decoded. In other words, in such settings, content can be propagated for a longer period of time.

[0440] Content propagation is basically unavoidable when exploiting temporal redundancy in video codecs. However, content propagation can significantly deteriorate the efficiency of in-loop depth-based filtering. With content propagation, samples filtered in one frame can reach the next frame, where they can be filtered again.

[0441] As the transmission extends, the over-filtering phenomenon will become more and more serious. Fig.18 A- Fig.18 An example is shown in C. In this example, the sequence is encoded and decoded with VTM-11.0 using the low-latency B profile at QP 37. CNN-based loop filtering is applied to the reconstructed frames. Fig.18 A shows the original frame from the "Blow a Bubble" sequence. Fig.18 B shows the filtered result without iterative training, where over-filtering occurs on the wall and face regions as the POC increases. Fig.18 C shows the filtering results of iterative training, where both objective and subjective quality are improved, especially as POC increases.

[0442] To solve the content dissemination problem, the examples described in this article include a method of iterative loop filtering (iDAM) using an adaptive model. The following provides details of the design of iDAM, including the framework of iDAM, network structure, training process and filter selection.

[0443] Deep neural networks excel at learning hierarchical representations that are critical to solving specific tasks. In order to give the network maximum freedom to decide what to learn, an end-to-end formulation has been emphasized. Given this, the examples described in this paper place the proposed filters after the initial reconstruction, i.e., filters that are not processed by existing handcrafted filters. Fig.19 An example 1900 of a hybrid video coding scheme including the proposed filter according to an example of the present specification is illustrated. Fig.19As shown, the unfiltered reconstruction and the predicted samples are input to the example network 1900. Note that in the prior art, prediction is rarely used as auxiliary information for loop filtering, however it can be useful, especially for inter-frame strips. The benefit of using prediction is twofold. First, inter-frame prediction contains temporal information, making it equivalent or similar to having multiple frames as input. In addition, reusing inter-frame prediction can avoid an additional motion compensation process that would otherwise be necessary to align multiple frames. Second, inter-frame prediction indirectly helps solve the content propagation problem because the prediction, together with the reconstruction, can reflect the proportion of content propagated from the reference frame and serve as a clue for the network to learn the appropriate filtering strength.

[0444] like Fig.19 As shown, the output of the proposed filter will be processed by ALF and then put into the decoded picture buffer as a reference. DF and SAO are not used in this embodiment because in some cases, no additional benefits are observed from the experimental results.

[0445] Fig. 20 The proposed CNN 2000 is provided for filtering the luminance component in one example. Fig. 20 In , N is the spatial width (height), M is the number of feature maps, and R is the number of residual blocks. When the spatial width or resolution changes, this is shown at the bottom or below the feature map. Fig. 20 The CNN 2000 consists of three parts: input part, backbone part and output part.

[0446] The input is responsible for aggregating information from both reconstruction and prediction. To this end, separate features are extracted from each of the reconstructed and predicted frames, the feature maps are concatenated together along the channel dimension, and then reduced through a 1×1 convolutional layer for information aggregation and complexity reduction.

[0447] The network backbone consists of a convolutional layer and multiple sequentially stacked residual blocks. In order to increase the receptive field and save computation, the stride of the first layer can be set to 2. The use of residual blocks enables faster convergence.

[0448] The number of feature maps (M) and residual blocks (R) controls the network size. Currently, M and R are empirically set to 96 and 32, respectively. The output first maps the latent representation from the backbone to four sub-images, which are then shuffled to produce a high-resolution reconstruction.

[0449] Traditional tools, such as cross-component linear models (CCLM) and cross-component adaptive loop filters (CCALF), use luma information for chroma prediction and can provide benefits. Examples of this specification can feed luma information for loop filtering of chroma. Considering that the resolution of luma is higher than chroma in the YUV 4:2:0 format, features are first extracted from luma and chroma separately, and then the luma feature map is downsampled before concatenating it with the chroma feature map. With respect to the network backbone, the chroma components share the same as the luma components.

[0450] Fig.21A This is an example of ordinary or traditional training 2100, in which the VTM encoder, which is the reference software of VVC, is used to compress the training images (or videos) to obtain distortion reconstruction and other auxiliary information. This information is then paired with the corresponding ground truth to form training samples. When the trained models are integrated for testing, their effectiveness may be degraded due to the statistical shift between the training samples and the test samples. The reasons for the statistical shift are further described below.

[0451] First, content propagation is a major factor. Since CNN-based filtering is only applied at test time, the quality of reference frames becomes higher at test time than during training.

[0452] After inheriting the content from the reference frame, the quality of the current frame is also improved. Ignoring that the current frame may have been indirectly affected by loop filtering on the previous frame, one direct result is over-filtering.

[0453] Second, the improved quality in the reference frame can change the optimal segmentation structure of the current frame. For example, the encoder can prefer larger segmentation blocks. Since compression artifacts (e.g., blocking artifacts) are highly correlated with the segmentation structure, they will also be changed.

[0454] Third, improved quality in the reference frame means improved quality in the prediction samples.Since the prediction is used as side information during loop filtering, the trained model may behave in unexpected ways.

[0455] To address these issues, embodiments of the present specification include an iterative training scheme, which includes: Fig. 21B The three stages are shown. Fig. 21B is an example 2150 of iterative training according to an example of the present specification, wherein a VTM encoder is used to generate training samples in phase I, and then the trained model from the previous phase is integrated into the VTM for generating training samples in a subsequent or next phase. Fig.21A and Fig. 21B, the line labeled CNN_Intra indicates the integration of the intra-frame strip model, while the line labeled CNN_Inter indicates the integration of the inter-frame strip model.

[0456] In the training phase I, the VTM encoder compresses the training images in full intra configuration. The reconstructed images are used together with other side information to train the intra frame filters.

[0457] In training phase II, the training video is compressed using a random access setting, where intra frames are processed by the filters obtained in phase I. Note that since intra frames are compressed with higher quality, they will be more likely to be referenced by other frames. Therefore, the effect of CNN-based loop filtering will be reflected in all frames. By collecting training samples from phase II, the impact of filtering applied to reference frames is taken into account. After completing training phase II, filters for intermediate frames are obtained. However, directly deploying those filters may still lead to suboptimal results because the intermediate frames themselves are processed by CNN-based filtering in testing rather than in training.

[0458] To further reduce the gap, a training phase III is provided. Specifically, the training video is compressed again under a random access configuration, where intra frames are processed by the filters obtained in phase I, and intermediate frames are processed by the filters obtained in phase II. Then, training samples are collected to train the final inter-frame filter. Note that in phase III, unfiltered reconstructions are extracted for training instead of reconstructions filtered by CNN-based loop filtering. The purpose of applying CNN-based loop filtering is to make the statistics of the reference frames as close to their final states as possible.

[0459] It can be inferred that content propagation varies with the time domain layer. For example, frames at higher time domain layers are affected by more content propagated from previously encoded and decoded frames. At the same time, if blocks are not encoded and decoded by skip mode in high time domain layers, their transform coefficients are coarsely quantized, resulting in larger distortion. Therefore, frames at different time domain layers may prefer different filtering strengths. In addition, different areas of the frame with various contents may also require different filter strengths, which various contents can be encoded and decoded in different modes or predicted from different reference frames. To this end, an embodiment of the present specification includes a filter selection mechanism that provides candidate filters of different filtering strengths to be selected for a particular block. Without loss of generality, in one example, the candidate list may contain three models. Note that more candidates can improve performance, but potentially at the expense of higher coding complexity.

[0460] Fig.22AAn encoder strategy 2200 designed for the above filter selection scheme is shown. All blocks in the current frame need to be processed by three models first. Then, five costs (e.g., Cost_0 to Cost_4) are calculated and compared with each other to achieve the best rate-distortion performance. In Cost_0, CNN-based filters are disabled for all blocks. In Cost_i, where i=1, 2, 3, the i-th CNN-based filter is used for all blocks. In Cost_4, different blocks may prefer different models, and information about whether to use a CNN-based filter or which filter to use is signaled for each block.

[0461] Fig. 22B The associated decoder strategy 2250 is shown. On the decoder side, whether to use a CNN based filter or which filter to use for a block is based on the Model_Id parsed from the bitstream, as Fig. 22B Note that the filter selection does not impose any additional burden on the decoder compared to the single filter based scheme.

[0462] Considering that filters trained at low quality levels need to compensate more due to distortion, and vice versa, filters of different filtering strengths are obtained in the present specification by reusing models trained at different quality levels. In this way, training additional models for filter selection can be avoided. Regarding the granularity of filter selection, the granularity can depend on the resolution and the bit rate. For higher resolutions, the granularity will be coarser because the content tends to change more slowly. For higher bit rates, the granularity will be finer because more overhead can be provided, or the overhead is available.

[0463] In order to handle multiple quality levels and implement filter selection, multiple filters are trained corresponding to different quantization parameters (QP) at the sequence level. However, it requires a larger memory to store multiple models. In addition, when processing different blocks within a frame, filter selection requires model switching, which may be unfriendly to hardware implementations. To address these issues, an embodiment of the present specification may include a conditional loop filter 2300 that can use auxiliary input parameters to adjust the filter strength. In this way, a single model can handle multiple quality levels. In addition, the conditional filter can use parameter selection to mimic the model selection mechanism, thereby avoiding model switching.

[0464] like Fig.23A As shown, the overall structure of the conditional loop filter 2300 is similar to Fig. 20The structure shown in , but with two exceptions on the input and backbone, respectively. First, there is an additional input plane derived by flattening the QP into a 2-D QP map. Second, the QP is used as external attention. Specifically, the QP is expanded to a 1-D vector with multiple fully connected layers, as Fig. 23B As shown in Example 2350 of . This vector is then used as a channel-by-channel scaling factor to recalibrate the convolutional feature map. To reduce complexity, no external attention is added to each residual block. Instead, the residual blocks are evenly partitioned into four groups, and the output of each group is multiplied by the attention vector. In this example, two fully connected layers are used at each position.

[0465] Traditionally, the use of QP is not sufficient to effectively utilize the information indicated by the QP value. Therefore, the embodiments of the present specification additionally use QP as external attention. One of the basic principles behind this design is that QP has a high correlation with the distortion level and is therefore useful for controlling the amplitude of the intermediate feature map.

[0466] The proposed method makes the loop filtering process conditional on the QP both in terms of input and internal calculations. To process a specific quality level, the corresponding QP is input to the filter. To facilitate filter selection, filters with different filtering strengths are constructed by modifying the QP to the corresponding value.

[0467] A third exemplary embodiment is discussed.

[0468] This embodiment describes a CNN-based loop filtering method, in which adaptive model selection is implemented.

[0469] Fig.24A The backbone of the proposed CNN filtering method is shown in Fig.16A , but the number of residual blocks is equal to 8, instead of Fig.16A 32 of . In order to increase the receptive field and reduce the complexity, the proposed method contains a convolutional layer with a stride of 2 at the beginning. After passing through this layer, the spatial resolution of the feature map is reduced to half of the input size in both horizontal and vertical directions. The feature map output from the first convolutional layer then passes through several sequentially stacked residual blocks. The last convolutional layer takes the feature map from the last residual block as input and produces 4 feature maps of N×N. Finally, a shuffle layer is adopted to generate a filter image with the same spatial resolution as the input of the CNN (i.e., 2N×2N). Other details related to the network architecture are described below.

[0470] For all convolutional layers, a kernel size of 3 × 3 is used. For the inner convolutional layers, the number of feature maps is M (M = 96). For the activation function, PReLU is used.

[0471] Fig. 24B The structure of Res Block is shown in the figure, where the calculation process in the attention module can be written as F_out = F_in × f(Rec, Pred, QP) + F_in.

[0472] Different groups of models are trained for band I and band B respectively.

[0473] When training the CNN filters for intra slices, prediction, segmentation, boundary strength and QP information are additionally fed into the network.

[0474] When training the CNN filters for inter-frame strips, prediction, boundary strength and QP information are additionally fed into the network.

[0475] In the proposed embodiment, each slice or block can determine whether to apply a CNN-based filter. When a CNN-based filter is determined to be applied to a slice / block, the filter model can be decided from a candidate model list including three models. To this end, separate models are trained with QPs in {17, 22, 27, 32, 37, 42}. Given a test QP for encoding and decoding a sequence, denoted as q, the candidate model list includes models trained with QPs equal to {q, q-5, q-10}. The selection process is based on the rate-distortion cost on the encoder side. If necessary, an indication of the on / off control as well as the model index can be signaled in the bitstream.

[0476] In one example, the granularity of filter determination and model selection depends on the resolution and QP. In the proposed scheme, given a higher resolution and a larger QP, NN model determination and selection will be performed for a larger area.

[0477] The list of candidate models is different on different temporal layers. For the low temporal layer, if the ratio of intra-coded samples is not less than the threshold, the third candidate is replaced by the intra-strip NN filter model. For the high temporal layer, the third candidate, the model corresponding to q-10, is replaced by the NN model corresponding to q+5.

[0478] For full intra-frame configurations, model selection can be disabled while still maintaining on / off control.

[0479] When the proposed NN filter is applied to the reconstructed picture, the scaling factor can be signaled in the picture header for each color component. The difference between the input samples and the NN filtered samples (residual) can be scaled by the scaling factor before being added to the input samples.

[0480] A fourth exemplary embodiment is discussed.

[0481] An example syntax related to CNN-based loop filters is provided. The following Table 4-1 shows an example slice header syntax.

[0482] Table 4-1. Stripe Header Syntax

[0483]

[0484] Table 4-2 below shows an example codec tree unit syntax.

[0485] Table 4-2. Codec tree unit syntax

[0486]

[0487]

[0488]

[0489] A fifth exemplary embodiment is discussed.

[0490] An example syntax related to a CNN based loop filter is provided. Table 5-1 below shows an example sequence parameter set RBSP syntax.

[0491] Table 5-1. Stripe Header Syntax

[0492]

[0493] Table 5-2 below shows an example stripe header syntax.

[0494] Table 5-2. Stripe Header Syntax

[0495]

[0496]

[0497] Table 5-3 below shows an example codec tree unit syntax.

[0498] Table 5-3. Codec tree unit syntax

[0499]

[0500]

[0501]

[0502] Semantics

[0503] sps_cnnlf_enabled_flag equal to 1 specifies that the NN-based filter is enabled for CLVS. sps_cnnlf_enabled_flag equal to 0 specifies that the CNNLF is disabled for CLVS.

[0504] sps_cnnlf_infer_size_base specifies the base inference block size for NN-based filters of CLVS.

[0505] The variable cnnlfInferSize[3] is set equal to {sps_cnnlf_infer_size_base>>1,sps_cnnlf_infer_size_base,sps_cnnlf_infer_size_base<<1}.

[0506] sps_cnnlf_infer_size_entension specifies the size of the extended samples at the boundary of the NN inference block of CLVS.

[0507] sps_cnnlf_max_num_params specifies the maximum number of NN filter parameter candidates for each block of CLVS.

[0508] slice_cnnlf_mode[0] equal to 0 indicates that the NN filter will not be used for the luma component of any block in the slice. slice_cnnlf_mode[0] equal to i (i=1, 2, ..., sps_cnnlf_max_num_params) indicates that the i-th NN filter parameter candidate will be used for the luma component of all blocks in the slice. slice_cnnlf_mode[0] equal to sps_cnnlf_max_num_params+1 indicates that the parameter candidate for each block is determined by the parameter index of each block, i.e., cnnlfParamIdx.

[0509] slice_cnnlf_mode[1] equal to 0 indicates that the NN filter will not be used for the chroma components of any block in the slice. slice_cnnlf_mode[1] equal to i (i=1, 2, ..., sps_cnnlf_max_num_params) indicates that the i-th NN filter parameter candidate will be used for the chroma components of all blocks in the slice. slice_cnnlf_mode[0] equal to sps_cnnlf_max_num_params+1 indicates that the parameter candidate for each block is determined by the parameter index of each block, i.e., cnnlfParamIdx.

[0510] slice_cnnlf_scale_flag[paramIdx][chal] equal to 1 specifies that residual scaling is enabled for the parameter candidate indexed by paramIdx for the channel indexed by chal. [paramIdx][chal] equal to 0 specifies that residual scaling is disabled for the parameter candidate indexed by paramIdx for the channel indexed by chal.

[0511] nnScale[paramIdx][chal] specifies the scaling factor for the parameter candidate indexed by paramIdx for the channel indexed by chal.

[0512] useCnnlf equal to 1 specifies that the NN-based filter is enabled for the block. useCnnlf equal to 0 specifies that the NN-based filter is disabled for the block.

[0513] useFirstParam equal to 1 specifies that the first parameter candidate of the NN filter is used for the block. useFirstParam equal to 0 specifies that the first parameter candidate of the NN filter is not used for the block.

[0514] cnnlfParamIdxMinus2 specifies the index of the parameter candidate of the NN filter of the block.

[0515] The variable cnnlfParamIdx[chal][unitIdx] is set equal to cnnlfParamIdxMinus2+2

[0516] numInferUnits[chal] refers to the number of inference blocks for the channel indexed by chal in the stripe and is derived as follows.

[0517]

[0518] Now turn to Fig.25 , a block diagram of an exemplary video processing system 2500 in which various techniques disclosed herein may be implemented is shown. Various implementations may include some or all of the components of the video processing system 2500. The video processing system 2500 may include an input 2502 for receiving video content. The video content may be received in a raw or uncompressed format (e.g., 8 or 10 bit multi-component pixel values), or may be received in a compressed or encoded format. The input 2502 may represent a network interface, a peripheral bus interface, or a storage interface. Examples of network interfaces include wired interfaces, such as Ethernet, a passive optical network (PON), etc., and wireless interfaces, such as Wi-Fi or a cellular interface.

[0519] The video processing system 2500 may include a codec component 2504 that can implement various codecs or encoding methods described in this document. The codec component 2504 can reduce the average bit rate of the video from the input 2502 to the output of the codec component 2504 to generate a coded representation of the video. Therefore, codec technology is sometimes referred to as video compression or video transcoding technology. The output of the codec component 2504 can be stored or transmitted via the connected communication, as represented by component 2506. The stored or transmitted bitstream (or coded) representation of the video received at the input 2502 can be used by component 2508 to generate pixel values ​​or displayable video sent to the display interface 2510. The process of generating a video that a user can watch from the bitstream representation is sometimes referred to as video decompression. In addition, although some video processing operations are referred to as "codec" operations or tools, it will be understood that the codec tool or operation is used at the encoder, and the corresponding decoding tool or operation of the inverse codec result will be performed by the decoder.

[0520] Examples of peripheral bus interfaces or display interfaces may include Universal Serial Bus (USB) or High-Definition Multimedia Interface (HDMI) or DisplayPort, etc. Examples of storage interfaces include Serial Advanced Technology Attachment (SATA), Peripheral Component Interconnect (PCI), Integrated Drive Electronics (IDE) interface, etc. The techniques described in this document may be embodied in various electronic devices, such as mobile phones, laptop computers, smart phones, or other devices capable of performing digital data processing and / or video display.

[0521] Fig.26 2600. The device 2600 may be used to implement one or more of the methods described herein. The device 2600 may be embodied in a smart phone, a tablet computer, a computer, an Internet of Things (IoT) receiver, etc. The device 2600 may include one or more processors 2602, one or more memories 2604, and video processing hardware 2606 (also known as video processing circuitry). The (one or more) processors 2602 may be configured to implement one or more methods described in this document. The (one or more) memories 2604 may be used to store data and code for implementing the methods and techniques described herein. The video processing hardware 2606 may be used to implement some of the techniques described in this document in hardware circuitry. In some embodiments, the hardware 2606 may be partially or completely located within the processor 2602, for example, a graphics processor.

[0522] Fig. 27 2700 is a block diagram illustrating an exemplary video encoding and decoding system 2700 that may utilize the techniques of this disclosure. Fig. 27As shown, the video codec system 2700 may include a source device 2710 and a target device 2720. The source device 2710 generates encoded video data, which may be referred to as a video encoding device. The target device 2720 may decode the encoded video data generated by the source device 2710, which may be referred to as a video decoding device.

[0523] Source device 2710 may include a video source 2712 , a video encoder 2714 , and an input / output (I / O) interface 2716 .

[0524] The video source 2712 may include sources such as a video capture device, an interface for receiving video data from a video content provider, and / or a computer graphics system for generating video data, or a combination of such sources. The video data may include one or more pictures. The video encoder 2714 encodes the video data from the video source 2712 to generate a bit stream. The bit stream may include a bit sequence that forms a coded representation of the video data. The bit stream may include coded pictures and associated data. The coded pictures are coded representations of pictures. The associated data may include sequence parameter sets, picture parameter sets, and other syntax structures. The I / O interface 2716 may include a modulator / demodulator (modem) and / or a transmitter. The coded video data may be sent directly to the target device 2720 via the network 2730 via the I / O interface 2716. The coded video data may also be stored on a storage medium / server 2740 for access by the target device 2720.

[0525] The target device 2720 may include an I / O interface 2726 , a video decoder 2724 , and a display device 2722 .

[0526] The I / O interface 2726 may include a receiver and / or a modem. The I / O interface 2726 may obtain encoded video data from the source device 2710 or the storage medium / server 2740. The video decoder 2724 may decode the encoded video data. The display device 2722 may display the decoded video data to the user. The display device 2722 may be integrated with the target device 2720, or may be external to the target device 2720 that may be configured to interface with an external display device.

[0527] The video encoder 2714 and the video decoder 2724 may operate according to a video compression standard, such as the High Efficiency Video Codec (HEVC) standard, the Versatile Video Codec (VVC) standard, and other current and / or other standards.

[0528] Fig.28 2800 is a block diagram illustrating an example of a video encoder 2800, which may be Fig. 27The video encoder 2714 in the video codec system 2700 shown in FIG.

[0529] Video encoder 2800 may be configured to perform any or all of the techniques of this disclosure. Fig.28 In the example of , video encoder 2800 includes multiple functional components. The techniques described in this disclosure can be shared between various components of video encoder 2800. In some examples, the processor can be configured to perform any or all of the techniques described in this disclosure.

[0530] The functional components of the video encoder 2800 may include a segmentation unit 2801, a prediction unit 2802, a residual generation unit 2807, a transform unit 2808, a quantization unit 2809, an inverse quantization unit 2810, an inverse transform unit 2811, a reconstruction unit 2812, a buffer 2813 and an entropy coding unit 2814, and the prediction unit 2802 may include a mode selection unit 2803, a motion estimation unit 2804, a motion compensation unit 2805 and an intra-frame prediction unit 2806.

[0531] In other examples, the video encoder 2800 may include more, fewer, or different functional components. In one example, the prediction unit 2802 may include an intra-block copy (IBC) unit. The IBC unit may perform prediction in an IBC mode in which at least one reference picture is a picture in which the current video block is located.

[0532] Furthermore, some components, such as the motion estimation unit 2804 and the motion compensation unit 2805, may be highly integrated but are not shown for the purpose of explanation. Fig.28 In the example, they are represented separately.

[0533] The segmentation unit 2801 may segment the picture into one or more video blocks. Fig. 27 The video encoder 2714 and the video decoder 2724 may support various video block sizes.

[0534] The mode selection unit 2803 may, for example, select one of the intra or inter coding modes based on the error result, and provide the resulting intra or inter coded block to the residual generation unit 2807 to generate residual block data, and to the reconstruction unit 2812 to reconstruct the coded block for use as a reference picture. In some examples, the mode selection unit 2803 may select a combined intra and inter prediction CIIP mode, wherein the prediction is based on an inter prediction signal and an intra prediction signal. The mode selection unit 2803 may also select a resolution (e.g., sub-pixel or integer pixel precision) of the motion vector of the block in the case of inter prediction.

[0535] In order to perform inter-frame prediction on the current video block, the motion estimation unit 2804 may generate motion information of the current video block by comparing the current video block with one or more reference frames from the buffer 2813. The motion compensation unit 2805 may determine a predicted video block of the current video block based on the motion information of pictures other than the picture associated with the current video block from the buffer 2813 and the decoded samples.

[0536] The motion estimation unit 2804 and the motion compensation unit 2805 may perform different operations for the current video block, for example, depending on whether the current video block is in an I slice, a P slice, or a B slice. An I slice (or I-frame) is the least compressible, but does not require other video frames to decode. An S slice (or P frame) can use data from previous frames to decompress and is more compressible than an I frame. A B slice (or B frame) can use both previous and forward frames for data reference to obtain the highest amount of data compression.

[0537] In some examples, the motion estimation unit 2804 may perform unidirectional prediction on the current video block, and the motion estimation unit 2804 may search the reference pictures of list 0 or list 1 to find the reference video block of the current video block. The motion estimation unit 2804 may then generate a reference index indicating the reference picture in list 0 or list 1 that includes the reference video block and a motion vector indicating the spatial displacement between the current video block and the reference video block. The motion estimation unit 2804 may output the reference index, the prediction direction indicator, and the motion vector as the motion information of the current video block. The motion compensation unit 2805 may generate a predicted video block of the current block based on the reference video block indicated by the motion information of the current video block.

[0538] In other examples, the motion estimation unit 2804 may perform bidirectional prediction on the current video block, and the motion estimation unit 2804 may search for a reference video block of the current video block in the reference pictures in list 0, and may also search for another reference video block of the current video block in the reference pictures in list 1. The motion estimation unit 2804 may then generate a reference index indicating the reference pictures in list 0 and list 1 that include the reference video block and a motion vector indicating a spatial displacement between the reference video block and the current video block. The motion estimation unit 2804 may output the reference index and the motion vector of the current video block as the motion information of the current video block. The motion compensation unit 2805 may generate a predicted video block of the current video block based on the reference video block indicated by the motion information of the current video block.

[0539] In some examples, motion estimation unit 2804 may output the full set of motion information for use in a decoding process by a decoder.

[0540] In some examples, motion estimation unit 2804 may not output a full set of motion information for the current video. Instead, motion estimation unit 2804 may signal the motion information of the current video block with reference to the motion information of another video block. For example, motion estimation unit 2804 may determine that the motion information of the current video block is sufficiently similar to the motion information of a neighboring video block.

[0541] In one example, motion estimation unit 2804 may indicate a value in a syntax structure associated with a current video block that indicates to video decoder 2724 that the current video block has the same motion information as another video block.

[0542] In another example, the motion estimation unit 2804 may identify another video block and a motion vector difference (MVD) in a syntax structure associated with the current video block. The motion vector difference indicates the difference between the motion vector of the current video block and the motion vector of the indicated video block. The video decoder 2724 may use the motion vector of the indicated video block and the motion vector difference to determine the motion vector of the current video block.

[0543] As discussed above, the video encoder 2714 may predictively signal motion vectors. Two examples of predictive signaling techniques that may be implemented by the video encoder 2714 include Advanced Motion Vector Prediction (AMVP) and Merge mode signaling.

[0544] The intra prediction unit 2806 may perform intra prediction on the current video block. When the intra prediction unit 2806 performs intra prediction on the current video block, the intra prediction unit 2806 may generate prediction data for the current video block based on decoded samples of other video blocks in the same picture. The prediction data for the current video block may include the predicted video block and various syntax elements.

[0545] The residual generation unit 2807 may generate residual data for the current video block by subtracting (e.g., indicated by a minus sign) the predicted video block of the current video block from the current video block. The residual data for the current video block may include residual video blocks corresponding to different sample components of the samples in the current video block.

[0546] In other examples, such as in skip mode, there may be no residual data for the current video block and the residual generation unit 2807 may not perform a subtraction operation.

[0547] Transform unit 2808 may generate one or more transform coefficient video blocks for the current video block by applying one or more transforms to the residual video block associated with the current video block.

[0548] After the transform unit 2808 generates the transform coefficient video block associated with the current video block, the quantization unit 2809 may quantize the transform coefficient video block associated with the current video block based on one or more quantization parameter (QP) values ​​associated with the current video block.

[0549] The inverse quantization unit 2810 and the inverse transform unit 2811 may apply inverse quantization and inverse transform to the transform coefficient video block, respectively, to reconstruct a residual video block from the transform coefficient video block. The reconstruction unit 2812 may add the reconstructed residual video block to corresponding samples from one or more predicted video blocks generated by the prediction unit 2802 to generate a reconstructed video block associated with the current block for storage in the buffer 2813.

[0550] After the reconstruction unit 2812 reconstructs the video block, a loop filtering operation may be performed to reduce video blocking artifacts in the video block.

[0551] The entropy encoding unit 2814 may receive data from other functional components of the video encoder 2800. When the entropy encoding unit 2814 receives data, the entropy encoding unit 2814 may perform one or more entropy encoding operations to generate entropy-coded data and output a bitstream including the entropy-coded data.

[0552] Fig.29 is a block diagram illustrating an example of a video decoder 2900, which may be Fig. 27 The video decoder 2724 in the video codec system 2700 is shown.

[0553] Video decoder 2900 may be configured to perform any or all of the techniques of this disclosure. Fig.29 In the example of , video decoder 2900 includes multiple functional components. The techniques described in this disclosure can be shared between various components of video decoder 2900. In some examples, the processor can be configured to perform any or all of the techniques described in this disclosure.

[0554] exist Fig.29 In the example of , the video decoder 2900 includes an entropy decoding unit 2901, a motion compensation unit 2902, an intra-frame prediction unit 2903, an inverse quantization unit 2904, an inverse transform unit 2905, a reconstruction unit 2906 and a buffer 2907. In some examples, the video decoder 2900 can perform substantially the same operations as those described with respect to the video encoder 2714 ( Fig. 27 ) is a decoding pass that is the inverse of the encoding pass described.

[0555] The entropy decoding unit 2901 may retrieve an encoded bitstream. The encoded bitstream may include entropy-encoded video data (e.g., encoded blocks of video data). The entropy decoding unit 2901 may decode the entropy-encoded video data, and the motion compensation unit 2902 may determine motion information including motion vectors, motion vector precision, reference picture list indexes, and other motion information from the entropy-decoded video data. For example, the motion compensation unit 2902 may determine this information by performing AMVP and Merge mode signaling.

[0556] The motion compensation unit 2902 may generate motion compensated blocks, possibly performing interpolation based on interpolation filters.Identifiers of interpolation filters to be used with sub-pixel precision may be included in the syntax elements.

[0557] The motion compensation unit 2902 may calculate interpolation of sub-integer pixels of a reference block using interpolation filters as used by the video encoder 2714 during encoding of the video block. The motion compensation unit 2902 may determine the interpolation filters used by the video encoder 2714 from received syntax information and use the interpolation filters to produce a predictive block.

[0558] The motion compensation unit 2902 may use some of the syntax information to determine the size of blocks used to encode (one or more) frames and / or (one or more) slices of the encoded video sequence, partition information describing how each macroblock of a picture of the encoded video sequence is partitioned, a mode indicating how to encode each partition, one or more reference frames (and reference frame lists) for each inter-frame coded block, and other information used to decode the encoded video sequence.

[0559] The intra prediction unit 2903 may form a prediction block from spatially neighboring blocks using, for example, an intra prediction mode received in the bitstream. The inverse quantization unit 2904 inversely quantizes (i.e., back-quantizes) the quantized video block coefficients provided in the bitstream and decoded by the entropy decoding unit 2901. The inverse transform unit 2905 applies an inverse transform.

[0560] The reconstruction unit 2906 may sum the residual block with the corresponding prediction block generated by the motion compensation unit 2902 or the intra prediction unit 2903 to form a decoded block. If necessary, a deblocking filter may also be applied to filter the decoded block in order to remove blocking artifacts. The decoded video block is then stored in a buffer 2907, which provides reference blocks for subsequent motion compensation / intra prediction and also produces decoded video for presentation on a display device.

[0561] Fig.303000 is a flow chart of an example method 3000 for video processing. The method 3000 determines, at step 3002, to apply an NN filter to unfiltered samples of a video unit to generate filtered samples. The NN filter includes an NN filter model generated based on segmentation information of the video unit. The use of the NN filter is indicated by one or more syntax elements in the bitstream. At step 3004, conversion is performed between a video media data file and a bitstream based on the EDRAP samples on the filtered samples. The conversion may include encoding at an encoder, decoding at a decoder, or a combination thereof.

[0562] It should be noted that the method 3000 may be implemented in an apparatus for processing video data, the apparatus comprising a processor and a non-transitory memory having instructions thereon, such as the encoder 500, the video encoder 2800, and / or the video decoder 2900. In this case, the instructions, when executed by the processor, cause the processor to perform the method 3000. In addition, the method 3000 may be performed by a non-transitory computer-readable medium comprising a computer program product for use by a video codec device. The computer program product comprises computer executable instructions stored on a non-transitory computer-readable medium, such that when executed by the processor, the video codec device performs the method 3000.

[0563] A list of some preferred solutions for some embodiments is provided below.

[0564] The following solutions illustrate exemplary embodiments of the techniques discussed in this disclosure.

[0565] 1. A method implemented by a video codec device, comprising: applying a neural network (NN) filter to unfiltered samples of a video unit to generate filtered samples, wherein the NN filter includes an NN filter model generated based on segmentation information of the video unit, and wherein use of the NN filter is indicated by one or more syntax elements in a bitstream; and performing conversion between a video media file and a bitstream based on the filtered samples.

[0566] 2. The method of solution 1, wherein the one or more syntax elements include a sequence parameter set (SPS) convolutional neural network loop filter (CNNLF) enable flag (sps_cnnlf_enabled_flag), and wherein sps_cnnlf_enabled_flag equal to 1 specifies that the NN-based filter is enabled for a codec layer video sequence (CLVS), and sps_cnnlf_enabled_flag equal to 0 specifies that the CNNLF is disabled for CLVS.

[0567] 3. A method according to any of solutions 1-2, wherein one or more syntax elements include a spsCNNLF inference size base (sps_cnnlf_infer_size_base), wherein sps_cnnlf_infer_size_base specifies the base inference block size of the NN-based filter of the CLVS, and wherein the CNNLF inference size (cnnlfInferSize[3]) variable is set equal to {sps_cnnlf_infer_size_base>>1, sps_cnnlf_infer_size_base, sps_cnnlf_infer_size_base<<1}, wherein >> is a right shift and << is a left shift.

[0568] 4. A method according to any one of solutions 1-3, wherein the one or more syntax elements include SPSCNNLF inference size extension (sps_cnnlf_infer_size_entension), and wherein sps_cnnlf_infer_size_entension specifies the size of the extended samples at the boundary of the NN inference block of the CLVS.

[0569] 5. A method according to any one of solutions 1-4, wherein one or more syntax elements include an SPSCNNLF maximum number parameter (sps_cnnlf_max_num_params), and wherein sps_cnnlf_max_num_params specifies a maximum number of NN filter parameter candidates for each block of the CLVS.

[0570] 6. A method according to any one of solutions 1-5, wherein the one or more syntax elements include a first slice CNNLF mode (slice_cnnlf_mode[0]), and wherein slice_cnnlf_mode[0] is equal to 0 to indicate that the NN filter is not used for the luma component of any block in the slice, slice_cnnlf_mode[0] is equal to i to indicate that the i-th NN filter parameter candidate is used for the luma components of all blocks in the slice, wherein i is equal to 1, 2, ..., sps_cnnlf_max_num_params, and slice_cnnlf_mode[0] is equal to sps_cnnlf_max_num_params+1 to indicate that the parameter candidate for each block is determined by the parameter index (cnnlfParamIdx) of each block.

[0571] 7. A method according to any one of solutions 1-6, wherein one or more syntax elements include a second slice CNNLF mode (slice_cnnlf_mode[1]), and wherein slice_cnnlf_mode[1] is equal to 0 to indicate that the NN filter is not used for the chroma components of any block in the slice, slice_cnnlf_mode[1] is equal to i to indicate that the i-th NN filter parameter candidate is used for the chroma components of all blocks in the slice, wherein i is equal to 1, 2, ..., sps_cnnlf_max_num_params, and slice_cnnlf_mode[1] is equal to sps_cnnlf_max_num_params+1 to indicate that the parameter candidate for each block is determined by the parameter index (cnnlfParamIdx) of each block.

[0572] 8. A method according to any one of solutions 1-7, wherein one or more syntax elements include a slice CNNLF scaling flag (slice_cnnlf_scale_flag) having a parameter index parameter (paramIdx) and a channel index parameter (chal), wherein slice_cnnlf_scale_flag[paramIdx][chal] is equal to 1 to specify that residual scaling is enabled for the parameter candidate indexed by paramIdx for the channel indexed by chal, and wherein [paramIdx][chal] is equal to 0 to specify that residual scaling is disabled for the parameter candidate indexed by paramIdx for the channel indexed by chal.

[0573] 9. A method according to any one of solutions 1-8, wherein one or more syntax elements include an NN scale (nnScale) having paramIdx and chal, and wherein nnScale[paramIdx][chal] specifies a scaling factor of a parameter candidate indexed by paramIdx for a channel indexed by chal.

[0574] 10. A method according to any one of solutions 1-9, wherein one or more syntax elements include use CNNLF (useCnnlf), and wherein useCnnlf equal to 1 specifies that the NN-based filter is enabled for the block, and useCnnlf equal to 0 specifies that the NN-based filter is disabled for the block.

[0575] 11. A method according to any one of solutions 1-10, wherein one or more syntax elements include using a first parameter (useFirstParam), and wherein useFirstParam equal to 1 specifies that the first parameter candidate of the NN filter is used for the block, and useFirstParam equal to 0 specifies that the first parameter candidate of the NN filter is not used for the block.

[0576] 12. A method according to any one of solutions 1-11, wherein one or more syntax elements include a CNNLF parameter index minus two (cnnlfParamIdxMinus2), wherein cnnlfParamIdxMinus2 specifies the index of a parameter candidate for the NN filter of the block, and wherein a variable CNNLF parameter index (cnnlfParamIdx[chal][unitIdx]) having a chal and a unit index is set equal to cnnlfParamIdxMinus2+2.

[0577] 13. A method according to any of solutions 1-12, wherein the number of inference units with chal (numInferUnits[chal]) refers to the number of inference blocks of the channel indexed by chal in the stripe and is derived as follows:

[0578]

[0579]

[0580] where sh_slice_type is the slice type in the slice header, cnnlfInferSizeLuma is the CNNLF inferred size luma variable, cnnlfInferSizeChroma is the CNNLF inferred size chroma variable, SliceQpY is the slice luma quantization parameter, pps_pic_width_in_luma_samples is the PPS picture width of luma samples in the picture parameter set (PPS), and pps_pic_height_in_luma_samples is the PPS picture height of luma samples in the PPS.

[0581] 14. A method according to any of solutions 1-13, wherein the use of the NN filter is indicated in the SPS in the bitstream as follows:

[0582] if(sps_cnnlf_enabled_flag){

[0583] sps_cnnlf_infer_size_base

[0584] sps_cnnlf_infer_size_extension

[0585] sps_cnnlf_max_num_params

[0586] }

[0587] 15. The method according to any of solutions 1-14, wherein the use of the NN filter is indicated in a slice header in the bitstream as follows:

[0588] if(sps_cnnlf_enabled_flag){

[0589] for(chal=0;chal<(sps_chroma_format_idc!=0?2:1);chal++){

[0590] slice_cnnlf_mode[chal]

[0591] if (slice_cnnlf_mode[chal]) {

[0592] if(slice_cnnlf_mode[chal]==sps_cnnlf_max_num_params+1){

[0593] for(paramIdx=0;paramIdx <sps_cnnlf_max_num_param;paramIdx++){

[0594] slice_cnnlf_scale_flag[paramIdx][chal]

[0595] if(slice_cnnlf_scale_flag[paramIdx][chal])

[0596] nnScale[paramIdx][chal]

[0597] }

[0598] }else{

[0599] slice_cnnlf_scale_flag[slice_cnnlf_mode[chal]-1][chal]

[0600] if(slice_cnnlf_scale_flag[slice_cnnlf_mode[chal]-1][chal])

[0601] nnScale[slice_cnnlf_mode[chal]-1][chal]

[0602] }

[0603] }

[0604] }

[0605] }

[0606] Where sps_chroma_format_idc is the SPS chroma format identification code.

[0607] 16. The method according to any of solutions 1-15, wherein the use of NN filters is indicated in the codec tree unit syntax in the bitstream as follows:

[0608] if(CtbAddrX==0&&CtbAddrY==0&&sps_cnnlf_enabled_flag){

[0609] for(chal=0;chal<(sps_chroma_format_idc!=0?2:1);chal++){

[0610] for(unitIdx=0;unitIdx <numInferUnits[chal];unitIdx++){

[0611] if(slice_cnnlf_mode[chal] <sps_cnnlf_max_num_params+1){

[0612] cnnlfParamIdx[chal][unitIdx]=slice_cnnlf_mode[chal]

[0613] }else{

[0614] useCnnlf

[0615] if(sps_cnnlf_max_num_params==1){

[0616] cnnlfParamIdx[chal][unitIdx]=useCnnlf? 1:0

[0617] }else if{!useCnnlf){

[0618] cnnlfParamIdx[chal][unitIdx]=0

[0619] }else{

[0620] useFirstParam

[0621] if(sps_cnnlf_max_num_params==2){

[0622] cnnlfParamIdx[chal][unitIdx]=useFirstParam? 1:2

[0623] }else if(useFirstParam){

[0624] cnnlfParamIdx[chal][unitIdx]=1

[0625]

[0626] Where CtbAddrX is the codec tree block (CTB) address on the X-axis, and CtbAddrY is the CTB address on the Y-axis.

[0627] 17. A device for processing video data, comprising: a processor; and a non-transitory memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to perform a method according to any one of solutions 1-16.

[0628] 18. A non-transitory computer-readable medium comprising a computer program product for use by a video codec device, the computer program product comprising computer-executable instructions, wherein the computer-executable instructions are stored on the non-transitory computer-readable medium, so that when executed by a processor, the video codec device performs a method according to any one of solutions 1-16.

[0629] 19. A non-temporary computer-readable recording medium storing a bitstream of a video generated by a method performed by a video processing device, wherein the method includes: determining to apply a neural network (NN) filter to unfiltered samples of a video unit to generate filtered samples, wherein the NN filter includes an NN filter model generated based on segmentation information of the video unit, and wherein use of the NN filter is indicated by one or more syntax elements in the bitstream; and generating a bitstream based on the determination.

[0630] 20. A method for storing a bitstream of a video, comprising: determining to apply a neural network (NN) filter to unfiltered samples of a video unit to generate filtered samples, wherein the NN filter includes an NN filter model generated based on segmentation information of the video unit, and wherein use of the NN filter is indicated by one or more syntax elements in the bitstream; generating a bitstream based on the determination; and storing the bitstream in a non-temporary computer-readable recording medium.

[0631] The following documents are incorporated herein by reference in their entirety:

[0632] [1] Johannes Ballé, Valero Laparra, and Eero P Simoncelli, “End-to-end optimization of nonlinear transform codes for perceptual quality,” PCS IEEE (2016), 1–5.

[0633] [2] Lucas Theis, Wenzhe Shi, Andrew Cunningham, and Ferenc Huszár, “Lossy image compression with compressive autoencoders,” arXiv preprint arXiv:1703.00395 (2017).

[0634] [3] Jiahao Li, Bin Li, Jizheng Xu, Ruiqin Xiong, and Wen Gao, “Fully Connected Network-Based Intra Prediction for Image Coding,” IEEE Transactions on Image Processing, July 27, 2018, pp. 3236–3247.

[0635] [4] Yuanying Dai, Dong Liu, and Feng Wu, “A convolutional neural network approach for post-processing in HEVC intra coding,” MMM. Springer, 28–39.

[0636] [5] Rui Song, Dong Liu, Houqiang Li, and Feng Wu, “Neural network-based arithmetic coding of intra prediction modes in HEVC,” IEEE International Conference on Visual Communications and Image Processing, 1–4.

[0637] [6] J. Pfaff, P. Helle, D. Maniry, S. Kaltenstadler, W. Samek, H. Schwarz, D. Marpe, and T. Wiegand, “Neural network based intra prediction for video coding,” in Applications of Digital Image Processing XLI, vol. 10752. International Society for Optical Engineering, 1075213 (2018).

[0638] The disclosed and other solutions, examples, embodiments, modules and functional operations described in this document can be implemented in digital electronic circuits, or in computer software, firmware or hardware (including the structures disclosed in this document and their structural equivalents), or in a combination of one or more of them. The disclosed and other embodiments can be implemented as one or more computer program products, i.e., one or more modules of computer program instructions encoded on a computer-readable medium, which are used to be executed by a data processing device or control the operation of the data processing device. The computer-readable medium can be a machine-readable storage device, a machine-readable storage substrate, a memory device, a combination of substances that affect machine-readable propagation signals, or a combination of one or more of them. The term "data processing device" includes all devices, equipment and machines for processing data, including, for example, a programmable processor, a computer, or multiple processors or computers. In addition to hardware, the device may also include code that creates an execution environment for the computer program in question, for example, code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them. A propagation signal is an artificially generated signal generated to encode information for transmission to a suitable receiver device, such as a machine-generated electrical signal, an optical signal, or an electromagnetic signal.

[0639] A computer program (also known as a program, software, software application, script, or code) can be written in any form of programming language (including compiled or interpreted languages), and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. A program may be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files storing one or more modules, subroutines, or code portions). A computer program may be deployed to execute on one computer or on multiple computers located at one site or distributed across multiple sites and interconnected by a communication network.

[0640] The processes and logic flows described in this document can be performed by one or more programmable processors executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows can also be performed by special purpose logic circuits, and the apparatus can also be implemented as special purpose logic circuits, such as field programmable gate arrays (FPGAs) or application specific integrated circuits (ASICs).

[0641] Processors suitable for executing computer programs include, for example, general-purpose and special-purpose microprocessors, and any one or more processors of any type of digital computer. Typically, the processor will receive instructions and data from a read-only memory or a random access memory or both. The basic elements of a computer are a processor for executing instructions and one or more memory devices for storing instructions and data. Typically, a computer will also include one or more mass storage devices (e.g., magnetic disks, magneto-optical disks, or optical disks) for storing data, or operably coupled to receive data from or transfer data to or from the one or more mass storage devices. However, a computer does not require such a device. Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, including, for example, semiconductor memory devices, such as erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), and flash memory devices; magnetic disks, such as internal hard disks or removable disks; magneto-optical disks; and compact disk read-only memory (CD ROM) and digital versatile disk read-only memory (DVD-ROM) disks. The processor and the memory can be supplemented by, or incorporated in, special purpose logic circuitry.

[0642] Although this patent document contains many details, these details should not be interpreted as limitations on any subject matter or the scope of possible protection, but rather as descriptions of features specific to a particular embodiment of a particular technology. Certain features described in the context of a separate embodiment in this patent document may also be implemented in combination in a single embodiment. On the contrary, the various features described in the context of a single embodiment may also be implemented separately in multiple embodiments or in any suitable sub-combination. In addition, although features may be described above as working in certain combinations and even initially claimed as such, one or more features from the claimed combination may be excluded from the combination in some cases, and the claimed combination may be directed to a sub-combination or a variation of the sub-combination.

[0643] Similarly, although operations are depicted in a particular order in the drawings, this should not be understood as requiring that such operations be performed in the particular order shown or in a sequential order, or that all of the operations shown be performed, to achieve the desired results. In addition, the separation of various system components in the embodiments described in this patent document should not be understood as requiring such separation in all embodiments.

[0644] Only a few implementations and examples are described, and other implementations, enhancements, and variations may be made based on what is described and shown in this patent document.

[0645] Although this patent document contains many details, these details should not be interpreted as limitations on any subject matter or the scope of possible protection, but rather as descriptions of features specific to a particular embodiment of a particular technology. Certain features described in the context of a separate embodiment in this patent document may also be implemented in combination in a single embodiment. On the contrary, the various features described in the context of a single embodiment may also be implemented separately in multiple embodiments or in any suitable sub-combination. In addition, although features may be described above as working in certain combinations and even initially claimed as such, one or more features from the claimed combination may be excluded from the combination in some cases, and the claimed combination may be directed to a sub-combination or a variation of the sub-combination.

[0646] Similarly, although operations are depicted in a particular order in the drawings, this should not be understood as requiring that such operations be performed in the particular order shown or in a sequential order, or that all of the operations shown be performed, to achieve the desired results. In addition, the separation of various system components in the embodiments described in this patent document should not be understood as requiring such separation in all embodiments.

[0647] Only a few implementations and examples are described, and other implementations, enhancements, and variations may be made based on what is described and shown in this patent document.

Claims

1. A method implemented by a video encoding and decoding device, comprising: applying a neural network (NN) filter to unfiltered samples of a video unit to generate filtered samples, wherein the NN filter comprises an NN filter model generated based on segmentation information of the video unit, and wherein use of the NN filter is indicated by one or more syntax elements in a bitstream; and Conversion between a video media file and the bitstream is performed based on the filtered samples.

2. The method according to claim 1, wherein: The one or more syntax elements include a sequence parameter set (SPS) convolutional neural network loop filter (CNNLF) enabled flag (sps_cnnlf_enabled_flag), and wherein sps_cnnlf_enabled_flag equal to 1 specifies that the NN-based filter is enabled for a codec layer video sequence (CLVS), and sps_cnnlf_enabled_flag equal to 0 specifies that the CNNLF is disabled for the CLVS.

3. The method according to any one of claims 1 to 2, wherein: The one or more syntax elements include a SPSCNNLF inference size base (sps_cnnlf_infer_size_base), wherein sps_cnnlf_infer_size_base specifies a base inference block size for NN-based filters of CLVS, and wherein a CNNLF inference size (cnnlfInferSize[3]) variable is set equal to {sps_cnnlf_infer_size_base>>1, sps_cnnlf_infer_size_base, sps_cnnlf_infer_size_base<<1}, wherein >> is a right shift and << is a left shift.

4. The method according to any one of claims 1 to 3, wherein: The one or more syntax elements include an SPSCNNLF inference size extension (sps_cnnlf_infer_size_entension), and wherein the sps_cnnlf_infer_size_entension specifies a size of extended samples at a boundary of a NN inference block of the CLVS.

5. The method according to any one of claims 1 to 4, wherein: The one or more syntax elements include an SPSCNNLF maximum number parameter (sps_cnnlf_max_num_params), and wherein sps_cnnlf_max_num_params specifies a maximum number of NN filter parameter candidates for each block of the CLVS.

6. The method according to any one of claims 1 to 5, wherein: The one or more syntax elements include a first slice CNNLF mode (slice_cnnlf_mode[0]), and wherein slice_cnnlf_mode[0] is equal to 0 to indicate that the NN filter is not used for the luma component of any block in the slice, slice_cnnlf_mode[0] is equal to i to indicate that the i-th NN filter parameter candidate is used for the luma components of all blocks in the slice, wherein i is equal to 1, 2, ..., sps_cnnlf_max_num_params, and slice_cnnlf_mode[0] is equal to sps_cnnlf_max_num_params+1 to indicate that the parameter candidate for each block is determined by the parameter index (cnnlfParamIdx) of each block.

7. The method according to any one of claims 1 to 6, wherein: The one or more syntax elements include a second slice CNNLF mode (slice_cnnlf_mode[1]), and wherein slice_cnnlf_mode[1] is equal to 0 to indicate that the NN filter is not used for the chroma components of any block in the slice, slice_cnnlf_mode[1] is equal to i to indicate that the i-th NN filter parameter candidate is used for the chroma components of all blocks in the slice, wherein i is equal to 1, 2, ..., sps_cnnlf_max_num_params, and slice_cnnlf_mode[1] is equal to sps_cnnlf_max_num_params+1 to indicate that the parameter candidate for each block is determined by the parameter index (cnnlfParamIdx) of each block.

8. The method according to any one of claims 1 to 7, wherein: The one or more syntax elements include a slice CNNLF scaling flag (slice_cnnlf_scale_flag) having a parameter index parameter (paramIdx) and a channel index parameter (chal), wherein slice_cnnlf_scale_flag[paramIdx][chal] equal to 1 specifies that residual scaling is enabled for the parameter candidate indexed by paramIdx for the channel indexed by chal, and wherein [paramIdx][chal] equal to 0 specifies that residual scaling is disabled for the parameter candidate indexed by paramIdx for the channel indexed by chal.

9. The method according to any one of claims 1 to 8, wherein: The one or more syntax elements include a NN scale (nnScale) having paramIdx and chal, and wherein nnScale[paramIdx][chal] specifies a scaling factor for a parameter candidate indexed by paramIdx for a channel indexed by chal.

10. The method according to any one of claims 1 to 9, wherein: The one or more syntax elements include use CNNLF (useCnnlf), and wherein useCnnlf equal to 1 specifies that the NN-based filter is enabled for the block, and useCnnlf equal to 0 specifies that the NN-based filter is disabled for the block.

11. The method according to any one of claims 1 to 10, wherein: The one or more syntax elements include useFirstParam, and wherein useFirstParam equal to 1 specifies that the first parameter candidate of the NN filter is used for the block, and useFirstParam equal to 0 specifies that the first parameter candidate of the NN filter is not used for the block.

12. The method according to any one of claims 1 to 11, wherein: The one or more syntax elements include a CNNLF parameter index minus two (cnnlfParamIdxMinus2), wherein cnnlfParamIdxMinus2 specifies an index of a parameter candidate for the NN filter of the block, and wherein a variable CNNLF parameter index (cnnlfParamIdx[chal][unitIdx]) having a chal and a unit index is set equal to cnnlfParamIdxMinus2+2.

13. The method according to any one of claims 1 to 12, wherein: The number of inference units with chal (numInferUnits[chal]) refers to the number of inference blocks for the channel indexed by chal in the stripe and is derived as follows: where sh_slice_type is the slice type in the slice header, cnnlfInferSizeLuma is the CNNLF inferred size luma variable, cnnlfInferSizeChroma is the CNNLF inferred size chroma variable, SliceQpY is the slice luma quantization parameter, pps_pic_width_in_luma_samples is the PPS picture width of luma samples in the picture parameter set (PPS), and pps_pic_height_in_luma_samples is the PPS picture height of luma samples in the PPS.

14. The method according to any one of claims 1 to 13, wherein: The use of the NN filter is indicated in the SPS in the bitstream as follows:

15. The method according to any one of claims 1 to 14, wherein: The use of the NN filter is indicated in the slice header in the bitstream as follows: Where sps_chroma_format_idc is the SPS chroma format identification code.

16. The method according to any one of claims 1 to 15, wherein: The use of the NN filter is indicated in the codec tree unit syntax in the bitstream as follows: Where CtbAddrX is the codec tree block (CTB) address on the X-axis, and CtbAddrY is the CTB address on the Y-axis.

17. An apparatus for processing video data, comprising: processor; and a non-transitory memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to perform the method according to any one of claims 1-16.

18. A non-transitory computer-readable medium comprising a computer program product for use by a video codec device, the computer program product comprising computer-executable instructions, wherein the computer-executable instructions are stored on the non-transitory computer-readable medium so that when executed by a processor, the video codec device performs the method according to any one of claims 1-16.

19. A non-transitory computer-readable recording medium storing a bit stream of a video generated by a method performed by a video processing device, wherein the method comprises: determining to apply a neural network (NN) filter to unfiltered samples of a video unit to generate filtered samples, wherein the NN filter comprises an NN filter model generated based on segmentation information of the video unit, and wherein use of the NN filter is indicated by one or more syntax elements in a bitstream; and A bitstream is generated based on the determination.

20. A method for storing a bitstream of a video, comprising: determining to apply a neural network (NN) filter to unfiltered samples of a video unit to generate filtered samples, wherein the NN filter comprises an NN filter model generated based on segmentation information of the video unit, and wherein use of the NN filter is indicated by one or more syntax elements in a bitstream; generating a bitstream based on the determining; as well as The bit stream is stored in a non-transitory computer-readable recording medium.

Images