Filling method for filter in loop based on neural network

Abstract

The invention provides a filling method for an in-loop filter based on a neural network. The method is implemented by a video encoding and decoding device. The method comprises: determining, in real time, filling dimensions for filling samples to be applied to a video unit of a video for in-loop filtering, wherein d1, d2, d3 and d4 represent filling dimensions corresponding to top, bottom, left and right boundaries, respectively, of the video unit; and performing a conversion between the video unit and the bitstream of the video based on the determined filling dimension. A corresponding video encoding and decoding apparatus and a non-transitory computer-readable recording medium are also disclosed.

Description

[0001] Cross-references to related applications

[0002] This application is a divisional application of U.S. Patent Application No. 202210553172.6, filed May 20, 2022, which claims priority and benefit to U.S. Provisional Patent Application No. 63 / 191,121, filed May 20, 2021. All of the above applications are incorporated herein by reference as a part of the disclosure of this application. Technical Field

[0003] This invention generally relates to video encoding and decoding, and more specifically, to in-loop filters in image / video encoding and decoding. Background Technology

[0004] Digital video consumes the largest share of bandwidth in the internet and other digital communication networks. As the number of connected user devices capable of receiving and displaying video increases, the bandwidth demand for digital video is expected to continue to grow. Summary of the Invention

[0005] The disclosed aspects / embodiments provide a technique for real-time determination of the padding dimension of padding samples to be applied to video units for in-loop filtering. Therefore, the size of the padding region can be adjusted to accommodate different types of multi-layer bitstreams and / or different stripes of different temporal layers. Consequently, the video encoding / decoding process is improved compared to conventional video encoding / decoding techniques.

[0006] The first aspect relates to a method implemented by an encoding / decoding device. The method includes: determining in real-time the padding dimension of padding samples for in-loop filtering of video to be applied to a video unit, wherein… , , and These represent the padding dimensions corresponding to the top, bottom, left, and right boundaries of the video unit, respectively; and the conversion between the video unit and the video bitstream is performed based on the determined padding dimensions.

[0007] Optionally, in any of the foregoing aspects, another implementation of that aspect provides , , and Each has a different value; , , and Each has the same value; or = =

[0008] Optionally, in any of the preceding aspects, another implementation of the aspect provides that the indication corresponding to the padding dimension is included in a bitstream.

[0009] Optionally, in any of the preceding aspects, another implementation of the aspect provides that at least one of the padding dimensions is based on a position of the video unit relative to a parent video unit from which the video unit is split, and wherein a padding dimension corresponding to an outer boundary of the parent video unit is different than a padding dimension corresponding to an inner boundary of the parent video unit.

[0010] Optionally, in any of the preceding aspects, another implementation of the aspect provides that at least one of the padding dimensions is based on a size, color format, channel type, slice type, or split tree type of the video unit.

[0011] Optionally, in any of the preceding aspects, another implementation of the aspect provides that at least one of the padding dimensions is based on which temporal layer the video unit belongs to, or based on reference picture information.

[0012] Optionally, in any of the preceding aspects, another implementation of the aspect provides that the at least one padding dimension is based on a type of neural network (NN) in-loop filtering method applied to the video unit or whether a NN in-loop filtering method is applied to the video unit.

[0013] Optionally, in any of the preceding aspects, another implementation of the aspect provides that the video unit has a video unit size that depends on at least one of a slice resolution and a quantization parameter (QP), and wherein the parent video unit is a slice.

[0014] Optionally, in any of the preceding aspects, another implementation of the aspect provides that the padding dimensions define a padding region around the video unit, and wherein the method further comprises padding the padding region with padding samples when at least one neighboring sample outside the video unit is not available.

[0015] Optionally, in any of the preceding aspects, another implementation of the aspect provides that a boundary of the video unit is padded with padding samples when neighboring samples outside the video unit and along the boundary are not available.

[0016] Optionally, in any of the preceding aspects, another implementation of the aspect provides that the padding dimensions define a padding region around the video unit, and wherein the method further comprises padding the padding region using a padding method comprising zero padding, reflection padding, copy padding, constant padding, or mirror padding.

[0017] Optionally, in any of the preceding aspects, another implementation of the aspect provides that the padding method used to pad the padding region is based on a size of the video unit.

[0018] Optionally, in any of the preceding aspects, another implementation of the aspect provides a padding method for generating padding samples outside of a video unit is based on a type of a neural network (NN) filtering method applied to the video unit, or based on whether a NN filtering method is applied to the video unit.

[0019] Optionally, in any of the preceding aspects, another implementation of the aspect provides a padding method for generating padding samples outside of a video unit is based on a type of a neural network (NN) filtering method applied to the video unit, or based on whether a NN filtering method is applied to the video unit.

[0020] Optionally, in any of the preceding aspects, another implementation of the aspect provides a granularity of a neural network (NN) filter model to be applied to the padding samples is predefined, contained in a bitstream, derived, or based on dimensions of the video unit.

[0021] Optionally, in any of the preceding aspects, another implementation of the aspect provides a granularity of a neural network (NN) filter model to be applied to the padding samples is based on a quantization parameter (QP) and a resolution of the video unit.

[0022] Optionally, in any of the preceding aspects, another implementation of the aspect provides a binarization of a neural network (NN) filter model index corresponding to the NN filter model to be applied to the padding samples is based on a maximum number allowed for a higher level than the video unit, wherein the higher level is a slice, picture, or sequence level, and wherein the NN filter model index is binarized as a truncated unary code or a truncated binary code.

[0023] Optionally, in any of the preceding aspects, another implementation of the aspect provides the conversion comprises encoding the video unit into a bitstream.

[0024] Optionally, in any of the preceding aspects, another implementation of the aspect provides the conversion comprises decoding the video unit from a bitstream.

[0025] An apparatus for processing video data, comprising a processor and a non-transitory memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to: determine, in real-time, padding dimensions for padding samples to be applied to a video unit of a video for in-loop filtering, wherein 、 、 and respectively represent padding dimensions corresponding to top, bottom, left, and right boundaries of the video unit; and perform a conversion between the video unit and a video bitstream based on the determined padding dimensions.

[0026] A non-transitory computer-readable recording medium storing a bitstream of a video generated by a method performed by a video processing apparatus, wherein the method comprises: determining padding dimensions for padding samples to be applied to a video unit of the video for in-loop filtering in real time, wherein 、 、 and respectively represent padding dimensions for top, bottom, left and right boundaries of the video unit; and generating the bitstream based on the determined padding dimensions.

[0027] For the sake of clarity, any of the foregoing embodiments can be combined with any one or more of the other foregoing embodiments to create a new embodiment within the scope of the present disclosure.

[0028] These and other features will become more apparent from the following detailed description in conjunction with the accompanying drawings and claims. BRIEF DESCRIPTION OF DRAWINGS

[0029] For a more complete understanding of the present disclosure, reference is now made to the following brief description of the drawings and detailed description in conjunction with the accompanying drawings and specific embodiments.

[0030] Figure 1 is an example of a raster-scan slice partition of a picture.

[0031] Figure 2 is an example of a rectangular slice partition of a picture.

[0032] Figure 3 is an example of a picture partitioned into tiles, bricks, and rectangular slices.

[0033] Figure 4A is an example of a coding tree block (CTB) that crosses a bottom picture boundary.

[0034] Figure 4B is an example of a CTB that crosses a right picture boundary.

[0035] Figure 4C is an example of a CTB that crosses a bottom-right picture boundary.

[0036] Figure 5 is an example of an encoder block diagram.

[0037] Figure 6 is an illustration of samples within an 8x8 sample block.

[0038] Figure 7 is an example of a pixel involving filter on / off decision and strong / weak filter selection.

[0039] Figure 8Four one-dimensional (1-D) directional modes are shown for edge offset (EO) sample classification.

[0040] Figure 9 An example of a geometric-alternative loop filter (GALF) filter shape is shown.

[0041] Figure 10 An example of relative coordinates for 5x5 diamond filter support is shown.

[0042] Figure 11 Another example of relative coordinates for 5x5 diamond filter support is shown.

[0043] Figure 12A is an example architecture of a proposed convolutional neural network (CNN) filter.

[0044] Figure 12B is an example of a residual block (ResBlock) construction.

[0045] Figure 13 is a diagram illustrating an example of uni-directional inter prediction.

[0046] Figure 14 is a diagram illustrating an example of bi-directional inter prediction.

[0047] Figure 15 is a diagram illustrating an example of layer-based prediction.

[0048] Figure 16 Padding video units are illustrated, where , , and are padding dimensions for top, bottom, left, and right boundaries, respectively.

[0049] Figure 17 Mirror padding is illustrated, where the gray blocks represent padding samples.

[0050] Figure 18 is a block diagram showing an example video processing system.

[0051] Figure 19 is a block diagram of a video processing device.

[0052] Figure 20 is a block diagram illustrating an example of a video coding system.

[0053] Figure 21 is a block diagram illustrating an example of a video encoder.

[0054] Figure 22 is a block diagram illustrating an example of a video decoder.

[0055] Figure 23 is a method for coding video data according to embodiments of the disclosure. DETAILED DESCRIPTION

[0056] It should be understood at the outset that although illustrative implementations of one or more embodiments are provided below, the disclosed systems and / or methods can be implemented using any number of techniques, whether currently known or in existence. The disclosure should in no way be limited to the illustrative implementations, drawings, and techniques illustrated below, including the exemplary design and implementations set forth herein, but can be modified in any manner within the scope of the appended claims along with their full scope of equivalents.

[0057] The use of H.266 terminology in some descriptions is merely for ease of understanding and is not intended to limit the scope of the disclosed technology. Thus, the technology described herein is applicable to other video codec protocols and designs as well.

[0058] Video coding standards have evolved mainly through the development of the well-known International Telecommunication Union-Telecommunication (ITU-T) and International Organization for Standardization (ISO) / International Electrotechnical Commission (IEC) standards. The ITU-T produced H.261 and H.263, the ISO / IEC produced Motion Pictures Expert Group (MPEG)-1 and MPEG-4 Visual, and the two organizations jointly produced the H.262 / MPEG-2 Video and H.264 / MPEG-4 Advanced Video Coding (AVC) and H.265 / High Efficiency Video Coding (HEVC) standards.

[0059] Since H.262, video coding standards have been based on a hybrid video coding structure where temporal prediction plus transform coding is utilized. To explore future video coding technologies beyond HEVC, the Video Coding Experts Group (VCEG) and MPEG jointly formed the Joint Video Exploration Team (JVET) in 2015. Since then, many new methods have been adopted by the JVET and put into the reference software named Joint Exploration Model (JEM).

[0060] In April 2018, the Joint Video Team (JVT) of VCEG (Q6 / 16) and ISO / IEC JTC1 SC29 / WG11 (MPEG) was formed to work on the Versatile Video Coding (VVC) standard, targeting a 50% bitrate reduction compared to HEVC. The VVC version 1 was completed in July 2020.

[0061] Color spaces and chroma subsampling are discussed. A color space, also called a color model (or color system), is an abstract mathematical model that simply describes a range of colors as a tuple of numbers, usually 3 or 4 values or color components (e.g., red green blue (RGB)). At a fundamental level, a color space is an exposition of a coordinate system and a subspace.

[0062] For video compression, the most commonly used color spaces are YCbCr and RGB. Y’CbCr or Y Pb / Cb Pr / Cr, also written as YC B C R or Y’C B C R , is a family of color spaces used as part of the color image pipeline in video and digital photography systems. Y’ is the luminance component, C B and C R , also called Cb and Cr, are the blue-difference and red-difference chrominance components. Y’ (with the prime) is different from Y, Y being luminance, which means that the light intensity is based on gamma-corrected RGB primary non-linear encoding. Each color component (e.g., R, B, G, Y, etc.) can be referred to as a color channel or color channel type.

[0063] Chroma subsampling is the practice of encoding an image with lower resolution for chrominance information than for luminance by exploiting the fact that the human visual system is less sensitive to color differences than to luminance differences.

[0064] Color formats (such as 4:4:4, 4:2:2, and 4:2:0) are discussed.

[0065] For 4:4:4 chroma subsampling, the three Y’CbCr components have the same sampling rate, so there is no chroma subsampling. This scheme is sometimes used for high-end film scanners and movie post-production.

[0066] For 4:2:2 chroma subsampling, two chroma components are sampled at half the rate of the luminance sampling: horizontal chroma resolution is halved. This reduces the bandwidth of the uncompressed video signal by one third, but there is little visual difference.

[0067] For 4:2:0 chroma subsampling, compared to 4:1:1, the horizontal sampling is doubled, but in this scheme, the vertical resolution is halved because the Cb and Cr channels are only sampled on every other line. Thus, the data rate is the same. Cb and Cr are subsampled by a factor of two in the horizontal and vertical directions, respectively. The 4:2:0 scheme has three variants with different horizontal and vertical addressing.

[0068] In MPEG-2, Cb and Cr are co-sited in the horizontal direction. Cb and Cr are located between pixels in the vertical direction (in the interstitial gaps). In Joint Photographic Experts Group (JPEG) / JPEG File Interchange Format (JFIF), H.261, and MPEG-1, Cb and Cr are located in the interstitial gaps between alternating luma samples. In 4:2:0 DV, Cb and Cr are co-sited in the horizontal direction. In the vertical direction, they are co-sited on alternating lines.

[0069] Definitions of video units are provided. A picture is divided into one or more tile rows and one or more tile columns. A tile is a sequence of coding tree units (CTUs) covering a rectangular region of a picture. A tile is divided into one or more bricks, each consisting of multiple CTU rows within the tile. A tile that is not divided into multiple bricks is also referred to as a brick. However, a brick that is a true subset of a tile is not referred to as a tile. A slice either contains multiple tiles of a picture or contains multiple bricks of a tile.

[0070] Two slice modes are supported, namely, a raster-scan slice mode and a rectangular slice mode. In the raster-scan slice mode, a slice contains a sequence of tiles in a raster-scan order of tiles of a picture. In the rectangular slice mode, a slice contains multiple bricks of a picture that collectively form a rectangular region of the picture. The bricks within a rectangular slice are arranged in the brick raster-scan order of the slice.

[0071] Figure 1 is an example of a raster-scan slice partitioning of a picture 100, where the picture is divided into 12 tiles 102 and 3 raster-scan slices 104. As shown, each tile 102 and raster-scan slice 104 contains multiple CTUs 106.

[0072] Figure 2 is an example of a rectangular slice partitioning of a picture 200 according to the VVC specification, where the picture is divided into 24 tiles 202 (6 tile columns 203 and 4 tile rows 205) and 9 rectangular slices 204. As shown, each tile 202 and rectangular slice 204 contains multiple CTUs 206.

[0073] Figure 3 is an example of a picture 300 partitioned into tiles, bricks, and rectangular slices according to the VVC specification, where the picture 300 is divided into four tiles 302 (two tile columns 303 and two tile rows 305), eleven bricks 304 (the top-left tile contains one brick, the top-right tile contains five bricks, the bottom-left tile contains two bricks, and the bottom-right tile contains three bricks), and four rectangular slices 306.

[0074] CTU and coding tree block (CTB) sizes are discussed. In VVC, the coding tree unit (CTU) size, which is signaled in the sequence parameter set (SPS) by the syntax element log2_ctu_size_minus2, can be as small as 4x4. The sequence parameter set (SPS) SPS syntax is as follows.

[0075] seq_parameter_set_rbsp( ) { descriptor sps_decoding_parameter_set_id u(4) sps_video_parameter_set_id u(4) sps_max_sub_layers_minus1 u(3) sps_reserved_zero_5bits u(5) profile_tier_level( sps_max_sub_layers_minus1 ) gra_enabled_flag u(1) sps_seq_parameter_set_id ue(v) chroma_format_idc ue(v) if( chroma_format_idc = = 3 ) separate_colour_plane_flag u(1) pic_width_in_luma_samples ue(v) pic_height_in_luma_samples ue(v) conformance_window_flag u(1) if( conformance_window_flag ) { conf_win_left_offset ue(v) conf_win_right_offset ue(v) conf_win_top_offset ue(v) conf_win_bottom_offset ue(v) } bit_depth_luma_minus8 ue(v) bit_depth_chroma_minus8 ue(v) log2_max_pic_order_cnt_lsb_minus4 ue(v) sps_sub_layer_ordering_info_present_flag u(1) for( i = ( sps_sub_layer_ordering_info_present_flag? 0 : sps_max_sub_layers_minus1 ); i <= sps_max_sub_layers_minus1; i++ ) { sps_max_dec_pic_buffering_minus1[ i ] ue(v) sps_max_num_reorder_pics[ i ] ue(v) sps_max_latency_increase_plus1[ i ] ue(v) } long_term_ref_pics_flag u(1) sps_idr_rpl_present_flag u(1) rpl1_same_as_rpl0_flag u(1) for( i = 0; i <!rpl1_same_as_rpl0_flag? 2 : 1; i++ ) { num_ref_pic_lists_in_sps[ i ] ue(v) for( j = 0; j < num_ref_pic_lists_in_sps[ i ]; j++) ref_pic_list_struct( i, j ) } qtbtt_dual_tree_intra_flag u(1) log2_ctu_size_minus2 ue(v) log2_min_luma_coding_block_size_minus2 ue(v) partition_constraints_override_enabled_flag u(1) sps_log2_diff_min_qt_min_cb_intra_slice_luma ue(v) sps_log2_diff_min_qt_min_cb_inter_slice ue(v) sps_max_mtt_hierarchy_depth_inter_slice ue(v) sps_max_mtt_hierarchy_depth_intra_slice_luma ue(v) if( sps_max_mtt_hierarchy_depth_intra_slice_luma!= 0 ) { sps_log2_diff_max_bt_min_qt_intra_slice_luma ue(v) sps_log2_diff_max_tt_min_qt_intra_slice_luma ue(v) } if( sps_max_mtt_hierarchy_depth_inter_slices!= 0 ) { sps_log2_diff_max_bt_min_qt_inter_slice ue(v) sps_log2_diff_max_tt_min_qt_inter_slice ue(v) } if( qtbtt_dual_tree_intra_flag ) { sps_log2_diff_min_qt_min_cb_intra_slice_chroma ue(v) sps_max_mtt_hierarchy_depth_intra_slice_chroma ue(v) if ( sps_max_mtt_hierarchy_depth_intra_slice_chroma!= 0 ) { sps_log2_diff_max_bt_min_qt_intra_slice_chroma ue(v) sps_log2_diff_max_tt_min_qt_intra_slice_chroma ue(v) } } … rbsp_trailing_bits( ) }

[0076] log2_ctu_size_minus2 plus 2 specifies the luma coding tree block size of each ctu.

[0077] log2_min_luma_coding_block_size_minus2 plus 2 specifies the minimum luma coding block size.

[0078] The variables CtbLog2SizeY, CtbSizeY, MinCbLog2SizeY, MinCbSizeY, MinTbLog2SizeY, MaxTbLog2SizeY, MinTbSizeY, MaxTbSizeY, PicWidthInCtbsY, PicHeightInCtbsY, PicSizeInCtbsY, PicWidthInMinCbsY, PicHeightInMinCbsY, PicSizeInMinCbsY, PicSizeInMinCbsY, PicSizeInSamplesY, PicWidthInSamplesC, and PicHeightInSamplesC are derived as follows.

[0079] CtbLog2SizeY = log2_ctu_size_minus2 + 2 (7-9)

[0080] CtbSizeY = 1 « CtbLog2SizeY (7-10)

[0081] MinCbLog2SizeY = log2_min_luma_coding_block_size_minus2 + 2 (7-11)

[0082] MinCbSizeY = 1 « MinCbLog2SizeY (7-12)

[0083] MinTbLog2SizeY = 2 (7-13)

[0084] MaxTbLog2SizeY = 6 (7-14)

[0085] MinTbSizeY = 1 << MinTbLog2SizeY(7-15)

[0086] MaxTbSizeY = 1 << MaxTbLog2SizeY(7-16)

[0087] PicWidthInCtbsY = Ceil( pic_width_in_luma_samples ÷ CtbSizeY )(7-17)

[0088] PicHeightInCtbsY = Ceil( pic_height_in_luma_samples ÷ CtbSizeY )(7-18)

[0089] PicSizeInCtbsY = PicWidthInCtbsY PicHeightInCtbsY(7-19)

[0090] PicWidthInMinCbsY = pic_width_in_luma_samples / MinCbSizeY(7-20)

[0091] PicHeightInMinCbsY = pic_height_in_luma_samples / MinCbSizeY(7-21)

[0092] PicSizeInMinCbsY = PicWidthInMinCbsY PicHeightInMinCbsY(7-22)

[0093] PicSizeInSamplesY = pic_width_in_luma_samples pic_height_in_luma_samples(7-23)

[0094] PicWidthInSamplesC = pic_width_in_luma_samples / SubWidthC(7-24)

[0095] PicHeightInSamplesC = pic_height_in_luma_samples / SubHeightC(7-25)

[0096] Figure 4A is an example of a CTB crossing the bottom picture boundary. Figure 4B is an example of a CTB crossing the right picture boundary. Figure 4C is an example of a CTB crossing the bottom-right picture boundary. In Figures 4A-4C K = M, L < N; K < M, L = N; K < M, L < N, respectively.

[0097] Reference is made to Figures 4A-4C CTUs in picture 400 are discussed. Assume a CTB maximum coding unit (LCU) size, denoted by M x N (typically, M is equal to N, as defined in HEVC / VVC), and for CTBs located at picture (or slice or tile or other kind of type, taking picture boundaries as an example) boundaries, K x L samples are within the picture boundaries, with K < M or L < N. For Figures 4A-4C For those CTBs 402 depicted in

[0098] A coding flow of a typical video encoder / decoder (also called codec) is discussed. Figure 5 is an example of an encoder block diagram of VVC, which contains three in-loop filters: a deblocking filter (DF), a sample adaptive offset (SAO) filter, and an adaptive loop filter (ALF). Unlike DF which uses a pre-defined filter, SAO filter and ALF utilize original samples of the current picture to reduce the mean square error between original samples and reconstructed samples by adding offsets and applying a finite impulse response (FIR) filter, respectively, with the offsets and filter coefficients signaled by the coded side information. ALF is located at the last processing stage of each picture and can be regarded as a tool trying to capture and fix artifacts caused by preceding stages.

[0099] Figure 5 is a schematic diagram of an encoder 500. The encoder 500 is suitable for implementing VVC techniques. The encoder 500 comprises three in-loop filters, namely a deblocking filter (DF) 502, a sample adaptive offset (SAO) filter 504, and an ALF 506. Unlike DF 502 which uses a pre-defined filter, SAO filter 504 and ALF 506 utilize original samples of the current picture to reduce the mean square error between original samples and reconstructed samples by adding offsets and applying a FIR filter, respectively, with the offsets and filter coefficients signaled by the coded side information. ALF 506 is located at the last processing stage of each picture and can be regarded as a tool trying to capture and fix artifacts caused by preceding stages.

[0100] The encoder 500 also includes an intra-frame prediction component 508 and a motion estimation / compensation (ME / MC) component 510 configured to receive input video. The intra-frame prediction component 508 is configured to perform intra-frame prediction, while the ME / MC component 510 is configured to perform inter-frame prediction using a reference image obtained from a reference image buffer 512. Residual blocks from inter-frame or intra-frame prediction are fed to a transform component 514 and a quantization component 516 to generate quantized residual transform coefficients, which are then fed to an entropy codec component 518. The entropy codec component 518 entropy codes the prediction results and the quantized transform coefficients and sends them to a video decoder (not shown). The quantization component output from the quantization component 516 can be fed to an inverse quantization component 520, an inverse transform component 522, and a reconstruction (REC) component 524. The REC component 524 is capable of outputting images to a DF 502, a SAO 504, and an ALF 506 for filtering before these images are stored in the reference image buffer 512.

[0101] The input to the DF 502 is the reconstructed samples before the in-loop filters. First, vertical edges in the image are filtered. Then, using the samples modified by the vertical edge filtering process as input, horizontal edges in the image are filtered. Vertical and horizontal edges in the CTB of each CTU are processed separately on a codec unit basis. Vertical edges of the codec blocks in the codec unit are filtered starting from the left-hand edge of the codec block and proceeding geometrically towards the right-hand side of the codec block. Horizontal edges of the codec blocks in the codec unit are filtered, starting from the top edge of the codec block and proceeding geometrically towards the bottom edge of the codec block.

[0102] Figure 6 This is a diagram 600 showing sample point 602 within an 8×8 sample point block 604. As shown, diagram 600 includes horizontal and vertical block boundaries 606 and 608 on the 8×8 grid, respectively. Furthermore, diagram 600 depicts a non-overlapping block 610 of the 8×8 sample points, which can be de-blocked in parallel.

[0103] Boundary decisions were discussed. The filter was applied to 8×8 block boundaries. Furthermore, it must be a transform block boundary or an encoding / decoding sub-block boundary (e.g., due to the use of affine motion prediction, Alternate Temporal Motion Vector Prediction (ATMVP)). For those boundaries that are not such, the filter was disabled.

[0104] The calculation of boundary strength is discussed. For transform block boundaries / encoder / decoder sub-block boundaries, if they are located in an 8×8 grid, the transform block boundaries / encoder / decoder sub-block boundaries can be filtered, and the bS[xD] of the edge... i ][ yD j (where [xD])i ][ yD j ] are defined in Table 1 and Table 2, respectively.

[0105] Table 1. Boundary strength (when SPS Intra Block Copy (IBC) is disabled)

[0106] Priority Case Y U V 5 At least one adjacent block is intra 2 2 2 4 TU boundary and at least one adjacent block has non-zero transform coefficients 1 1 1 3 The number of reference pictures or MVs (1 for uni-prediction, 2 for bi-prediction) of adjacent blocks are different 1 N / A N / A 2 The absolute difference between motion vectors belonging to the same reference picture of adjacent blocks is greater than or equal to one integer luma sample 1 N / A N / A 1 Otherwise 0 0 0

[0107] Table 2. Boundary strength (when SPS IBC is enabled)

[0108] Priority Case Y U V 8 At least one adjacent block is intra 2 2 2 7 TU boundary and at least one adjacent block has non-zero transform coefficients 1 1 1 6 The prediction modes of adjacent blocks are different (e.g., one is IBC and one is inter) 1 5 Both IBC and the absolute difference between motion vectors belonging to adjacent blocks are greater than or equal to one integer luma sample 1 N / A N / A 4 The number of reference pictures or MVs (1 for uni-prediction, 2 for bi-prediction) of adjacent blocks are different 1 N / A N / A 3 The absolute difference between motion vectors belonging to the same reference picture of adjacent blocks is greater than or equal to one integer luma sample 1 N / A N / A 1 Otherwise 0 0 0

[0109] The deblocking decision for the luma component is discussed.

[0110] Figure 7 is an example 700 of pixels involved in the filter on / off decision and strong / weak filter selection. The wider and stronger luma filter is only used when condition 1, condition 2 and condition 3 are all TRUE. Condition 1 is the "large block condition". This condition checks whether the samples on the P side and the Q side belong to a large block, denoted by variables bSidePisLargeBlk and bSideQisLargeBlk, respectively. The definitions of bSidePisLargeBlk and bSideQisLargeBlk are as follows.

[0111] bSidePisLargeBlk = ((edge type is vertical and p0 belongs to a CU with width >= 32) || (edge type is horizontal and p0 belongs to a CU with height >= 32))? TRUE : FALSE

[0112] bSideQisLargeBlk = ((edge type is vertical and q0 belongs to a CU with width >= 32) || (edge type is horizontal and q0 belongs to a CU with height >= 32))? TRUE : FALSE

[0113] Based on bSidePisLargeBlk and bSideQisLargeBlk, condition 1 is defined as follows.

[0114] condition 1 = (bSidePisLargeBlk || bSideQisLargeBlk)? TRUE : FALSE

[0115] Next, if condition 1 is TRUE, condition 2 will be further checked. First, the following variables are derived.

[0116] dp0, dp3, dq0, dq3 are first derived as in HEVC

[0117] if (p side is greater than or equal to 32)

[0118] dp0 = ( dp0 + Abs( p50 - 2 p40 + p30 ) + 1 ) >> 1

[0119] dp3 = ( dp3 + Abs( p53 - 2 p43 + p33 ) + 1 ) >> 1

[0120] if (qSide is larger or equal to 32)

[0121] dq0 = ( dq0 + Abs( q50 - 2 q40 + q30 ) + 1 ) >> 1

[0122] dq3 = ( dq3 + Abs( q53 - 2 q43 + q33 ) + 1 ) >> 1

[0123] Cond2 = (d < beta)? true: false

[0124] where d = dp0 + dq0 + dp3 + dq3.

[0125] If Cond1 and Cond2 are valid, further check if any block uses subblock.

[0126] If (bSideP is LargeBlk)

[0127] {

[0128] If (mode block P == SUBBLOCKMODE)

[0129] Sp =5

[0130] else

[0131] Sp =7

[0132] }

[0133] else

[0134] Sp = 3

[0135] If (bSideQ is LargeBlk)

[0136] {

[0137] If (mode block Q == SUBBLOCKMODE)

[0138] Sq = 5

[0139] else

[0140] Sq = 7

[0141] }

[0142] else

[0143] Sq = 3

[0144] Finally, if both condition 1 and condition 2 are valid, the proposed deblocking method will check condition 3 (the large block strong filter condition), which is defined as follows.

[0145] In condition 3 StrongFilterCondition, the following variables are derived.

[0146] dpq is derived as in HEVC.

[0147] sp3 = Abs( p3 - p0 ) is derived as in HEVC,

[0148] If (p side is greater than or equal to 32)

[0149] If (Sp == 5)

[0150] sp3 = ( sp3+ Abs( p5 - p3) + 1) >> 1

[0151] else

[0152] sp3 = ( sp3+ Abs( p7 - p3) + 1) >> 1

[0153] sq3 = Abs( q0 - q3 ) is derived as in HEVC

[0154] if (q side is greater than or equal to 32)

[0155] if (Sq == 5)

[0156] sq3 = ( sq3+ Abs( q5 - q3) + 1) >> 1

[0157] else

[0158] sq3 = ( sq3+ Abs( q7 - q3) + 1) >> 1

[0159] StrongFilterCondition = (dpq is less than (β >> 2), sp3 + sq3 is less than (3 β>> 5), Abs( p0 - q0 ) less than (5 t C + 1 ) » 1 )? True : False.

[0160] A stronger luma deblocking filter (designed for larger blocks) is discussed.

[0161] When the samples on either side of the boundary belong to large blocks, a bilinear filter is used. A sample belonging to a large block is defined as when the width of the vertical edge >= 32 and the height of the horizontal edge >= 32.

[0162] The bilinear filter is shown as follows.

[0163] In the above HEVC deblocking, the block boundary samples pi for i = 0 to Sp-1 and qj for j = 0 to Sq-1 (pi and qj are the i-th sample in the row for filtering the vertical edge, or the i-th sample in the column for filtering the horizontal edge) are then replaced by the linear interpolation as follows.

[0164]

[0165]

[0166] where, The term is position dependent clipping, as described below; and the following gives , , and .

[0167] Deblocking control for chroma is discussed.

[0168] A strong chroma filter is used on both sides of the block boundary. Here, the chroma filter is selected when both sides of the chroma edge are greater than or equal to 8 (chroma position) and the following three conditions are satisfied. The first is the decision of the boundary strength and large block. The proposed filter can be applied when the block width or height in the chroma sample domain orthogonal to the block edge is equal to or greater than 8. The second and third decisions are basically the same as the HEVC luma deblocking decisions, which are the on / off decision and the strong filter decision, respectively.

[0169] In the first decision, the boundary strength (bS) is modified for the chroma filter and the conditions are checked sequentially. If the conditions are satisfied, the remaining decisions with lower priority will be skipped.

[0170] Chroma deblocking is performed when bS is equal to 2 or when bS is equal to 1 when a large block boundary is detected.

[0171] The second and third decisions are substantially the same as the HEVC luma strong filter decisions, as follows.

[0172] In the second decision: d is then derived as in HEVC luma deblocking. The second decision will be true when d is less than β.

[0173] In the third decision, StrongFilterCondition is derived as follows.

[0174] As derived in HEVC, sp3 = Abs( p3 - p0 ),

[0175] As derived in HEVC, sq3 = Abs( q0 - q3 ),

[0176] As in the HEVC design, StrongFilterCondition = (dpq is less than (β » 2), sp3 + sq3 is less than (β » 3), Abs( p0 - q0 ) is less than (5 t C + 1 ) » 1).

[0177] Strong deblocking filters for chroma are discussed. The following strong deblocking filters for chroma are defined.

[0178] p2' = (3 p3 + 2 p2 + p1 + p0 + q0 + 4) » 3

[0179] p1' = (2 p3 + p2 + 2 p1 + p0 + q0 + q1 + 4) » 3

[0180] p0' = (p3 + p2 + p1 + 2 p0 + q0 + q1 + q2 + 4) » 3

[0181] The proposed chroma filters perform deblocking on a 4x4 grid of chroma samples.

[0182] Position-dependent clipping (tcPD) is discussed. Position-dependent clipping tcPD is applied to the output samples of the luma filtering process that involves the strong and long filters of the 7, 5, and 3 samples at the boundary. Assuming a quantization error distribution, it is suggested to increase the clipping values of samples that are expected to have higher quantization noise, and thus are expected to have a larger deviation of the reconstructed sample value from the true sample value.

[0183] For each P or Q boundary filtered with asymmetric filter, the position dependent threshold table is selected from two tables (i.e., Tc7 and Tc3 in the following tables) provided to the decoder as side information according to the result of the decision process in the boundary strength calculation.

[0184] Tc7 = { 6, 5, 4, 3, 2, 1, 1}; Tc3 = { 6, 4, 2};

[0185] tcPD = (Sp == 3)? tc3 : Tc7;

[0186] tcQD = (Sq == 3)? tc3 : Tc7;

[0187] For P or Q boundary filtered with short symmetric filter, lower amplitude position dependent threshold is applied.

[0188] Tc3 = { 3, 2, 1};

[0189] After the threshold is defined, the filtered p' i and q' i sample values are clipped according to tcP and tcQ clipping values.

[0190] p'' i = Clip3(p' i + tcP i , p' i - tcP i , p' i );

[0191] q'' j = Clip3(q' j + tcQ j , q' j - tcQ j , q' j );

[0192] where p' i and q' i are the filtered sample values, p'' i and q'' j are the clipped output sample values, tcP i and tcQ j are the clipping thresholds derived from VVC tc parameters and tcPD and tcQD. The function Clip3 is the clipping function specified in VVC.

[0193] Subblock deblocking adjustment is discussed.

[0194] To implement parallel-friendly deblocking using long filter and subblock deblocking, the long filter is restricted to modify at most 5 samples on the side where subblock deblocking is used (affine or ATMVP or decoder-side motion vector refinement (DMVR)), as shown in the luma control of long filter. In addition, subblock deblocking is adjusted so that subblock boundaries on 8x8 grid close to CU or implicit transform unit (TU) boundaries are restricted to modify at most two samples on each side.

[0195] The following applies to subblock boundaries that are not aligned with CU boundaries.

[0196] If (mode block Q == SUBBLOCKMODE && edge!=0) {

[0197] if (!(implicitTU && (edge == (64 / 4))))

[0198] if (edge == 2 || edge == (orthogonalLength - 2) || edge == (56 / 4) || edge == (72 / 4))

[0199] Sp = Sq = 2;

[0200] else

[0201] Sp = Sq = 3;

[0202] else

[0203] Sp = Sq = bSideQisLargeBlk? 5:3

[0204] }

[0205] where edge equal to 0 corresponds to a CU boundary, edge equal to 2 or equal to orthogonalLength - 2 corresponds to a subblock boundary 8 samples away from a CU boundary, and so on. implicitTU is true if implicit splitting of TUs is used.

[0206] Sample adaptive offset is discussed. The input to SAO is the deblocked reconstructed samples (DB). The concept of SAO is to reduce the average sample distortion of a region by first classifying the region samples into multiple categories using a selected classifier, obtaining an offset for each category, and then adding the offset to each sample of the category, where the classifier index and the region’s offset are coded in the bitstream. In HEVC and VVC, a region (the unit for SAO parameter signaling) is defined as a CTU.

[0207] HEVC employs two types of SAO that can meet the low complexity requirement. The two types are edge offset (EO) and band offset (BO), which are discussed in detail below. The index of the SAO type is coded (in the range of [0, 2]). For EO, the sample classification is based on a comparison between the current sample and neighboring samples according to one-dimensional directional modes: horizontal, vertical, 135° diagonal, and 45° diagonal.

[0208] Figure 8 Four one-dimensional (1-D) directional modes 800 for EO sample classification are shown: horizontal (EO class = 0), vertical (EO class = 1), 135° diagonal (EO class = 2), and 45° diagonal (EO class = 3).

[0209] For a given EO class, each sample within a CTB is classified into one of five categories. The current sample value, labeled "c", is compared to its two neighboring sample values along the selected 1-D mode. The classification rule for each sample is summarized in Table 3. Categories 1 and 4 are associated with local valleys and local peaks along the selected 1-D mode, respectively. Categories 2 and 3 are associated with concave and convex corners along the selected one-dimensional pattern, respectively. If the current sample does not belong to EO categories 1-4, it belongs to category 0 and no SAO is applied.

[0210] Table 3: Sample classification rule for edge offset

[0211] Category Condition 1 c < a and c < b 2 (c < a && c==b) ||(c == a && c < b) 3 (c > a && c==b) || (c == a && c > b) 4 c > a && c > b 5 None of the above

[0212] A geometry-based adaptive loop filter in the Joint Exploration Model (JEM) is discussed. The input to the DB is the reconstructed samples after DB and SAO. The sample classification and filtering process are based on the reconstructed samples after DB and SAO.

[0213] In JEM, a geometry-based adaptive loop filter (GALF) with block-based filter adaptation is applied. For the luma component, one of 25 filters is selected for each 2x2 block according to the direction and activity of local gradients.

[0214] Filter shapes are discussed. Figure 9 An example of GALF filter shapes 900 is shown, including a 5x5 diamond on the left, a 7x7 diamond on the right, and a 9x9 diamond in the middle. In JEM, up to three diamond filter shapes can be selected for the luma component (as shown in Figure 9The index is signaled at picture level to indicate the filter shape for luma components. Each square represents one sample, and Ci (i is 0-6 (left), 0-12 (middle), 0-20 (right)) represents the coefficient applied to that sample. For chroma components in the picture, a 5x5 diamond is always used.

[0215] Block classification is discussed. Each block is classified into one of 25 classes. The classification index C is derived based on its directional and activity quantized values, as follows.

[0216] (1)

[0217] To calculate and , first the gradients in horizontal, vertical and two diagonal directions are calculated using 1-D Laplacian.

[0218] (2)

[0219] (3)

[0220] (4)

[0221] (5)

[0222] The index refers to the coordinates of the top-left sample in the block, and indicates the reconstructed sample at coordinates . The

[0223] maximum and minimum of the gradients in horizontal and vertical directions are then set as:

[0224] , , (6) And the maximum and minimum of the gradients in two diagonal directions are set as:

[0225]

[0226] , , (7) To derive the value of directional

[0227] , these values are compared with each other and with two thresholds :

[0228] First, if ​and both true, then is set to .

[0229] Second step, if , continue from step 3; otherwise continue from step 4.

[0230] Third step, if , then is set to ; otherwise is set to .

[0231] Fourth step, if , then is set to 4; otherwise is set to 3.

[0232] The activity value is calculated as follows:

[0233] (8)

[0234] is further quantized to the range 0 to 4, inclusive, and the quantized value is denoted as .

[0235] For both chroma components in a picture, the classification method is not applied, i.e., a single set of ALF coefficients is applied for each chroma component.

[0236] Geometric transformations of the filter coefficients are discussed.

[0237] Figure 10 An example of relative coordinates 1000 for a 5x5 diamond filter support is shown, for diagonal, vertical flip and rotation (from left to right), respectively.

[0238] Before filtering each 2x2 block, the filter coefficients associated with coordinates (k, l) are applied a geometric transformation such as rotation or diagonal and vertical flip. This is equivalent to applying these transformations to the samples in the filter support region. The idea is to make different blocks for which ALF is applied more similar by arranging the directionality of the different blocks.

[0239] Three geometric transformations are introduced, including diagonal, vertical flip and rotation:

[0240] Diagonal:

[0241] Vertical flip:

[0242] Rotation: (9)

[0243] where is the size of the filter, is the coefficient coordinate such that position is at the top-left corner, position is at the bottom-right corner. Depending on the gradient values computed for the block, a transform is applied to the filter coefficients . Table 4 summarizes the mapping between the transform and the four gradients of the four directions.

[0244] Table 4: Mapping of the gradients computed for a block and the transform

[0245] Gradient value Transform g d2 < g d1 and g h < g v ]]> No transform g d2 < g d1 and g v < g h ]]> Diagonal g d1 < g d2 and g h < g v ]]> Vertical flip g d1 < g d2 and g v < g h ]]> Rotation

[0246] Filter parameter signaling is discussed. In JEM, GALF filter parameters are signaled for the first CTU, i.e. after the slice header of the first CTU and before the SAO parameters. Up to 25 sets of luma filter coefficients can be signaled. To reduce the bit overhead, different categories of filter coefficients can be merged. Furthermore, the GALF coefficients of a reference picture are stored and allowed to be reused as GALF coefficients of the current picture. The current picture can choose to use the stored GALF coefficients of a reference picture and bypass the GALF coefficient signaling. In this case, only the index to one of the reference pictures is signaled and the stored GALF coefficients of the indicated reference picture are inherited by the current picture.

[0247] To support GALF temporal prediction, a candidate list of GALF filter sets is maintained. At the beginning of the decoding of a new sequence, the candidate list is empty. After the decoding of a picture, the corresponding filter set can be added to the candidate list. Once the size of the candidate list reaches the maximum allowed value (i.e. 6 in the current JEM), the new filter set overwrites the oldest set in the decoding order, i.e. the first-in-first-out (FIFO) rule is applied to update the candidate list. To avoid duplication, a set is only added to the list if the corresponding picture does not use GALF temporal prediction. To support temporal scalability, there are multiple candidate lists of filter sets and each candidate list is associated with a temporal layer. More specifically, each array assigned by the temporal layer index (TempIdx) can consist of filter sets with previously decoded pictures equal to lower TempIdx. For example, the k-th array is assigned to be associated with TempIdx equal to k and the k-th array only contains filter sets from pictures with TempIdx smaller than or equal to k. After a certain picture is coded, the filter set associated with the picture will be used to update those arrays associated with TempIdx equal to or higher.

[0248] Temporal prediction of the GALF coefficients is used for inter coded frames to minimize the signaling overhead. For intra frames, temporal prediction is not available and a set of 16 fixed filters is assigned to each class. To indicate the use of a fixed filter, a flag is signaled for each class and, if needed, the index of the selected fixed filter is also signaled. Even when a fixed filter is selected for a given class, the coefficients of the adaptive filter can still be transmitted for that class, in which case the coefficients of the filter that will be applied to reconstruct the picture are the sum of the two sets of coefficients.

[0249] The filtering process for the luma component can be controlled at the CU level. A flag is signaled to indicate whether GALF is applied to the luma component of a CU. For the chroma components, whether GALF is applied is only indicated at the picture level.

[0250] The filtering process is discussed. At the decoder side, when GALF is enabled for a block, every sample within that block is filtered, resulting in a sample value as shown below where L denotes the filter length, denotes the filter coefficients, and denotes the decoded filter coefficients.

[0251] (10)

[0252] Figure 11 Another example of relative coordinates 1100 for a 5x5 diamond filter support is shown assuming the coordinates (i, j) of the current sample are (0, 0). The samples in different coordinates that are filled with the same color are multiplied with the same filter coefficient.

[0253] The geometric transform based adaptive loop filter (GALF) in VVC is discussed. In VVC test model 4.0 (VTM4.0), the filtering process of the adaptive loop filter is performed as follows:

[0254] , (11)

[0255] where sample is the input sample, is the filtered output sample (i.e., the filter result), denotes the filter coefficients. In practice, in VTM4.0, it is

[0256] , (12)

[0257] where L denotes the filter length, and where are filter coefficients with fixed-point precision.

[0258] The current design of GALF in VVC has the following major changes compared to JEM:

[0259] 1) The adaptive filter shape is removed. For luma component, only 7x7 filter shape is allowed, for chroma component, only 5x5 filter shape is allowed.

[0260] 2) The signaling of ALF parameters is removed from slice / picture level to CTU level.

[0261] 3) The calculation of class index is performed at 4x4 level instead of 2x2 level. In addition, the sub-sampled Laplacian calculation method for ALF classification is utilized as proposed in JVET-L0147. More specifically, there is no need to calculate horizontal / vertical / 45 diagonal / 135 degree gradient for each sample within one block. Instead, 1:2 sub-sampling is used.

[0262] The non-linear ALF in current VVC is discussed with respect to filter reconstruction.

[0263] Equation (11) can be re-expressed as the following expression without affecting the coding efficiency:

[0264] , (13)

[0265] where, is the same filter coefficient as in equation (11) [except that equals 1 in equation (13), while it equals in equation (11)].

[0266] Using the above filter formula of equation (13), VVC introduces non-linearity to make ALF more effective by using a simple clipping function to reduce their impact when the neighboring sample values ) differ too much from the current sample value being filtered ).

[0267] More specifically, the ALF filter is modified as follows:

[0268] , (14)

[0269] where is a clipping function, is a clipping parameter depending on the filter coefficient. The encoder performs optimization to find the best value .

[0270] In the JVET-N0242 implementation, a clipping parameter is specified for each ALF filter Each filter coefficient is signaled with one clipping value. This means that up to 12 clipping values can be signaled in the bitstream for each luma filter and up to 6 clipping values for each chroma filter.

[0271] To limit the signaling cost and encoder complexity, only 4 fixed values are used, which are the same for inter and intra.

[0272] Because the variance of local differences is typically higher for luma than for chroma, two different sets of clipping values are applied for luma and chroma filters. A maximum sample value in each set is also introduced (here 1024 for 10-bit bit-depth) so that clipping can be disabled when not necessary.

[0273] Table 5 provides the set of clipping values used in the JVET-N0242 tests. These 4 values are chosen by roughly equally dividing the full range of sample values for luma (coded in 10 bits) and the range from 4 to 1024 for chroma in the log domain.

[0274] More precisely, the luma table of clipping values is obtained by the following formula:

[0275] AlfClip L , where M = 2 10 and N = 4 (15)

[0276] Similarly, the chroma table of clipping values is obtained according to the following formula:

[0277] AlfClip C , where M = 2 10 , N = 4 and A = 4 (16)

[0278] Table 5: authorized clipping values

[0279] Intra / inter slice Luma { 1024, 181, 32, 6} Chroma { 1024, 161, 25, 4}

[0280] The selected clipping values are coded in the “alf_data” syntax element by using a Golomb coding scheme corresponding to the index of the clipping value in the above table 5. The coding scheme is the same as for the coding of the filter indices.

[0281] Convolutional neural network based loop filters for video coding are discussed.

[0282] In deep learning, convolutional neural networks (CNNs, or ConvNets) are a class of deep neural networks most commonly used for analyzing visual images. They have very successful applications in image and video recognition / processing, recommender systems, image classification, medical image analysis, and natural language processing.

[0283] CNNs are regularized versions of multilayer perceptrons. Multilayer perceptrons generally mean fully connected networks, i.e., each neuron in one layer is connected to all neurons in the next layer. The "full connectivity" of these networks makes them prone to overfitting the data. Typical regularization methods include adding some form of weight magnitude measure to the loss function. CNNs take a different regularization approach: they exploit hierarchical patterns in the data and assemble more complex patterns using smaller and simpler ones. Thus, CNNs are at the lower end of the spectrum in terms of connectivity and complexity.

[0284] CNNs use relatively little preprocessing compared to other image classification / processing algorithms. This means that the network learns the filters that are hand-designed in traditional algorithms. This independence from prior knowledge and human effort in feature design is a major advantage.

[0285] Deep learning based image / video compression generally has two meanings: purely neural network based end-to-end compression and traditional framework enhanced by neural network. The neural network based end-to-end compression is discussed in the following: Johannes Balle, Valero Laparra, and Eero P. Simoncelli, “End-to-end optimization of nonlinear transform codes for perceptual quality,” 2016 Picture Coding Symposium (PCS), pp. 1-5, Institute of Electrical and Electronics Engineers (IEEE), and Lucas Theis, Wenzhe Shi, Andrew Cunningham, and Ferenc Huszar, “Lossy image compression with compressive autoencoders,” arXiv preprint arXiv:1703.00395 (2017).The following discuss traditional frameworks enhanced by neural networks: Jiahao Li, Bin Li, Jizheng Xu, Ruiqin Xiong, and Wen Gao, “Fully Connected Network-Based IntraPrediction for Image Coding” IEEE Transactions on Image Processing 27, 7(2018), 3236–3247; Yuanying Dai, Dong Liu, and Feng Wu, “A convolutional neural network approach for post-processing in HEVC intra coding” MMM. Springer, 28–39; Rui Song, Dong Liu, Houqiang Li, and Feng Wu, “Neural network-based arithmetic coding of intra prediction modes in HEVC” VCIP. IEEE, 1–4; and J. Pfaff, P. Helle, D. Maniry, S. Kaltenstadler, W. Samek, H. Schwarz, D. Marpe, and T. Wiegand, “Neural network based intraprediction for video coding”, Applications of Digital Image Processing XLI, Vol. 10752. International Society for Optics and Photonics, 1075213.

[0286] End-to-end compression usually adopts an autoencoder-like structure, implemented by either convolutional neural networks or recurrent neural networks. While relying purely on neural networks for image / video compression can avoid any manual optimization or hand-crafted design, the compression efficiency can not be satisfactory. Therefore, works in the second type of compression assist neural networks by replacing or enhancing certain modules to boost traditional compression frameworks. In this way, they can inherit the advantages of highly optimized traditional frameworks. For example, fully connected network-based intra prediction for image coding in HEVC as discussed in Jiahao Li, Bin Li, Jizheng Xu, Ruiqin Xiong, and Wen Gao, “Fully Connected Network-Based Intra Prediction for Image Coding” IEEE Transactions on Image Processing 27, 7 (2018), p. 3236-3247.

[0287] In addition to intra prediction, deep learning is also used to enhance other modules. For example, the in-loop filter of HEVC is replaced by a convolutional neural network in Yuanying Dai, Dong Liu, and Feng Wu, “A convolutional neural network approach for post-processing in HEVC intra coding” MMM. Springer, 28-39. Rui Song, Dong Liu, Houqiang Li, and Feng Wu, “Neural network-based arithmetic coding of intra prediction modes in HEVC” VCIP. IEEE, 1-4, which achieved satisfactory results. Neural networks are applied to improve the arithmetic coding engine.

[0288] In-loop filter based on convolutional neural network is discussed. In lossy image / video compression, the reconstructed frame is an approximation of the original frame, because the quantization process is irreversible, resulting in distortion of the reconstructed frame. To alleviate this distortion, a convolutional neural network can be trained to learn the mapping from the distorted frame to the original frame. In practice, training must be performed before deploying the CNN-based in-loop filter.

[0289] Training is discussed. The purpose of the training process is to find the best values of the parameters including weights and biases.

[0290] First, a codec (e.g., HM, JEM, VTM, etc.) is used to compress the training dataset to generate distorted reconstructed frames. Then, the reconstructed frames are fed into the CNN, and the cost is calculated using the output of the CNN and the ground truth frame (original frame). Commonly used cost functions include the sum of absolute differences (SAD) and the mean squared error (MSE). Next, the gradient of the cost with respect to each parameter is derived through a backpropagation algorithm. With the gradient, the parameter values can be updated. The above process is repeated until the convergence criterion is met. After the training is completed, the derived best parameters are saved for the inference phase.

[0291] The convolution process is discussed. In the convolution process, the filter moves over the picture from left to right and top to bottom, changing column by one pixel horizontally and row by one pixel vertically. The amount of movement between the application of the filter to the input image is called the stride, and it is almost always symmetric in the height and width dimensions. The default stride in two dimensions is (1, 1) for both height and width movement.

[0292] Figure 12A is an example architecture 1200 of the proposed CNN filter, Figure 12B is an example of the construction 1250 of a residual block (ResBlock). In most deep convolutional neural networks, the residual block is used as a basic module and is stacked several times to build the final network, where in one example, the residual block is obtained by combining a convolution layer, a ReLU / PReLU activation function, and a convolution layer, as shown in Figure 12B .

[0293] Inference is discussed. In the inference phase, the distorted reconstructed frames are fed into the CNN and processed by the CNN model whose parameters have been determined in the training phase. The input samples of the CNN can be the reconstructed samples before or after DB, or the reconstructed samples before or after SAO, or the reconstructed samples before or after ALF.

[0294] There are the following problems with the current CNN-based in-loop filter. First, the padding size is fixed for different types of different slices (e.g., intra mode slice (I slice), uni-prediction slice (P slice), and bi-prediction slice (B slice)). Second, the padding size is fixed for different temporal layers.

[0295] Techniques are disclosed herein that address one or more of the foregoing problems. For example, the invention provides techniques for determining, in real-time, padding dimensions for padding samples to be applied to a video unit for in-loop filtering. Thus, the size of the padding region can be adjusted to accommodate different types of multi-layer bitstreams and / or different slices of temporal layers. As a result, the video coding process is improved relative to conventional video coding techniques.

[0296] The following detailed embodiments should be considered illustrative of the general concepts explained herein. These embodiments should not be construed as being narrowly limited to the embodiments themselves. Moreover, these embodiments can be combined in any manner.

[0297] One or more neural network (NN) filter models are trained as part of in-loop filtering techniques or filtering techniques used in post-processing stages for reducing distortions caused during compression. Samples with different characteristics are processed by different neural network filter models. The invention sets forth how to decide padding dimensions for different video units to achieve better performance and how to handle samples located at the boundaries of video units.

[0298] In this disclosure, a NN filter can be any kind of NN filter, such as a convolutional neural network (CNN) filter. In the following discussion, a NN filter can also be referred to as a non-CNN filter, e.g., a filter using a machine learning based solution.

[0299] In the following discussion, a video unit can be a picture sequence, a picture, a slice, a tile, a brick, a subpicture, a CTU / CTB, a CTU / CTB row, one or more CUs / coding blocks (CBs), one or more CTUs / CTBs, one or more virtual pipeline data units (VPDUs), a subregion within a picture / slice / tile / brick. A parent video unit (also referred to as a parent video unit) represents a unit larger than a video unit. Typically, a parent video unit will contain several video units, e.g., when a video unit is a CTU, a parent video unit can be a slice, a CTU row, multiple CTUs, etc. In some embodiments, a video unit can be a sample / pixel.

[0300] Figure 13 FIG. 13 is a schematic diagram showing an example of uni-directional inter prediction 1300. The uni-directional inter prediction 1300 can be used to determine motion vectors for encoded and / or decoded blocks created when partitioning a picture.

[0301] Unidirectional inter prediction 1300 employs a reference frame 1330 having a reference block 1331 to predict a current block 1311 in a current frame 1310. As shown, the reference frame 1330 can be temporally subsequent to the current frame 1310 (e.g., as a subsequent reference frame), but in some examples can also be temporally prior to the current frame 1310 (e.g., as an earlier reference frame). The current frame 1310 is an example frame / picture that is being encoded / decoded at a particular time. The current frame 1310 contains an object in the current block 1311 that matches an object in the reference block 1331 of the reference frame 1330. The reference frame 1330 is a frame used as a reference to encode the current frame 1310, and the reference block 1331 is a block in the reference frame 1330 that contains an object that is also contained in the current block 1311 of the current frame 1310.

[0302] The current block 1311 is any coding unit being encoded / decoded at a specified point in the coding process. The current block 1311 can be an entire partitioned block, or a sub-block when an affine inter prediction mode is employed. The current frame 1310 is separated from the reference frame 1330 by a temporal distance (TD) 1333. The TD 1333 indicates an amount of time between the current frame 1310 and the reference frame 1330 in a video sequence, and can be measured in units of frames. Prediction information for the current block 1311 can reference the reference frame 1330 and / or the reference block 1331 by a reference index that indicates a direction and a temporal distance between frames. Within the temporal segment represented by the TD 1333, an object in the current block 1311 moves from a location in the current frame 1310 to another location in the reference frame 1330 (e.g., a location of the reference block 1331). For example, the object can move along a motion trajectory 1313, which is a direction in which the object moves over time. A motion vector 1335 describes a direction and magnitude of the object’s motion along the motion trajectory 1313 over the TD 1333. Thus, the encoded motion vector 1335, the reference block 1331, and a residual that includes a difference between the current block 1311 and the reference block 1331 provide information sufficient to reconstruct the current block 1311 and position the current block 1311 in the current frame 1310.

[0303] Figure 14 is a diagram illustrating an example of bi-directional inter prediction 1400. Bi-directional inter prediction 1400 can be used to determine motion vectors for encoded and / or decoded blocks created when partitioning a picture.

[0304] Bidirectional inter prediction 1400 is similar to unidirectional inter prediction 1300, but employs a pair of reference frames to predict a current block 1411 in a current frame 1410. Thus, current frame 1410 and current block 1411 are substantially similar to current frame 1310 and current block 1311, respectively. Current frame 1410 is positioned in the time domain between a previous reference frame 1420 that precedes current frame 1410 in the video sequence and a subsequent reference frame 1430 that follows current frame 1410 in the video sequence. Previous reference frame 1420 and subsequent reference frame 1430 are substantially similar to reference frame 1330 in other respects.

[0305] Current block 1411 is matched to a previous reference block 1421 in previous reference frame 1420 and a subsequent reference block 1431 in subsequent reference frame 1430. This matching indicates that, over the course of the video sequence, the object moves along motion trajectory 1413 and from a position at previous reference block 1421 to a position at subsequent reference block 1431 via current block 1411. Current frame 1410 is separated from previous reference frame 1420 by some previous temporal distance (TD0) 1423, and is separated from subsequent reference frame 1430 by some subsequent temporal distance (TD1) 1433. TD0 1423 indicates, in units of frames, an amount of time between previous reference frame 1420 and current frame 1410 in the video sequence. TD1 1433 indicates, in units of frames, an amount of time between current frame 1410 and subsequent reference frame 1430 in the video sequence. Thus, the object moves along motion trajectory 1413 from previous reference block 1421 to current block 1411 over a period of time indicated by TD0 1423. The object also moves along motion trajectory 1413 from current block 1411 to subsequent reference block 1431 over a period of time indicated by TD1 1433. Prediction information for current block 1411 can reference previous reference frame 1420 and / or previous reference block 1421 and subsequent reference frame 1430 and / or subsequent reference block 1431 by indicating a pair of reference indices for direction and temporal distance between frames.

[0306] A previous motion vector (MV0) 1425 describes a direction and magnitude of motion of the object along motion trajectory 1413 over TD0 1423 (e.g., between previous reference frame 1420 and current frame 1410). A subsequent motion vector (MV1) 1435 describes a direction and magnitude of motion of the object along motion trajectory 1413 over TD1 1433 (e.g., between current frame 1410 and subsequent reference frame 1430). In this way, in bidirectional inter prediction 1400, current block 1411 can be coded and reconstructed by employing previous reference block 1421 and / or subsequent reference block 1431, MV0 1425, and MV1 1435.

[0307] In one embodiment, inter prediction and / or bi-directional inter prediction can be performed on a sample-by-sample (e.g., pixel-by-pixel) basis rather than on a block-by-block basis. That is, a motion vector can be determined for each sample in the current block 1411 that points to each sample in the previous reference block 1421 and / or the subsequent reference block 1431. In such an embodiment, Figure 14 The previous motion vector 1425 and the subsequent motion vector 1435 depicted in FIG. 15 represent multiple motion vectors corresponding to multiple samples in the current block 1411, the previous reference block 1421, and the subsequent reference block 1431.

[0308] In merge mode and advanced motion vector prediction (AMVP) mode, a candidate list is generated by adding candidate motion vectors to the candidate list in the order defined by the mode of the candidate list determination. Such candidate motion vectors can include motion vectors according to uni-directional inter prediction 1300, bi-directional inter prediction 1400, or a combination thereof. Specifically, when neighboring blocks are encoded, motion vectors are generated for these blocks. Such motion vectors are added to the candidate list for the current block, and a motion vector for the current block is selected from the candidate list. The motion vector can then be signaled as an index to the selected motion vector in the candidate list. A decoder can use the same process as the encoder to build the candidate list, and can determine the selected motion vector from the candidate list based on the signaled index. Thus, the candidate motion vectors include motion vectors generated according to uni-directional inter prediction 1300 and / or bi-directional inter prediction 1400, depending on which method is used when encoding such neighboring blocks.

[0309] A slice is an integer number of complete tiles or an integer number of consecutive complete coding tree units (CTU) lines within one tile of a picture that is exclusively contained in a single network abstraction layer (NAL) unit. When a slice contains one or more video units generated using intra prediction, the slice can be referred to as an I slice or I slice type. When a slice contains one or more video units generated using uni-directional inter prediction as shown in Figure 13 When a slice contains one or more video units generated using bi-directional inter prediction as shown in Figure 14 When a slice contains one or more video units generated using bi-directional inter prediction as shown in

[0310] Figure 15 is a diagram showing an example of layer-based prediction 1500. Layer-based prediction 1500 is compatible with uni-directional inter prediction and / or bi-directional inter prediction, but also performs between pictures in different layers.

[0311] Layer-based prediction 1500 is applied between pictures 1511, 1512, 1513, and 1514 in a different layer (also referred to as a temporal layer) and pictures 1515, 1516, 1517, and 1518. In the example shown, pictures 1511, 1512, 1513, and 1514 are part of layer N+1 1532, and pictures 1515, 1516, 1517, and 1518 are part of layer N 1531. A layer, such as layer N 1531 and / or layer N+1 1532, is a group of pictures that are all associated with similar values of characteristics, such as similar size, quality, resolution, signal-to-noise ratio, capacity, etc. In the example shown, layer N+1 1532 is associated with larger picture sizes than layer N 1531. Thus, in this example, pictures 1511, 1512, 1513, and 1514 in layer N+1 1532 have larger picture sizes (e.g., larger height and width, and thus more samples) than pictures 1515, 1516, 1517, and 1518 in layer N 1531. However, such pictures can be separated between layer N+1 1532 and layer N 1531 by other characteristics. While only two layers, layer N+1 1532 and layer N 1531, are shown, a group of pictures can be separated into any number of layers based on associated characteristics. Layer N+1 1532 and layer N 1531 can also be represented by a layer identifier (ID). A layer ID is a data item associated with a picture and indicates that the picture is part of the indicated layer. Thus, each picture 1511-1518 can be associated with a corresponding layer identifier (ID) to indicate which of layer N+1 1532 or layer N 1531 includes the corresponding picture.

[0312] Pictures 1511-1518 in different layers 1531-1532 are configured to be displayed alternately. As such, pictures 1511-1518 in different layers 1531-1532 can share the same temporal ID and can be included in the same access unit (AU) 1506. As used herein, an AU is a set of one or more coded pictures associated with the same display time domain for output from a decoded picture buffer (DPB). For example, a decoder can decode and display picture 1515 at the current display time if smaller pictures are desired, or a decoder can decode and display picture 1511 at the current display time if larger pictures are desired. As such, pictures 1511-1514 at the higher layer N+1 1532 contain substantially the same picture data as corresponding pictures 1515-1518 at the lower layer N 1531 (although the picture sizes are different). Specifically, picture 1511 contains substantially the same picture data as picture 1515, picture 1512 contains substantially the same picture data as picture 1516, and so on.

[0313] Pictures 1511-1518 can be coded by referring to other pictures 1511-1518 in the same layer N 1531 or layer N+1 1532. Coded pictures to another picture in the same layer results in inter-prediction 1523, which is compatible with uni-directional inter-prediction and / or bi-directional inter-prediction. Inter-prediction 1523 is depicted by solid arrows. For example, picture 1513 can be coded with inter-prediction 1523 by using one or both of pictures 1511, 1512, and / or 1514 in layer N+1 1532 as reference, where one picture is referenced for uni-directional inter-prediction and / or two pictures are referenced for bi-directional inter-prediction. Further, picture 1517 can be coded with inter-prediction 1523 by using one or both of pictures 1515, 1516, and / or 1518 in layer N 1531 as reference, where one picture is referenced for uni-directional inter-prediction and / or two pictures are referenced for bi-directional inter-prediction. When a picture is used as a reference picture for another picture in the same layer when performing inter-prediction 1523, the picture can be referred to as a reference picture. For example, picture 1512 can be a reference picture used to code picture 1513 according to inter-prediction 1523. In a multi-layer context, inter-prediction 1523 can also be referred to as intra-layer prediction. As such, inter-prediction 1523 is a mechanism of coding a sample of a current picture by referring to an indicated sample in a reference picture that is different from the current picture, where the reference picture and the current picture are in the same layer.

[0314] Pictures 1511-1518 can also be coded by referring to other pictures 1511-1518 in different layers. This process is referred to as inter-layer prediction 1521 and is represented by dashed arrows. Inter-layer prediction 1521 is a mechanism of coding a sample of a current picture by referring to an indicated sample in a reference picture, where the current picture and the reference picture are in different layers and thus have different layer IDs. For example, a picture in a lower layer N 1531 can be used as a reference picture to code a corresponding picture in a higher layer N+1 1532. As a specific example, picture 1511 can be coded according to inter-layer prediction 1521 with reference to picture 1515. In this case, picture 1515 is used as an inter-layer reference picture. An inter-layer reference picture is a reference picture used for inter-layer prediction 1521. In most cases, inter-layer prediction 1521 is constrained such that a current picture (e.g., picture 1511) can only use inter-layer reference pictures (e.g., picture 1515) that are included in the same AU 1506 and are in a lower layer. When multiple layers (e.g., more than two) are available, inter-layer prediction 1521 can code / decode a current picture based on multiple inter-layer reference pictures that are lower in level than the current picture.

[0315] The video encoder can use layer-based prediction 1500 to encode the pictures 1511-1518 through many different combinations and / or permutations of inter-prediction 1523 and inter-layer prediction 1521. For example, picture 1515 can be coded according to intra-prediction. Pictures 1516-1518 can then be coded according to inter-prediction 1523 by using picture 1515 as a reference picture. In addition, picture 1511 can be coded according to inter-layer prediction 1521 by using picture 1515 as an inter-layer reference picture. Pictures 1512-1514 can then be coded according to inter-prediction 1523 by using picture 1511 as a reference picture. In this way, reference pictures can be used as single-layer reference pictures and inter-layer reference pictures for different coding mechanisms. By coding a higher layer N+1 1532 picture based on a lower layer N 1531 picture, the higher layer N+1 1532 can avoid using intra-prediction, which has much lower coding efficiency than inter-prediction 1523 and inter-layer prediction 1521. In this way, the low coding efficiency of intra-prediction can be limited to the minimum / lowest quality pictures and, thus, to the least amount of video data coded. Pictures used as reference pictures and / or inter-layer reference pictures can be indicated in entries of a reference picture list(s) included in a reference picture list structure.

[0316] Figure 15 Each AU 1506 can contain several pictures. For example, one AU 1506 can contain pictures 1511 and 1515. Another AU 1506 can contain pictures 1512 and 1516. In fact, each AU 1506 is a set of one or more coded pictures (e.g., for display to a user) associated with the same display time (e.g., same temporal ID) for output from a decoded picture buffer (DPB). Each access unit delimiter (AUD) 1508 is an indicator or data structure used to indicate the start of an AU (e.g., AU 1506) or a boundary between AUs.

[0317] The previous H.26x video codec family has provided support for scalability in profiles that are independent of single-layer coding. Scalable Video Coding (SVC) is a scalable extension of AVC / H.264, supporting spatial, temporal, and quality scalability. For SVC, a flag is signaled in each macroblock (MB) in an enhancement layer (EL) picture to indicate whether the EL MB is predicted using a colocated block from a lower layer. The prediction from the colocated block can include texture, motion vector, and / or coding mode. Implementations of SVC cannot directly reuse the unmodified H.264 / AVC implementation in their design. The SVC EL macroblock syntax and decoding process are different from the H.264 / AVC syntax and decoding process.

[0318] Scalable HEVC (SHVC) is an extension of the HEVC / H.265 standard, supporting spatial and quality scalability, Multiview HEVC (MV-HEVC) is an extension of HEVC / H.265, supporting multi-view scalability, and 3D HEVC (3D-HEVC) is an extension of HEVC / H.264, supporting a more advanced, more efficient three-dimensional (3D) video coding than MV-HEVC. Note that temporal scalability is included as a component of the single-layer HEVC codec. The design of the multi-layer extension of HEVC adopts the idea where the decoded pictures for inter-layer prediction only come from the same AU and are treated as long-term reference pictures (LTRPs), and are assigned reference indices in the reference picture lists together with other temporal reference pictures in the current layer. Inter-layer prediction (ILP) is implemented at the prediction unit (PU) level by setting the value of the reference index to refer to the inter-layer reference picture in the reference picture list.

[0319] It is worth noting that both the reference picture resampling and the spatial scalability features require resampling of a reference picture or a portion thereof. The reference picture resampling (RPR) can be implemented at the picture level or at the coding block level. However, when RPR is referred to as a coding feature, it is a feature for single-layer coding. Even so, from the codec design perspective, it is possible or even preferred to use the same resampling filter for the RPR feature for single-layer coding and the spatial scalability feature for multi-layer coding.

[0320] Copy or repeated padding can be used to extend a picture to a larger size. More specifically, a reference picture (e.g., Figure 13 the reference frame 1330 in FIG. 13A, or Figure 14The previous reference frame 1420 or the subsequent reference frame 1430 in which the reference block 1331 is located is extended to form a larger picture. For example, the boundary samples located at the left side boundary of the reference picture are copied to the left side of the reference picture, the boundary samples located at the right side boundary of the reference picture are copied to the right side of the reference picture, the boundary samples located at the top boundary of the reference picture are copied to the top of the reference picture, and the boundary samples located at the bottom boundary of the reference picture are copied to the bottom of the reference picture. These copied boundary samples located outside the reference picture are referred to as padded samples (also referred to as padding samples).

[0321] For current picture coding, when the motion vector (e.g., MV 1335) of the current block (e.g., current block 1311) points to (partially or completely) a reference block (e.g., reference block 1331) located outside the reference picture (e.g., reference frame 1330), the prediction block of the current block is generated from the padded samples outside the reference picture boundary.

[0322] Motion compensated padding is discussed in Y.W. Chen et al., “Description of SDR HDR and 360 video coding technology proposal by Qualcomm and Technicolor - low and high complexity versions,” JVET document JVET-J0021, 2018. When the decoder performs motion compensation, if the motion vector points to a block outside the reference frame boundary, a portion of the reference block is unavailable. To address this issue, padded samples can be used to extend or upscale the reference picture / frame. For each region of size 4xM or Mx4 along the boundary of the reference picture to be padded, M is the desired frame boundary extension, the motion vector is derived from the nearest 4x4 block within the frame. If the nearest 4x4 block is intra coded, a zero motion vector is used. If the nearest 4x4 block is coded with bi-directional inter prediction, only the motion vector pointing away from the frame boundary is used for the motion compensation of padding. After the motion vector derivation, the motion compensation is then performed to obtain the pixels in the padded region, while taking into account the average pixel value offset between the nearest 4x4 block and the corresponding block in its reference picture.

[0323] A discussion of padding size is provided.

[0324] Example 1

[0325] 1. Figure 16A padded video unit 1600 is shown, where 、 、 、 are padding dimensions for the top, bottom, left, and right boundaries 1604, respectively, of the padded video unit 1600. The padding dimensions collectively form a padding region 1602 around the padded video unit 1600. As described above, the padding region 1602 includes padding samples used to predict samples within the padded video unit 1600. Samples within the padded video unit 1600 that abut one or more boundaries 1604 of the padded video unit 1600 can be referred to as boundary samples or neighboring samples.

[0326] In one embodiment, the padding size related parameters (e.g., 、 、 、 ) are determined in real-time (also referred to as instantaneously). Real-time (or real-time) describes various operations in a computation or other process that guarantee a response time within a specified time (deadline), typically a relatively short time. A real-time process is typically a process that occurs within a time step of a defined maximum duration and is fast enough to affect the environment in which the process occurs, such as input to a computing system. In one embodiment, real-time means deciding and / or implementing dimensions in a prediction process.

[0327] a. In one example, 、 、 、 are different.

[0328] b. In one example, 、 、 、 are the same.

[0329] c. In one example, = = .

[0330] d. In one example, an indication of the padding size is signaled in the bitstream. In one embodiment, the indication is a hint, an initial parameter, or other value used to derive the padding size. The padding size refers to the size of the padding region 1602 and / or one or more padding dimensions.

[0331] e. In one example, at least one padding size related parameter is determined according to a position of the video unit relative to a parent video unit.

[0332] i. In one example, , , , At least one of them is different for video units located at the boundary of the parent video unit, and , , , The same applies to internal video units.

[0333] f. In one example, the padding size depends on the video unit size.

[0334] i. In one example, the padding size is the same for video units of different block sizes.

[0335] ii. In one example, a larger padding size is used for a larger video unit.

[0336] ii. In one example, the ratio of padding size to video unit size is a predefined constant.

[0337] g. In one example, the fill size depends on the color format (e.g., 4:4:4 or 4:2:0) and / or the color channel type.

[0338] i. In one example, the fill size is the same for different channel types.

[0339] ii. In one example, the fill size is set larger for the luminance component and smaller for the chrominance component.

[0340] h. In one example, the padding size depends on the stripe type and / or the split tree type (single-tree or dual-tree). A split tree is a structure that visually represents dividing a set into subsets such that every element in the set lies in exactly one of the subsets. In a single-tree, the split tree is shared by the luma and chroma components. In a dual-tree, there may be one split tree for the luma component and another for the chroma component.

[0341] i. In one example, the fill size is the same for different types of strips.

[0342] ii. In one example, the padding size is set to be larger for intra-frame stripes and smaller for inter-frame stripes.

[0343] i. In one example, the padding size depends on the temporal layer and / or reference image information.

[0344] i. In one example, the padding size is the same for different time-domain layers.

[0345] ii. In one example, the padding size is set larger for low temporal layers (e.g., layer 1531 in FIG. 15B) and smaller for high temporal layers (e.g., layer 1532 in FIG. 15B). Figure 15 Figure 15

[0346] j. In one example, the padding size of a first video unit depends on the kind of NN filtering method applied to the first video unit.

[0347] k. In one example, the padding size depends on other decoding information (e.g., properties of a video unit, such as percentage of intra coded samples).

[0348] l. In one example, the padding size depends on whether a NN filter is applied.

[0349] m. In one example, the padding dimension is set to 8 for internal boundaries and 0 for boundaries located at the parent video unit boundary. In one embodiment, the internal boundary is a boundary of a child video unit within a parent video unit and is created by partitioning the parent video unit.

[0350] n. In one example, a video unit is set to a block (e.g., a coding block) whose size depends on the slice resolution and quantization parameter (QP). Resolution can refer to spatial resolution (e.g., size of a display), temporal resolution (e.g., frame rate), or video quality (e.g., number of pixels displayed - 1920 x 1080, etc.).

[0351] o. In one example, a parent video unit (also referred to as a parent video unit) is set to a slice.

[0352] A discussion of padding methods is provided.

[0353] Example 2

[0354] 2. The padding method (e.g., how to generate the values of the samples outside the video unit and / or whether to apply padding) can be decided in real time (also referred to as on-the-fly).

[0355] a. In one example, whether to apply padding can depend on whether at least one or all of the samples outside the video unit are available.

[0356] i. In one example, when all the samples in the area to be padded are available for all boundaries, such as the top boundary, bottom boundary, left boundary, and right boundary, these samples are used directly without padding.

[0357] ​​​​​ii. In one example, when at least one sample in the padded region is not available for all boundaries, all neighboring samples are padded.

[0358] b. In one example, whether to apply padding can depend on whether at least one or all of the samples outside the video unit along a given direction (e.g., top / bottom / left / right) are available.

[0359] i. In one example, if all neighboring samples are available for a particular boundary (e.g., top boundary, bottom boundary, left boundary, or right boundary), the neighboring samples are used directly without padding.

[0360] ii. In one example, when at least one sample in the padded region is not available for a particular boundary, all neighboring samples of that boundary are padded.

[0361] c. In one example, available samples in the padded region are used directly without padding, and unavailable samples in the padded region can be padded.

[0362] d. In one example, the padding method can be one of the methods from zero padding, reflection padding, copy padding, constant padding, and mirror padding. In one embodiment, zero padding occurs when a boundary of all zero-valued samples (or pixels) is added around the edges of the input picture. This adds zero padding outside the picture, hence the name zero padding. In one embodiment, during reflection padding, values are filled in the opposite direction of the edge of the “to be padded” shape (e.g., video unit) with the “reflection” or “mirror” of the values. In one embodiment, during copy padding, when the selected reference block is outside or across the frame boundary of the reference frame, the frame boundary pixels are repeated. In one embodiment, the initial picture for reconstruction is padded with a uniform intensity boundary, which is referred to as constant padding. Figure 17 Mirror padding 1700 is shown, where the gray blocks represent padded samples. The numbers within each block are provided to illustrate suitable patterns for implementing mirror padding.

[0363] i. In one example, to apply mirror padding, the values outside the boundary are obtained by mirroring the video unit across its boundary, as shown in Figure 17

[0364] e. The padding method can depend on the video unit size.

[0365] f. In one example, an indication of the padding method is signaled in the bitstream.

[0366] g. In one example, at least one padding method related parameter (e.g., padding dimension) is determined according to the location of the video unit relative to the parent video unit.​

[0367] h. In one example, the padding method of a first video unit depends on the kind of NN filtering method applied to the first video unit.

[0368] i. In one example, the padding method depends on other decoding information (e.g., the characteristics of a video unit, such as the percentage of intra coded samples).

[0369] j. In one example, the padding method depends on whether a NN filter is applied.

[0370] k. The padding method can depend on the channel type.

[0371] l. The padding method can depend on the slice type.

[0372] m. The padding method can depend on the temporal layer.

[0373] Example 3

[0374] 3. In one example, the samples in the padded region are the unfiltered version, i.e., before the NN filter is applied.

[0375] Example 4

[0376] 4. In one example, the samples in the padded region are the filtered version, i.e., after the NN filter is applied.

[0377] The interfering block size is discussed.

[0378] Example 5

[0379] 5. The granularity of the NN filter model selection can be different from the CTU size.

[0380] a. The granularity can be predefined, or an indication of the granularity can be signaled in the bitstream or derived on the fly.

[0381] b. In one example, the granularity depends on the QP and the resolution.

[0382] i. In one example, the granularity is coarser when the QP is larger or the resolution is higher.

[0383] ii. In one example, let q denote the sequence level QP and w denote the frame width. When q < 23, the granularity is 32 32. When 23 q < 29, w 832, the granularity is 32 32. When 23 q < 29, w 832, the granularity is 64 64. When q 29 and w 832, the granularity is 128 128. When q 29 and w 832, the granularity is 256 256.

[0384] Binarization of neural network filter model indices is discussed.

[0385] Example 6

[0386] Binarization of NN filter model indices for a first video unit, such as a coding block or region, can depend on a maximum number of models allowed at a level higher than the first video unit. Binarization is the process of dividing data into two groups and assigning one of two values to all members of the same group. This is typically achieved by defining a threshold and assigning the value 0 to all data points below the threshold and the value 1 to data points above the threshold.

[0387] a. In one example, the high level is slice / picture / sequence level.

[0388] b. In one example, the indication of the maximum number of models is signaled at the high level, or is pre-defined or derived on the fly.

[0389] i. In one example, the indication of the maximum number of models is signaled in the picture header / slice header / picture parameter set (PPS) / sequence parameter set (SPS) / adaptation parameter set (APS).

[0390] c. In one example, the NN filter model indices can be binarized into a truncated code, such as a truncated unary code or a truncated binary code. Unary coding is an entropy coding that represents natural numbers n with n ones followed by a zero. Truncated unary coding is a variant of unary coding. Truncated binary or truncated binary coding is an entropy coding that is typically used for uniform probability distributions with a finite alphabet.

[0391] Figure 18 is a block diagram illustrating an example video processing system 1800 in which various techniques disclosed herein can be implemented. Various implementations can include some or all of the components of the video processing system 1800. The video processing system 1800 can include an input 1802 for receiving video content. The video content can be received in a raw or uncompressed format, e.g., 8 or 10-bit multi-component pixel values, or can be in a compressed or encoded format. The input 1802 can represent a network interface, a peripheral bus interface, or a storage interface. Examples of network interfaces include wired interfaces such as Ethernet, passive optical networks (PONs), etc., and wireless interfaces such as Wi-Fi or cellular interfaces.

[0392] The video processing system 1800 can include a codec component 1804, which can implement various coding or encoding methods described in the present document. The codec component 1804 can reduce the average bitrate of a video from the input 1802 to the output of the codec component 1804 to produce a coded representation of the video. The coding techniques are thus sometimes called video compression or video transcoding techniques. The output of the codec component 1804 can be stored, or transmitted via a connected communication, as represented by component 1806. The component 1808 can generate pixel values or displayable video from a stored or transmitted bitstream (or coded) representation of the video received at the input 1802 to send to a display interface 1810. The process of generating user-viewable video from a bitstream representation is sometimes called video decompression. Also, while certain video processing operations are referred to as “coding” operations or tools, it should be understood that the encoding tools or operations are used at an encoder and corresponding decoding tools or operations inverse to the encoding results will be performed by a decoder.

[0393] Examples of peripheral bus interfaces or display interfaces can include Universal Serial Bus (USB) or High Definition Multimedia Interface (HDMI) or DisplayPort, etc. Examples of storage interfaces include SATA (Serial Advanced Technology Attachment), Peripheral Component Interconnect (PCI), Integrated Drive Electronics (IDE) interface, etc. The techniques described in the present document can be implemented in various electronic devices, such as mobile phones, laptops, smart phones, or other devices capable of performing digital data processing and / or video display.

[0394] Figure 19 is a block diagram of a video processing apparatus 1900. The video processing apparatus 1900 can be used to implement one or more methods described herein. The video processing apparatus 1900 can be embodied in a smart phone, a tablet computer, a computer, an Internet of Things (IoT) receiver, etc. The video processing apparatus 1900 can include one or more processors 1902, one or more memories 1904, and video processing hardware 1906 (also referred to as video processing circuitry). The processor(s) 1902 can be configured to implement one or more methods described in the present document. The memory(ies) 1904 can be used for storing data and code used for implementing the methods and techniques described herein. The video processing hardware 1906 can be used to implement, in hardware circuitry, some of the techniques described in the present document. In some embodiments, the video processing hardware 1906 can be partially or entirely within the processor 1902, such as a graphics processor.

[0395] Figure 20 is a block diagram illustrating an example of a video coding system 2000 that can utilize the techniques of this disclosure. As Figure 20As shown, video coding system 2000 can include a source device 2010 and a destination device 2020. Source device 2010 generates encoded video data that can be referred to as a video encoding device. Destination device 2020 can decode the encoded video data generated by source device 2010 that can be referred to as a video decoding device.

[0396] Source device 2010 can include a video source 2012, a video encoder 2014, and an input / output (VO) interface 2016.

[0397] Video source 2012 can include a source such as a video capture device, an interface to receive video data from a video content provider, and / or a computer graphics system for generating video data, or a combination of such sources. Video data can comprise one or more pictures. Video encoder 2014 encodes video data from video source 2012 to generate a bitstream. The bitstream can include a sequence of bits that form a coded representation of the video data. The bitstream can include coded pictures and associated data. A coded picture is a coded representation of a picture. Associated data can include sequence parameter sets, picture parameter sets, and other syntax structures. VO interface 2016 can include a modulator / demodulator (modem) and / or a transmitter. Encoded video data can be transmitted directly to destination device 2020 via VO interface 2016 by network 2030. The encoded video data can also be stored onto a storage medium / server 2040 for access by destination device 2020.

[0398] Destination device 2020 can include VO interface 2026, video decoder 2024, and display device 2022.

[0399] VO interface 2026 can include a receiver and / or a modem. VO interface 2026 can acquire encoded video data from source device 2010 or storage medium / server 2040. Video decoder 2024 can decode the encoded video data. Display device 2022 can display the decoded video data to a user. Display device 2022 can be integrated with destination device 2020, or can be external to destination device 2020, which can be configured to interface with an external display device.

[0400] Video encoder 2014 and video decoder 2024 can operate according to a video compression standard, such as High Efficiency Video Coding (HEVC) standard, Versatile Video Coding (VVC) standard, and other current and / or further standards.

[0401] Figure 21 is a block diagram illustrating an example of a video encoder 2100, which can be the video encoder 2014 of FIG. 21Figure 20 Video encoder 2014 in video coding system 2000 described above.

[0402] Video encoder 2100 can be configured to perform any or all of the techniques of this disclosure. In Figure 21 In an example, video encoder 2100 includes a plurality of functional components. The techniques described in this disclosure can be shared among the various components of video encoder 2100. In some examples, a processor can be configured to perform any or all of the techniques described in this disclosure.

[0403] The functional components of video encoder 2100 can include a partition unit 2101, a prediction unit 2102, a residual generation unit 2107, a transform unit 2108, a quantization unit 2109, an inverse quantization unit 2110, an inverse transform unit 2111, a reconstruction unit 2112, a buffer 2113, and an entropy encoding unit 2114. Prediction unit 2102 can include a mode selection unit 2103, a motion estimation unit 2104, a motion compensation unit 2105, and an intra-prediction unit 2106.

[0404] In other examples, video encoder 2100 can include more, less, or different functional components. In an example, prediction unit 2102 can include an intra-block copy (IBC) unit. The IBC unit can perform prediction in an IBC mode in which at least one reference picture is the picture in which the current video block is located.

[0405] Furthermore, some components, such as motion estimation unit 2104 and motion compensation unit 2105, can be highly integrated, but are represented separately for illustrative purposes. Figure 21 In an example, video encoder 2100 includes a plurality of functional components. The techniques described in this disclosure can be shared among the various components of video encoder 2100. In some examples, a processor can be configured to perform any or all of the techniques described in this disclosure.

[0406] Partition unit 2101 can partition a picture into one or more video blocks. Figure 20 Video encoder 2014 and video decoder 2024 can support various video block sizes.

[0407] Mode selection unit 2103 can select one of the coding modes (intra or inter), e.g., based on error results, and provide the resulting intra or inter coded block to residual generation unit 2107 to generate residual block data and to reconstruction unit 2112 to reconstruct the coded block for use as a reference picture. In some examples, mode selection unit 2103 can select a combined intra and inter prediction (CIIP) mode in which the prediction is based on both an inter prediction signal and an intra prediction signal. In the case of inter prediction, mode selection unit 2103 can also select a resolution for the motion vectors (e.g., sub-pixel or integer pixel precision) for the block.

[0408] To perform inter prediction on a current video block, motion estimation unit 2104 can generate motion information for the current video block by comparing one or more reference frames from buffer 2113 to the current video block. Motion compensation unit 2105 can determine a predicted video block for the current video block based on the motion information and decoded samples from pictures other than the picture associated with the current video block from buffer 2113.

[0409] Motion estimation unit 2104 and motion compensation unit 2105 can perform different operations on a current video block, e.g., depending on whether the current video block is an I slice, a P slice, or a B slice. I slices (or I frames) are the least compressible, but do not require other video frames for decoding. P slices (or P frames) can be decompressed using data in previous frames, and are more easily compressed than I frames. B slices (or B frames) can use both previous and future frames as data references to achieve the highest amount of data compression.

[0410] In some examples, motion estimation unit 2104 can perform uni-prediction on a current video block, and motion estimation unit 2104 can search for a reference video block for the current video block in reference pictures in list 0 or list 1. Motion estimation unit 2104 can then generate a reference index indicating a reference picture in list 0 or list 1 that contains the reference video block, and a motion vector indicating a spatial displacement between the current video block and the reference video block. Motion estimation unit 2104 can output the reference index, the prediction direction indicator, and the motion vector as motion information for the current video block. Motion compensation unit 2105 can generate a predicted video block for the current block based on the reference video block indicated by the motion information for the current video block.

[0411] In other examples, motion estimation unit 2104 can perform bi-prediction on a current video block, motion estimation unit 2104 can search for a reference video block for the current video block in reference pictures in list 0, and can also search for another reference video block for the current video block in reference pictures in list 1. Motion estimation unit 2104 can then generate a reference index indicating reference pictures in list 0 and list 1 that contain the reference video blocks, and a motion vector indicating a spatial displacement between the reference video blocks and the current video block. Motion estimation unit 2104 can output the reference index and the motion vector as motion information for the current video block. Motion compensation unit 2105 can generate a predicted video block for the current video block based on the reference video blocks indicated by the motion information for the current video block.

[0412] In some examples, motion estimation unit 2104 can output a complete set of motion information for decoding processing by a decoder.

[0413] In some examples, the motion estimation unit 2104 can not output a full set of motion information for the current video. Instead, the motion estimation unit 2104 can signal the motion information for the current video block with reference to the motion information of another video block. For example, the motion estimation unit 2104 can determine that the motion information for the current video block is sufficiently similar to the motion information of a neighboring video block.

[0414] In one example, the motion estimation unit 2104 can indicate a value in a syntax structure associated with the current video block, the value indicating to the video decoder 2024 that the current video block has the same motion information as another video block.

[0415] In another example, the motion estimation unit 2104 can identify another video block and a motion vector difference (MVD) in a syntax structure associated with the current video block. The motion vector difference indicates a difference between the motion vector of the current video block and the motion vector of the indicated video block. The video decoder 2024 can use the motion vector of the indicated video block and the motion vector difference to determine the motion vector of the current video block.

[0416] As described above, the video encoder 2014 can predictively signal motion vectors. Two examples of predictive signaling techniques that can be implemented by the video encoder 2014 include advanced motion vector prediction (AMVP) and merge mode signaling.

[0417] The intra prediction unit 2106 can perform intra prediction on the current video block. When the intra prediction unit 2106 performs intra prediction on the current video block, the intra prediction unit 2106 can generate prediction data for the current video block based on decoded samples of other video blocks in the same picture. The prediction data for the current video block can include a predicted video block and various syntax elements.

[0418] The residual generation unit 2107 can generate residual data for the current video block by subtracting (e.g., indicated by a negative sign) the predicted video block for the current video block from the current video block. The residual data for the current video block can include a residual video block corresponding to different sample components of samples in the current video block.

[0419] In other examples, there can be no residual data for the current video block, such as in skip mode, and the residual generation unit 2107 can not perform the subtraction operation.

[0420] The transform unit 2108 can generate one or more blocks of transform coefficients for the current video block by applying one or more transforms to the residual video block associated with the current video block.

[0421] After the transform unit 2108 generates a transform coefficient video block associated with the current video block, the quantization unit 2109 can quantize the transform coefficient video block associated with the current video block based on one or more quantization parameter (QP) values associated with the current video block.

[0422] The inverse quantization unit 2110 and the inverse transform unit 2111 can apply inverse quantization and inverse transform, respectively, to the transform coefficient video block to reconstruct a residual video block from the transform coefficient video block. The reconstruction unit 2112 can add the reconstructed residual video block to corresponding samples from one or more prediction video blocks generated by the prediction unit 2102 to produce a reconstructed video block associated with the current block for storage in the buffer 2113.

[0423] After the reconstruction unit 2112 reconstructs the video block, in-loop filtering operations can be performed to reduce video block artifacts in the video block.

[0424] The entropy encoding unit 2114 can receive data from other functional components of the video encoder 2100. When the entropy encoding unit 2114 receives data, the entropy encoding unit 2114 can perform one or more entropy encoding operations to generate entropy encoded data and output a bitstream that includes the entropy encoded data.

[0425] Figure 22 is a block diagram illustrating an example of a video decoder 2200 that can be Figure 20 the video decoder 2024 in the video coding system 2000 described in

[0426] The video decoder 2200 can be configured to perform any or all of the techniques of this disclosure. In Figure 22 In examples, the video decoder 2200 includes a plurality of functional components. The techniques described in this disclosure can be shared amongst the various components of the video decoder 2200. In some examples, a processor can be configured to perform any or all of the techniques described in this disclosure.

[0427] In Figure 22 In examples, the video decoder 2200 includes an entropy decoding unit 2201, a motion compensation unit 2202, an intra prediction unit 2203, an inverse quantization unit 2204, an inverse transform unit 2205, a reconstruction unit 2206, and a buffer 2207. In some examples, the video decoder 2200 can perform a decoding process generally reciprocal to the encoding process described with respect to the video encoder 2014 Figure 20 ) described above.

[0428] Entropy decoding unit 2201 can retrieve an encoded bitstream. The encoded bitstream can include entropy coded video data (e.g., encoded video data blocks). Entropy decoding unit 2201 can decode the entropy coded video data and, from the entropy decoded video data, motion compensation unit 2202 can determine motion information, including motion vectors, motion vector precision, reference picture list index, and other motion information. Motion compensation unit 2202 can determine such information, for example, by performing AMVP and merge mode signaling.

[0429] Motion compensation unit 2202 can generate a motion compensated block, possibly performing interpolation based on an interpolation filter. An identifier of the interpolation filter to be used at sub-pixel precision can be included in the syntax elements.

[0430] Motion compensation unit 2202 can use the interpolation filter as used by video encoder 2014 during encoding of the video block to calculate the interpolation of sub-integer pixels of the reference block. Motion compensation unit 2202 can determine the interpolation filter used by video encoder 2014 from the received syntax information and use the interpolation filter to generate the prediction block.

[0431] Motion compensation unit 2202 can use some of the syntax information to determine the size of the blocks used to encode the frame(s) and / or slice(s) of the coded video sequence, partitioning information describing how each macroblock of a picture of the coded video sequence was partitioned, modes indicating how each partition was encoded, one or more reference frames (and reference frame lists) for each inter-coded block, and other information to decode the coded video sequence.

[0432] Intra prediction unit 2203 can use, for example, intra prediction modes received in the bitstream to form a prediction block from spatially neighboring blocks. Inverse quantization unit 2204 inverse quantizes, i.e., de-quantizes, quantized video block coefficients provided in the bitstream and decoded by entropy decoding unit 2201. Inverse transform unit 2205 applies an inverse transform.

[0433] Reconstruction unit 2206 can add the residual block to the corresponding prediction block generated by motion compensation unit 2202 or intra prediction unit 2203 to form a decoded video block. If desired, a deblocking filter can also be applied to the decoded video block to filter out blockiness artifacts. The decoded video blocks are then stored in buffer 2207, which provides reference blocks for subsequent motion compensation / intra prediction, and also generates decoded video for presentation on a display device.

[0434] Figure 23is a method 2300 for coding video data according to embodiments of the present disclosure. The method 2300 can be performed by a coding device (e.g., an encoder) having a processor and a memory. The method 2300 can be implemented when determining padding dimensions to be used for padding samples to be applied to a video unit during in-loop filtering is needed.

[0435] In block 2302, the coding device determines in real-time padding dimensions to be used for padding samples applied to a video unit of a video for in-loop filtering. In one embodiment, 、 、 and represent padding dimensions corresponding to top, bottom, left and right boundaries of the video unit, respectively.

[0436] In block 2304, the coding device performs a conversion between the video unit and a video bitstream based on the generated filtered samples. When implemented in an encoder, the conversion includes receiving a media file (e.g., the video unit) and encoding the media file into a bitstream. When implemented in a decoder, the conversion includes receiving a bitstream including a media file, and decoding the bitstream to obtain the media file.

[0437] In one embodiment, the method 2300 can utilize or incorporate one or more features or processes of other methods disclosed herein.

[0438] Next, a list of some embodiment preferred solutions is provided.

[0439] The following solution shows an example embodiment (e.g., Example 1) of the techniques discussed in the present disclosure.

[0440] 1. A method of video processing, comprising: for a conversion between a video block of a video and a bitstream of the video, determining padding dimensions around the video block to apply in-loop filtering to the video block according to a rule; and performing the conversion based on the determining, wherein the dimensions defined by 、 、 and are padding dimensions for top, bottom, left and right boundaries, respectively.

[0441] 2. The method of item 1, wherein the rule specifies 、 、 、 are different from each other.

[0442] 3. A method of video processing, comprising determining, for a conversion between a video block of a video and a bitstream of the video, whether or how to generate padding samples outside the video block to apply in-loop filtering to the video block according to a rule, and performing the conversion based on the determination.

[0443] 4. The method of clause 3, wherein the rule specifies whether to generate padding samples depends on whether one or all of the samples outside the video unit are available.

[0444] 5. The method of clause 3, wherein the rule specifies whether to generate padding samples depends on whether one or all of the samples outside the video unit along a given direction are available.

[0445] 6. The method of any of clauses 3-5, wherein the rule specifies, before applying in-loop filtering, to use available samples outside the video block without padding, and to pad unavailable samples outside the video block.

[0446] 7. The method of any of clauses 3-6, wherein the padding samples are samples before applying a neural network (NN) filter.

[0447] 8. The method of any of clauses 3-7, wherein the padding samples are samples after applying a neural network (NN) filter.

[0448] 9. A method of video processing, comprising determining, for a conversion between a video comprising video pictures and a bitstream representation of the pictures, a granularity of a neural network (NN) filter model used for in-loop filtering during the conversion based on a rule, and performing the conversion based on the determination, wherein the rule defines the granularity of the NN filter model independently of a size of a video block.

[0449] 10. The method of clause 9, wherein the rule defines the granularity is pre-defined.

[0450] 11. The method of any of clauses 9-10, wherein the rule defines the granularity depends on a quantization parameter (QP) or a frame resolution.

[0451] 12. A method of video processing, comprising performing a conversion between a video comprising a video region containing a video unit and a bitstream of the video, wherein the conversion includes applying an in-loop neural network (NN) filter to the first video unit according to a rule, wherein the rule specifies that indexing of a number of models of the NN filter according to a maximum number of models is allowed to be binarized at a level of the video region.

[0452] 13. The method of clause 12, wherein the video region is a slice or a picture or a sequence.

[0453] 14. The method of any of clauses 12-13, wherein the maximum number of models allowed at the video region level is predetermined or determined during the conversion.

[0454] 15. The method of any of clauses 12-13, wherein the binarization of the index uses a truncated code.

[0455] 16. The method of any of clauses 1-15, wherein the conversion comprises generating a bitstream from the video.

[0456] 17. The method of any of clauses 1-15, wherein the conversion comprises generating a video from the bitstream.

[0457] 18. A video decoding apparatus comprising a processor configured to implement the method of one or more of clauses 1-17.

[0458] 19. A video encoding apparatus comprising a processor configured to implement the method of one or more of clauses 1-17.

[0459] 20. A computer program product having computer code stored thereon, the code, when executed by a processor, causing the processor to implement the method of any of clauses 1-17.

[0460] 21. A computer readable medium having a bitstream stored thereon, the bitstream being generated by the method of any of clauses 1-17.

[0461] 22. A method of generating a bitstream, comprising: generating the bitstream using one or more of clauses 1-17, and writing the bitstream to a computer readable medium.

[0462] 23. The method, apparatus or system described in the present document.

[0463] The following documents are incorporated by reference in their entirety:

[0464] [1] Johannes Ballé, Valero Laparra, and Eero P. Simoncelli, “End-to-end optimization of nonlinear transform codes for perceptual quality,” in PCS IEEE (2016), 1-5.

[0465] [2] Lucas Theis, Wenzhe Shi, Andrew Cunningham, and Ferenc Huszar, “Lossy image compression with compressive autoencoders,” arXiv preprint arXiv: 1703.00395 (2017).

[0466] [3] Jiahao Li, Bin Li, Jizheng Xu, Ruiqin Xiong, and Wen Gao, “Fully Connected Network-Based Intra Prediction for Image Coding,” IEEE Transactions on Image Processing 27, 7 (2018), 3236-3247.

[0467] [4] Yuanying Dai, Dong Liu, and Feng Wu, “A convolutional neural network approach for post-processing in HEVC intra coding,” MMM. Springer, 28-39.

[0468] [5] Rui Song, Dong Liu, Houqiang Li, and Feng Wu, “Neural network-based arithmetic coding of intra prediction modes in HEVC,” VCIP IEEE (2017), 1-4.

[0469] [6] J. Pfaff, P. Helle, D. Maniry, S. Kaltenstadler, W. Samek, H. Schwarz, D. Marpe, and T. Wiegand, “Neural network based intra prediction for video coding,” Applications of Digital Image Processing XLI, Vol. 10752. International Society for Optics and Photonics, 1075213 (2018).

[0470] The disclosed and other solutions, examples, embodiments, modules and functional operations described herein can be implemented in digital electronic circuitry, or in computer software, firmware, or hardware, including the structures disclosed in this document and their structural equivalents, or in combinations of one or more of them. The disclosed and other embodiments can be implemented as one or more computer program products, i.e., one or more modules of computer program instructions encoded on a computer readable medium for execution by, or to control the operation of, data processing apparatus. The computer readable medium can be a machine-readable storage device, a machine-readable storage substrate, a memory device, a composition of matter effecting a machine- readable propagated signal, or a combination of one or more of them. The term “data processing apparatus” encompasses all apparatus, devices, and machines for processing data, including by way of example a programmable processor, a computer, or multiple processors or computers. The apparatus can include, in addition to hardware, code that creates an execution environment for the computer program in question, e.g., code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them. The propagated signal is an artificially generated signal, e.g., a machine-generated electrical, optical, or electromagnetic signal, that is generated to encode information for transmission to suitable receiver apparatus.

[0471] A computer program, which can also be referred to or referred to as a program, software, a software application, an app, a script, or code, can be written in any form of programming language, including compiled or interpreted languages, and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. A program can be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files that store one or more modules, sub programs, or portions of code). A computer program can be deployed to be executed on one computer or on multiple computers that are located at one site or distributed across multiple sites and are interconnected by a communication network.

[0472] The processes and logic flows described herein can be performed by one or more programmable processors executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows can also be performed by special purpose logic circuitry, and / or devices that are

[0473] Processors suitable for the execution of a computer program include, by way of example, both general and special purpose microprocessors, and any one or more processors of any kind of digital computer. Generally, a processor will receive instructions and data from a read-only memory or a random access memory or both. The essential elements of a computer are a processor for performing instructions and one or more memory devices for storing instructions and data. Generally, a computer will also include, or be operatively coupled to receive data from or transfer data to, or both, one or more mass storage devices for storing data, e.g., magnetic, magneto-optical disks, or optical disks. However, a computer need not have such devices. Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media and memory devices, including by way of example semiconductor memory devices, e.g., erasable programmable read only memory (EPROM), electrically erasable programmable read only memory (EEPROM), and flash memory devices; magnetic disks, e.g., internal hard disks or removable disks; magneto-optical disks; and compact disc read only memories (CD-ROMs) and digital versatile disc read only memories (DVD-ROMs). The processor and the memory can be supplemented by, or incorporated in, special purpose logic circuitry.

[0474] While this patent document contains many details, these should not be construed as limiting the scope of any subject matter or of any embodiments of the subject matter claimed, but as merely describing features that are particular embodiments of the particular technology. Certain features described in the context of separate embodiments can also be implemented in combination with each other. Conversely, various features described in the context of a single embodiment can also be implemented separately or in any appropriate subcombination. Moreover, although features can be described above as acting in certain combinations and even initially claimed as such, one or more features from a claimed combination can in some cases be excised from the combination and the claimed combination can be directed to a subcombination or variation of a subcombination.

[0475] Similarly, while operations are described in a particular, sequential order, this should not be understood as requiring that the operations be performed in that particular order, or in sequential order at all, or that all operations be performed, to achieve desirable results. Further, the separation of various system components in the embodiments described in this patent document should not be understood as requiring such separation in all embodiments.

[0476] Only a few implementations and examples are described and other implementations, enhancements and variations can be made based on what is described and illustrated in this patent document.

Claims

1. A method for processing video data, comprising: For the conversion between video units and the bitstream of the video, it is determined that the granularity of the selected neural network (NN) filter model to be applied to the padding samples corresponding to the video unit is different from the size of the codec tree (CTU) unit; The conversion is performed based on the determination.

2. The method according to claim 1, wherein, The granularity is predefined; or The granularity indication is transmitted via signal in the bit stream, or the granularity indication is derived in real time.

3. The method according to claim 1, wherein, The granularity depends on the quantization parameter (QP) and the resolution of the video unit.

4. The method according to claim 3, wherein, The larger the QP or the higher the resolution, the coarser the granularity.

5. The method according to claim 1, wherein, When q < 23, the particle size is 32 × 32; Wherein, when 23≤q<29 and w≤832, the particle size is 32×32; Wherein, when 23≤q<29 and w>832, the particle size is 64×64; Wherein, when q≥29 and w≤832, the particle size is 128×128; Specifically, when q ≥ 29 and w > 832, the particle size is 256 × 256. Where q indicates the sequence-level quantization parameter (QP), and w indicates the frame width.

6. The method according to claim 1, wherein, The filling method used to generate the filling samples outside the video unit is determined in real time.

7. The method according to claim 6, wherein, Whether the filling method is applied depends on whether at least one or all samples outside the video unit are available.

8. The method according to claim 6, wherein, When all samples in the fill area surrounding the video unit are available at the top, bottom, left, and right boundaries, the samples are used directly without being filled; or When at least one sample point in the fill area surrounding the video unit is unavailable at the top, bottom, left, and right boundaries, all neighboring sample points are filled.

9. The method according to claim 6, wherein, Whether the filling method is applied depends on whether at least one or all samples are available outside the video unit along a given direction.

10. The method according to claim 9, wherein, When all neighboring samples are available for a specific boundary, the neighboring samples are used directly without being filled; or Where at least one sample point in the filling area around the video unit is unavailable at a specific boundary, all neighboring sample points of the specific boundary are filled.

11. The method according to claim 6, wherein, Available samples in the filling area around the video unit are used directly without being filled, while unavailable samples in the filling area around the video unit are filled.

12. The method according to claim 6, wherein, The filling method includes one of zero filling, reflection filling, copy filling, constant filling, and mirror filling; Wherein, when the filling method is the mirror filling, the value outside the boundary of the video unit is obtained by mirroring the video unit along the edge of the video unit.

13. The method according to claim 6, wherein, The filling method of the video unit is based on the size of the video unit, the type of neural network filtering method applied to the video unit, decoding information, whether a neural network filter is applied, channel type, stripe type, or the temporal layer to which the video unit belongs; The filling method is indicated by a signal transmitted in the bit stream. In this method, at least one relevant parameter is determined based on the position of the video unit relative to a parent video unit, wherein the parent video unit is segmented to obtain the video unit, and the relevant parameter includes a filling dimension.

14. The method according to claim 1, wherein, The fill dimension of the fill sample is determined in real time, where d1, d2, d3 and d4 represent the fill dimensions corresponding to the top boundary, bottom boundary, left boundary and right boundary of the video unit, respectively. Among them, d1, d2, d3 and d4 are different, or d1, d2, d3 and d4 are the same, or d1=d2 and d3=d4.

15. The method according to claim 1, wherein, The samples in the padding region surrounding the video unit are unfiltered samples before the application of the neural network (NN) filter; or The sample points in the filling area surrounding the video unit are filtered sample points after applying a neural network (NN) filter.

16. The method according to claim 1, wherein, The binarization of the neural network (NN) filter model index corresponding to the NN filter model used for filling samples corresponding to the video unit is based on the maximum number allowed at a level higher than the video unit, where the level is the strip level, image level, or sequence level. The indication of the maximum number is either transmitted via signaling at that level or is predefined or derived in real time. The maximum number of indications is transmitted via signals in the image header, strip header, image parameter set (PPS), sequence parameter set (SPS), or adaptive parameter set (APS), and The NN filter model index is binarized into a rounding unary code or a rounding binary code.

17. The method according to claim 1, wherein, The conversion includes encoding the video unit into the bitstream.

18. The method according to claim 1, wherein, The conversion includes decoding the video unit from the bitstream.

19. An apparatus for processing video data, comprising a processor and a non-transitory memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to: For the conversion between video units and the bitstream of the video, it is determined that the granularity of the selected neural network (NN) filter model to be applied to the padding samples corresponding to the video unit is different from the size of the codec tree (CTU) unit; The conversion is performed based on the determination.

20. A non-transitory computer-readable recording medium storing a bitstream of video generated by a method performed by a video processing apparatus, wherein the method comprises: For a video unit of the video, it is determined that the granularity of the selected neural network (NN) filter model to be applied to the filler samples corresponding to the video unit is different from the size of the codec tree (CTU) unit; The bitstream of the video is generated based on the determination.

Images