External attention in neural network-based video coding

Abstract

A method implemented by a video coding device includes applying a neural network (NN) filter to unfiltered samples of a video unit to generate filtered samples. The NN filter is based on a NN filter model configured to obtain an attention based on a coding parameter input. The method also includes performing a conversion between a video media file and a bitstream based on the generated filtered samples.

Description

[0001] Cross-references to related applications

[0002] This patent application claims the benefit of U.S. Provisional Patent Application No. 63 / 216,923, filed June 30, 2021, entitled “External Attention In Neural Network-Based Coding Tools For Video Coding,” which is incorporated herein by reference. Technical Field

[0003] This disclosure generally relates to image and video encoding and decoding. Background Technology

[0004] Digital video accounts for the largest share of bandwidth usage on the internet and other digital communication networks. As the number of connected user devices capable of receiving and displaying video increases, the bandwidth demand for digital video is expected to continue to grow. Summary of the Invention

[0005] The disclosed aspects / embodiments provide one or more neural network (NN) filter models trained as encoding / decoding tools to improve the efficiency of video encoding / decoding. NN-based encoding / decoding tools can be used to replace or enhance one or more modules implemented by a video encoder / decoder (also referred to as a codec). For example, NN models can be trained to provide additional intra-frame prediction modes, additional inter-frame prediction modes, transform kernels, and / or loop filters. Furthermore, NN models can be generated or designed as attention mechanisms using external information such as prediction, partitioning, and quantization parameters (QP).

[0006] The first aspect relates to a method implemented by an encoding / decoding device. The method includes applying a neural network (NN) filter to unfiltered samples of a video unit to generate filtered samples, wherein the NN filter is based on an NN filter model configured to acquire attention based on encoding / decoding parameter inputs. The method also includes performing a conversion between a video media file and a bitstream based on the generated filtered samples.

[0007] Optionally, in any of the foregoing aspects, another implementation of the aspect provides that the encoding / decoding parameter input includes one or more selected from the group consisting of: a segmentation scheme for the video unit; a prediction mode for the video unit; a quantization parameter associated with the video unit; and a boundary strength parameter for the boundary of the video unit.

[0008] Optionally, in any of the preceding aspects, another embodiment of the aspect provides that the NN filter uses a convolution layer to extract features from the codec parameter input; and uses the extracted features as attention in the NN filter model.

[0009] Optionally, in any of the preceding aspects, another embodiment of the aspect provides that the intermediate feature map of the NN filter model is to be recalibrated by attention, and the attention is obtained by concatenating the codec parameter input with the intermediate feature map to provide a concatenated result, and feeding the concatenated result into a convolution layer of the NN filter.

[0010] Optionally, in any of the preceding aspects, another embodiment of the aspect provides that the attention is obtained using a two-layer convolutional neural network, and the attention is a single-channel feature map having a spatial resolution identical to that of the intermediate feature map of the NN filter model to be recalibrated by the attention.

[0011] Optionally, in any of the preceding aspects, another embodiment of the aspect provides that the attention is obtained using a two-layer convolutional neural network, and the attention is a single-channel feature map having a spatial resolution identical to that of the intermediate feature map of the NN filter model to be recalibrated by the attention. N ×W×H where N is the number of channels, W is the channel width, and H is the channel height, and wherein the obtained attention is given as A, where A ∈ R W×H .

[0012] Optionally, in any of the preceding aspects, another embodiment of the aspect provides that φ represents the recalibrated intermediate feature map, and applying the attention comprises providing the recalibrated intermediate feature map according to φ i，j，k = G i，j，k × A j，k where 1≤i≤N, where 1≤j≤W, and where 1≤k≤H.

[0013] Optionally, in any of the preceding aspects, another embodiment of the aspect provides that φ represents the recalibrated intermediate feature map, and applying the attention comprises providing the recalibrated intermediate feature map according to φ i，j，k = G i，j，k × f(A j，k ) where 1≤i≤N, where 1≤j≤W, where 1≤k≤H, and where f represents a mapping function applied to each element of the attention.

[0014] Optionally, in any of the preceding aspects, another embodiment of the aspect provides that the mapping function f comprises a sigmoid function or a hyperbolic tangent function.

[0015] Optionally, in any of the preceding aspects, another embodiment of such aspects provides using different A or f for different channels of the intermediate feature map.

[0016] Optionally, in any of the preceding aspects, another embodiment of such aspects provides that φ represents the recalibrated intermediate feature map, and applying attention comprises: i，j，k = G i，j，k x f(A j，k ) + G i，j，k to provide the recalibrated intermediate feature map, where 1≤i≤N, where 1≤j≤W, where 1≤k≤H, and where f represents a mapping function applied to each element of the attention.

[0017] Optionally, in any of the preceding aspects, another embodiment of such aspects provides that the mapping function f comprises a sigmoid function or a hyperbolic tangent function.

[0018] Optionally, in any of the preceding aspects, another embodiment of such aspects provides using different A or f for different channels of the intermediate feature map.

[0019] Optionally, in any of the preceding aspects, another embodiment of such aspects provides that the attention is applied to a designated layer inside the NN filter model.

[0020] Optionally, in any of the preceding aspects, another embodiment of such aspects provides that the NN filter model comprises a residual block, and the attention is applied only to the feature map from the last layer of each residual block.

[0021] Optionally, in any of the preceding aspects, another embodiment of such aspects provides that the NN filter is one or more selected from a group comprising: an adaptive loop filter, a deblocking filter, and a sample adaptive offset filter.

[0022] Optionally, in any of the preceding aspects, another embodiment of such aspects provides that the converting comprises generating a bitstream from the video media file.

[0023] Optionally, in any of the preceding aspects, another embodiment of such aspects provides that the converting comprises parsing the bitstream to obtain the video media file.

[0024] A second aspect relates to an apparatus for coding video data comprising a processor and a non-transitory memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to: apply a neural network (NN) filter to unfiltered samples of a video unit to generate filtered samples, wherein the NN filter is based on a NN filter model configured to obtain an attention based on a coding parameter input; and convert between a video media file and a bitstream based on the generated filtered samples.

[0025] A third aspect relates to a non-transitory computer readable medium storing a video bitstream generated by a method performed by a video processing apparatus. The method performed by the video processing apparatus comprises applying a neural network (NN) filter to unfiltered samples of a video unit to generate filtered samples, wherein the NN filter is based on a NN filter model configured to obtain an attention based on a coding parameter input; and generating the bitstream based on the generated filtered samples.

[0026] A fourth aspect relates to a method performed by a video processing apparatus for storing a video bitstream. The method performed by the video processing apparatus comprises applying a neural network (NN) filter to unfiltered samples of a video unit to generate filtered samples, wherein the NN filter is based on a NN filter model configured to obtain an attention based on a coding parameter input; and generating the bitstream based on the generated filtered samples.

[0027] For purposes of clarity, any of the foregoing embodiments can be combined with any one or more of the other foregoing embodiments, to create new embodiments within the scope of the present disclosure.

[0028] These and other features will be more clearly understood from the following detailed description taken in conjunction with the accompanying drawings and claims. BRIEF DESCRIPTION OF DRAWINGS

[0029] For a more complete understanding of the present disclosure, reference is now made to the following brief description of the drawings taken in conjunction with the detailed description below.

[0030] Figure 1 is an example of a raster scan slice partition of a picture.

[0031] Figure 2 is an example of a rectangular slice partition of a picture.

[0032] Figure 3 is an example of a picture partitioned into tiles, bricks, and rectangular slices.

[0033] Figure 4A is an example of a coding tree block (CTB) across a bottom picture boundary.

[0034] Figure 4B is an example of CTBs across the right picture boundary.

[0035] Figure 4C is an example of CTBs across the bottom picture boundary.

[0036] Figure 5 is an example of an encoder block diagram.

[0037] Figure 6 is an illustration of samples within an 8x8 sample block.

[0038] Figure 7 is an example of pixels involved in filter on / off decision and strong / weak filter selection.

[0039] Figure 8 Four one-dimensional (1-D) directional modes for EO sample classification are shown.

[0040] Figure 9 An example of a geometric transform based adaptive loop filter (GALF) filter shape is shown.

[0041] Figure 10 An example of relative coordinates for 5x5 diamond filter support is shown.

[0042] Figure 11 Another example of relative coordinates for 5x5 diamond filter support is shown.

[0043] Figure 12A is an example architecture of the proposed CNN filter.

[0044] Figure 12B is an example of the construction of a Residual Block (ResBlock).

[0045] Figure 13 is an example of a process to generate filtered samples based on a neural network filter model receiving coding parameters as input, according to various examples.

[0046] Figure 14 is an example of applying attention obtained using external information, such as coding parameters, to feature maps of a neural network filter model to provide re-calibrated feature maps, according to various examples.

[0047] Figure 15A is a schematic block diagram of an architecture of a neural network filtering method, according to various examples, and Figure 15B is a schematic block diagram of an attention residual block used in the architecture of Figure 15A

[0048] Figure 16 ​is a block diagram illustrating an example video processing system.

[0049] Figure 17 is a block diagram of a video processing device.

[0050] Figure 18 is a block diagram illustrating an example of a video coding system.

[0051] Figure 19 is a block diagram illustrating an example of a video encoder.

[0052] Figure 20 is a block diagram illustrating an example of a video decoder.

[0053] Figure 21 is a method for coding video data according to embodiments of the disclosure. DETAILED DESCRIPTION

[0054] It should be understood at the outset that although illustrative implementations of one or more embodiments are provided below, the disclosed systems and / or methods can be implemented using any number of techniques, whether currently known or in existence. The disclosure should in no way be limited to the illustrative implementations, drawings, and techniques illustrated below, including the exemplary design and implementations set forth herein, but can be modified in various ways within the scope of the appended claims and their equivalents.

[0055] The use of versatile video coding (VVC) (also known as H.266) terminology in certain descriptions is merely for ease of understanding and is not intended to limit the scope of the disclosed technology. As such, the technology described herein is also applicable to other video codec protocols and designs.

[0056] This patent document relates to video coding. In particular, this specification relates to loop filters in image / video coding. The disclosed examples can be applied individually or in various combinations for video bitstreams coded using existing video coding standards such as the VVC standard, the High Efficiency Video Coding (HEVC) standard, or a yet to be finalized standard (e.g., the third generation of Audio Video Coding standard (AVS3)). The disclosed examples can also be applicable to future video coding standards, future video codecs, or as a post-processing method outside of the encoding / decoding process.

[0057] Video coding standards have evolved mainly through the development of the well-known International Telecommunication Union - Telecommunication (ITU-T) and International Organization for Standardization (ISO) / International Electrotechnical Commission (IEC) standards. The ITU-T developed H.261 and H.263, ISO / IEC developed Motion Pictures Expert Group (MPEG)-1 and MPEG-4 Visual, and the two organizations jointly developed the H.262 / MPEG-2 Video, H.264 / MPEG-4 Advanced Video Coding (AVC), and H.265 / High Efficiency Video Coding (HEVC) standards.

[0058] Since H.262, video coding standards are based on a hybrid video coding structure where temporal prediction plus transform coding is employed. To explore future video coding technologies beyond HEVC, the Joint Video Exploration Team (JVET) was founded by the Video Coding Experts Group (VCEG) and MPEG jointly in 2015. Since then, many new methods have been adopted by the JVET and applied to the reference software named Joint Exploration Model (JEM).

[0059] In April 2018, the Joint Video Team (JVT) was created between VCEG (Q6 / 16) and ISO / IEC JTC1 SC29 / WG11 (MPEG) to work on the VVC standard with the goal of 50% bitrate reduction compared to HEVC. VVC version 1 was finalized in July 2020.

[0060] Color spaces and chroma subsampling are discussed. A color space, also called a color model (or color system), is an abstract mathematical model that simply describes a range of colors as a tuple of numbers, typically 3 or 4 values or color components (e.g., red green blue (RGB)). Fundamentally, a color space is an exposition of a coordinate system and a subspace.

[0061] For video compression, the most commonly used color spaces are YCbCr and RGB. YCbCr, Y’CbCr, or Y Pb / Cb Pr / Cr, also written as YC B C R or Y’C B C R , is a family of color spaces used as part of the color image pipeline in video and digital photography systems. Y’ is the luma component, C B and C R are the blue-difference and red-difference chroma components. Y’ (with the prime) is different from Y, Y is the luminance, which means that the light intensity is non-linearly encoded based on the RGB primaries with gamma correction.

[0062] Chroma subsampling is a practice of encoding images with lower resolution for chroma information than for luma information, exploiting the fact that the human visual system is less sensitive to color differences than to luminance differences.

[0063] For 4:4:4 chroma subsampling, each of the three Y’CbCr components has the same sampling rate, so there is no chroma subsampling. This scheme is sometimes used for high-end film scanners and film post-production.

[0064] For 4:2:2 chroma subsampling, the two chroma components are sampled at half the luma sampling rate: horizontal chroma resolution is halved. This reduces the bandwidth of the uncompressed video signal by one third, but there is little visual difference.

[0065] For 4:2:0 chroma subsampling, compared to 4:1:1, horizontal sampling is doubled, but vertical resolution is halved because in this scheme the Cb and Cr channels are only sampled on every other line. Thus, the data rate is the same. Cb and Cr are subsampled by a factor of 2 in the horizontal and vertical directions, respectively. There are three variants of the 4:2:0 scheme with different horizontal and vertical siting.

[0066] In MPEG-2, Cb and Cr are co-sited in the horizontal direction. Cb and Cr are siting-gapped (gap-sited) between pixels in the vertical direction. In Joint Photographic Experts Group (JPEG) / JPEG File Interchange Format (JFIF), H.261, and MPEG-1, Cb and Cr are gap-sited, located in the middle of alternate luma samples. In 4:2:0 DV, Cb and Cr are co-sited in the horizontal direction. In the vertical direction, they are co-sited on alternate lines.

[0067] Definitions of video units are provided. A picture is divided into one or more tile rows and one or more tile columns. A tile is a sequence of coding tree units (CTUs) covering a rectangular region of the picture. A tile is divided into one or more bricks, each consisting of multiple CTU rows within the tile. A tile that is not partitioned into multiple bricks is also referred to as a brick. However, a brick that is a proper subset of a tile is not referred to as a tile. A slice includes multiple tiles of a picture or multiple bricks of a tile.

[0068] Two slice modes are supported, namely, a raster-scan slice mode and a rectangular slice mode. In the raster-scan slice mode, a slice includes a sequence of tiles in a raster scan of tiles of a picture. In the rectangular slice mode, a slice includes multiple bricks of a picture that collectively form a rectangular region of the picture. The bricks within a rectangular slice are arranged in the order of brick raster scan of the slice.

[0069] Figure 1is an example of raster-scan slice partitioning of picture 100, where the picture is divided into twelve tiles 102 and three raster-scan slices 104. As shown, each tile 102 and slice 104 includes multiple CTUs 106.

[0070] Figure 2 is an example of rectangular slice partitioning of picture 200 according to the VVC specification, where the picture is divided into twenty-four tiles 202 (six tile columns 203 and four tile rows 205) and nine rectangular slices 204. As shown, each tile 202 and rectangular slice 204 contains multiple CTUs 206.

[0071] Figure 3 is an example of partitioning picture 300 into tiles, tiles, and rectangular slices according to the VVC specification, where the picture is divided into four tiles 302 (two tile columns 303 and two tile rows 305), eleven tiles 304 (the top-left tile contains one tile, the top-right tile contains five tiles, the bottom-left tile contains two tiles, and the bottom-right tile contains three tiles), and four rectangular slices 306.

[0072] CTU and coding tree block (CTB) sizes are discussed. In VVC, the coding tree unit (CTU) size, signaled in the sequence parameter set (SPS) by the syntax element log2_ctu_size_minus2, can be as small as 4x4. The sequence parameter set (SPS) raw byte sequence payload (RBSP) syntax is as follows.

[0073]

[0074]

[0075]

[0076] log2_ctu_size_minus2 plus 2 specifies the luma coding tree block size of each CTU.

[0077] log2_min_luma_coding_block_size_minus2 plus 2 specifies the minimum luma coding block size.

[0078] The variables CtbLog2SizeY, CtbSizeY, MinCbLog2SizeY, MinCbSizeY, MinTbLog2SizeY, MaxTbLog2SizeY, MinTbSizeY, MaxTbSizeY, PicWidthInCtbsY, PicHeightInCtbsY, PicSizeInCtbsY, PicWidthInMinCbsY, PicHeightInMinCbsY, PicSizeInMinCbsY, PicSizeInSamplesY, PicWidthInSamplesC, and PicHeightInSamplesC are derived as follows:

[0079] CtbLog2SizeY = log2_ctu_size_minus2 + 2 (7-9)

[0080] CtbSizeY = 1 « CtbLog2SizeY (7-10)

[0081] MinCbLog2SizeY = log2_min_luma_coding_block_size_minus2 + 2 (7-11)

[0082] MinCbSizeY = 1 « MinCbLog2SizeY (7-12)

[0083] MinTbLog2SizeY = 2 (7-13)

[0084] MaxTbLog2SizeY = 6 (7-14)

[0085] MinTbSizeY = 1 « MinTbLog2SizeY (7-15)

[0086] MaxTbSizeY = 1 « MaxTbLog2SizeY (7-16)

[0087] PicWidthInCtbsY = Ceil(pic_width_in_luma_samples ÷ CtbSizeY) (7-17)

[0088] PicHeightInCtbsY = Ceil(pic_height_in_luma_samples ÷ CtbSizeY) (7-18)

[0089] PicSizeInCtbsY = PicWidthInCtbsY * PicHeightInCtbsY (7-19)

[0090] PicWidthInMinCbsY = pic_width_in_luma_samples / MinCbSizeY (7-20)

[0091] PicHeightInMinCbsY = pic_height_in_luma_samples / MinCbSizeY (7-21)

[0092] PicSizeInMinCbsY = PicWidthInMinCbsY * PicHeightInMinCbsY (7-22)

[0093] PicSizeInSamplesY = pic_width_in_luma_samples * pic_height_in_luma_samples (7-23)

[0094] PicWidthInSamplesC = pic_width_in_luma_samples / SubWidthC (7-24)

[0095] PicHeightInSamplesC = pic_height_in_luma_samples / SubHeightC (7-25)

[0096] Figure 4A is an example of a CTB across the bottom picture border. Figure 4B is an example of a CTB across the right picture border. Figure 4C is an example of a CTB across the bottom-right picture border. In Figures 4A-4C , there are K=M, L<N; K

[0097] Referring to Figures 4A-4C CTUs in picture 400 are discussed. Assume that the CTB / maximum coding unit (LCU) size is denoted by MxN (typically M is equal to N as defined in HEVC / VVC), and for a CTB located at the picture (or slice or tile or other type, taking picture border as an example) border, KxL samples are within the picture border, where K Figures 4A-4C For those CTBs 402 depicted in, the CTB size is still equal to MxN, however, the bottom / right side boundary of the CTB is outside of picture 400.

[0098] The codec process of a typical video encoder / decoder (a.k.a. codec) is discussed. Figure 5 is an example of an encoder block diagram of VVC, which contains three in-loop filters: a Deblocking Filter (DF), a Sample Adaptive Offset (SAO), and an Adaptive Loop Filter (ALF). Unlike DF, which uses a pre-defined filter, SAO and ALF exploit the original samples of the current picture by adding an offset and applying a Finite Impulse Response (FIR) filter, respectively, to reduce the mean square error between the original and reconstructed samples with the help of coded side information signaling the offset and filter coefficients. ALF is located at the last processing stage of each picture and can be regarded as a tool that tries to capture and fix artifacts established by previous stages.

[0099] Figure 5 is a schematic diagram of an encoder 500. The encoder 500 is suitable for implementing VVC techniques. The encoder 500 includes three in-loop filters, namely, a Deblocking Filter (DF) 502, a Sample Adaptive Offset (SAO) 504, and an ALF 506. Unlike DF 502, which uses a pre-defined filter, SAO 504 and ALF 506 exploit the original samples of the current picture by adding an offset and applying a FIR filter, respectively, to reduce the mean square error between the original and reconstructed samples with the help of coded side information signaling the offset and filter coefficients. ALF 506 is located at the last processing stage of each picture and can be regarded as a tool that tries to capture and fix artifacts established by previous stages.

[0100] The encoder 500 also includes an intra prediction component 508 and a motion estimation / compensation (ME / MC) component 510 configured to receive an input video. The intra prediction component 508 is configured to perform intra prediction, while the ME / MC component 510 is configured to perform inter prediction with reference pictures obtained from a reference picture buffer 512. Residual blocks from either inter or intra prediction are fed to a transform component 514 and a quantization component 516 to generate quantized residual transform coefficients, which are fed to an entropy coding component 518. The entropy coding component 518 entropy codes the prediction results and quantized transform coefficients and sends them to a video decoder (not shown). Quantized components output from the quantization component 516 can be fed to an inverse quantization component 520, an inverse transform component 522, and a reconstruction (REC) component 524. The REC component 524 is capable of outputting images to the DF 502, the SAO 504, and the ALF 506 for filtering before the images are stored in the reference picture buffer 512.

[0101] The input to DF 502 is the reconstructed samples before the in-loop filter. The vertical edges in the picture are filtered first. Then, the horizontal edges in the picture are filtered using the samples modified by the vertical edge filtering process as input. The vertical and horizontal edges in the CTBs of each CTU are processed separately on a coding unit basis. The vertical edges of the coding blocks in a coding unit are filtered starting from the left edge of the coding blocks and proceeding through the edges to the right side of the coding blocks in their geometric order. The horizontal edges of the coding blocks in a coding unit are filtered starting from the top edge of the coding blocks and proceeding through the edges to the bottom side of the coding blocks in their geometric order.

[0102] Figure 6 is an illustration 600 of samples 602 within an 8x8 block of samples 604. As shown, the illustration 600 includes horizontal block boundaries 606 and vertical block boundaries 608 on an 8x8 grid, respectively. In addition, the illustration 600 depicts a non-overlapping block 610 of 8x8 samples that can be de-blocked in parallel.

[0103] Boundary decision is discussed. The filter is applied to 8x8 block boundaries. In addition, it must be a transform block boundary or a coding sub-block boundary (e.g., due to the use of affine motion prediction, optional temporal motion vector prediction (ATMVP)). For those boundaries that are not such boundaries, the filter is disabled.

[0104] Boundary strength calculation is discussed. For a transform block boundary / coding sub-block boundary, if it is located in an 8x8 grid, the transform block boundary / coding sub-block boundary can be filtered and the bS[xD i ][yD j ](where [xD i ][yD j ] denotes the coordinates) is defined in Table 1 and Table 2, respectively.

[0105] Table 1. Boundary strength (when SPS IBC is disabled)

[0106]

[0107] Table 2. Boundary strength (when SPS IBC is enabled)

[0108]

[0109] Deblocking decision for the luma component is discussed.

[0110] Figure 7is an example 700 of pixels involved in filter on / off decision and strong / weak filter selection. The wider and stronger luma filter is used only when condition 1, condition 2 and condition 3 are all true. Condition 1 is "large block condition". This condition checks whether the samples on the P side and the Q side belong to a large block, denoted by variables bSidePisLargeBlk and bSideQisLargeBlk, respectively. The definitions of bSidePisLargeBlk and bSideQisLargeBlk are as follows.

[0111] bSidePisLargeBlk = ((edge type is vertical and p0 belongs to a CU with width >= 32) || (edge type is horizontal and p0 belongs to a CU with height >= 32))? TRUE : FALSE

[0112] bSideQisLargeBlk = ((edge type is vertical and q0 belongs to a CU with width >= 32) || (edge type is horizontal and q0 belongs to a CU with height >= 32))? TRUE : FALSE

[0113] Based on bSidePisLargeBlk and bSideQisLargeBlk, condition 1 is defined as follows.

[0114] Condition 1 = (bSidePisLargeBlk || bSideQisLargeBlk)? TRUE : FALSE

[0115] Next, if condition 1 is true, condition 2 will be further checked. First, the following variables are derived.

[0116] First, dp0, dp3, dq0, dq3 are derived as in HEVC.

[0117] If (P side >= 32)

[0118] dp0 = (dp0 + Abs(p50 - 2*p40 + p30) + 1) » 1

[0119] dp3 = (dp3 + Abs(p53 - 2*p43 + p33) + 1) » 1

[0120] If (Q side >= 32)

[0121] dq0 = (dq0 + Abs(q50 - 2*q40 + q30) + 1) » 1

[0122] dq3 = (dq3 + Abs(q53 - 2*q43 + q33) + 1) » 1

[0123] Condition 2 = (d < beta)? TRUE : FALSE

[0124] where d = dp0+ dq0+ dp3+ dq3.

[0125] If both Condition 1 and Condition 2 are valid, then further check whether any block uses sub-blocks.

[0126]

[0127]

[0128] Finally, if both Condition 1 and Condition 2 are valid, then the proposed deblocking method will check Condition 3 (Strong Filter Condition for large blocks), which is defined as follows.

[0129] In Condition 3 StrongFilterCondition, the following variables are derived.

[0130] dpq is derived as in HEVC.

[0131]

[0132] As in HEVC, StrongFilterCondition = (dpq < (beta » 2), sp3+ sq3 < (3*beta » 5), and Abs(p0-q0) < (5*beta » 1))? TRUE : FALSE. C

[0133] The strong deblocking filter for luma (designed for large blocks) is discussed.

[0134] When the samples on either side of the boundary belong to large blocks, the bilinear filter is used. The samples belonging to large blocks are defined when the width of the vertical edge >= 32 and the height of the horizontal edge >= 32.

[0135] The bilinear filter is listed as follows.

[0136] Then, in the above HEVC deblocking, for the block boundary sample p i and q j are replaced by linear interpolation, p i and q j are the i-th sample in a row for vertical edge filtering, or the j-th sample in a column for horizontal edge filtering, as follows.

[0137] p i ' = (f i * Middle s，t + (64 - f i ​) * P s + 32) » 6), clipped to p i ± tcPD i

[0138] q j ' = (g j * Middle s，t + (64 - g j ) * Q s + 32) » 6), clipped to q j ± tcPD j

[0139] where tcPD i and tcPD j terms are position dependent clippings described below and g j , f i , Middle s，t , P s and Qs are given below.

[0140] Chroma deblocking control is discussed.

[0141] A strong chroma filter is used on both sides of the block boundary. Here, when both sides of the chroma edge are greater than or equal to 8 (chroma position), the chroma filter is selected and the following decision with three conditions is satisfied. The first is a decision for the boundary strength and large block. The proposed filter can be applied when the block width or height in the chroma sample domain orthogonal to the block edge is equal to or greater than 8. The second and third are basically the same as the HEVC luma deblocking decision, open / close decision and strong filter decision, respectively.

[0142] In the first decision, the boundary strength (bS) is modified for the chroma filter and the conditions are checked sequentially. If the condition is satisfied, the rest of the lower priority conditions are skipped.

[0143] Chroma deblocking is performed when bS is equal to 2 or bS is equal to 1 when a large block boundary is detected.

[0144] The second and third conditions are basically the same as the HEVC luma strong filter decision as follows.

[0145] In the second condition: d is derived as in HEVC luma deblocking. The second condition is true when d is less than β.

[0146] In the third condition, StrongFilterCondition is derived as follows.

[0147] dpq is derived as in HEVC luma deblocking

[0148] sp3 = Abs(p3 - p0), derived as in HEVC

[0149] sq3 = Abs(q0 - q3), derived as in HEVC

[0150] As in the HEVC design, StrongFilterCondition = (dpq < (beta » 2), sp3 + sq3 < (beta » 3), Abs(p0 - q0) < (5 * t C + 1) » 1).

[0151] The strong de-blocking filter for chroma is discussed. The following strong de-blocking filter for chroma is defined.

[0152] p2' = (3 * p3 + 2 * p2 + p1 + p0 + q0 + 4) » 3

[0153] p1' = (2 * p3 + p2 + 2 * p1 + p0 + q0 + q1 + 4) » 3

[0154] p0' = (p3 + p2 + p1 + 2 * p0 + q0 + q1 + q2 + 4) » 3

[0155] The proposed chroma filter performs de-blocking on a 4x4 grid of chroma samples.

[0156] The position dependent clipping (tcPD) is discussed. The position dependent clipping tcPD is applied to the output samples of the luma filtering process which involves the strong and long filters modifying 7, 5 and 3 samples at the boundaries. Assuming a quantization error distribution, it is proposed to increase the clipping value for samples which are expected to have a higher quantization noise, thus an expected larger deviation of the reconstructed sample value from the true sample value.

[0157] For each P or Q boundary filtered with an asymmetric filter, depending on the result of the decision process in the boundary strength calculation, a position dependent threshold table is selected from two tables provided to the decoder as side information, namely the table Tc7 and Tc3 tabulated below.

[0158] Tc7 = {6, 5, 4, 3, 2, 1, 1}; Tc3 = {6, 4, 2};

[0159] tcPD = (Sp == 3)? Tc3 : Tc7;

[0160] tcQD = (Sq == 3)? Tc3 : Tc7;

[0161] For P or Q boundaries filtered with a short symmetric filter, a lower amplitude position dependent threshold is applied.

[0162] Tc3 = {3, 2, 1};

[0163] After defining the threshold, the filtered p' is p' is p' p' p'' p''''''''''''''''"" ""...' ... i and q' j The sample values ​​are clipped.

[0164] p” i =Clip3(p' i +tcP i ,p' i –tcP i ,p' i );

[0165] q” j =Clip3(q' j +tcQ j ,q' j –tcQ j ,q' j );

[0166] Where p' i and q' j These are the filtered sample values, p” i and q” j It is the output sample value after cropping, tcP i and tcQ j The clipping threshold is derived from the VVC tc parameters, tcPD, and tcQD. The Clip3 function is the clipping function defined in VVC.

[0167] Now we discuss sub-block deblocking adjustments. To achieve parallel-friendly deblocking using long filters and sub-block deblocking, the long filter is restricted to modifying a maximum of 5 samples on the side using sub-block deblocking (AFFINE or ATMVP or decoder-side motion vector refinement (DMVR)), as shown in the long filter's brightness control. Furthermore, sub-block deblocking is adjusted such that sub-block boundaries on the 8×8 grid near the codec unit (CU) or implicit TU boundary are restricted to modifying a maximum of two samples on each side.

[0168] The following applies to sub-block boundaries that are not aligned with the CU boundary.

[0169]

[0170]

[0171] An edge of 0 corresponds to the CU boundary, an edge of 2 or orthogonalLength-2 corresponds to 8 sample points of the sub-block boundary from the CU boundary, and so on. If implicit partitioning of TU is used, then implicit TU is true.

[0172] Sample Adaptive Offset (SAO) is discussed. The input to SAO is the de-blocked (DB) reconstructed samples. The concept of SAO is to reduce the average sample distortion of a region by first classifying the region samples into multiple categories using a selected classifier, obtaining an offset for each category, and then adding the offset to each sample of the category, where the classifier index and the region offset are coded in the bitstream. In HEVC and VVC, a region (the unit for which SAO parameters are signaled) is defined as a CTU.

[0173] Two types of SAO are employed in HEVC that can satisfy the low complexity requirement. The two types are Edge Offset (EO) and Band Offset (BO), which are discussed in detail below. The index of the SAO type is coded (in the range of [0, 2]). For EO, the sample classification is based on a comparison between the current sample and its neighbors according to one-dimensional (ID) directional patterns: horizontal, vertical, 135° diagonal, and 45° diagonal.

[0174] Figure 8 The four ID directional patterns 800 for EO sample classification are shown: horizontal (EO classification = 0), vertical (EO classification = 1), 135° diagonal (EO classification = 2), and 45° diagonal (EO classification = 3).

[0175] For a given EO classification, each sample within a CTB is classified into one of five categories. The current sample value, labeled “c”, is compared to its two neighbors along the selected ID pattern. The classification rule for each sample is summarized in Table 3. Categories 1 and 4 are associated with local valleys and local peaks along the selected ID pattern, respectively. Categories 2 and 3 are associated with corners and cusps along the selected ID pattern, respectively. If the current sample does not belong to EO categories 1-4, it belongs to category 0 and no SAO is applied.

[0176] Table 3: Sample classification rule for Edge Offset

[0177]

[0178] Geometric-based adaptive loop filter in Joint Exploration Model (JEM) is discussed. The input to DB is the reconstructed samples after DB and SAO. The sample classification and filtering process is based on the reconstructed samples after DB and SAO.

[0179] In JEM, a geometric-based adaptive loop filter (GALF) with block-based filter adaptation is applied. For the luma component, one of 25 filters is selected for each 2x2 block based on the local gradient direction and activity.

[0180] Filter shapes are discussed. Figure 9 An example of GALF filter shapes 900 is shown, including a 5x5 diamond on the left, a 7x7 diamond in the middle, and a 9x9 diamond on the right. In JEM, up to three diamond filter shapes can be selected for the luma component (as shown in Figure 9 The index is signaled at picture level to indicate the filter shape used for the luma component. Each square represents a sample, and Ci (i is 0-6 (left), 0-12 (middle), 0-20 (right)) represents the coefficient applied to that sample. For the chroma components in a picture, a 5x5 diamond is always used.

[0181] Block classification is discussed. Each 2x2 block is classified into one of 25 classes. The classification index C is derived based on its directionality D and significance quantized values are derived as follows.

[0182]

[0183] To compute D and The gradients in horizontal, vertical and two diagonal directions are first computed using a 1-D Laplacian operator.

[0184]

[0185]

[0186]

[0187]

[0188] The indices i and j refer to the coordinates of the top-left sample in the 2x2 block, and R(i,j) represents the reconstructed sample at coordinate (i,j).

[0189] The maximum and minimum of the gradients in horizontal and vertical directions are then set as:

[0190]

[0191] The maximum and minimum of the gradients in the two diagonal directions are set as:

[0192]

[0193] To derive the value of directionality D, these values are compared with each other and with two thresholds t1 and t2:

[0194] Step 1. If and are true, then D is set to 0.

[0195] Step 2. If Then continue from step 3; otherwise continue from step 4.

[0196] Step 3. If Then D is set to 2; otherwise D is set to 1.

[0197] Step 4. If Then D is set to 4; otherwise D is set to 3.

[0198] The significance value A is computed as follows:

[0199]

[0200] A is further quantized to the range 0 to 4, and the quantized value is denoted as

[0201] For two chroma components in a picture, the classification method is not applied, i.e., a single set of ALF coefficients is applied for each chroma component.

[0202] Geometric transformations of filter coefficients are discussed.

[0203] Figure 10 Relative coordinates 1000 are shown for 5x5 diamond filter support - diagonal, vertical flip, and rotation (from left to right), respectively.

[0204] Before filtering each 2x2 block, a geometric transformation such as rotation or diagonal anti-flip and vertical flip is applied to the filter coefficients f(k,l) associated with the coordinates (k,l) depending on the gradient value computed for that block. This is equivalent to applying these transformations to the samples in the filter support region. The idea is to make the different blocks for which ALF is applied more similar by aligning their directionality.

[0205] Three geometric transformations are introduced, including diagonal, vertical flip, and rotation:

[0206] Diagonal: f D (k,l) = f(l,k),

[0207] Vertical flip: f V (k,l) = f(k,K-l-1), (9)

[0208] Rotation: f R (k,l) = f(K-l-1,k)

[0209] where K is the size of the filter and 0 < k, l < K - 1 are the coefficient coordinates such that position (0, 0) is at the top-left corner and position (K - 1, K - 1) is at the bottom-right corner. A transform is applied to the filter coefficients f(k, l) according to the gradient value computed for the block. Table 4 summarizes the mapping between the transform and the four gradients of the four directions.

[0210] Table 4: Mapping between the gradients computed for a block and the transform

[0211]

[0212] The signaling of the filter parameters is discussed. In JEM, the GALF filter parameters are signaled for the first CTU, i.e., after the slice header of the first CTU and before the SAO parameters. Up to 25 sets of luma filter coefficients can be signaled. To reduce the bit overhead, the filter coefficients of different categories can be merged. In addition, the GALF coefficients of a reference picture are stored and allowed to be reused as the GALF coefficients of the current picture. The current picture can choose to use the stored GALF coefficients of a reference picture and bypass the GALF coefficient signaling. In this case, only the index of the reference picture is signaled and the stored GALF coefficients of the indicated reference picture are inherited for the current picture.

[0213] To support GALF temporal prediction, a candidate list of GALF filter sets is maintained. At the beginning of the decoding of a new sequence, the candidate list is empty. After the decoding of a picture, the corresponding filter set can be added to the candidate list. Once the size of the candidate list reaches the maximum allowed value (i.e., 6 in the current JEM), the new filter set overrides the oldest set in the decoding order, that is, the first-in-first-out (FIFO) rule is applied to update the candidate list. To avoid duplication, a set is only added to the list if the corresponding picture does not use GALF temporal prediction. To support temporal scalability, there are multiple candidate lists of filter sets and each candidate list is associated with a temporal layer. More specifically, each array assigned by the temporal layer index (TempIdx) can consist of filter sets with previously decoded pictures equal to lower TempIdx. For example, the k-th array is assigned to be associated with TempIdx equal to k and the k-th array only contains filter sets from pictures with TempIdx less than or equal to k. After a certain picture is coded, the filter set associated with the picture will be used to update those arrays associated with TempIdx equal to or higher than the TempIdx of the picture.

[0214] Temporal prediction of the GALF coefficients is used for inter coded frames to minimize the signaling overhead. For intra frames, temporal prediction is not available and a set of 16 fixed filters is assigned to each class. To indicate the use of a fixed filter, a flag for each class is signaled and, if needed, the index of the selected fixed filter is also signaled. Even when a fixed filter is selected for a given class, the coefficients of the adaptive filter f(k, l) can still be transmitted for that class, in which case the coefficients of the filter that will be applied to the reconstructed image are the sum of the two sets of coefficients.

[0215] The filtering process for the luma component can be controlled at the CU level. A flag is signaled to indicate whether GALF is applied to the luma component of a CU. For the chroma components, whether GALF is applied is only indicated at the picture level.

[0216] The filtering process is discussed. At the decoder side, when GALF is enabled for a block, each sample R(i, j) within the block is filtered, resulting in a sample value R'(i, j) as shown below, where L denotes the filter length, f m，n denotes the filter coefficients, and f(k, l) denotes the decoded filter coefficients.

[0217]

[0218] Figure 11 An example of the relative coordinates for a 5x5 diamond filter support is shown assuming the coordinates (i, j) of the current sample are (0, 0). The samples in different coordinates that are filled with the same color are multiplied by the same filter coefficient.

[0219] The geometric transform based adaptive loop filter (GALF) in VVC is discussed. In VVC test model 4.0 (VTM4.0), the filtering process of the adaptive loop filter is performed as follows:

[0220] O(x, y) =∑ (i，j) w(i, j).I(x+i, y+j) (11)

[0221] where the sample I(x+i, y+j) is the input sample, O(x, y) is the filtered output sample (i.e., the filtering result), and w(i, j) denotes the filter coefficient. In practice, in VTM4.0, it is implemented using integer arithmetic for fixed-point precision calculation.

[0222]

[0223] where L denotes the filter length, and where w(i, j) is the filter coefficient of fixed-point precision.

[0224] Compared to the design in JEM, the current design of GALF in VVC has the following major changes:

[0225] 1) The adaptive filter shape is removed. Only 7x7 filter shape is allowed for luma component and only 5x5 filter shape is allowed for chroma component.

[0226] 2) The signaling of ALF parameters is moved from slice / picture level to CTU level.

[0227] 3) The calculation of class index is performed at 4x4 level instead of 2x2. In addition, the method of sub-sampling Laplacian for ALF classification is utilized as proposed in JVET-L0147. More specifically, instead of calculating the horizontal / vertical / 45 diagonal / 135 degree gradient for each sample within a block, 1:2 sub-sampling is used.

[0228] Regarding the filtered reconstruction, the non-linear ALF in current VVC is discussed.

[0229] Equation (11) can be re-expressed as the following expression without affecting the coding efficiency:

[0230] O(x, y) = I(x, y) +∑ (i，j)≠(0，0) w(i, j).(I(x+i, y+j)-I(x, y)) (13)

[0231] where w(i, j) is the same filter coefficient as in equation (11) [except w(0, 0), which is equal to 1 in equation (13) and to 1-∑ (i，j)≠(0，0) w(i, j) in equation (11)].

[0232] Using the above filter formula (13), VVC introduces non-linearity to make ALF more effective by using a simple clipping function to reduce the impact when the neighboring sample values (I(x+i, y+j)) differ too much from the filtered current sample value (I(x, y)).

[0233] More specifically, the ALF filter is modified as follows:

[0234] O'(x, y) = I(x, y) +∑ (i，j)≠(0，0) w(i, j).K(I(x+i, y+j)-I(x, y), k(i, j)) (14)

[0235] where K(d, b) = min(b, max(-b, d)) is the clipping function and k(i, j) is the clipping parameter depending on the (i, j) filter coefficient. The encoder performs optimization to find the best k(i, j).

[0236] In the JVET-N0242 implementation, a clipping parameter k(i,j) is specified for each ALF filter and one clipping value is signaled for each filter coefficient. This means that up to 12 clipping values can be signaled in the bitstream for each luma filter and up to 6 clipping values for each chroma filter.

[0237] To limit the signaling cost and encoder complexity, only 4 fixed values are used, which are the same for INTER and INTRA slices.

[0238] Because the variance of local differences for luma is usually higher than the variance of local differences for chroma, two different sets of luma and chroma filters are applied. A maximum sample value in each set (here 1024 for 10-bit bit-depth) is also introduced so that clipping can be disabled when not necessary.

[0239] Table 5 provides the set of clipping values used in the JVET-N0242 tests. These 4 values are chosen by roughly equally dividing the full range of sample values for luma (coded in 10 bits) and the range from 4 to 1024 for chroma in the log domain.

[0240] More precisely, the luma table of clipping values is obtained by the following formula:

[0241]

[0242] Similarly, the chroma table of clipping values is obtained by the following formula:

[0243]

[0244] Table 5: Authorized clipping values

[0245] INTRA / INTER slice group luminance {1024,181,32,6} chrominance {1024,161,25,4}

[0246] The selected clipping values corresponding to the clipping value indices in Table 5 above are coded in the "alf_data" syntax element by using a Golomb coding scheme. This coding scheme is the same as the one used for the filter indices.

[0247] Convolutional neural network based loop filters for video coding are discussed.

[0248] In deep learning, a convolutional neural network (CNN or ConvNet) is a class of deep neural networks, most commonly applied to analyzing visual imagery. They have very successful applications in image and video recognition / processing, recommender systems, image classification, medical image analysis, natural language processing.

[0249] CNNs are regularized versions of multilayer perceptrons. Multilayer perceptrons typically mean fully connected networks, i.e., each neuron in one layer is connected to all neurons in the next layer. The "full connectivity" of these networks makes them prone to overfitting the data. Typical regularization methods include adding some form of weight magnitude measure to the loss function. CNNs take a different regularization approach: they exploit the hierarchical patterns in the data and use the advantage of assembling more complex patterns out of smaller and simpler ones. Thus, CNNs are at the lower end of the spectrum in terms of connectivity and complexity.

[0250] CNNs use relatively little preprocessing compared to other image classification / handling algorithms. This means that the network learns the filters that are hand-designed in traditional algorithms. This independence from existing knowledge and manual effort in feature design is a major advantage.

[0251] Deep learning based image / video compression generally has two meanings: purely neural network based end-to-end compression, and traditional framework enhanced by neural network. Purely neural network based end-to-end compression is discussed in Johannes Ballé, Valero Laparra, and Eero P. Simoncelli, “End-to-end optimization of nonlinear transform codes for perceptual quality,” 2016 Picture Coding Symposium (PCS), pp. 1-5, Institute of Electrical and Electronics Engineers (IEEE), and Lucas Theis, Wenzhe Shi, Andrew Cunningham, and Ferenc Huszar, “Lossy image compression with compressive autoencoders,” arXiv preprint arXiv:1703.00395 (2017).Fully Connected Network-Based Intra Prediction for Image Coding, IEEE Transactions on Image Processing 27, 7 (2018), 3236-3247, Yuanying Dai, Dong Liu, and Feng Wu, A convolutional neural network approach for post-processing in HEVC intra coding, MMM. Springer, 28-39, Rui Song, Dong Liu, Houqiang Li, and Feng Wu, Neural network-based arithmetic coding of intra prediction modes in HEVC, VCIP. IEEE, 1-4, and J. Pfaff, P. Helle, D. Maniry, S. Kaltenstadler, W. Samek, H. Schwarz, D. Marpe, and T. Wiegand, Neural network based intra prediction for video coding, Digital Image Processing Applications XLI, Vol. 10752, International Society for Optics and Photonics, 1075213, discuss a traditional framework enhanced by neural networks.

[0252] The first type usually adopts an autoencoder-like structure, implemented by either convolutional neural networks or recurrent neural networks. Although relying purely on neural networks for image / video compression can avoid any manual optimization or hand-crafted design, the compression efficiency can not be satisfactory. Therefore, researches devoted to the second type aim to enhance the traditional compression framework by replacing or augmenting certain modules with the aid of neural networks. In this way, they can inherit the advantages of the highly optimized traditional framework. For example, the fully connected network for intra prediction proposed in Jiahao Li, Bin Li, Jizheng Xu, Ruiqin Xiong, and Wen Gao, “Fully Connected Network-Based Intra Prediction for Image Coding,” IEEE Transactions on Image Processing 27, 7 (2018), 3236-3247, discussed in HEVC.

[0253] In addition to intra prediction, deep learning is also used to enhance other modules. For example, the in-loop filter of HEVC is replaced by a convolutional neural network, and satisfactory results are achieved in Yuanying Dai, Dong Liu, and Feng Wu, “A convolutional neural network approach for post-processing in HEVC intra coding,” MMM. Springer, 28-39. In Rui Song, Dong Liu, Houqiang Li, and Feng Wu, “Neural network-based arithmetic coding of intra prediction modes in HEVC,” VCIP. IEEE, 1-4, research applies neural networks to improve the arithmetic coding engine.

[0254] In-loop filtering based on convolutional neural networks is discussed. In lossy image / video compression, the reconstructed frame is an approximation of the original frame, because the quantization process is irreversible, resulting in distortion of the reconstructed frame. To alleviate this distortion, a convolutional neural network can be trained to learn the mapping from the distorted frame to the original frame. In practice, training must be performed before deploying the CNN-based in-loop filter.

[0255] Training is discussed. The goal of the training process is to find the best values of the parameters, including weights and biases.

[0256] First, a codec (e.g., HM, JEM, VTM, etc.) is used to compress the training dataset to generate distorted reconstructed frames. Then, the reconstructed frames are fed to the CNN and the cost is computed using the output of the CNN and the ground truth (original) frames. Commonly used cost functions include the sum of absolute differences (SAD) and the mean squared error (MSE). Next, the gradient of the cost with respect to each parameter is derived through a backpropagation algorithm. With the gradient, the parameter values can be updated. The above process is repeated until a convergence criterion is met. After the training is completed, the derived optimal parameters are saved for the inference phase.

[0257] The convolution process is discussed. During convolution, the filter moves over the image from left to right and top to bottom, changing one column of pixels when moving horizontally and one row of pixels when moving vertically. The amount of movement between applying the filter to the input image is called the stride, and it is almost always symmetric in the height and width dimensions. The default stride(s) in two dimensions is (1, 1) for both height and width movement.

[0258] Figure 12A is an example architecture 1200 of the proposed CNN filter, and Figure 12B is an example of the construction 1250 of a residual block (ResBlock). In most deep convolutional neural networks, the residual block is used as a basic module and is stacked several times to construct the final network, where in one example, the residual block is obtained by combining a convolutional layer, a ReLU / PReLU activation function, and a convolutional layer, as shown in Figure 12B .

[0259] Inference is discussed. During the inference phase, the distorted reconstructed frames are fed to the CNN and processed by the CNN model whose parameters have been determined in the training phase. The input samples of the CNN can be the reconstructed samples before DB or after DB, or the reconstructed samples before SAO or after SAO, or the reconstructed samples before ALF or after ALF.

[0260] There are certain issues with current CNN-based in-loop filtering. For example, the NN model does not use or otherwise leverage external information (e.g., including information from a video codec such as coding parameters and / or coding syntax, which is information other than information generated by the NN model itself) as attention or as an attention mechanism. Thus, information including various coding parameters such as prediction (e.g., prediction mode, motion vector, etc.), partitioning (e.g., segmentation information), etc. can not be fully utilized, such as to recalibrate feature maps.

[0261] Techniques are disclosed herein that address one or more of the foregoing problems. For example, the present disclosure provides one or more neural network (NN) filter models trained as a coding tool to improve the efficiency of video coding. The NN-based coding tool can be used to replace or otherwise augment modules included in a video codec. For example, the NN model can be used as an additional intra prediction mode, an inter prediction mode, a transform kernel, or an in-loop filter. The present disclosure also sets forth how to design the NN model by using external information such as prediction, partition, QP, etc. (as described further below) as attention or attention mechanism, which is further described below. It should be noted that the NN model can be used as any coding tool such as NN-based intra / inter prediction, NN-based super resolution, NN-based motion compensation, NN-based reference generation, NN-based fractional pixel interpolation, NN-based in-loop / post filtering, etc.

[0262] The following list of embodiments should be considered as examples to explain the general concepts. The embodiments should not be interpreted in a narrow way. Furthermore, the embodiments can be combined in any way.

[0263] In the present disclosure, the NN model can include any kind of NN architecture such as a convolutional neural network (CNN) or a fully connected NN, or a combination of CNN and fully connected NN. In the following discussion, the NN model can also be referred to as a CNN model.

[0264] In the following discussion, a video unit can be a sequence, a picture, a slice, a tile, a subpicture, a CTU / CTB, a CTU / CTB row, one or more CUs / coding blocks (CBs), one or more CTUs / CTBs, one or more virtual pipeline data units (VPDUs), a sub-region within a picture / slice / tile / tile. A parent video unit denotes a unit larger than the video unit. Typically, the parent unit will contain several video units, for example, when the video unit is a CTU, the parent unit can be a slice, a CTU row, multiple CTUs, etc. In some embodiments, the video unit can be a sample / pixel.

[0265] Figure 13is an example of a process 1300 for generating filtered samples based on a NN filter model that receives coding parameters (e.g., external information for a NN filter model) as input. The NN filter model has an attention mechanism that is based on the coding parameter input. The attention mechanism is configured to generate or obtain attention, which is useful for mimicking cognitive attention and in fact enhances certain portions of data input to the NN filter while reducing other portions of data input to the NN filter. For example, the attention can be obtained by extracting attention from processing external information for the NN filter model, such as one or more coding parameters. The attention is configured to be applied to one or more feature maps, such as by weighting the feature maps according to the attention.

[0266] In Figure 13 In the illustrated process 1300, at least some unfiltered samples are provided as input to the NN filter. In an example, the unfiltered samples are samples (e.g., pixels) of a video unit that have not yet been subjected to any filtering or have not yet been subjected to a sufficient amount of filtering. Thus, the output of the NN filter can be filtered samples. The output of the NN filter is also based on a NN filter model that is generated using a coding parameter input. The coding parameter input can be related to a reconstruction of a video unit, a partitioning or segmentation scheme for a video unit, a prediction mode for a video unit, a quantization parameter (QP) associated with a video unit, and / or a boundary strength parameter for a boundary of a video unit.

[0267] For example, a convolutional layer can be used to extract features from the coding parameters (e.g., external information), or to extract features from both the coding parameters and internal information, such as features extracted internally within the NN filter model. At least one of the extracted features is used as the attention obtained in the NN filter model.

[0268] Figure 14 is an example of a process 1400 of applying attention obtained using external information, such as coding parameters, to a feature map of a NN filter model to provide a re-calibrated feature map, according to some examples. The feature map of the NN filter model is a result of applying a filter to an input image (or is a feature map provided as output of a prior layer). For example, at each layer of the NN filter model, the feature map is the output of that layer. Thus, a feature map produced by a layer internal to the NN filter model can be referred to as an intermediate feature map, while a feature map produced by a final layer of the NN filter model can be referred to as a final feature map. For example, the final output of the NN filter model can be a final feature map.

[0269] In Figure 14In the example of FIG. 14, the operations performed by the convolutional layer(s) of the NN filter model are denoted as a, and the external information (e.g., coding parameters) is denoted as E, and the internal information (e.g., feature maps extracted inside the NN filter model) is denoted as I. The attention obtained is denoted as A. The process 1400 applies the attention A to the feature maps G to generate the re-calibrated feature maps f.

[0270] Thus, the operation a is applied to the external information (e.g., coding parameters) or a concatenation of the external information and the internal information to extract or otherwise obtain the attention A. The attention A is applied to the feature maps G to generate the re-calibrated feature maps f. For example, applying the attention A to the feature maps G effectively re-weights the feature maps G (e.g., applies different weights to different elements of the feature maps G), and the re-calibrated feature maps f are the re-weighted feature maps resulting from applying the attention A to the feature maps G. In the process 1400, N is the number of channels, W is the channel width, and H is the channel height.

[0271] In the example of FIG. 14, the operations performed by the convolutional layer(s) of the NN filter model are denoted as a, and the external information (e.g., coding parameters) is denoted as E, and the internal information (e.g., feature maps extracted inside the NN filter model) is denoted as I. The attention obtained is denoted as A. The process 1400 applies the attention A to the feature maps G to generate the re-calibrated feature maps f. Figure 14 In the example of FIG. 14, the operations performed by the convolutional layer(s) of the NN filter model are denoted as a, and the external information (e.g., coding parameters) is denoted as E, and the internal information (e.g., feature maps extracted inside the NN filter model) is denoted as I. The attention obtained is denoted as A. The process 1400 applies the attention A to the feature maps G to generate the re-calibrated feature maps f. N×W×H The attention obtained is denoted as A, where A e R W×H The attention obtained is denoted as A, where A e R N×W×H In this example, the e notation indicates that G is an element of the set R W×H and A is an element of the set R

[0272] For example, R is the domain to which a particular element in the attention or feature maps belongs. R can be a domain that includes any real numbers or a domain that includes any integer real numbers. In some examples, the feature maps are two-dimensional or three-dimensional arrays. Thus, the feature maps G (such as the intermediate feature maps G, where G e R N×W×H ) indicates that each element of the feature maps G belongs to R, and there are NxWxH elements in the feature maps G. In this example, having values of N, W, and H indicates that the feature maps G are a three-dimensional array with a dimensionality of N, W, and H along each dimension, respectively. In another example, such as the attention A, where A e R W ×H , each element of the attention A belongs to R, and there are WxH elements in the attention A. In this example, having values of W and H indicates that the attention A is a two-dimensional array with a dimensionality of W and H along each dimension, respectively.

[0273] In one example, the re-calibrated feature maps are generated according to:

[0274] f i，j，k= G i，j，k × A j，k , 1≤i≤N, 1≤j≤W, and 1≤k≤H.

[0275] In another example, the recalibrated feature map is generated according to:

[0276] φ i，j，k = G i，j，k × f(A j，k ), 1≤i≤N, 1≤j≤W, 1≤k≤H, and f denotes a mapping function applied to each element of the attention.

[0277] In yet another example, the recalibrated feature map is generated according to:

[0278] φ i，j，k = G i，j，k × f(A j，k ) + G i，j，k , 1≤i≤N, 1≤j≤W, 1≤k≤H, and f denotes a mapping function applied to each element of the attention.

[0279] As mentioned above, the residual block serves as a basic module of the NN model, and multiple residual blocks can be stacked to build the final network. For example, a residual block can be obtained by combining a convolutional layer, a ReLU / PreLU activation function, and another convolutional layer, as shown in Figure 12B Thus, in at least some examples, the attention is only applied to the last layer of the residual block (e.g., the feature map output of the last layer), rather than each layer of the residual block. Examples of applying attention in this way are also shown in Figure 15B , which are further discussed below.

[0280] In another example, the attention is only applied to a specified layer of the NN filter model (e.g., the feature map output by the specified layer), rather than each layer of the NN filter model. The specified layer can be specified by the topology of the NN filter model.

[0281] A discussion of model selection is provided.

[0282] Example 1

[0283] 1. The NN filter model can obtain attention using external information such as reconstruction parameters, partitioning or splitting parameters, prediction parameters, boundary strength parameters, QP, etc. (generally, coding parameters related to the video unit being filtered) as input. For example, the NN filter model has an attention mechanism based on the coding parameter input.

[0284] a. In one example, the convolutional layers of the NN filter model are used to extract features from external information or both external information and internal information, where internal information refers to features extracted inside the network, and external information refers to other information that cannot be obtained from the features inside the network, such as coding parameters related to the video unit being filtered. At least one of these extracted features will be used as attention. As mentioned above, Figure 14 A diagram of the attention mechanism according to some examples is provided, where the operation performed by the convolutional layer(s) is denoted as a, the external information is denoted as E, the internal information is denoted as I, and the obtained attention is denoted as A.

[0285] i. In one example, E is one of the reconstruction parameters, partition or split parameters, prediction parameters, boundary strength parameters, QP, etc.

[0286] ii. In one example, E can be an arbitrary combination of the reconstruction parameters, partition or split parameters, prediction parameters, boundary strength parameters, QP, etc.

[0287] iii. In one example, I is the intermediate feature map(s) of the NN model that will be recalibrated by the obtained attention.

[0288] iv. In one example, A = a(E).

[0289] v. In one example, A = a(E, I), where E and I are first concatenated and then fed to a convolutional layer.

[0290] vi. In one example, A = a(E), where E is the concatenation of the reconstructed image and the partitioned image, a is a two-layer convolutional neural network, and A is a single-channel feature map whose spatial resolution is the same as the feature map to which A will be applied.

[0291] b. In one example, the obtained attention is used to recalibrate the intermediate feature maps. The intermediate feature maps of the NN model are denoted as G, where G e g N×W×H where N, W, and H are the number of channels, width, and height, respectively. The obtained attention is denoted as A, where A e R W×H represents the obtained attention. In this context, the e notation indicates that G is an element from the set given by R N×W×H and A is an element from the set given by R W×H .

[0292] i. In one example, the process of applying attention can be written as:

[0293] φ i，j，k = G i，j，k x A j，k, 1≤i≤N, 1≤j≤W, 1≤k≤H, where φ is the recalibrated feature map.

[0294] ii. In one example, the process of applying attention can be written as:

[0295] φ i，j，k = G i，j，k × f(A j，k ), 1≤i≤N, 1≤j≤W, 1≤k≤H, where φ is the recalibrated feature map, and f denotes a mapping function applied to each element of attention. The mapping function can be a sigmoid function, a hyperbolic tangent (e.g., tanh) function, etc. For example, a sigmoid function is a function that has a special “S” shape or sigmoid curve. The hyperbolic tangent function is one example of a sigmoid function.

[0296] 1. In one example, different A and / or different f can be used for different channels of the feature map.

[0297] iii. In one example, the process of applying attention can be written as:

[0298] φ i，j，k = G i，j，k × f(A j，k ) + G i，j，k , 1≤i≤N, 1≤j≤W, 1≤k≤H, where φ is the recalibrated feature map, and f denotes a mapping function applied to each element of attention. The mapping function can be a sigmoid function, a hyperbolic tangent (e.g., tanh) function, etc.

[0299] 1. In one example, different A and / or different f can be used for different channels of the feature map.

[0300] iv. In one example, the attention operation can be applied to a specified layer inside the network.

[0301] 1. In one example, when the network contains residual blocks, the attention operation is only applied to the feature maps from the last layer of each residual block.

[0302] Example 2

[0303] 2. In a second embodiment, the external attention mechanism described herein can be applied in various ways. For example, the proposed external attention mechanism can be used in any NN model for video coding, such as NN-based intra / inter prediction, NN-based super-resolution, NN-based motion compensation, NN-based reference generation, NN-based fractional pixel interpolation, NN-based in-loop / post filtering, etc.

[0304] a.In one example, an external attention mechanism is used for NN-based in-loop filtering.

[0305] The first example embodiment is discussed.

[0306] The first example embodiment proposes a convolutional neural network based in-loop filtering method, in which adaptive model selection is introduced. The proposed deep in-loop filter with adaptive model selection (DAM) method is developed from the previous paper JVET-V0100, and a new network structure is introduced to the codebase of VTM-11.0+NewMCTF. Compared with VTM-11.0+NewMCTF, the proposed method proves a BD rate reduction of {Y, Cb, Cr} under AI, RA and LDB configurations.

[0307] In this first example embodiment, a convolutional neural network (CNN) is used to construct an in-loop filter to reduce the distortion that occurs during compression. The network structure is revised from the network structure discussed in Yue Li, Li Zhang, Kai Zhang, “Deep in-loop filter with adaptive model selection,” JVET-V0100. Similar to JVET-V0100, a residual block is used as a basic module and is stacked several times to construct the final network. As a further development of JVET-V0100, an external attention mechanism is introduced to this paper, resulting in increased representation capacity under similar model size. In addition, to handle different types of content, individual networks are trained for different types of slices and quality levels.

[0308] The first example embodiment is related to Figure 12A and Figure 12B as shown. To improve the architecture, an Figure 15A and Figure 15B with an external attention mechanism is now introduced. Figure 15A is a schematic block diagram of an architecture 1500 of a NN filtering method according to various examples, and Figure 15B is a schematic block diagram showing the construction of an attention residual block 1550 used in the architecture 1500 of Figure 15A according to various examples.

[0309] Apart from the attention residual block 1550, the other parts of the architecture 1500 are the same as those in JVET-V0100. The computational process in the attention module 1550 can be written as:

[0310] F_out = F_in x f(Rec, Pred) + F_in

[0311] where F_in and F_out represent the input and output of the attention module 1550, respectively. Rec and Pred represent the reconstruction and prediction, respectively. In this example, f comprises 2 convolutional layers with an activation function applied after the first convolutional layer. The objective of f is to generate a spatial attention map from external information and then recalibrate the feature map F_in.

[0312] In the technique of the proposed first example embodiment, each slice or block can determine whether a CNN-based filter is applied or not. When it is determined that a CNN-based filter is applied to a slice / block, it can further decide which model in a candidate model list comprising three models. For this, each model is trained with QPs in {17, 22, 27, 32, 37, 42}. Given a test QP (denoted as q) for a sequence, the candidate model list comprises models trained with QPs equal to {q, q-5, q-10}. The selection process is based on the rate-distortion cost at the encoder side. If needed, an indication of the on / off control and the model index are signaled in the bitstream. Further details on the model selection are provided as follows.

[0313] The granularity of the filter determination and model selection depends on the resolution and QP. In the proposed scheme, given a higher resolution and a larger QP, the determination and selection will be performed on a larger region.

[0314] The candidate model list is different on different temporal layers. For low temporal layers, the third candidate is replaced by the in-slice NN filter model if the ratio of samples coded by intra is not smaller than a threshold. For high temporal layers, the third candidate (i.e., the model corresponding to q-10) is replaced by the model corresponding to q+5.

[0315] For the all-intra configuration, the model selection is disabled, but the on / off control is still preserved.

[0316] Open-source machine learning frameworks such as PyTorch are useful for performing inference of the CNN filters proposed in VTM. Table 1-1 provides the network information in the inference stage as suggested in S. Liu, A. Segall, E. Alshina, R.-L. Liao, “JVET common test conditions and evaluation procedures for neural network-based video coding technology,” JVET-V2016.

[0317] Table 1-1. Network information for NN-based video coding tool test in inference phase

[0318]

[0319] PyTorch can also be used as a training platform. DIV2K (from https: / / data.vision.ee.ethz.ch / cvl / DIV2K / , R. Timofte, E. Agustsson, S. Gu, J. Wu, A. Ignatov, L. V. Gool) and BVI-DVC (from Ma, Di, Fan Zhang, and David R. Bull, “BVI-DVC: A Training Database for Deep Video Compression,” arXiv preprint arXiv:2003.13552 (2020)) datasets are used for training CNN filters for I- and B- slices, respectively. Different CNN models are trained to adapt to different QP points. Table 1-2 provides network information in the training phase, as suggested in S. Liu, A. Segall, E. Alshina, R.-L. Liao, “JVET common test conditions and evaluation procedures for neural network-based video coding technology,” JVET-V2016.

[0320] Table 1-2. Network information for NN-based video coding tool test in training phase

[0321]

[0322] Based on the common test conditions recommended in "JVET common test conditions and evaluation procedures for neural network-based video coding technology" (JVET-V2016) by S. Liu, A. Segall, E. Alshina, and R.-L. Liao, the proposed CNN-based in-loop filtering method was tested on VTM-11.0+NewMCTF (from https: / / vcgit.hhi.fraunhofer.de / jvet / VVCSoftware_VTM / - / tags / VTM-11.0). NewMCTF patches are from https: / / vcgit.hhi.fraunhofer.de / jvet-ahg-nnvc / nnvc-ctc / - / tree / master / Software%20Patches.

[0323] Deblocking filtering and SAO were disabled when ALF (and CCALF) were placed after the proposed CNN-based filtering. Tables 1-3 to 1-5 show the test results. In the AI ​​configuration, the proposed method reduced the BD rates of Y, Cb, and Cr by an average of 9.12%, 22.39%, and 22.60%, respectively. In the RA configuration, the proposed method reduced the BD rates of Y, Cb, and Cr by an average of 12.32%, 27.48%, and 27.22%, respectively. In the LDB configuration, the proposed method reduced the BD rates of Y, Cb, and Cr by an average of %, %, and %, respectively.

[0324] Table 1-3. Performance (RA) of the proposed method on VTM11.0+NewMCTF

[0325]

[0326] Table 1-4. Performance (LDB) of the proposed method on VTM11.0+NewMCTF

[0327]

[0328]

[0329] Table 1-5. Performance of the proposed method on VTM11.0+NewMCTF (AI)

[0330]

[0331] The first example embodiment proposes a CNN-based in-loop filtering method. The proposed CNN-based filtering method demonstrates useful coding gain.

[0332] Turning now to Figure 16 which is a block diagram illustrating an example video processing system 1600 in which various techniques disclosed herein can be implemented. Various implementations can include some or all of the components of video processing system 1600. Video processing system 1600 can include an input 1602 for receiving video content. The video content can be received in a raw or uncompressed format, e.g., 8 or 10 bit multi-component pixel values, or can be in a compressed or coded format. Input 1602 can represent a network interface, a peripheral bus interface, or a storage interface. Examples of network interfaces include wired interfaces such as Ethernet, passive optical networks (PONs), etc., and wireless interfaces such as Wi-Fi or cellular interfaces.

[0333] Video processing system 1600 can include a coding component 1604 that can implement various coding or encoding methods described in this document. Coding component 1604 can reduce the average bitrate of video from input 1602 to an output of coding component 1604 to produce a coded representation of the video. Coding techniques are thus sometimes referred to as video compression or video transcoding techniques. The output of coding component 1604 can be stored, or transmitted via a communication connection as represented by component 1606. The stored or communicated bitstream (or coded) representation of the video received at input 1602 can be used by component 1608 to generate pixel values or a displayable video to display interface 1610. The process of generating user-viewable video from a bitstream representation is sometimes referred to as video decompression. Moreover, while certain video processing operations are referred to as “coding” operations or tools, it will be understood that coding tools or operations are used at an encoder and corresponding decoding tools or operations that reverse the results of the coding will be performed by a decoder.

[0334] Examples of peripheral bus interfaces or display interfaces can include Universal Serial Bus (USB), or High Definition Multimedia Interface (HDMI), or DisplayPort, etc. Examples of storage interfaces include SATA (Serial Advanced Technology Attachment), Peripheral Component Interconnect (PCI), Integrated Drive Electronics (IDE) interfaces, etc. The techniques described in this document can be embodied in various electronic devices, such as mobile telephones, laptop computers, smart phones, or other devices capable of performing digital data processing and / or video display.

[0335] Figure 17is a block diagram of a video processing device 1700. The device 1700 can be used to implement one or more of the methods described herein. The device 1700 can be embodied in a smartphone, tablet computer, computer, Internet of Things (IoT) receiver, etc. The device 1700 can include one or more processors 1702, one or more memories 1704, and video processing hardware 1706 (a.k.a., video processing circuitry). The processor(s) 1702 can be configured to implement one or more methods described in the present document. The memory(ies) 1704 can be used for storing data and code used for implementing the methods and techniques described herein. The video processing hardware 1706 can be used to implement, in hardware circuitry, some of the techniques described in the present document. In some embodiments, the hardware 1706 can be partially or entirely within the processor 1702 (e.g., a graphics processor).

[0336] Figure 18 is a block diagram illustrating an example video coding system 1800 that can utilize the techniques of this disclosure. As shown in Figure 18 the video coding system 1800 can include a source device 1810 and a destination device 1820. The source device 1810 generates encoded video data, where the source device 1810 can be referred to as a video encoding device. The destination device 1820 can decode the encoded video data generated by the source device 1810, and the destination device 1820 can be referred to as a video decoding device.

[0337] The source device 1810 can include a video source 1812, a video encoder 1814, and an input / output (I / O) interface 1816.

[0338] The video source 1812 can include a source such as a video capture device, an interface to receive video data from a video content provider, and / or a computer graphics system for generating video data, or a combination of such sources. The video data can comprise one or more pictures. The video encoder 1814 encodes the video data from the video source 1812 to generate a bitstream. The bitstream can include a sequence of bits that form a coded representation of the video data. The bitstream can include coded pictures and associated data. A coded picture is a coded representation of a picture. The associated data can include sequence parameter sets, picture parameter sets, and other syntax structures. The I / O interface 1816 can include a modulator / demodulator (modem) and / or a transmitter. The encoded video data can be transmitted directly to the destination device 1820 via the I / O interface 1816 and network 1830. The encoded video data can also be stored onto a storage medium / server 1840 for access by the destination device 1820.

[0339] The destination device 1820 can include an I / O interface 1826, a video decoder 1824, and a display device 1822.

[0340] The I / O interface 1826 can include a receiver and / or a modem. The I / O interface 1826 can obtain encoded video data from the source device 1810 or the storage medium / server 1840. The video decoder 1824 can decode the encoded video data. The display device 1822 can display the decoded video data to a user. The display device 1822 can be integrated with the destination device 1820, or can be external to the destination device 1820, which can be configured to interface with an external display device.

[0341] The video encoder 1814 and the video decoder 1824 can operate according to a video compression standard, such as High Efficiency Video Coding (HEVC) standard, Versatile Video Coding (VVC) standard, and other current and / or further standards.

[0342] Figure 19 is a block diagram illustrating an example of a video encoder 1900 that can be Figure 18 the video encoder 1814 in the video coding system 1800 shown.

[0343] The video encoder 1900 can be configured to perform any or all of the techniques of this disclosure. In Figure 19 examples, the video encoder 1900 includes a plurality of functional components. The techniques described in this disclosure can be shared among the various components of the video encoder 1900. In some examples, a processor can be configured to perform any or all of the techniques described in this disclosure.

[0344] The functional components of the video encoder 1900 can include a partitioning unit 1901, a prediction unit 1902 (which can include a mode select unit 1903, a motion estimation unit 1904, a motion compensation unit 1905, and an intra-prediction unit 1906), a residual generation unit 1907, a transform unit 1908, a quantization unit 1909, an inverse quantization unit 1910, an inverse transform unit 1911, a reconstruction unit 1912, a buffer 1913, and an entropy encoding unit 1914.

[0345] In other examples, the video encoder 1900 can include more, less, or different functional components. In examples, the prediction unit 1902 can include an intra-block copy (IBC) unit. The IBC unit can perform prediction in IBC mode in which at least one reference picture is the picture in which the current video block is located.

[0346] Furthermore, some components, such as the motion estimation unit 1904 and the motion compensation unit 1905, can be highly integrated, but are represented separately for illustrative purposes. Figure 19 in examples.

[0347] Segmentation unit 1901 can segment an image into one or more video blocks. Figure 18 The video encoder 1814 and video decoder 1824 can support various video block sizes.

[0348] The mode selection unit 1903 can select one of the encoding / decoding modes (e.g., intra-frame or inter-frame) based on the error result, and provide the resulting intra-frame or inter-frame codec block to the residual generation unit 1907 to generate residual block data, and to the reconstruction unit 1912 to reconstruct the coded block for use as a reference picture. In some examples, the mode selection unit 1903 can select a combination of intra-frame and inter-frame prediction modes (CIIP), where prediction is based on inter-frame prediction signals and intra-frame prediction signals. In the case of inter-frame prediction, the mode selection unit 1903 can also select the resolution of the block's motion vector (e.g., sub-pixel or integer pixel precision).

[0349] To perform inter-frame prediction on the current video block, motion estimation unit 1904 can generate motion information for the current video block by comparing one or more reference frames from buffer 1913 with the current video block. Motion compensation unit 1905 can determine the predicted video block for the current video block based on the motion information and decoded samples of images from buffer 1913 other than the image associated with the current video block.

[0350] The motion estimation unit 1904 and motion compensation unit 1905 can perform different operations on the current video block, depending on whether the current video block is in an I-band, P-band, or B-band. I-bands (or I-frames) are the least compressible but do not require other video frames for decoding. S-bands (or P-frames) can use data from previous frames for decompression and are more compressible than I-frames. B-bands (or B-frames) can use previous and forward frames for data reference to achieve the highest level of data compression.

[0351] In some examples, motion estimation unit 1904 can perform unidirectional prediction on the current video block, and can search for reference images in list 0 or list 1 for reference video blocks of the current video block. Motion estimation unit 1904 can then generate a reference index indicating the reference image in list 0 or list 1, which contains the reference video block and a motion vector indicating the spatial displacement between the current video block and the reference video block. Motion estimation unit 1904 can output the reference index, prediction direction indicator, and motion vector as motion information for the current video block. Motion compensation unit 1905 can generate a predicted video block for the current block based on the reference video block indicated by the motion information of the current video block.

[0352] In other examples, the motion estimation unit 1904 can perform bi-prediction for the current video block, the motion estimation unit 1904 can search for a reference video block for the current video block in a reference picture in list 0 and also search for another reference video block for the current video block in list 1. The motion estimation unit 1904 can then generate a reference index that indicates the reference picture in list 0 and list 1 that contains the reference video blocks and a motion vector that indicates a spatial displacement between the reference video blocks and the current video block. The motion estimation unit 1904 can output the reference index and the motion vector for the current video block as motion information for the current video block. The motion compensation unit 1905 can generate a predicted video block for the current video block based on the reference video blocks indicated by the motion information for the current video block.

[0353] In some examples, the motion estimation unit 1904 can output a full set of motion information for a current video block for use in decoding processing at the decoder

[0354] In some examples, the motion estimation unit 1904 can not output a full set of motion information for a current video block. Instead, the motion estimation unit 1904 can signal the motion information for the current video block with reference to motion information for another video block. For example, the motion estimation unit 1904 can determine that the motion information for the current video block is sufficiently similar to the motion information for a neighboring video block.

[0355] In one example, the motion estimation unit 1904 can indicate a value in a syntax structure associated with the current video block, the value indicating to the video decoder 1024 that the current video block has the same motion information as another video block.

[0356] In another example, the motion estimation unit 1904 can identify another video block and a motion vector difference (MVD) in a syntax structure associated with the current video block. The motion vector difference indicates a difference between a motion vector for the current video block and a motion vector for the indicated video block. The video decoder 1824 can use the motion vector for the indicated video block and the motion vector difference to determine the motion vector for the current video block.

[0357] As discussed above, the video encoder 1814 can predictively signal motion vectors. Two examples of predictive signaling techniques that can be implemented by the video encoder 1814 include advanced motion vector prediction (AMVP) and Merge mode signaling.

[0358] The intra prediction unit 1906 can perform intra prediction for the current video block. When the intra prediction unit 1906 performs intra prediction for the current video block, the intra prediction unit 1906 can generate prediction data for the current video block based on decoded samples of other video blocks in the same picture. The prediction data for the current video block can include a predicted video block and various syntax elements.

[0359] Residual generation unit 1907 can generate residual data for the current video block by subtracting (e.g., indicated by the minus sign) the prediction video block(s) from the current video block. The residual data for the current video block can include residual video blocks corresponding to different sample components of samples in the current video block.

[0360] In other examples, e.g., in skip mode, there can be no residual data for the current video block, and residual generation unit 1907 can not perform the subtraction operation.

[0361] Transform unit 1908 can generate one or more transform coefficient video blocks for the current video block by applying one or more transforms to the residual video blocks associated with the current video block.

[0362] After transform unit 1908 generates the transform coefficient video blocks associated with the current video block, quantization unit 1909 can quantize the transform coefficient video blocks associated with the current video block based on one or more quantization parameter (QP) values associated with the current video block.

[0363] Inverse quantization unit 1910 and inverse transform unit 1911 can apply inverse quantization and inverse transform, respectively, to the transform coefficient video blocks to reconstruct the residual video blocks from the transform coefficient video blocks. Reconstruction unit 1912 can add the reconstructed residual video blocks to corresponding samples from the one or more prediction video blocks generated by prediction unit 1902 to produce a reconstructed video block associated with the current block for storage in buffer 1913.

[0364] After reconstruction unit 1912 reconstructs the video block, loop filtering operations can be performed to reduce blocking artifacts in the video block.

[0365] Entropy encoding unit 1914 can receive data from other functional components of video encoder 1900. When entropy encoding unit 1914 receives data, entropy encoding unit 1914 can perform one or more entropy encoding operations to generate entropy encoded data and output a bitstream that includes the entropy encoded data.

[0366] Figure 20 FIG. 20 is a block diagram illustrating an example of a video decoder 2000 that can be Figure 18 the video decoder 1824 in the video coding system 1800 shown.

[0367] Video decoder 2000 can be configured to perform any or all of the techniques of this disclosure. In Figure 20In an example of the video decoder 2000, the video decoder 2000 includes a plurality of functional components. The techniques described in this disclosure can be shared among the various components of the video decoder 2000. In some examples, the processor can be configured to perform any or all of the techniques described in this disclosure.

[0368] In Figure 20 In an example of the video decoder 2000, the video decoder 2000 includes an entropy decoding unit 2001, a motion compensation unit 2002, an intra prediction unit 2003, an inverse quantization unit 2004, an inverse transformation unit 2005, a reconstruction unit 2006, and a buffer 2007. In some examples, the video decoder 2000 can perform a decoding process generally reciprocal to the encoding process described with respect to the video encoder 1814 Figure 18

[0369] The entropy decoding unit 2001 can retrieve an encoded bitstream. The encoded bitstream can include entropy encoded video data (e.g., encoded blocks of video data). The entropy decoding unit 2001 can decode the entropy encoded video data, and from the entropy decoded video data, the motion compensation unit 2002 can determine motion information including motion vectors, motion vector precision, reference picture list indices, and other motion information. The motion compensation unit 2002 can determine such information, for example, by performing AMVP and Merge mode signaling.

[0370] The motion compensation unit 2002 can generate a motion compensated block, possibly performing interpolation based on an interpolation filter. An identifier of the interpolation filter to be used at sub-pixel precision can be included in the syntax elements.

[0371] The motion compensation unit 2002 can use the interpolation filter as used by the video encoder 1814 during encoding of the video block to calculate the interpolation of sub-integer pixels of the reference block. The motion compensation unit 2002 can determine the interpolation filter used by the video encoder 1814 from the received syntax information and use the interpolation filter to generate the predicted block.

[0372] The motion compensation unit 2002 can use some of the syntax information to determine the size of the blocks used to encode the frame(s) and / or slice(s) of the encoded video sequence, partitioning information describing how each macroblock of a picture of the encoded video sequence is partitioned, modes indicating how each partition is encoded, one or more reference frames (and reference frame lists) for each inter-coded block, and other information used to decode the encoded video sequence.

[0373] ​The intra prediction unit 2003 can use, for example, an intra prediction mode received in the bitstream to form a prediction block from spatially neighboring blocks. The inverse quantization unit 2004 inverse quantizes, i.e., dequantizes, quantized video block coefficients provided in the bitstream and decoded by the entropy decoding unit 2001. The inverse transform unit 2005 applies an inverse transform.

[0374] The reconstruction unit 2006 can add the residual block to the corresponding prediction block generated by the motion compensation unit 2002 or the intra prediction unit 2003 to form a decoded block. If desired, a deblocking filter can also be applied to filter the decoded block in order to remove blocking artifacts. The decoded video blocks are then stored in the buffer 2007 to provide reference blocks for subsequent motion compensation / intra prediction, and also to produce decoded video for presentation on a display device.

[0375] Figure 21 A method 2100 for coding video data according to embodiments of the disclosure is provided. The method 2100 can be performed by a coding device (e.g., an encoder) having a processor and a memory. The method 2100 can be implemented to provide a NN filter model that uses external information (e.g., coding parameters) as an attention mechanism. Such a NN filter model allows the use of such external information to recalibrate one or more feature maps generated by the NN filter.

[0376] At block 2102, the coding device applies a neural network (NN) filter to unfiltered samples of a video unit to generate filtered samples. The NN filter is based on a NN filter model having an attention mechanism that is based on coding parameter input. In embodiments, the unfiltered samples are samples (or pixels) that have not yet undergone any filtering process or have not yet been sufficiently filtered. For example, the unfiltered samples have not yet undergone any NN filter. As another example, the unfiltered samples have not yet undergone a NN filter, an adaptive loop filter (ALF), a deblocking filter (DF), a sample adaptive offset (SAO) filter, or a combination thereof.

[0377] At block 2104, the coding device converts between a video media file and a bitstream based on the generated filtered samples.

[0378] When implemented in an encoder, the converting includes receiving a media file (e.g., a video unit) and encoding the filtered samples into a bitstream. When implemented in a decoder, the converting includes receiving a bitstream including the filtered samples, and decoding the bitstream to obtain the filtered samples.

[0379] In embodiments, the method 2100 can utilize or incorporate one or more features or processes of other methods disclosed herein.

[0380] In one example, a non-transitory computer-readable medium stores a bitstream of a video generated by a method, such as all or part of method 2100, performed by a video processing device (e.g., video processing device 1700 as described above). For example, filtered samples can be generated by applying a NN filter to unfiltered samples of a video unit to generate the bitstream. As described above, the NN filter is based on a NN filter model generated using a quality indicator (QI) input and the bitstream is generated based on the filtered samples.

[0381] Next, a list of preferred solutions for some embodiments is provided.

[0382] The following solutions show example embodiments (e.g., Example 1) of the techniques discussed in this disclosure.

[0383] 1. A method of video processing, comprising performing a conversion between a video comprising a video block and a bitstream of the video based on a rule, wherein the conversion includes filtering at least some samples of the video block using a neural network (NN) filter that uses one or more NN models, and wherein the rule specifies that the one or more NN models are equipped with an attention mechanism based on external information of the video.

[0384] 2. The method of solution 1, wherein the rule specifies that a convolution layer of the NN is used to extract features from the external information, and uses the extracted features E as an attention A in the one or more NN models.

[0385] 3. The method of solution 2, wherein the rule specifies that the internal information I is used to determine A.

[0386] 4. The method of solution 3, wherein I comprises an intermediate feature map of the one or more NN models.

[0387] 5. The method of any of solutions 2-4, wherein the rule specifies that A is obtained only from E, without using I.

[0388] 6. The method of any of solutions 2-4, wherein the rule specifies that A is obtained by concatenating E and I.

[0389] 7. The method of any of solutions 2-6, wherein the rule specifies that A is obtained using a two-layer convolutional neural network.

[0390] 8. The method of any of solutions 2-7, wherein the conversion includes recalibrating an intermediate feature map of the video using the attention A.

[0391] 9. The method according to solution 8, wherein the intermediate feature map of the one or more NN models is represented as G e R N ×W×H where N, W and H are the number of channels, width and height, respectively, and wherein the attention A is represented as A e R W×H .

[0392] 10. The method according to solution 9, wherein applying the attention is: φ i，j，k = G i，j，k x A j，k , 1 < i < N, 1 < j < W, 1 < k < H, where φ is the recalibrated feature map.

[0393] 11. The method according to claim 10, wherein the process of applying the attention is: φ i，j，k = G i，j，k x f(A j，k ), 1 < i < N, 1 < j < W, 1 < k < H, where φ is the recalibrated feature map, and f represents a mapping function applied to each element of the attention.

[0394] 12. The method according to solution 11, wherein the function is a sigmoid function or a tanh function.

[0395] 13. The method according to solutions 11-12, wherein different A and / or different f are used for different channels of the feature map.

[0396] 14. The method according to solution 10, wherein applying the attention is: φ i，j，k = G i，j，k x f(A j，k ) + G i，j，k , 1 < i < N, 1 < j < W, 1 < k < H, where φ is the recalibrated feature map, and f represents a mapping function applied to each element of the attention.

[0397] 15. The method according to solution 14, wherein different A and / or different f can be used for different channels of the feature map.

[0398] 16. The method according to solutions 10-15, wherein the attention operation can be applied to a specified layer within the NN.

[0399] 17. The method according to any of solutions 10-16, wherein the rule is that when the NN contains residual blocks, the attention operation is applied only to the feature map from the last layer of each residual block.

[0400] 18. The method according to solution 14, wherein the function is a sigmoid function or a tan function.

[0401] 19. The method of any of solutions 1-18, wherein the external information comprises a partitioning scheme for the video block.

[0402] 20. The method of solutions 1-18, wherein the external information comprises a prediction mode for the video block.

[0403] 21. The method of any of solutions 1-18, wherein the external information comprises a quantization parameter associated with the video block.

[0404] 22. The method of any of solutions 1-21, wherein the external information comprises a boundary strength parameter for a boundary of the video block.

[0405] 23. The method of any of solutions 1-22, wherein the conversion comprises generating a bitstream from the video.

[0406] 24. The method of any of solutions 1-22, wherein the conversion comprises generating the video from the bitstream.

[0407] 25. A video decoding apparatus comprising a processor configured to implement a method recited in one or more of solutions 1 to 24.

[0408] 26. A video encoding apparatus comprising a processor configured to implement a method recited in one or more of solutions 1 to 24.

[0409] 27. A computer program product having computer code stored thereon, the code, when executed by a processor, causing the processor to implement a method recited in any of solutions 1 to 24.

[0410] 28. A computer readable medium having a bitstream stored thereon, the bitstream being generated by a method recited in any of solutions 1 to 24.

[0411] 29. A method of generating a bitstream, comprising generating the bitstream using one or more of solutions 1-24, and writing the bitstream to a computer readable medium.

[0412] 30. The method, apparatus or system described in the present document.

[0413] The disclosed and other solutions, examples, embodiments, modules and functional operations described herein can be implemented in digital electronic circuitry, or in computer software, firmware, or hardware, including the structures disclosed in this document and their structural equivalents, or in combinations of one or more of them. The disclosed and other embodiments can be implemented as one or more computer program products, i.e., one or more modules of computer program instructions encoded on a computer readable medium for execution by, or to control the operation of, data processing apparatus. The computer readable medium can be a machine-readable storage device, a machine-readable storage substrate, a memory device, a combination of one or more of them, or a combination of one or more of them. The term "data processing apparatus" encompasses all apparatus, devices, and machines for processing data, including by way of example a programmable processor, a computer, or multiple processors or computers. The apparatus can also include, in addition to a hardware part, code that creates an execution environment for the computer program in question, e.g., code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them. The propagated signal is an artificially generated signal, e.g., a machine-generated electrical, optical, or electromagnetic signal, that is generated to encode information for transmission to suitable receiver apparatus.

[0414] A computer program (which can also be referred to or referred to as a program, software, software application, script, or code) can be written in any form of programming language, including compiled or interpreted languages, and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. A program can be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files that store one or more modules, sub programs, or portions of code). A computer program can be deployed to be run on one computer or on multiple computers that are located at one site or distributed across multiple sites and are interconnected by a communication network.

[0415] The processes and logic flows described in this document can be performed by one or more programmable processors executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows can also be performed by, and apparatus can also be implemented as, special purpose logic circuitry, e.g., an FPGA (field programmable gate array) or an ASIC (application specific integrated circuit).

[0416] Processors suitable for the execution of a computer program include, by way of example, both general and special purpose microprocessors, and any one or more processors of any kind of digital computer. Generally, a processor will receive instructions and data from a read-only memory or a random access memory or both. The essential elements of a computer are a processor for executing instructions and one or more memory devices for storing instructions and data. Generally, a computer will also include, or be operatively coupled to receive data from or transfer data to, or both, one or more mass storage devices for storing data, e.g., magnetic, magneto-optical disks, or optical disks. However, a computer need not have such devices. Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media and memory devices, including by way of example semiconductor memory devices, e.g., erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), and flash memory devices; magnetic disks, e.g., internal hard disks or removable disks; magneto-optical disks; and compact disc read-only memory (CD-ROM) and digital versatile disc read-only memory (DVD-ROM) disks. The processor and the memory can be supplemented by, or incorporated in, special purpose logic circuitry.

[0417] Although the patent document contains many details, these should not be construed as limiting the subject matter or the scope of any potential patent to the precise embodiments presented in the patent document. Certain features described in the patent document in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment can also be implemented separately or in any suitable subcombination. Moreover, although features can be described above as acting in certain combinations and even initially claimed as such, one or more features from a claimed combination can in some cases be excised from the combination and the claimed combination can be directed to a subcombination or variation of a subcombination.

[0418] Similarly, while operations are depicted in the drawings in a particular order, this should not be understood as requiring such order nor that all illustrated operations be performed, to achieve desirable results. Moreover, the separation of various system components in the embodiments described in this patent document should not be understood as requiring such separation in all embodiments.

[0419] Only a few implementations and examples are described and other implementations, enhancements and variations can be made based on what is described and illustrated in this patent document.

[0420] While this patent document contains many details, these should not be construed as limiting the scope of any subject matter or of any embodiment, but as merely describing features that are specific to certain embodiments of the specific technology described in this patent document. Certain features described in the context of separate embodiments in this patent document can also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment can also be implemented separately or in any appropriate subcombination. Moreover, although features can be described above as acting in certain combinations and even initially claimed as such, one or more features from a claimed combination can in some cases be excised from the combination and the claimed combination can be directed to a subcombination or variation of a subcombination.

[0421] Similarly, while operations are depicted in the drawings in a particular order, this should not be understood as requiring any particular order among the operations or that all of the operations be performed, to achieve desirable results. Additionally, the separation of various system components in the embodiments described in this patent document should not be understood as requiring such separation in all embodiments.

[0422] Only a few implementations and examples are described and other implementations, enhancements and variations can be made based on what is described and illustrated in this patent document.

Claims

1. A method implemented by a video encoding / decoding device, comprising: A neural network (NN) filter is applied to unfiltered samples of a video unit to generate filtered samples, wherein the NN filter is based on an NN filter model configured to acquire attention based on encoding / decoding parameter inputs; and The conversion between video media files and bitstreams is performed based on the generated filtered samples. The intermediate feature map of the NN filter model is recalibrated by the attention, which is obtained by concatenating the encoding / decoding parameter input with the intermediate feature map to provide a concatenation result, and feeding the concatenation result into the convolutional layer of the NN filter.

2. The method according to claim 1, wherein, The encoding / decoding parameter inputs include one or more selected from the group consisting of: a segmentation scheme for the video unit; a prediction mode for the video unit; quantization parameters associated with the video unit; and boundary strength parameters for the boundaries of the video unit.

3. The method according to claim 1, further comprising: The convolutional layer using the NN filter extracts features from the encoding / decoding parameter input; as well as The extracted features are used as attention in the NN filter model.

4. The method according to claim 3, wherein, The attention is obtained using a two-layer convolutional neural network, wherein the attention is a single-channel feature map having the same spatial resolution as the intermediate feature map of the NN filter model that will be recalibrated by the attention.

5. The method of claim 3, further comprising using the attention to recalibrate the intermediate feature maps of the NN filter model, wherein, The intermediate feature map of the NN filter model is given as G, where ,in, It refers to the number of channels. It is the channel width, and It is the channel height, and the attention gained is given as... ,in .

6. The method according to claim 5, wherein, This represents the recalibrated intermediate feature map, wherein applying the attention includes: according to To provide recalibrated intermediate feature maps, where 1 i , of which 1 j And one of them k .

7. The method according to claim 5, wherein, This represents the recalibrated intermediate feature map, wherein applying the attention includes: according to To provide recalibrated intermediate feature maps, where 1 i , of which 1 j , of which 1 k , and among them, This represents the mapping function applied to each element of the attention.

8. The method according to claim 7, wherein, The mapping function This includes the sigmoid function or the hyperbolic tangent function.

9. The method according to claim 7, wherein, Different methods are used for different channels of the intermediate feature map. or .

10. The method according to claim 5, wherein, This represents the recalibrated intermediate feature map, wherein applying the attention includes: according to + To provide recalibrated intermediate feature maps, where 1 i , of which 1 j , of which 1 k , and among them, This represents the mapping function applied to each element of the attention.

11. The method according to claim 10, wherein, The mapping function This includes the sigmoid function or the hyperbolic tangent function.

12. The method according to claim 10, wherein, Different methods are used for different channels of the intermediate feature map. or .

13. The method according to any one of claims 5-12, wherein, The attention is applied to a specified layer within the NN filter model.

14. The method according to claim 13, wherein, The NN filter model comprises residual blocks, wherein the attention is applied only to the feature map from the last layer of each residual block.

15. The method according to any one of claims 1-12, wherein, The NN filter is selected from one or more of the following: adaptive loop filter, deblocking filter, and sample adaptive offset filter.

16. The method according to any one of claims 1-12, wherein, The conversion includes generating the bitstream based on the video media file.

17. The method according to any one of claims 1-12, wherein, The conversion includes parsing the bitstream to obtain the video media file.

18. An apparatus for encoding and decoding video data, comprising a processor and a non-transitory memory having instructions thereon, wherein, When executed by a processor, the instructions cause the processor to perform the method according to any one of claims 1 to 17.

19. A non-transitory computer-readable medium having a computer program and a bit stream stored thereon, wherein, When the computer program is executed by the video processing device, it generates the bitstream using the method described in any one of claims 1 to 17.

Images