Subblock-based adaptive interpolation filter in digital video coding

Abstract

The invention discloses a subblock-based adaptive interpolation filter in digital video coding. In block-based video compression, an interpolation filter may be used in a motion compensation process or in block prediction to spatially mix pixels in a predicted block and generate a reconstructed block. However, the content within a block may vary significantly across the block, and the use of a single interpolation filter type for the entire block may not be sufficient to provide effective motion compensation. In order to more effectively address this switch, a sub-block adaptive interpolation filtering method may be implemented in a video codec to improve the quality of the reconstructed block while being able to maintain a smaller file size. Sub-block adaptive interpolation filtering may be implemented by using a different interpolation filter type for each sub-block of the block. Sub-block adaptive interpolation filtering may result in improved motion compensation and higher video quality.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This non-provisional application claims priority to and / or benefits from a provisional application, serial number 63 / 589,260, filed on October 10, 2023, entitled “SUBBLOCK-BASED ADAPTIVE INTERPOLATION FILTER IN DIGITAL VIDEO CODING”. This provisional application is hereby incorporated by reference in its entirety. Background Art

[0002] Video compression is a technique that makes video files smaller and easier to transmit over the Internet. There are different methods and algorithms for video compression, which have different performance and tradeoffs. Video compression involves encoding and decoding. Encoding is the process of transforming (uncompressed) video data into a compressed format. Decoding is the process of recovering video data from a compressed format. Encoder-decoder systems are called codecs. BRIEF DESCRIPTION OF THE DRAWINGS

[0003] The embodiments will be easily understood by the following detailed description in conjunction with the accompanying drawings. To facilitate this description, similar reference numerals represent similar structural elements. In the figures of the accompanying drawings, the embodiments are shown by way of example and not limitation.

[0004] Figure 1 An encoding system and multiple decoding systems according to some embodiments of the present disclosure are shown.

[0005] Figure 2 An exemplary encoder for encoding video frames and outputting an encoded bitstream according to some embodiments of the present disclosure is shown.

[0006] Figure 3 An exemplary decoder for decoding an encoded bitstream and outputting a decoded video according to some embodiments of the present disclosure is shown.

[0007] Figure 4 A process for block encoding of a frame according to some embodiments of the present disclosure is shown.

[0008] Figure 5 A process for block decoding of a frame according to some embodiments of the present disclosure is shown.

[0009] Figure 6 Exemplary partition shapes according to some embodiments of the present disclosure are shown.

[0010] Figure 7 Examples of filter sets according to some embodiments of the present disclosure are shown.

[0011] Figure 8 Another example of a filter set according to some embodiments of the present disclosure is shown.

[0012] Fig. 9 Describes some embodiments of the present disclosure Figure 8 The filter set in .

[0013] Fig.10 An example of a partition shape and filter type selection process according to some embodiments of the present disclosure is shown.

[0014] Fig.11 Depicted is a flow chart of an exemplary method for decoding an encoded bitstream according to some embodiments of the present disclosure.

[0015] Fig.12 Depicted is a flow chart of an exemplary method for encoding a video according to some embodiments of the present disclosure.

[0016] Fig.13 A block diagram of an exemplary computing device is depicted in accordance with some embodiments of the present disclosure. DETAILED DESCRIPTION Overview

[0017] Video coding or video compression is the process of compressing video data for storage, transmission, and playback. Video compression can involve taking large amounts of raw video data and applying one or more compression techniques to reduce the amount of data required to represent the video while maintaining an acceptable level of visual quality. Video compression is a technique for efficiently storing and transmitting video content over limited bandwidth networks.

[0018] A video includes a video frame or one or more (time) sequences of frames. A frame has a frame index indicating the position of the frame within the video or one or more sequences. A frame may include an image or a single still image. A frame may have millions of pixels. For example, a frame of an uncompressed 4K video may have a resolution of 3840×2160 pixels. A pixel may have a luma / luminance value and a chroma / chrominance value. In video compression, a frame may be partitioned into blocks for block-based processing or block-based compression. A block may have a much smaller size, such as 512×512 pixels, 256×256 pixels, 128×128 pixels, 64×64 pixels, 32×32 pixels, 16×16 pixels, 8×8 pixels, 4×4 pixels, etc. A block may include a square or rectangular area of ​​a frame.

[0019] In video compression, motion compensation is a tool for reducing temporal redundancy between video frames. Motion compensation may involve using a motion estimation process or algorithm to predict a block in the current frame based on a reference block in a previously encoded frame, followed by interpolation of the predicted block (e.g., by applying an interpolation filter) to generate a reconstructed block.

[0020] In video compression, block prediction can be used to reduce spatial redundancy within a video frame. Block prediction may involve predicting a block in the current frame based on a reference block in the same frame using a vector estimation process or algorithm, followed by interpolation of the predicted block (e.g., by applying an interpolation filter) to generate a reconstructed block.

[0021] Interpolation filters can be used in motion compensation processes or in block prediction to spatially mix pixels in predicted blocks and generate reconstructed blocks. Interpolation filters can help reduce distortion or artifacts in reconstructed blocks. Interpolation filters can help make reconstructed blocks better match original or source blocks. In some cases, a fixed set of interpolation filter types can be used for each block size in a video codec, and the encoder can select a filter type for the entire block based on the characteristics of the video content and the block. Interpolation filter types may include filters that can derive sub-pixel information. Interpolation filter types may include multi-tap finite impulse response (FIR) filters. Interpolation filter types can assist fractional motion estimation. Interpolation filter types may include filters based on neural networks that implement suitable kernels. Interpolation filter types may include affine motion filters. Interpolation filter types may include resampling filters. Interpolation filter types may include smoothing filters (smooth / smoothing filter). Interpolation filter types may include sharpening filters (sharp / sharpening filter). Interpolation filter types may include conventional filters. The normal filter, the smoothing filter, and the sharpening filter may have different filter coefficients to achieve different filtering effects.

[0022] However, the content within a block may vary significantly across blocks, and using a single interpolation filter type for the entire block may not be sufficient to provide effective motion compensation. In addition, the block sizes used in modern video codecs may be very large, up to 256x256 pixels, 512x512 pixels, or larger. This may increase the need for effective filtering methods to address spatial variations within the block. Therefore, using a single interpolation filter type for such large block sizes may not be sufficient to provide effective motion compensation because the content within the block and its spatial characteristics may vary significantly across blocks.

[0023] One approach to addressing this concern involves dividing blocks into smaller blocks during block partitioning in block-based compression, and treating the smaller blocks as independent blocks in the encoded bitstream. Smaller blocks will be encoded as separate blocks, and different interpolation filter types can be selected for the smaller blocks. In other words, this approach allows different interpolation filters to be selected for each smaller block. However, this approach comes at a cost. Additional signaling (e.g., additional bits) will be required to signal the decoding parameters of the smaller blocks. When different signaling is used for smaller blocks, entropy can increase, which can lead to inefficient entropy coding.

[0024] In order to more effectively address this concern, a sub-block adaptive interpolation filtering method can be implemented in a video codec to improve the quality of the reconstructed block while being able to keep the file size small. Sub-block adaptive interpolation filtering can be implemented by using a different interpolation filter type for each sub-block of the block. Sub-block adaptive interpolation filtering can result in improved motion compensation and higher video quality.

[0025] As used herein, a sub-block refers to a region, part, or area of ​​a block. A sub-block is not a smaller block partitioned from a larger block or super-block during block partitioning in block-based compression. In the coded bitstream, a sub-block is not encoded as a block with its own, separate, or separate block header or signaling information.

[0026] A technical task for implementing sub-block adaptive interpolation filtering in video compression is to develop an efficient sub-block adaptive filtering method that can effectively improve motion compensation and video quality in video decoding while minimizing computational and signaling overheads.

[0027] One aspect of sub-block adaptive filtering techniques involves enabling sub-block based interpolation filter selection in an encoder to improve motion compensation processes and enhance video quality. A video encoder may be allowed to select (1) a partition shape from a set of predefined partition shapes, and (2) for each partition of the selected partition shape, an interpolation filter type or an interpolation filter type from a set of options. The video encoder may optimize for filter cost, which may include one or more of the following: distortion cost and signaling cost.

[0028] Another aspect of the sub-block adaptive filtering technique relates to effectively encoding interpolation filter information in the encoded bitstream. In some cases, the index of (one or more) interpolation filter types and block partition shapes can be signaled in the bitstream as interpolation filter information. In some cases, the interpolation filter option set may include one or more options involving applying different interpolation filter types in different areas of the block. The index of the interpolation filter option can be signaled in the bitstream as interpolation filter information. The interpolation filter information can be used by the decoder in the motion compensation process.

[0029] Sub-block adaptive filtering techniques can be used in any video codec that uses interpolation filters for motion compensation in inter-frame prediction or block prediction in intra-frame prediction. Video codec standards can be used in a wide range of applications, including, for example, video streaming, video conferencing, broadcasting, and the like. Some examples of video codec standards that can employ sub-block adaptive filtering techniques include AVC (Advanced Video Coding), HEVC (High Efficiency Video Coding), AV1 (AOMedia Video 1), and VVC (Versatile Video Coding). AVC (also known as "ITU-T H.264 (08 / 21)") was approved on August 22, 2021. HEVC (also known as "H.265ITU-T H.265 (V9) (09 / 2023)") was approved on September 13, 2023. AV1 is a video decoding codec designed for video transmission on the Internet. Version 1.1.1 of the "AV1 Bitstream & Decoding Process Specification" with errata was last modified on January 18, 2019. VVC (also known as "ITU-TH.266 (V3) (09 / 2023)") was approved on September 29, 2023.

[0030] The method may be incorporated into hardware and / or software that supports interpolation filtering, for example, in the motion compensation process, such as SVT-AV1 (Scalable Video Technology AV1 Encoder), SVT-VP9 (Scalable Video Technology VP9 Encoder), and SVT-HEVC (Scalable Video Technology HEVC Encoder). The method may enable more efficient motion compensation by allowing different interpolation filter types to be used for different regions, parts, or areas of a predicted block, which may better capture local characteristics of the video content and result in higher video quality.

[0031] The adaptive sub-block filtering techniques described herein may be a solution to the sub-block filtering problem in video coding, where the benefits of using different filters for different regions of a block cannot be easily or effectively achieved by simply dividing the block into smaller blocks due to the computational and signaling overhead involved. The benefits of the proposed adaptive sub-block filtering techniques may include improved compression efficiency and reduced computational complexity compared to alternative techniques. The technique may allow adaptive sub-block filtering based on the characteristics of each block, and may improve the visual quality of the encoded video. The method may be implemented in any video coding standard that supports motion compensation with interpolation filtering. The method may be implemented in any video coding standard that supports block prediction with interpolation filtering. Video Compression

[0032] Figure 1 The encoding system 130 and one or more decoding systems 150 according to some embodiments of the present disclosure are shown. 1…D .

[0033] The encoding system 130 may be Fig.13 The encoding system 130 may be implemented on a computing device 1300 of a computer system. The encoding system 130 may be implemented in the cloud or in a data center. The encoding system 130 may be implemented on a device for capturing video. The encoding system 130 may be implemented on an independent computing system. The encoding system 130 may perform an encoding process in video compression. The encoding system 130 may receive a video (e.g., an uncompressed video, an initial video, a raw video, etc.) including a sequence of video frames 104. The video frames 104 may include image frames or images that make up the video. The video may have a frame rate or frames per second (FPS), which defines the number of frames per second of the video. The higher the FPS, the more realistic and smooth the video looks. Typically, for a natural, realistic viewing experience for human viewers, the FPS is greater than 24 frames per second. Examples of videos may include TV series, movies, short films, short videos (e.g., less than 15 seconds long), video capture gaming experiences, computer screen content, video conferencing content, live event broadcast content, sports content, surveillance video, videos shot using a mobile computing device (e.g., a smart phone), etc. In some cases, the video may include a mix or combination of different types of videos.

[0034] The encoding system 130 may include an encoder 102 that receives a video frame 104 and encodes the video frame 104 into an encoded bitstream 180. Figure 2 An exemplary implementation of encoder 102 is shown in FIG.

[0035] The encoded bitstream 180 may be compressed, which means that the encoded bitstream 180 may be smaller in size than the video frame 104. The encoded bitstream 180 may include a series of bits, for example, with 0s and 1s. The encoded bitstream 180 may have header information, payload information, and footer information, which may be encoded as bits in the bitstream. The header information may provide information about one or more of the following items: the format of the encoded bitstream 180, the encoding process implemented in the encoder 102, the parameters of the encoder 102, and the metadata of the encoded bitstream 180. For example, the header information may include one or more of the following items: resolution information, frame rate, aspect ratio, color space, etc. The payload information may include data representing the content of the video frame 104, such as sample frames, symbols, syntax elements, etc. For example, the payload information may include bits encoding one or more of the motion predictor, transform coefficients, prediction mode, and quantization level of the video frame 104. The footer information may indicate the end of the encoded bitstream 180. The footer information may include other information including one or more of: a checksum, an error correction code, and a signature.The format of the encoded bitstream 180 may vary depending on the specifications of the encoding and decoding processes (ie, codecs).

[0036] The encoded bitstream 180 may include packets, wherein the encoded video data and signaling information may be packaged. An exemplary format is an open bitstream unit (OBU), which is used in an AV1 encoded bitstream. The OBU may include a header and a payload. The header may include information about the OBU, such as information indicating the type of the OBU. Examples of OBU types may include a sequence header OBU, a frame header OBU, a metadata OBU, a temporal delimiter OBU, and a tile group OBU. The payload in the OBU may carry quantized transform coefficients and syntax elements, which may be used in a decoder to correctly decode the encoded video data, thereby regenerating a video frame.

[0037] The encoded bitstream 180 may be transmitted via the network 140 to one or more decoding systems 150 1…D The network 140 may be the Internet. The network 140 may include one or more of the following: a cellular data network, a wireless data network, a wired data network, a cable Internet network, a fiber optic network, a satellite Internet network, and the like.

[0038] A number D of decoding systems 150 are shown. 1…D . Decoding system 150 1…D At least one of the Fig.13 The system 150 is implemented on the computing device 1300.1…D Examples of may include personal computers, mobile computing devices, gaming devices, augmented reality devices, mixed reality devices, virtual reality devices, televisions, etc. Decoding system 150 1…D Each of the decoding processes in the video compression can be performed. Decoding system 150 1…D Each of the decoders may include a decoder (eg, decoder 1 ... D 162 1…D ) and one or more display devices (eg, display devices 1 ... D 164 1…D ).exist Figure 3 An exemplary implementation of a decoder (eg, decoder 11621) is shown in FIG.

[0039] For example, decoding system 1 1501 may include decoder 1 1621 and display device 1 1641. Decoder 1 1621 may implement a decoding process for video compression. Decoder 1 1621 may receive encoded bitstream 180 and produce decoded video 1681. Decoded video 1681 may include a series of video frames, which may be a version or a reconstructed version of video frame 104 encoded by encoding system 130. Display device 1 1641 may output decoded video 1681 for display to one or more human viewers or users of decoding system 11501.

[0040] For example, decoding system 2 1502 may include decoder 2 1622 and display device 2 1642. Decoder 2 1622 may implement a decoding process for video compression. Decoder 2 1622 may receive the encoded bitstream 180 and generate decoded video 1682. Decoded video 1682 may include a series of video frames, which may be a version or a reconstructed version of video frames 104 encoded by encoding system 130. Display device 2 1642 may output decoded video 1682 for display to one or more human viewers or users of decoding system 2 1502.

[0041] For example, decoding system D 150 D Decoder D 162 may be included D and display device D 164 D . Decoder D 162 D It can realize the decoding process of video compression. Decoder D 162 D The coded bitstream 180 may be received and the decoded video 168 may be generated. D . Decoded video 168 D A series of video frames may be included, which may be versions or reconstructed versions of video frames 104 encoded by encoding system 130. Display device D 164 D Can output decoded video 168 D, for display to the decoding system D150 D One or more human viewers or users.

[0042] As discussed herein, encoder 102 may be modified to implement Figure 4 , Fig.10 and Fig.12 The operation shown in FIG. 162 is as follows. 1…D The decoder can be modified to implement Fig.10 The encoder 102 and the decoder may implement the same Figure 4-12 Operations related to the sub-block adaptive interpolation filtering technology shown in . Video Encoder

[0043] Figure 2 An encoder 102 for encoding a video frame and outputting an encoded bitstream according to some embodiments of the present disclosure is shown. The encoder 102 may include one or more of the following: signal processing operations and data processing operations, including inter-frame and intra-frame prediction, transforms, quantization, in-loop filtering, and entropy decoding. The encoder 102 may include a reconstruction loop involving inverse quantization and inverse transforms to ensure that the decoder will see the same reference blocks and frames. The encoder 102 may receive a video frame 104 and encode the video frame 104 into an encoded bitstream 180. The encoder 102 may include one or more of the following: partitioning 206, transform and quantization 214, inverse transform and inverse quantization 218, in-loop filter 228, motion estimation 234, inter-frame prediction 236, intra-frame prediction 238, and entropy decoding 216.

[0044] Partition 206 can divide the frames in video frame 104 into blocks of pixels. Different codecs can allow different variable ranges of block sizes. In one codec, a frame can be partitioned into blocks of 128×128 or 64×64 pixels by partition 206. In some cases, a frame can be partitioned into blocks of 256×256 or 512×512 pixels by partition 206. Large blocks can be referred to as super blocks. Partition 206 can further divide each super block using a multi-way partition tree structure. In some cases, the partitioning of the super block can be further recursively divided (e.g., down to blocks of 4×4 size) by partitioning 206 using a multi-way partition tree structure. In another codec, a frame can be partitioned into coding tree units of 128×128 pixels by partition 206. Partition 206 can divide the coding tree unit into four coding units using a quadtree partition structure. Partition 206 can further recursively divide the coding unit using a quadtree partition structure. Partition 206 may use multiple types of tree structures (e.g., quadtree, binary tree, or ternary tree structures) to (further) subdivide the coding units. The smallest coding unit may have a size of 4×4. In some codecs, the coding units of luma pixels may be subdivided into smaller coding units (e.g., more tree structure subdivisions are performed) than the coding units of chroma pixels (e.g., the tree structure subdivisions are stopped earlier). Partition 206 may output initial samples 208, e.g., as pixel blocks. The operations performed in partition 206 create blocks of different sizes from a super block and are not to be confused with partition operations used to create sub-blocks (e.g., regions, areas, or portions) of a single block.

[0045] Intra-prediction 238 may predict samples of a block based on reconstructed predicted samples of previously encoded spatial neighboring blocks of the same frame. Intra-prediction 238 may receive reconstructed predicted samples 226 (of previously encoded spatial neighboring blocks of the same frame). Reconstructed predicted samples 226 may be generated by adder 222 based on reconstructed predicted residual 224 and predicted samples 212. Intra-prediction 238 may determine a suitable predictor for predicting samples based on reconstructed predicted samples of previously encoded spatial neighboring blocks of the same frame. Intra-prediction 238 may generate predicted samples 212 generated using a suitable predictor. Intra-prediction 238 may output or identify neighboring blocks and a predictor for generating predicted samples 212. The identified neighboring blocks and predictors may be encoded in the encoded bitstream 180 to enable a decoder to reconstruct blocks using the same neighboring blocks and predictors. In a codec, intra prediction 238 can support multiple different predictors, such as 56 different predictors. Some predictors (such as directional predictors) can capture different spatial redundancies in directional textures. By extrapolating the pixel values ​​of adjacent blocks along a certain direction, the directional predictor can be used in intra prediction 238 to predict the pixel values ​​of the block. The intra prediction 238 of different codecs can support different predictor sets to utilize different spatial modes in the same frame. The example of predictor can include direct current (DC), plane, Paeth, smooth, smooth vertical, smooth horizontal, recursive filtering mode based on, from brightness to chroma (chroma-from-luma), block copy, palette, multiple reference lines, sub-partitioning within the frame, matrix-based intra prediction (matrix coefficients can be defined by offline training using neural networks), wide-angle prediction, cross-component linear model, template matching, etc. In some cases, intra prediction 238 can perform block prediction, wherein the predicted block can be generated according to the reconstructed adjacent blocks of the same frame. Optionally, some type of interpolation filter may be applied to the predicted block to blend the pixels of the predicted block. The predicted samples 212 may be generated by translating neighboring blocks (within the same frame) according to a vector and optionally applying an interpolation filter to the neighboring blocks, using a vector compensation process in intra prediction 238 to predict the pixel values ​​of the block. The intra prediction 238 may output or identify the vector applied in generating the predicted samples 212. The intra prediction 238 may output or identify the type of interpolation filter applied in generating the predicted samples 212.

[0046] Motion estimation 234 and inter prediction 236 may predict samples of a block based on samples of a previously encoded frame (e.g., a reference frame in a decoded picture buffer 232). Motion estimation 234 and inter prediction 236 may perform motion compensation, which may involve identifying a suitable reference block and a suitable motion predictor (or vector) for the block and, optionally, an interpolation filter to be applied to the reference block. Motion estimation 234 may receive initial samples 208 from partition 206. Motion estimation 234 may receive samples (e.g., samples of a previously encoded frame or reference frame) from a decoded picture buffer 232. Motion estimation 234 may use multiple reference frames to determine one or more suitable motion predictors. A motion predictor may include a motion vector that captures the movement of a block between frames in a video. Motion estimation 234 may output or identify one or more reference frames and one or more suitable motion predictors. Inter prediction 236 may apply one or more suitable motion predictors determined in motion estimation 234 and one or more reference frames to generate predicted samples 212. The identified reference frame(s) and motion predictor(s) may be encoded in the encoded bitstream 180 to enable a decoder to reconstruct the block using the same reference frame(s) and motion predictor(s). In one codec, motion estimation 234 may implement a single reference frame prediction mode, where a single reference frame using a corresponding motion predictor is used for inter prediction 236. Motion estimation 234 may implement a composite reference frame prediction mode, where two reference frames using two corresponding motion predictors are used for inter prediction 236. In one codec, motion estimation 234 may implement techniques for searching and identifying good reference frame(s) that can produce the most efficient motion predictor. Techniques in motion estimation 234 may include searching for good reference frame(s) candidates spatially (within the same frame) and temporally (in previously encoded frames). Techniques in motion estimation 234 may include searching a deep spatial neighborhood to find a spatial candidate pool. Techniques in motion estimation 234 may include generating a temporal candidate pool using a temporal motion field estimation mechanism. Techniques in motion estimation 234 may use a motion field estimation process. Afterwards, the temporal and spatial candidates may be ranked and a suitable motion predictor may be determined.In one codec, inter prediction 236 may support multiple different motion predictors.Examples of predictors may include geometric motion vectors (complex non-linear motion), warp motion compensation (affine transforms that capture non-translational object movement), overlapped block motion compensation, advanced compound prediction (compound wedge prediction, differential modulation mask prediction, compound prediction based on frame distance, and compound inter-frame intra-frame prediction), dynamic spatial and temporal motion vector references, affine motion compensation (capturing higher order motion such as rotation, scaling, and sheering), adaptive motion vector resolution mode, geometric partitioning mode, bidirectional optical flow, prediction refinement using optical flow, bidirectional prediction using weights, extended merge prediction, etc. Optionally, some type of interpolation filter may be applied to the predicted block to blend the pixels of the predicted block. The pixel values ​​of the block may be predicted using the motion predictor / vector determined in the motion compensation process in motion estimation 234 and inter-frame prediction 236 and optionally applying an interpolation filter. The inter-frame prediction 236 may output or identify the motion predictor / vector applied in generating the predicted samples 212. The inter prediction 236 may output or identify the type of interpolation filter applied in generating the predicted samples 212 .

[0047] Mode selection 230 may be informed by components such as motion estimation 234 to determine whether inter-prediction 236 or intra-prediction 238 may be more efficient for encoding a block. Inter-prediction 236 may output predicted samples 212 of a predicted block. Inter-prediction 236 may output a selected predictor and a selected interpolation filter (if applicable) that may be used to generate a predicted block. Intra-prediction 238 may output predicted samples 212 of a predicted block. Intra-prediction 238 may output a selected predictor and a selected interpolation filter (if applicable) that may be used to generate a predicted block. Regardless of the mode, a predicted residual 210 may be generated by subtractor 220 by subtracting predicted samples 212 from initial samples 208.

[0048] Transform and quantize 214 may receive predicted residual 210. Predicted residual 210 may be generated by subtractor 220, which takes initial sample 208 and subtracts predicted sample 212 to output predicted residual 210. Predicted residual 210 may be referred to as prediction error (e.g., error between initial sample and predicted sample 212) for intra prediction 238 and inter prediction 236. The prediction error has a smaller range of values ​​than the initial sample and may be coded with fewer bits in the encoded bitstream 180. Transform and quantize 214 may include one or more of transform and quantize. Transform may include converting predicted residual 210 from spatial domain to frequency domain. Transform may include applying one or more transform kernels. Examples of transform kernels may include discrete cosine transforms (DCTs) in horizontal and vertical forms, asymmetric discrete sine transforms (ADSTs), flip ADSTs and identity transforms (IDTXs), multiple transform selections, low-frequency inseparable transforms, sub-block transforms, non-square transforms, DCT-VIII, discrete sine transforms VII (DST-VII), discrete wavelet transforms (DWTs), and the like. The transform may convert the predicted residual 210 into transform coefficients. Quantization may quantize the transformed coefficients, for example, by reducing the precision of the transform coefficients. Quantization may include using a quantization matrix (e.g., a linear and nonlinear quantization matrix). The elements in the quantization matrix may be larger for higher frequency bands and smaller for lower frequency bands, meaning that higher frequency coefficients are more coarsely quantized and lower frequency coefficients are more finely quantized. Quantization may include dividing each transform coefficient by the corresponding element in the quantization matrix and rounding to the nearest integer. In practice, the quantization matrix may implement different quantization parameters (QPs) for different frequency bands and chroma planes, and spatial prediction may be used. An appropriate quantization matrix may be selected and signaled for each frame and encoded in the encoded bitstream 180. Transform and quantization 214 may output quantized transform coefficients and syntax elements 278 that indicate the coding mode and parameters used in the encoding process implemented in encoder 102.

[0049] The inverse transform and inverse quantization 218 may apply the inverse operations performed in the transform and quantization 214 to produce a reconstructed predicted residual 224 as part of a reconstruction path to produce a decoded picture buffer 232 for the encoder 102. The inverse transform and inverse quantization 218 may receive the quantized transform coefficients and syntax elements 278. The inverse transform and inverse quantization 218 may perform one or more inverse quantization operations, such as applying an inverse quantization matrix, to obtain unquantized / original transform coefficients. The inverse transform and inverse quantization 218 may perform one or more inverse transform operations (e.g., inverse transform (e.g., inverse DCT, inverse DWT, etc.)) to obtain a reconstructed predicted residual 224. A reconstruction path is provided in the encoder 102 to generate reference blocks and frames, which are stored in the decoded picture buffer 232. The reference blocks and frames may match the blocks and frames to be generated in the decoder. The reference blocks and frames are used as reference blocks and frames by motion estimation 234, inter-frame prediction 236, and intra-frame prediction 238.

[0050] The in-loop filter 228 may implement a filter to remove artifacts introduced by the encoding process in the encoder 102 (e.g., the processing performed by the partition 206 and the transform and quantization 214). The in-loop filter 228 may receive the reconstructed predicted samples 226 from the adder 222 and output the frame to the decoded picture buffer 232. Examples of filters may include a constrained low-pass filter, a directional deringing filter, an edge-oriented conditional replacement filter, a loop restoration filter, a Wiener filter, a self-guided restoration filter, a constrained directional enhancement filter, a luma map with chroma scaling, a sample adaptive offset filter, an adaptive loop filter, a cross-component adaptive loop filter, and the like.

[0051] Entropy coding 216 may receive quantized transform coefficients and syntax elements 278 (e.g., referred to herein as symbols) and perform entropy coding. Entropy coding 216 may generate and output a coded bit stream 180. Entropy coding 216 may utilize statistical redundancy and apply a lossless algorithm to encode symbols and produce a compressed bit stream, such as a coded bit stream 180. Entropy coding 216 may implement a certain version of arithmetic coding. Different versions may have different advantages and disadvantages. In one codec, entropy coding 216 may implement (symbol to symbol) adaptive multi-symbol arithmetic coding. In another codec, entropy coding 216 may implement context-based adaptive binary arithmetic coder (CABAC). Binary arithmetic coding is different from multi-symbol arithmetic coding. Binary arithmetic coding encodes only one bit at a time, for example, with a binary value of 0 or 1. Binary arithmetic coding may first convert each symbol into a binary representation (e.g., using a fixed number of bits per symbol). Handling only binary values ​​0 or 1 simplifies computation and reduces complexity. Binary arithmetic coding may assign probabilities to each binary value (e.g., the chance that the bit has a binary value of 0 and the chance that the bit has a binary value of 1). Multi-symbol arithmetic coding performs encoding on an alphabet with at least two or three symbol values ​​and assigns probabilities to each symbol value in the alphabet. Multi-symbol arithmetic coding may encode more bits at a time, which may result in fewer operations for encoding the same amount of data. Multi-symbol arithmetic coding may require more computation and storage (because probability estimates may be updated for each element in the alphabet). In multi-symbol arithmetic coding, maintaining and updating the probability of each possible symbol value (e.g., cumulative probability estimates) may be more complex (e.g., complexity grows with the size of the alphabet). Multi-symbol arithmetic coding should not be confused with binary arithmetic coding because the two different entropy coding processes are implemented in different ways and may result in different encoded bit streams for the same set of quantized transform coefficients and syntax elements 278.

[0052] As discussed herein, encoder 102 may be modified to implement Figure 4 , Fig.10 and Fig.12 The encoder 102 can implement the operation shown in Figure 4 , Figure 6-10 and Fig.12 Operations related to the sub-block adaptive interpolation filtering technology shown in . Video Decoder

[0053] Figure 3A decoder 1 1621 is shown that decodes an encoded bitstream and outputs a decoded video according to some embodiments of the present disclosure. The decoder 1 1621 may include one or more of the following: signal processing operations and data processing operations, including entropy decoding, inverse transform, inverse quantization, inter-frame and intra-frame prediction, in-loop filtering, etc. The decoder 1 1621 may have signal and data processing operations that mirror the operations performed in the encoder. The decoder 1 1621 may apply the signal and data processing operations signaled in the encoded bitstream 180 to reconstruct the video. The decoder 1 1621 may receive the encoded bitstream 180 and generate and output a decoded video 1681 having multiple video frames. The decoded video 1681 may be provided to one or more display devices for display to one or more human viewers. The decoder 1 1621 may include one or more of the following: entropy decoding 302, inverse transform and inverse quantization 218, in-loop filter 228, inter-frame prediction 236, and intra-frame prediction 238. Some of the functionality has been previously implemented in encoders (such as Figure 2 The encoder 102 is described and used in the embodiment of the present invention.

[0054] Entropy decoding 302 may decode the encoded bitstream 180 and output symbols that have been coded in the encoded bitstream 180. The symbols may include quantized transform coefficients and syntax elements 278. Entropy decoding 302 may reconstruct the symbols from the encoded bitstream 180.

[0055] The inverse transform and inverse quantization 218 may receive the quantized transform coefficients and syntax elements 278 and perform the operations performed in the encoder. The inverse transform and inverse quantization 218 may output a reconstructed predicted residual 224. The adder 222 may receive the reconstructed predicted residual 224 and the predicted samples 212 and generate a reconstructed predicted sample 226. The inverse transform and inverse quantization 218 may output a syntax element 278 with signaling information for notifying / indicating / controlling operations in the decoder 1 1621, such as mode selection 230, intra-frame prediction 238, inter-frame prediction 236, and in-loop filter 228.

[0056] Depending on the prediction mode signaled in the coded bitstream 180 (eg, as quantized transform coefficients and syntax elements in the syntax elements 278 ), intra prediction 238 or inter prediction 236 may be applied to generate the predicted samples 212 .

[0057] The adder 222 may sum the predicted samples 212 of the decoded reference block and the reconstructed predicted residual 224 to produce reconstructed predicted samples 226 of the reconstructed block. For intra prediction 238, the decoded reference block may be in the same frame as the block being decoded or reconstructed. For inter prediction 236, the decoded reference block may be in a different (reference) frame in the decoded picture buffer 232.

[0058] The intra prediction 238 may apply a predictor or vector (e.g., based on the signaled predictor information) to a reconstructed block, which may be generated using a decoded reference block of the same frame. The intra prediction 238 may apply a suitable interpolation filter type (e.g., based on the signaled interpolation filter information) to the reconstructed block to generate predicted samples 212.

[0059] The inter prediction 236 may apply a predictor or vector (e.g., according to the signaled predictor information) to a reconstructed block, which may be generated using a decoded reference block from a different frame of the decoded picture buffer 232. The inter prediction 236 may apply a suitable interpolation filter type (e.g., according to the signaled interpolation filter information) to the reconstructed block to generate predicted samples 212.

[0060] The in-loop filter 228 may receive the reconstructed predicted samples and output decoded video 1681 .

[0061] As discussed herein, decoder 1 1621 (and other decoders) may be modified to implement Figure 5 and Fig.11 The decoder 1 1621 (and other decoders) can be implemented with Figure 5-9 and Fig.11 Operations related to the sub-block adaptive interpolation filtering technology shown in . Block prediction in inter prediction and intra prediction

[0062] If used Figure 2-3 As described above, intra prediction 238 and / or inter prediction 236 in the encoder may implement some form of prediction based on reference blocks. Intra prediction 238 and / or inter prediction 236 may optionally apply an interpolation filter type to the block to blend the pixels in the predicted block. Intra prediction 238 may encode the block using spatial redundancy and use reference blocks in the same frame. Inter prediction 236 may encode the block using temporal redundancy and use reference blocks in different frames.

[0063] Figure 4A process 400 for encoding a block of a frame is shown according to some embodiments of the present disclosure. An encoder may implement the process 400. The process 400 may encode an initial block 404. The initial block 404 may include a block of pixels or samples (e.g., luma samples, chroma samples, etc.). The initial block 404 may originate from an initial, uncompressed video frame of an uncompressed video. The goal of the process 400 is to compress the initial block 404 (e.g., a current block to be encoded) to generate encoded video data that can be used by a decoder to reconstruct the initial block 404 as closely as possible (e.g., to achieve as little visual quality loss as possible) while utilizing fewer bits than the bits of the initial block 404 (e.g., to achieve compression).

[0064] In find reference block and predictor 406, a search process may be implemented in the encoder to find a suitable reference block and a suitable predictor that can be used to predict the initial block 404 based on the reference block. Find reference block and predictor 406 may receive one or more options for the initial block 404 and the reference block and one or more options for the predictor. Find reference block and predictor 406 may determine a suitable reference block and predictor 402 for the initial block 404. Suitability may depend on the available options for the reference block and the available options for the predictor. Suitability may depend on whether the reference block and predictor may produce the best or desired match with the initial block 404.

[0065] In determining interpolation filter 408, a selection process may be implemented in the encoder to determine the appropriate interpolation filter type to be applied to the reference block. Determine interpolation filter 408 may receive initial block 404 and reference block and predictor 402. Determine interpolation filter 408 may determine interpolation filter 482. Suitability may depend on the available options for interpolation filter type. Suitability may depend on whether the interpolation filter type can achieve the best or desired visual quality. Suitability may depend on whether the interpolation filter type produces the best or desired match with initial block 404.

[0066] In some cases, determining the interpolation filters 408 may occur before the lookup reference blocks and predictor 406. In some cases, determining the interpolation filters 408 may occur after the lookup reference blocks and predictor 406. In some cases, determining the interpolation filters 408 may occur simultaneously with (or in parallel with) the lookup reference blocks and predictor 406. In some cases, determining the interpolation filters 408 may be combined with the lookup reference blocks and predictor 406 (e.g., performing the search or selection process of the predictor in the determine reference blocks and predictor 402 with a transform operation that includes prediction and interpolation filtering).

[0067] In determining residual data 410, the predictor found in finding reference block and predictor 406 may be applied to the reference block found in finding reference block and predictor 406 to produce a predicted reference block. In determining residual data 410, the interpolation filter type determined in determining interpolation filter 408 may be applied to the predicted reference block to produce a filtered predicted reference block. In determining residual data 410, residual data 492 may be determined by differencing the initial block 404 and the filtered predicted reference block. Determining residual data 410 may receive the initial block 404 and the reference block and predictor 402 and the interpolation filter 482. Determining residual data 410 may determine and output residual data 492.

[0068] In block encoding 412, the initial block 404 is encoded. The block encoding 412 may receive the reference block and predictor 402, the interpolation filter 482, and the residual data 492. The block encoding 412 may produce an encoded block 460 (in compressed form) that encodes the initial block 404. The block encoding 412 may encode the initial block 404 using syntax elements that signal or identify the reference block and predictor 402, the interpolation filter 482. The block encoding 412 may encode the initial block 404 by applying a transform to the residual data 492. The block encoding 412 may apply a transform utilizing Figure 2 The other operations discussed above are performed by transform and quantization 214 and entropy decoding 216 to produce the encoded data in the encoded block 460.

[0069] Figure 5 A process 500 for decoding a block of a frame according to some embodiments of the present disclosure is shown. A decoder may implement the process 500. The process 500 may be performed on (e.g., Figure 4 The coded block may include information of one or more of the following items: a reference block 502, a predictor 508, an interpolation filter 510, and residual data 504. The reference block 502, the predictor 508, the interpolation filter 510, and the residual data 504 may be decoded by performing the process 400 using Figure 3 The entropy decoding 302 and inverse transform and inverse quantization 218 discussed above are performed to extract the encoded bitstream. The goal of process 500 is to reconstruct or recover the original blocks of the frame from the encoded data.

[0070] In block decoding 520, the encoded data of reference block 502 may be decoded into a block of pixel data. Block decoding 520 may receive the encoded data of reference block 502 or an encoded reference block. Block decoding 520 may output the (decoded) reference block 502 as a block of pixel data.

[0071] In apply predictor 522, predictor 508 may be applied to (decoded) reference block 502. Apply predictor 522 may receive predictor 508 and (decoded) reference block 502. Apply predictor 522 may output predicted block 506.

[0072] In applying interpolation filter 524, interpolation filter 510 may be applied to predicted block 506. Applying interpolation filter 524 may receive predicted block 506 and interpolation filter 510. Applying interpolation filter 524 may output filtered block 576.

[0073] In some cases, applying interpolation filters 524 may occur before applying predictor 522. In some cases, applying interpolation filters 524 may occur after applying predictor 522. In some cases, applying interpolation filters 524 may occur simultaneously with (or in parallel with) applying predictor 522. In some cases, applying interpolation filters 524 and applying predictor 522 may be combined (e.g., applying predictor 508 with a transform operation that includes prediction and interpolation filtering).

[0074] In reconstruction 526, a reconstructed block 546 (e.g., a portion of a reconstructed frame) may be recovered or reconstructed based on the encoded data of the block. The reconstructed block 546 may include a block of pixel data and may be used to reconstruct a portion of a reconstructed frame of a video. Reconstruction 526 may receive filtered block 576 and residual data 504. Reconstruction 526 may add residual data 504 to filtered block 576. Reconstruction 526 may output reconstructed block 546 based on filtered block 576 and residual data 504. Exemplary adaptive sub-block interpolation filtering techniques in encoding and decoding

[0075] During a video encoding and / or decoding process, adaptive sub-block interpolation filtering may be applied to motion compensation of a block of luminance and / or chrominance samples. Adaptive sub-block interpolation filtering involves determining interpolation filter information that may indicate different interpolation filter types being applied to different regions, areas, or portions of a block. For example, the interpolation filter information in the encoded bitstream may indicate a first interpolation filter type for a first region of a predicted block and a second interpolation filter type that is different from the first interpolation filter type for a second region of the predicted block.

[0076] exist Figure 4 In the context of , the adaptive sub-block interpolation filter technique may enhance the determined interpolation filter 408 of the encoding process 400 . Fig.10 and Fig.12An exemplary process for determining interpolation filters 408 is shown in . The determined interpolation filters 482 may include a variety of different interpolation filter types being applied to different regions, areas, or portions of a predicted reference block.

[0077] exist Figure 5 In the context of , the adaptive sub-block interpolation filter technique may enhance the application of the interpolation filter 524 of the decoding process 500 . Fig.11 An exemplary process of applying interpolation filters 524 is shown in . Interpolation filters 510 may include a variety of different interpolation filter types being applied to different areas, regions, or portions of a predicted block. Example partition shapes for dividing a block into one or more regions, zones or sections

[0078] Adaptive sub-block interpolation filtering may allow a video encoder to select (1) a partition shape from a set of predefined partition shapes, and (2) for each partition of the selected partition shape, an interpolation filter type from a set of filter type options. In some cases, adaptive sub-block interpolation filtering may allow a video encoder to select an interpolation filter type from a set of interpolation filter type options. The interpolation filter type options may include one or more single interpolation filter type options. The single interpolation filter type may specify that the same interpolation filter type is applied to the entire block. The interpolation filter type options may include one or more multiple interpolation filter type options. Multiple interpolation filter types may specify that different interpolation filter types are applied to different areas / regions / portions of a block.

[0079] Figure 6 Exemplary partition shapes according to some embodiments of the present disclosure are shown. Although square blocks are shown, it should be understood that the blocks can be rectangular. The exemplary partition shapes show different ways of dividing blocks (of pixel data) into different regions, zones or portions. Figure 6 14 different partition shapes are shown, for example, partition_shape0, partition_shape1, partition_shape2, partition_shape3, partition_shape4, partition_shape5, partition_shape6, partition_shape7, partition_shape8, partition_shape9, partition_shape10, partition_shape11, partition_shape12 and partition_shape13.

[0080] In some embodiments, a block may include a single region. In some embodiments, a block may have multiple regions, such as a first region and a second region.

[0081] In some cases, the first region may include the upper half of the decoded reference block, and the second region may include the lower half of the decoded reference block. An example is shown as partition_shape2.

[0082] In some cases, the first region may include the left half of the decoded reference block, and the second region may include the right half of the decoded reference block. An example is shown as partition_shape1.

[0083] In some cases, the first region may include a first third of the decoded reference block, and the second region may include a second third of the decoded reference block.

[0084] In some cases, the first region may include a first quarter of the decoded reference block, and the second region may include a second quarter of the decoded reference block. Examples are shown as partition_shape3, partition_shape4, partition_shape6, partition_shape7, partition_shape8, partition_shape9, partition_shape10, partition_shape11, partition_shape12, partition_shape13.

[0085] partition_shape0 has a region that includes the entire block (unpartitioned). partition_shape1 has two half regions that are divided vertically. partition_shape2 has two half regions that are divided horizontally. patition_shape3 has four quarter regions. patition_shape4 has one half region and two quarter regions. patition_shape5 has one half region and two quarter regions. patition_shape6 has one half region and two quarter regions. patition_shape7 has one half region and two quarter regions. patition_shape8 has one half region and two quarter regions. patition_shape9 has one half region and two quarter regions. patition_shape10 has one half region and two quarter regions. patition_shape11 has one half region and two quarter regions. patition_shape12 has four quarter regions. patition_shape13 has four quarter regions. Exemplary Interpolation Filter Information

[0086] Figure 7 An example of a filter set according to some embodiments of the present disclosure is shown. As an example, Figure 7 Four different interpolation filter types are shown, such as F(0), F(1), F(2), and F(3). A filter set may have a plurality of different interpolation filter types. A filter index may indicate one of the different interpolation filter types in a filter set. Interpolation filter information in an encoded bitstream may include a filter index.

[0087] The index of the filter type(s) and the block partition shape may be signaled in the bitstream as interpolation filter information to be used by a decoder (e.g., in a motion compensation process or a block prediction process) to reconstruct luma and / or chroma samples. The interpolation filter information in the encoded bitstream may include a partition shape index and one or more filter indices. The partition shape index may indicate a manner in which a block is divided into one or more regions (e.g., at least a first region and a second region). The one or more filter indices may individually indicate an interpolation filter type, e.g., a first interpolation filter type being applied to a first region and a second interpolation filter type being applied to a second region. The one or more filter indices may be in the form of a list or an array.

[0088] In some embodiments, the decoder may read a compressed bitstream generated by the encoder. The decoder may determine from the compressed bitstream a partition shape index indicating the shape of the partition, such as a particular way of dividing a block into sub-blocks (e.g., regions, zones, or portions). The partition shape index may indicate Figure 6 The decoder may determine a filter index indicating a specific interpolation filter type for each sub-block (e.g., region, area, or portion) based on the compressed bitstream. The decoder may apply the corresponding filter type indicated by the filter index to the corresponding sub-block created using the partition shape indicated by the partition shape index.

[0089] For example, the decoder may read from the compressed bitstream in the interpolation filter information: the partition shape index is equal to 2. The partition shape index may correspond to a partition shape that horizontally divides the block into a top horizontal half region or sub-block and a bottom horizontal half region or sub-block (e.g., as indicated by Figure 6). The decoder may read a first filter index indicating an interpolation filter type for the top horizontal half region and a second filter index indicating an interpolation filter type for the bottom horizontal half region from the compressed bitstream in the interpolation filter information. The decoder may apply the interpolation filter type corresponding to the first filter index to the top horizontal half region. The decoder may apply the interpolation filter type corresponding to the second filter index to the bottom horizontal half region.

[0090] In some embodiments, a method of signaling the use of adaptive sub-block interpolation filtering in the video decoding process is to add a signal (e.g., seq_sub_interpolation_is_allowed) in the coded bitstream (e.g., in the sequence header). The signal may indicate whether adaptive sub-block filtering is allowed in the entire video sequence. If the signal is set to 1, for example, the encoder may use this method for any picture in the sequence.

[0091] In some embodiments, the interpolation filter information in the encoded bitstream may include a sequence header signal to indicate whether multiple interpolation filter types are allowed for a given block of a frame sequence. In other words, the sequence header signal may indicate whether a given block in a frame sequence may have different interpolation filter types to be applied to different regions of the given block.

[0092] In some cases, if the encoder can use the method for any frame / picture / slice in a sequence, another signal (e.g., pic_sub_interpolation_is_allowed) can be added to the coded bitstream as interpolation filter information, for example, in a frame / picture / slice header. This signal in the frame / picture / slice header can further specify whether adaptive sub-block filtering is allowed for the current frame, picture, or slice. In some cases, pic_sub_interpolation_is_allowed can be added to the frame / picture / slice header even if seq_sub_interpolation_is_allowed is not used for the sequence.

[0093] In some embodiments, the interpolation filter information in the encoded bitstream may include a frame header signal to indicate whether multiple interpolation filter types are allowed for a given block of a frame. In other words, the sequence header signal may indicate whether a given block in a frame may have different interpolation filter types to be applied to different regions of the given block.

[0094] A signal (e.g., sub_interpolation_allowed_block_sizes[]) may be added to the coded bitstream as interpolation filter information indicating for which block sizes adaptive sub-block interpolation filtering is allowed. The signal may include an array of binary values, with one element for each possible block size. For example, a value of 1 for a particular block size may indicate that the adaptive sub-block filtering method is allowed for that size, while a value of 0 may indicate that it is not allowed. The signal may include multiple binary values ​​corresponding to different block sizes. In some cases, the signal sub_interpolation_allowed_block_sizes[] may be combined with the frame header signal pic_sub_interpolation_is_allowed. In some cases, the signal sub_interpolation_allowed_block_sizes[] may be combined with the sequence header signal seq_sub_interpolation_is_allowed.

[0095] In some embodiments, the interpolation filter information in the encoded bitstream may include a block size signal to indicate one or more block sizes that allow multiple interpolation filter types and one or more additional block sizes that do not allow multiple interpolation filter types. The block size signal may be included in other sequence level signaling. The block size signal may be included in other frame level signaling.

[0096] At the block level, for each block in a frame, picture, or slice that allows the use of this adaptive sub-block interpolation filtering method, one or more signals may be added to the block header (e.g., partition_shape and filter_type[]). The partition_shape signal may specify the shape of the partition to be used for the block, and may take a value from a set of partition shape options from which the encoder can select. The filter_type[] signal may specify the interpolation filter type to be used for each sub-block (e.g., region, area, or portion) within the block, and may take a value from a set of interpolation filter type options from which the encoder can select. Exemplary interpolation filter type options are shown in Figure 6-7 Shown in.

[0097] At the decoder, the following operations can be performed: 1. The decoder may read the sequence header signal (eg, seq_sub_interpolation_is_allowed) and may determine whether adaptive sub-block interpolation filtering is allowed for the (entire) video sequence. 2. If the signal in the sequence header allows adaptive sub-block interpolation filtering, then for each frame, picture or slice, the decoder can read the frame / picture / slice header signal (e.g., pic_sub_interpolation_is_allowed[]) and determine whether adaptive sub-block interpolation filtering is allowed for the current frame, picture or slice. 3. The decoder may further read a block size signal (eg, sub_interpolation_allowed_block_sizes[]) to determine for which block sizes adaptive sub-block interpolation filtering is allowed. 4. If the signal in the frame / picture / slice header allows adaptive sub-block interpolation filtering, then for each block in the frame / picture / slice that has a size that allows the use of adaptive interpolation techniques, the decoder can read the block header signal (e.g., partition_shape and filter_type[]), and can determine the partition shape to be used for the block and the interpolation filter type to be used for each sub-block (e.g., region, area, or part) within the block. 5. For each block that allows adaptive sub-block interpolation filtering, the decoder may apply the selected partition shape and interpolation filter type indicated by, for example, the partition_shape signal and the filter_type[] signal, respectively, to perform sub-block filtering on the luminance and / or chrominance samples of the block during the motion compensation process or the block prediction process. Otherwise, for each block that does not allow adaptive sub-block filtering, the decoder may apply a single or default interpolation filter to the entire block during the motion compensation process or the block prediction process.

[0098] Figure 8 Another example of a filter set according to some embodiments of the present disclosure is shown. Fig. 9 Describes some embodiments of the present disclosure Figure 8 As an example, Figure 8-9Seven different interpolation filter type options are shown, for example, F(0), F(1), F(2), F(3), F(4), F(5) and F(6). The different interpolation filter type options may be signaled using corresponding filter indexes. The different interpolation filter type options may include one or more options, each of which indicates that the same interpolation filter type is applied to the entire block. The different interpolation filter type options may include one or more options, each of which indicates that different interpolation filter types are applied to different regions / regions / portions of the block. The interpolation filter type options shown signal how the block is partitioned into one or more regions, regions or portions of the block, and which interpolation filter type is applied to each of the one or more regions, regions or portions of the block, rather than separately signaling the partition type and one or more interpolation filter types to be applied to one or more regions, regions or portions of the block. This way of signaling both the partition shape and the corresponding interpolation filter type together may result in fewer options and variability, but this way of signaling requires fewer bits than separately signaling the partition shape and the filter index.

[0099] Filter index 0F(0) may indicate a normal interpolation filter type to be applied to the entire block. Filter index 1F(1) may indicate a smoothing interpolation filter type to be applied to the entire block. Filter index 2F(2) may indicate a sharpening interpolation filter type to be applied to the entire block. Filter index 3F(3) may indicate a normal interpolation filter type to be applied to the upper half of the block and a smoothing interpolation filter type to be applied to the lower half of the block. Filter index 4F(4) may indicate a smoothing interpolation filter type to be applied to the upper half of the block and a normal interpolation filter type to be applied to the lower half of the block. Filter index 5F(5) may indicate a normal interpolation filter type to be applied to the left half of the block and a smoothing interpolation filter type to be applied to the right half of the block. Filter index 6F(6) may indicate a smoothing interpolation filter type to be applied to the left half of the block and a normal interpolation filter type to be applied to the right half of the block.

[0100] At the block level, for each block in a frame, picture, or slice that is allowed to use this adaptive sub-block interpolation filtering method, one or more signals may be added to the block header (e.g., filter_type). The filter_type signal may specify the interpolation filter type option to be used for the block, and may take a value from a set of interpolation filter type options from which the encoder can select. The interpolation filter type options may involve applying different interpolation filter types to different regions, areas, or portions of the block. Exemplary interpolation filter type options are shown in Figure 8-9 Shown in.

[0101] At the decoder, the following operations can be performed: 1. For each block in a frame / picture / slice, the decoder can read the block header signal (e.g., filter_type) and can determine the interpolation filter type option to be used for the block. The interpolation filter type option may involve applying different interpolation filter types to different regions, areas, or portions of the block. 2. For each block in a frame / picture / slice, the decoder may apply an interpolation filter type option indicated by, for example, a filter_type signal. If the interpolation filter type indicates sub-block filtering, the decoder may perform sub-block filtering on the luma and / or chroma samples of the block during a motion compensation process or a block prediction process.

[0102] In some embodiments, interpolation filter type signaling may be implemented to further reduce the number of bits that may be required to indicate appropriate adaptive sub-block interpolation filtering. In some embodiments, the interpolation filter information may include a signal indicating that the interpolation filter information is the same as another (or previous / adjacent reconstructed block). In some embodiments, the interpolation filter information may include a filter index residual signal indicating a residual filter index value relative to the filter index of another reconstructed block. Exemplary encoder implementation of adaptive sub-block interpolation filtering in an encoder

[0103] Fig.10 An example of a partition shape and filter type selection process 1000 is shown according to some embodiments of the present disclosure. The process 1000 may be implemented in an encoder.

[0104] In a block prediction operation, for each interpolation filter type, the encoder may generate a motion compensated block (or predicted block) using a selected reference block and a selected predictor. The encoder may generate a predicted block using motion compensation or block prediction. The encoder may determine a predicted block based on the reference block and the predictor, wherein the predicted block has at least a first region and a second region. The encoder may determine a predictor for the reference block and the initial or source block. The predicted block may have more regions. As depicted, for the purpose of evaluating the filter cost, the predicted block has four quarter regions.

[0105] In the filtering operation, the encoder may apply a specific interpolation filter type to the predicted block to produce a filtered predicted block. Examples of interpolation filter types may include low-pass filters, high-pass filters, or some other type of interpolation filter type. Different interpolation filter types (e.g., F(0), F(1), F(2), ..., and F(n)) may be applied to produce different filtered predicted blocks, for example, B(0), B(1), B(2), ..., and B(n). The encoder may apply a first interpolation filter type to a first region of a predicted block to obtain a first region of a first filtered predicted block (e.g., the first region of B(0)). The encoder may apply a first interpolation filter type to a second region of a predicted block to obtain a second region of a first filtered predicted block (e.g., the second region of B(0)). The encoder may apply a second interpolation filter type to a first region of a predicted block to obtain a first region of a second filtered predicted block (e.g., the first region of B(1)). The encoder may apply a second interpolation filter type to a second region of the predicted block to obtain a second filtered second region of the predicted block (eg, a second region of B(1)).

[0106] In determining filter costs 1002, for each sub-block (e.g., a region, area, or portion of a block), the encoder may perform one or more of the following: distortion calculation, filter rate estimation, filter cost calculation. For each sub-block, the encoder may utilize the determined filter costs in selecting optimal filter type 1004 to select the best interpolation filter type option (including sub-block partition shapes and different interpolation filter types being applied to the sub-blocks).

[0107] The encoder may determine a filter cost based on the initial block, the first region of the first filtered predicted block, the second region of the first filtered predicted block, the first region of the second filtered predicted block, and the second region of the second filtered predicted block. Based on the filter cost, the encoder may determine interpolation filter information for encoding a predicted block (e.g., a block that the predicted block is intended to predict).

[0108] The distortion calculation may include calculating the difference between an initial or source video block (e.g., depicted as S) and each filtered reference block (e.g., B(0), B(1), B(2), ..., or B(n)) generated in the first operation. This difference may be referred to as distortion or distortion cost. For a filtered predicted block (e.g., B(0)), a sub-block based distortion may be obtained by subtracting the filtered predicted block (e.g., B(0)) from the initial video block S, thereby obtaining different distortion costs for the sub-blocks (e.g., D0(0), D0(1), D0(2), and D0(3)). The distortion cost of the sub-block is related to the distortion caused by the specific interpolation filter type applied to obtain the filtered predicted block (e.g., B(0)). For a filtered predicted block (e.g., B(1)), a sub-block based distortion may be obtained by subtracting the filtered predicted block (e.g., B(1)) from the initial video block S, thereby obtaining different distortion costs for the sub-blocks (e.g., D1(0), D1(1), D1(2), and D1(3)). The distortion cost of the sub-block is related to the distortion caused by the specific interpolation filter type applied to obtain the filtered predicted block (e.g., B(1)). For a filtered predicted block (e.g., B(n)), a sub-block based distortion may be obtained by subtracting the filtered predicted block (e.g., B(n)) from the initial video block S, thereby obtaining different distortion costs for the sub-blocks (e.g., Dn(0), Dn(1), Dn(2), and Dn(3)). The distortion cost of the sub-block is related to the distortion caused by the specific interpolation filter type applied to obtain the filtered predicted block (e.g., B(n)). The total distortion cost for a particular interpolation filter type option may be calculated based on a sum or combination of corresponding sub-block based distortion costs. The corresponding sub-block based distortion cost for a particular interpolation filter type option will include a collection of sub-block based distortion costs associated with applying a particular interpolation filter type to a particular sub-block according to the particular interpolation filter type option. In some cases, a particular interpolation filter type option may apply different interpolation filter types to different sub-blocks, or in other words, apply adaptive sub-block interpolation filtering.

[0109] The encoder may determine a first distortion cost, e.g., D0(0), based on a first region of the first filtered predicted block and the initial block. The encoder may determine a second distortion cost, e.g., D0(1), based on a second region of the first filtered predicted block and the initial block. The encoder may determine a third distortion cost, e.g., D1(0), based on a first region of the second filtered predicted block and the initial block. The encoder may determine a fourth distortion cost, e.g., D1(1), based on a second region of the second filtered predicted block and the initial block. Fig.10 As shown, the additional distortion cost can be determined.

[0110] Filter rate estimation may include estimating the number of bits that may be used to signal the type of interpolation filter being used in the bitstream. Filter rate estimation may include determining a signaling cost for signaling different interpolation filter types. Estimating the number of bits to be used may allow a video encoder to balance video quality and the number of bits used to encode the video. Determination and minimization of signaling cost may be performed at a frame level, e.g., determining the signaling cost for an entire frame. Determination and minimization of signaling cost may be performed at a sequence level, e.g., determining the signaling cost for an entire sequence of frames.

[0111] The encoder may determine a first signaling cost for a first option of interpolation filter information, the first option indicating different interpolation filter types for the predicted block. The encoder may determine a second signaling cost for a second option of interpolation filter information, the second option indicating a single interpolation filter type for the predicted block.

[0112] Determining filter costs 1002 may include calculating the cost of each filter by considering the distortion and filter rate estimated in the distortion calculation and the filter rate estimation, respectively. The cost of each filter may include a weighted sum of the distortion cost and the signaling cost. The encoder may determine a first filter cost in the filter costs based on a first signaling cost and a first distortion cost of a first option of the interpolation filter information. The encoder may determine a second filter cost in the filter costs based on a second signaling cost and a second distortion cost of a second option of the interpolation filter information.

[0113] Selecting the best filter type 1004 may include selecting the best filter type for different sub-blocks of the block based on the filter cost calculation. Selecting the best filter type 1004 may include selecting the interpolation filter type option for the block based on the filter cost calculation. Selecting the best filter type 1004 may include selecting the partition shape option and one or more interpolation filter types based on the filter cost calculation to apply to the sub-blocks of the block. The encoder may reconstruct the cost of each partition shape and select the best partition shape that provides the lowest cost. The encoder may determine the optimal filter cost in the filter cost. The encoder may determine the interpolation filter information corresponding to the optimal filter cost. The encoder may signal the selected interpolation filter type option in the bitstream so that the decoder can use them in the motion compensation process or the block prediction process during video playback. The encoder may signal the selected block partition shape and (one or more) filter types in the bitstream so that the decoder can use them in the motion compensation process or the block prediction process during video playback. Adaptive sub-block based interpolation filtering method

[0114] Fig.11A flowchart of an exemplary method 1100 for decoding an encoded bitstream according to some embodiments of the present disclosure is depicted. The method 1100 may be implemented in a decoder described and illustrated herein. The method 1100 may be illustrated Figure 5 An example of process 500 in FIG. 1100 may be performed by Fig.13 The computing device 1300 is used to execute.

[0115] In 1102, a decoder may receive an encoded bitstream. The encoded bitstream may include an encoded reference block, interpolation filter information, and residual data of the encoded block. The interpolation filter information may indicate a first interpolation filter type for a first region of the predicted block and a second interpolation filter type for a second region of the predicted block that is different from the first interpolation filter type.

[0116] At 1104 , the decoder may decode the encoded reference block, eg, to obtain a decoded reference block.

[0117] At 1106 , the decoder may generate a predicted block based on the decoded reference block.

[0118] In 1108 , the decoder may apply a first interpolation filter type to the first region of the predicted block to obtain a filtered first region of the block.

[0119] In 1110 , the decoder may apply a second interpolation filter type to a second region of the predicted block to obtain a filtered second region of the block.

[0120] In 1112, the decoder may output a reconstructed block of a reconstructed frame based on the filtered block and the residual data.

[0121] In some embodiments, the encoded bitstream further includes predictor information indicating the predictor.The decoder may generate a predicted block by applying the predictor to the decoded reference block.

[0122] In some embodiments, the method 1100 may be used for intra prediction.The decoded reference block and the reconstructed block are both in a reconstructed frame.

[0123] In some embodiments, method 1100 may be used for inter-frame prediction. The decoded reference block may be part of a reference frame. The reference frame and the reconstructed frame may have different frame indices. The reference frame and the reconstructed frame are different frames of the video.

[0124] Fig.12 A flowchart of an exemplary method 1200 for encoding a video according to some embodiments of the present disclosure is depicted. The method 1200 may be implemented in an encoder as described and illustrated herein. The method 1200 may be illustrated Figure 4The method 1200 may be performed by Fig.13 The computing device 1300 is used to execute.

[0125] In 1202, the encoder may determine a predicted block based on a reference block and a predictor. The predicted block has at least a first region and a second region.

[0126] In 1204 , the encoder may apply a first interpolation filter type to a first region of a predicted block to obtain a first filtered first region of the predicted block.

[0127] In 1206 , the encoder may apply the first interpolation filter type to the second region of the predicted block to obtain a first filtered second region of the predicted block.

[0128] In 1208 , the encoder may apply a second interpolation filter type to the first region of the predicted block to obtain a second filtered predicted first region of the block.

[0129] In 1210 , the encoder may apply a second interpolation filter type to a second region of the predicted block to obtain a second filtered second region of the predicted block.

[0130] In 1212, the encoder may determine a filter cost based on the initial block, the first region of the first filtered predicted block, the second region of the first filtered predicted block, the first region of the second filtered predicted block, and the second region of the second filtered predicted block.

[0131] In 1214, the encoder may determine interpolation filter information for encoding the predicted block based on the filter cost. The encoder may write the interpolation filter information into the encoded bitstream. Exemplary Computing Devices

[0132] Fig.13 1300 according to some embodiments of the present disclosure. One or more computing devices 1300 may be used to implement the functionality described in conjunction with the figures and herein. The multiple components shown in the figures may be included in the computing device 1300, but any one or more of these components may be omitted or duplicated as appropriate for the application. In some embodiments, some or all of the components included in the computing device 1300 may be attached to one or more motherboards. In some embodiments, some or all of these components are manufactured onto a single system-on-chip (SoC) die. Additionally, in various embodiments, the computing device 1300 may not include Fig.13The computing device 1300 may include one or more of the components shown, and the computing device 1300 may include an interface circuit unit for coupling to one or more of the components. For example, the computing device 1300 may not include the display device 1306, and may include a display device interface circuit unit (e.g., a connector and a driver circuit unit) to which the display device 1306 may be coupled. In another set of examples, the computing device 1300 may not include the audio input device 1318 or the audio output device 1308, and may include an audio input or output device interface circuit unit (e.g., a connector and a supporting circuit unit) to which the audio input device 1318 or the audio output device 1308 may be coupled.

[0133] The computing device 1300 may include a processing device 1302 (e.g., one or more processing devices, one or more of the same type of processing devices, one or more of different types of processing devices). The processing device 1302 may include an electronic circuit unit that processes electronic data from a data storage element (e.g., a register, a memory, a resistor, a capacitor, a qubit unit) to transform the electronic data into other electronic data that can be stored in the register and / or the memory. Examples of the processing device 1302 may include a central processing unit (CPU), a graphics processing unit (GPU), a quantum processor, a machine learning processor, an artificial intelligence processor, a neural network processor, an artificial intelligence accelerator, an application specific integrated circuit (ASIC), an analog signal processor, an analog computer, a microprocessor, a digital signal processor, a field programmable gate array (FPGA), a tensor processing unit (TPU), a data processing unit (DPU), etc.

[0134] The computing device 1300 may include a memory 1304, which may itself include one or more storage devices, such as volatile memory (e.g., DRAM), non-volatile memory (e.g., read-only memory (ROM)), high bandwidth memory (HBM), flash memory, solid-state memory, and / or a hard drive. The memory 1304 includes one or more non-transitory computer-readable storage media. In some embodiments, the memory 1304 may include a memory that shares a die with the processing device 1302. In some embodiments, the memory 1304 includes one or more non-transitory computer-readable media that stores instructions executable to perform the operations described herein, such as Figure 1-12The operations shown, process 400, process 500, process 1000, method 1100, and method 1200. Memory 1304 may include one or more non-transitory computer-readable media storing instructions executable to perform operations associated with adaptive sub-block interpolation filtering. Memory 1304 may include one or more non-transitory computer-readable media storing instructions executable to perform operations associated with determining interpolation filter information for a block. Memory 1304 may include one or more non-transitory computer-readable media storing instructions executable to perform operations of applying interpolation filter information to a block. Memory 1304 may include one or more non-transitory computer-readable media storing one or more of the following: input frames of an encoder, intermediate data structures calculated by an encoder, a bitstream generated by an encoder, a bitstream received by a decoder, an intermediate data structure calculated by a decoder, and a reconstructed frame generated by a decoder. Memory 1304 may include one or more non-transitory computer-readable media storing one or more of the following: Figure 4 The memory 1304 may include one or more non-transitory computer-readable media that stores one or more of the following: Figure 5 The instructions stored in one or more non-transitory computer-readable media may be executed by the processing device 1302. In some embodiments, the memory 1304 may store data, such as data structures, binary data, bits, metadata, files, blobs, etc., as described in conjunction with the figures and herein. Exemplary data that may be stored in the memory 1304 are depicted. As depicted, the memory 1304 may store one or more data.

[0135] In some embodiments, computing device 1300 may include communication device 1312 (e.g., one or more communication devices). For example, communication device 1312 may be configured to manage wired and / or wireless communications to transfer data to and from computing device 1300. The term "wireless" and its derivatives may be used to describe circuits, devices, systems, methods, techniques, communication channels, etc. that can transfer data through a non-solid medium by using modulated electromagnetic radiation. The term does not imply that the associated devices do not include any wires, although in some embodiments they may not include any wires. Communication device 1312 may implement any of a variety of wireless standards or protocols, including but not limited to IEEE standards including Wi-Fi (Institute of Electrical and Electronics Engineers (IEEE) 802.10 series), IEEE 802.16 standards (e.g., IEEE 802.16-2005 amendment), Long Term Evolution (LTE) project together with any amendments, updates and / or revisions (e.g., Advanced LTE project, Ultra Mobile Broadband (UMB) project (also known as "3GPP2"), etc.). IEEE 802.16 compatible broadband wireless access (BWA) networks are often referred to as WiMAX networks, an acronym that stands for Worldwide Interoperability for Microwave Access, which is a certification mark for products that have passed the IEEE 802.16 standard conformance and interoperability testing. The communication device 1312 may operate according to a Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Universal Mobile Telecommunications System (UMTS), High Speed ​​Packet Access (HSPA), Evolved HSPA (E-HSPA), or LTE network. The communication device 1312 may operate according to Enhanced Data GSM Evolution (EDGE), GSM Evolution Radio Access Network (GERAN), Universal Terrestrial Radio Access Network (UTRAN), or Evolved UTRAN (E-UTRAN). The communication device 1312 may operate according to Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Digital Enhanced Cordless Telecommunications (DECT), Evolution Data Optimized (EV-DO), and derivative protocols thereof, as well as any other wireless protocols designated as 4G, 4G, 5G, and above. In other embodiments, the communication device 1312 may operate according to other wireless protocols. The computing device 1300 may include an antenna 1322 to facilitate wireless communications and / or receive other wireless communications (such as radio frequency transmissions). The computing device 1300 may include receiver circuitry and / or transmitter circuitry. In some embodiments, the communication device 1312 may manage wired communications, such as electrical, optical, or any other suitable communication protocol (e.g., Ethernet). As described above, the communication device 1312 may include multiple communication chips.For example, the first communication device 1312 may be dedicated to shorter range wireless communications such as Wi-Fi or Bluetooth, while the second communication device 1312 may be dedicated to longer range wireless communications such as Global Positioning System (GPS), EDGE, GPRS, CDMA, WiMAX, LTE, EV-DO, etc. In some embodiments, the first communication device 1312 may be dedicated to wireless communications, while the second communication device 1312 may be dedicated to wired communications.

[0136] Computing device 1300 may include a power supply / power circuit unit 1314. Power supply / power circuit unit 1314 may include one or more energy storage devices (e.g., batteries or capacitors) and / or circuit units for coupling components of computing device 1300 to an energy source (e.g., DC power, AC power, etc.) separate from computing device 1300.

[0137] Computing device 1300 may include a display device 1306 (or corresponding interface circuitry as discussed above). For example, display device 1306 may include any visual indicator, such as a heads-up display, a computer monitor, a projector, a touch screen display, a liquid crystal display (LCD), a light emitting diode display, or a flat panel display.

[0138] Computing device 1300 may include audio output device 1308 (or corresponding interface circuitry as discussed above). For example, audio output device 1308 may include any device that generates an audible indicator, such as a speaker, headphones, or earbuds.

[0139] Computing device 1300 may include audio input device 1318 (or corresponding interface circuitry as discussed above). Audio input device 1318 may include any device that generates a signal representing sound, such as a microphone, a microphone array, or a digital musical instrument (e.g., a musical instrument with a Musical Instrument Digital Interface (MIDI) output).

[0140] Computing device 1300 may include GPS device 1316 (or corresponding interface circuitry as discussed above). GPS device 1316 may communicate with a satellite-based system and may receive the location of computing device 1300, as is known in the art.

[0141] The computing device 1300 may include a sensor 1330 (or one or more sensors). The computing device 1300 may include corresponding interface circuit units as discussed above. The sensor 1330 may sense a physical phenomenon and convert the physical phenomenon into an electrical signal that can be processed by, for example, the processing device 1302. Examples of the sensor 1330 may include: a capacitive sensor, an inductive sensor, a resistive sensor, an electromagnetic field sensor, a light sensor, a camera, an imager, a microphone, a pressure sensor, a temperature sensor, a vibration sensor, an accelerometer, a gyroscope, a strain sensor, a moisture sensor, a humidity sensor, a distance sensor, a range sensor, a time of flight sensor, a pH sensor, a particle sensor, an air quality sensor, a chemical sensor, a gas sensor, a biosensor, an ultrasonic sensor, a scanner, and the like.

[0142] Computing device 1300 may include other output devices 1310 (or corresponding interface circuitry as discussed above). Examples of other output devices 1310 may include an audio codec, a video codec, a printer, a wired or wireless transmitter for providing information to other devices, a tactile output device, a gas output device, a vibration output device, a lighting output device, a home automation controller, or an additional storage device.

[0143] Computing device 1300 may include other input devices 1320 (or corresponding interface circuitry as discussed above). Examples of other input devices 1320 may include an accelerometer, a gyroscope, a compass, an image capture device, a keyboard, a cursor control device such as a mouse, a stylus, a touchpad, a barcode reader, a quick response (QR) code reader, any sensor, or a radio frequency identification (RFID) reader.

[0144] The computing device 1300 may have any desired form factor, such as a handheld or mobile computer system (e.g., a cellular phone, a smart phone, a mobile Internet device, a music player, a tablet computer, a laptop computer, a netbook computer, a personal digital assistant (PDA), an ultra-mobile personal computer, a remote control, a wearable device, a helmet, glasses, footwear, electronic clothing, etc.), a desktop computer system, a server or other networked computing component, a printer, a scanner, a monitor, a set-top box, an entertainment control unit, a vehicle control unit, a digital camera, a digital video recorder, an Internet of Things device, or a wearable computer system. In some embodiments, the computing device 1300 may be any other electronic device that processes data. Select Example

[0145] Example 1 provides a method, which includes: receiving an encoded bitstream, wherein: the encoded bitstream includes an encoded reference block, interpolation filter information, and residual data of the encoded block; and the interpolation filter information indicates a first interpolation filter type for a first area of ​​a predicted block and a second interpolation filter type for a second area of ​​the predicted block, which is different from the first interpolation filter type; decoding the encoded reference block; generating the predicted block based on the decoded reference block; applying the first interpolation filter type to the first area of ​​the predicted block to obtain a first area of ​​the filtered block; applying the second interpolation filter type to the second area of ​​the predicted block to obtain a second area of ​​the filtered block; and outputting a reconstructed block of a reconstructed frame based on the filtered block and the residual data.

[0146] Example 2 provides the method of Example 1, wherein: the encoded bitstream further includes predictor information indicating a predictor; and generating the predicted block includes applying the predictor to the decoded reference block.

[0147] Example 3 provides the method of Example 1 or 2, wherein: the decoded reference block and the reconstructed block are both in the reconstructed frame.

[0148] Example 4 provides the method of Example 1 or 2, wherein: the decoded reference block is part of a reference frame; and the reference frame and the reconstructed frame have different frame indices.

[0149] Example 5 provides the method of any one of Examples 1-4, wherein the interpolation filter information includes a filter index.

[0150] Example 6 provides a method of any one of Examples 1-5, wherein the interpolation filter information includes: a partition shape index indicating a manner for dividing the predicted block into at least the first region and the second region; and one or more filter indices indicating the first interpolation filter type and the second interpolation filter type.

[0151] Example 7 provides the method of any one of Examples 1-6, wherein the interpolation filter information includes a signal indicating that the interpolation filter information is the same as another reconstructed block.

[0152] Example 8 provides the method of any one of Examples 1-6, wherein the interpolation filter information comprises a filter index residual signal for indicating a residual filter index value relative to a filter index of a further reconstructed block.

[0153] Example 9 provides the method of any of Examples 1-8, wherein the interpolation filter information includes a sequence header signal for indicating whether multiple interpolation filter types are allowed for a given block of a frame sequence, wherein the frame sequence includes the reconstructed frame and additional frames.

[0154] Example 10 provides the method of any of Examples 1-9, wherein the interpolation filter information comprises a frame header signal indicating whether multiple interpolation filter types are allowed for a given block of the reconstructed frame.

[0155] Example 11 provides the method of any of Examples 1-10, wherein the interpolation filter information includes a block size signal for indicating one or more block sizes for which multiple interpolation filter types are allowed and one or more additional block sizes for which multiple interpolation filter types are not allowed.

[0156] Example 12 provides the method of any one of Examples 1-11, wherein: the first region includes an upper half of the decoded reference block; and the second region includes a lower half of the decoded reference block.

[0157] Example 13 provides the method of any one of Examples 1-11, wherein: the first region includes a left half of the decoded reference block; and the second region includes a right half of the decoded reference block.

[0158] Example 14 provides the method of any one of Examples 1-11, wherein: the first region includes a first third of the decoded reference block; and the second region includes a second third of the decoded reference block.

[0159] Example 15 provides the method of any one of Examples 1-11, wherein: the first region includes a first quarter of the decoded reference block; and the second region includes a second quarter of the decoded reference block.

[0160] Example 16 provides a method, comprising determining a predicted block of an initial block based on a reference block and a predictor, wherein the predicted block has at least a first region and a second region; applying a first interpolation filter type to the first region of the predicted block to obtain the first region of a first filtered predicted block; applying the first interpolation filter type to the second region of the predicted block to obtain the second region of the first filtered predicted block; applying a second interpolation filter type to the first region of the predicted block to obtain the first region of a second filtered predicted block; applying the second interpolation filter type to the second region of the predicted block to obtain the second region of the second filtered predicted block; determining a filter cost based on the initial block, the first region of the first filtered predicted block, the second region of the first filtered predicted block, the first region of the second filtered predicted block, and the second region of the second filtered predicted block; and determining interpolation filter information for encoding the predicted block based on the filter cost.

[0161] Example 17 provides the method of Example 16, wherein determining the filtering cost includes: determining a first distortion cost based on the first area of ​​the first filtered predicted block and the initial block; determining a second distortion cost based on the second area of ​​the first filtered predicted block and the initial block; determining a third distortion cost based on the first area of ​​the second filtered predicted block and the initial block; and determining a fourth distortion cost based on the second area of ​​the second filtered predicted block and the initial block.

[0162] Example 18 provides the method of Example 16 or 17, wherein determining the filter cost includes: determining a first signaling cost of a first option of the interpolation filter information, the first option indicating different interpolation filter types for the predicted block; and determining a second signaling cost of a second option of the interpolation filter information, the second option indicating a single interpolation filter type for the predicted block.

[0163] Example 19 provides a method of any one of Examples 16-18, wherein determining the filter cost includes: determining a first filter cost in the filter cost based on a first signaling cost and a first distortion cost of a first option of the interpolation filter information; and determining a second filter cost in the filter cost based on a second signaling cost and a second distortion cost of a second option of the interpolation filter information.

[0164] Example 20 provides the method of any one of Examples 16-19, wherein determining the interpolation filter information comprises: determining an optimal filter cost among the filter costs; and determining the interpolation filter information corresponding to the optimal filter cost.

[0165] Example 21 provides a device comprising: one or more processors for executing instructions; and one or more non-transitory computer-readable media storing instructions, which, when the instructions are executed by the one or more processors, cause the one or more processors to: receive an encoded bitstream, wherein: the encoded bitstream includes an encoded reference block, interpolation filter information, and residual data of the encoded block; and the interpolation filter information indicates a first interpolation filter type for a first region of a predicted block and a second interpolation filter type for a second region of the predicted block, which is different from the first interpolation filter type; decode the encoded reference block; generate the predicted block based on the decoded reference block; apply the first interpolation filter type to the first region of the predicted block to obtain a first region of the filtered block; apply the second interpolation filter type to the second region of the predicted block to obtain a second region of the filtered block; and output a reconstructed block of a reconstructed frame based on the filtered block and the residual data.

[0166] Example 22 provides the apparatus of Example 21, wherein: the encoded bitstream further comprises predictor information indicating a predictor; and generating the predicted block comprises applying the predictor to the decoded reference block.

[0167] Example 23 provides the apparatus of Example 21 or 22, wherein: the decoded reference block and the reconstructed block are both in the reconstructed frame.

[0168] Example 24 provides the apparatus of Example 21 or 22, wherein: the decoded reference block is part of a reference frame; and the reference frame and the reconstructed frame have different frame indices.

[0169] Example 25 provides the apparatus of any one of Examples 21-24, wherein the interpolation filter information includes a filter index.

[0170] Example 26 provides an apparatus of any of Examples 21-25, wherein the interpolation filter information includes a partition shape index indicating a manner for dividing the predicted block into at least the first region and the second region; and one or more filter indices indicating the first interpolation filter type and the second interpolation filter type.

[0171] Example 27 provides the apparatus of any one of Examples 21-26, wherein the interpolation filter information comprises a signal for indicating that the interpolation filter information is the same as another reconstructed block.

[0172] Example 28 provides the apparatus of any of Examples 21-26, wherein the interpolation filter information comprises a filter index residual signal for indicating a residual filter index value relative to a filter index of another reconstructed block.

[0173] Example 29 provides an apparatus of any of Examples 21-28, wherein the interpolation filter information includes a sequence header signal for indicating whether multiple interpolation filter types are allowed for a given block of a frame sequence, wherein the frame sequence includes the reconstructed frame and additional frames.

[0174] Example 30 provides the apparatus of any one of Examples 21 to 29, wherein the interpolation filter information comprises a frame header signal for indicating whether multiple interpolation filter types are allowed for a given block of the reconstructed frame.

[0175] Example 31 provides an apparatus of any one of Examples 21 to 30, wherein the interpolation filter information includes a block size signal for indicating one or more block sizes for which multiple interpolation filter types are allowed and one or more additional block sizes for which multiple interpolation filter types are not allowed.

[0176] Example 32 provides an apparatus of any one of Examples 21-31, wherein: the first region includes an upper half of the decoded reference block; and the second region includes a lower half of the decoded reference block.

[0177] Example 33 provides an apparatus of any one of Examples 21-31, wherein: the first region includes a left half of the decoded reference block; and the second region includes a right half of the decoded reference block.

[0178] Example 34 provides an apparatus of any one of Examples 21-31, wherein: the first region includes a first third of the decoded reference block; and the second region includes a second third of the decoded reference block.

[0179] Example 35 provides an apparatus of any one of Examples 21-31, wherein: the first region includes a first quarter of the decoded reference block; and the second region includes a second quarter of the decoded reference block.

[0180] Example 36 provides an apparatus comprising: one or more processors for executing instructions; and one or more non-transitory computer-readable media storing instructions, wherein the instructions, when executed by the one or more processors, cause the one or more processors to: determine a predicted block of an initial block based on a reference block and a predictor, wherein the predicted block has at least a first region and a second region; apply a first interpolation filter type to the first region of the predicted block to obtain a first filtered predicted block of the first region; apply the first interpolation filter type to the second region of the predicted block to obtain a second filtered predicted block of the first region; Applying a second interpolation filter type to the first region of the predicted block to obtain a first region of a second filtered predicted block; applying the second interpolation filter type to the second region of the predicted block to obtain a second region of the second filtered predicted block; determining a filter cost based on the initial block, the first region of the first filtered predicted block, the second region of the first filtered predicted block, the first region of the second filtered predicted block, and the second region of the second filtered predicted block; and determining interpolation filter information for encoding the predicted block based on the filter cost.

[0181] Example 37 provides the apparatus of Example 36, wherein determining the filtering cost comprises: determining a first distortion cost based on the first region of the first filtered predicted block and the initial block; determining a second distortion cost based on the second region of the first filtered predicted block and the initial block; determining a third distortion cost based on the first region of the second filtered predicted block and the initial block; and determining a fourth distortion cost based on the second region of the second filtered predicted block and the initial block.

[0182] Example 38 provides an apparatus of Example 36 or 37, wherein determining the filter cost comprises: determining a first signaling cost for a first option of the interpolation filter information, the first option indicating different interpolation filter types for the predicted block; and determining a second signaling cost for a second option of the interpolation filter information, the second option indicating a single interpolation filter type for the predicted block.

[0183] Example 39 provides an apparatus of any one of Examples 36-38, wherein determining the filter cost includes: determining a first filter cost in the filter cost based on a first signaling cost and a first distortion cost of a first option of the interpolation filter information; and determining a second filter cost in the filter cost based on a second signaling cost and a second distortion cost of a second option of the interpolation filter information.

[0184] Example 40 provides the apparatus of any one of Examples 36-39, wherein determining the interpolation filter information comprises: determining an optimal filter cost among the filter costs; and determining the interpolation filter information corresponding to the optimal filter cost.

[0185] Example 41 provides one or more non-transitory computer-readable media storing instructions that, when executed by one or more processors, cause the one or more processors to: receive an encoded bitstream, wherein: the encoded bitstream includes an encoded reference block, interpolation filter information, and residual data of the encoded block; and the interpolation filter information indicates a first interpolation filter type for a first region of a predicted block and a second interpolation filter type for a second region of the predicted block that is different from the first interpolation filter type; decode the encoded reference block; generate the predicted block based on the decoded reference block; apply the first interpolation filter type to the first region of the predicted block to obtain a first region of the filtered block; apply the second interpolation filter type to the second region of the predicted block to obtain a second region of the filtered block; and output a reconstructed block of a reconstructed frame based on the filtered block and the residual data.

[0186] Example 42 provides one or more non-transitory computer-readable media of Example 41, wherein: the encoded bitstream also includes predictor information indicating a predictor; and generating the predicted block includes applying the predictor to the decoded reference block.

[0187] Example 43 provides the one or more non-transitory computer-readable media of Example 41 or 42, wherein: the decoded reference block and the reconstructed block are both in the reconstructed frame.

[0188] Example 44 provides one or more non-transitory computer-readable media of Example 41 or 42, wherein: the decoded reference block is part of a reference frame; and the reference frame and the reconstructed frame have different frame indexes.

[0189] Example 45 provides one or more non-transitory computer-readable media of any of Examples 41-44, wherein the interpolation filter information comprises a filter index.

[0190] Example 46 provides one or more non-temporary computer-readable media of any of Examples 41-45, wherein the interpolation filter information includes: a partition shape index indicating a manner for dividing the predicted block into at least the first region and the second region; and one or more filter indices indicating the first interpolation filter type and the second interpolation filter type.

[0191] Example 47 provides one or more non-transitory computer-readable media of any of Examples 41-46, wherein the interpolation filter information includes a signal for indicating that the interpolation filter information is the same as another reconstructed block.

[0192] Example 48 provides one or more non-transitory computer-readable media of any of Examples 41-46, wherein the interpolation filter information includes a filter index residual signal for indicating a residual filter index value relative to a filter index of another reconstructed block.

[0193] Example 49 provides one or more non-transitory computer-readable media of any of Examples 41-48, wherein the interpolation filter information includes a sequence header signal for indicating whether multiple interpolation filter types are allowed for a given block of a frame sequence, wherein the frame sequence includes the reconstructed frame and additional frames.

[0194] Example 50 provides one or more non-transitory computer-readable media of any of Examples 41-49, wherein the interpolation filter information includes a frame header signal for indicating whether multiple interpolation filter types are allowed for a given block of the reconstructed frame.

[0195] Example 51 provides one or more non-transitory computer-readable media of any of Examples 41-50, wherein the interpolation filter information includes a block size signal for indicating one or more block sizes for which multiple interpolation filter types are allowed and one or more additional block sizes for which multiple interpolation filter types are not allowed.

[0196] Example 52 provides one or more non-transitory computer-readable media of any of Examples 41-51, wherein: the first region includes an upper half of the decoded reference block; and the second region includes a lower half of the decoded reference block.

[0197] Example 53 provides one or more non-transitory computer-readable media of any of Examples 41-51, wherein: the first region includes a left half of the decoded reference block; and the second region includes a right half of the decoded reference block.

[0198] Example 54 provides one or more non-transitory computer-readable media of any of Examples 41-51, wherein: the first region includes a first third of the decoded reference block; and the second region includes a second third of the decoded reference block.

[0199] Example 55 provides one or more non-transitory computer-readable media of any of Examples 41-51, wherein: the first region includes a first quarter of the decoded reference block; and the second region includes a second quarter of the decoded reference block.

[0200] Example 56 provides one or more non-transitory computer-readable media storing instructions that, when executed by one or more processors, cause the one or more processors to: determine a predicted block of an initial block based on a reference block and a predictor, wherein the predicted block has at least a first region and a second region; apply a first interpolation filter type to the first region of the predicted block to obtain a first filtered predicted block; apply the first interpolation filter type to the second region of the predicted block to obtain a second filtered predicted block; apply a second interpolation filter type to the first region of the predicted block The invention relates to a method for encoding a first area of ​​the first filtered predicted block to obtain a first area of ​​a second filtered predicted block; applying the second interpolation filter type to the second area of ​​the predicted block to obtain a second area of ​​the second filtered predicted block; determining a filter cost based on the initial block, the first area of ​​the first filtered predicted block, the second area of ​​the first filtered predicted block, the first area of ​​the second filtered predicted block, and the second area of ​​the second filtered predicted block; and determining interpolation filter information for encoding the predicted block based on the filter cost.

[0201] Example 57 provides one or more non-transitory computer-readable media of Example 56, wherein determining the filtering cost includes: determining a first distortion cost based on the first region of the first filtered predicted block and the initial block; determining a second distortion cost based on the second region of the first filtered predicted block and the initial block; determining a third distortion cost based on the first region of the second filtered predicted block and the initial block; and determining a fourth distortion cost based on the second region of the second filtered predicted block and the initial block.

[0202] Example 58 provides one or more non-transitory computer-readable media of Example 56 or 57, wherein determining the filter cost includes: determining a first signaling cost for a first option of the interpolation filter information, the first option indicating different interpolation filter types for the predicted block; and determining a second signaling cost for a second option of the interpolation filter information, the second option indicating a single interpolation filter type for the predicted block.

[0203] Example 59 provides one or more non-transitory computer-readable media of any one of Examples 56-58, wherein determining the filter cost includes: determining a first filter cost in the filter cost based on a first signaling cost and a first distortion cost of a first option of the interpolation filter information; and determining a second filter cost in the filter cost based on a second signaling cost and a second distortion cost of a second option of the interpolation filter information.

[0204] Example 60 provides one or more non-transitory computer-readable media of any of Examples 56-59, wherein determining the interpolation filter information comprises: determining an optimal filter cost among the filter costs; and determining the interpolation filter information corresponding to the optimal filter cost.

[0205] Example A provides an apparatus including components for performing any of the methods provided in Examples 1-20 or components for performing any of the methods provided in Examples 1-20.

[0206] Example B provides one or more non-transitory computer-readable media storing instructions that, when executed by one or more processors, cause the one or more processors to perform any of the methods provided in Examples 1-20 and the methods described herein.

[0207] Example C provides a device comprising: one or more processors for executing instructions, and one or more non-temporary computer-readable media storing the instructions, wherein the instructions, when executed by the one or more processors, cause the one or more processors to perform any of the methods provided in Examples 1-20 and the methods described herein.

[0208] Example D provides an encoder for generating an encoded bitstream using the operations described herein.

[0209] Example E provides an encoder for performing any of the methods provided in Examples 16-20.

[0210] Example F provides a decoder for decoding an encoded bitstream using the operations described herein.

[0211] Example H provides a decoder for performing any of the methods provided in Examples 1-15. Variants and other notes

[0212] Despite Figure 1-5 and Figure 10-12 Shown and referenced in Figure 1-5 and Figure 10-12 The operations of the described example methods are shown as occurring once each and in a particular order, but it will be appreciated that the operations may be performed in any suitable order and repeated as desired. In addition, one or more operations may be performed in parallel. In addition, Figure 1-5 and Figure 10-12 The operations shown in the drawings may be combined or may include more or less detail than described.

[0213] The above description of the illustrated implementation of the present disclosure (including the content described in the abstract) is not intended to be exhaustive or to limit the present disclosure to the disclosed precise form. Although specific implementations and examples of the present disclosure are described herein for illustrative purposes, various equivalent modifications are possible within the scope of the present disclosure as will be appreciated by those skilled in the relevant art. These modifications may be made to the present disclosure in accordance with the above detailed description.

[0214] For purposes of explanation, specific numbers, materials, and configurations are set forth in order to provide a thorough understanding of the illustrative implementations. However, it will be apparent to those skilled in the art that the present disclosure may be implemented without these specific details, and / or that the present disclosure may be implemented with only some of the described aspects. In other cases, well-known features are omitted or simplified so as not to obscure the illustrative implementations.

[0215] In addition, reference is made to the accompanying drawings which form a part of this document, and in which exemplified embodiments are shown by way of illustration. It is to be understood that other embodiments may be utilized, and structural or logical changes may be made without departing from the scope of the present disclosure. Therefore, the following detailed description should not be considered restrictive.

[0216] The various operations can be described as multiple discrete actions or operations in a manner that is most helpful for understanding the disclosed subject matter. However, the order of description should not be interpreted as implying that these operations are necessarily order-dependent. In particular, these operations may not be performed in the order presented. The described operations may be performed in an order different from the described embodiments. In additional embodiments, various additional operations may be performed, or the described operations may be omitted.

[0217] For purposes of this disclosure, the phrase "A or B" or the phrase "A and / or B" means (A), (B), or (A and B). For purposes of this disclosure, the phrase "A, B or C" or the phrase "A, B and / or C" means (A), (B), (C), (A and B), (A and C), (B and C), or (A, B and C). When used with reference to a measurement range, the term "between" includes the endpoints of the measurement range.

[0218] The description uses the phrases "in one embodiment" or "in an embodiment," each of which may refer to one or more of the same or different embodiments. As used with respect to embodiments of the present disclosure, the terms "comprising, including," "having," and the like are synonymous. The present disclosure may use perspective-based descriptions, such as "above," "below," "top," "bottom," and "side" to explain various features of the drawings, but these terms are merely for ease of discussion and do not imply a desired or required orientation. The drawings are not necessarily drawn to scale. Unless otherwise specified, the use of ordinal adjectives "first," "second," and "third," etc., describing common objects merely indicates that different instances of similar objects are referenced, and is not intended to imply that the objects so described must be in a given sequence in time, space, rank, or in any other manner.

[0219] In the following detailed description, various aspects of the illustrative implementations will be described using terms that are commonly employed by those skilled in the art to convey the substance of their work to others skilled in the art.

[0220] As described herein or known in the art, the terms "substantially," "close," "approximately," "close to," and "approximately" generally refer to within + / - 20% of a target value. Similarly, as described herein or known in the art, terms indicating the orientation of various elements (e.g., "coplanar," "perpendicular," "orthogonal," "parallel," or any other angle between elements) generally refer to within + / - 5-20% of a target value.

[0221] Furthermore, the terms "comprise, comprising, include, including," "have, having," or any other variation thereof, are intended to cover a non-exclusive inclusion. For example, a method, process, or apparatus that comprises a list of elements is not necessarily limited to only those elements but may include other elements not expressly listed or inherent to such method, process, or apparatus. Furthermore, the term "or" refers to an inclusive "or" rather than an exclusive "or."

[0222] The systems, methods, and devices of the present disclosure each have several innovative aspects, no single one of which is responsible for all of the desirable attributes disclosed herein.The details of one or more implementations of the subject matter described in this specification are set forth in the description and the drawings.

Claims

1. A method comprising: Receiving an encoded bitstream, where: The encoded bitstream includes an encoded reference block, interpolation filter information, and residual data of the encoded block; and The interpolation filter information indicates a first interpolation filter type for a first region of a predicted block and a second interpolation filter type for a second region of the predicted block that is different from the first interpolation filter type; decoding the encoded reference block; generating the predicted block based on the decoded reference block; applying the first interpolation filter type to the first region of the predicted block to obtain a filtered first region of the block; applying the second interpolation filter type to the second region of the predicted block to obtain a second region of the filtered block; and A reconstructed block of a reconstructed frame is output based on the filtered block and the residual data.

2. The method according to claim 1, wherein: The encoded bitstream further includes predictor information indicating a predictor; and Generating the predicted block comprises applying the predictor to the decoded reference block.

3. The method according to claim 1 or 2, wherein: The decoded reference block and the reconstructed block are both in the reconstructed frame.

4. The method according to claim 1 or 2, wherein: The decoded reference block is part of a reference frame; and The reference frame and the reconstructed frame have different frame indexes.

5. The method according to claim 1 or 2, wherein: The interpolation filter information includes a filter index.

6. The method according to claim 1 or 2, wherein: The interpolation filter information includes: a partition shape index indicating a manner for partitioning the predicted block into at least the first region and the second region; and One or more filter indices indicating the first interpolation filter type and the second interpolation filter type.

7. The method according to claim 1 or 2, wherein: The interpolation filter information includes: A signal indicating that the interpolation filter information is the same as another reconstructed block.

8. The method according to claim 1 or 2, wherein: The interpolation filter information includes: A filter index residual signal for indicating a residual filter index value relative to a filter index of another reconstructed block.

9. The method according to claim 1 or 2, wherein: The interpolation filter information includes: A sequence header signal for indicating whether multiple interpolation filter types are allowed for a given block of a sequence of frames, wherein the sequence of frames includes the reconstructed frame and further frames.

10. The method according to claim 1 or 2, wherein: The interpolation filter information includes: A frame header signal is used to indicate whether multiple interpolation filter types are allowed for a given block of the reconstructed frame.

11. The method according to claim 1 or 2, wherein: The interpolation filter information includes: A block size signal is used to indicate one or more block sizes that allow multiple interpolation filter types and one or more other block sizes that do not allow multiple interpolation filter types.

12. The method according to claim 1 or 2, wherein: The first region includes an upper half of the decoded reference block; and The second region includes a lower half of the decoded reference block.

13. The method according to claim 1 or 2, wherein: The first region includes a left half of the decoded reference block; and The second region includes a right half of the decoded reference block.

14. The method according to claim 1 or 2, wherein: The first region includes a first third of the decoded reference block; and The second region includes a second third of the decoded reference block.

15. The method according to claim 1 or 2, wherein: The first region includes a first quarter of the decoded reference block; and The second region includes a second quarter of the decoded reference block.

16. An apparatus comprising: one or more processors for executing instructions; and One or more non-transitory computer-readable media storing instructions that, when executed by the one or more processors, cause the one or more processors to: Receiving an encoded bitstream, where: The encoded bitstream includes an encoded reference block, interpolation filter information, and residual data of the encoded block; and The interpolation filter information indicates a first interpolation filter type for a first region of a predicted block and a second interpolation filter type for a second region of the predicted block that is different from the first interpolation filter type; decoding the encoded reference block; generating the predicted block based on the decoded reference block; applying the first interpolation filter type to the first region of the predicted block to obtain a filtered first region of the block; applying the second interpolation filter type to the second region of the predicted block to obtain a second region of the filtered block; and A reconstructed block of a reconstructed frame is output based on the filtered block and the residual data.

17. The device according to claim 16, wherein: The encoded bitstream further includes predictor information indicating a predictor; and Generating the predicted block comprises applying the predictor to the decoded reference block.

18. The device according to claim 16 or 17, wherein: The decoded reference block and the reconstructed block are both in the reconstructed frame.

19. The device according to claim 16 or 17, wherein: The decoded reference block is part of a reference frame; and The reference frame and the reconstructed frame have different frame indexes.

20. The device according to claim 16 or 17, wherein: The interpolation filter information includes a filter index.

21. The device according to claim 16 or 17, wherein: The interpolation filter information includes: a partition shape index indicating a manner for partitioning the predicted block into at least the first region and the second region; and One or more filter indices indicating the first interpolation filter type and the second interpolation filter type.

22. One or more non-transitory computer-readable media storing instructions that, when executed by one or more processors, cause the one or more processors to: Receiving an encoded bitstream, where: The encoded bitstream includes an encoded reference block, interpolation filter information, and residual data of the encoded block; and The interpolation filter information indicates a first interpolation filter type for a first region of a predicted block and a second interpolation filter type for a second region of the predicted block that is different from the first interpolation filter type; decoding the encoded reference block; generating the predicted block based on the decoded reference block; applying the first interpolation filter type to the first region of the predicted block to obtain a filtered first region of the block; applying the second interpolation filter type to the second region of the predicted block to obtain a second region of the filtered block; as well as A reconstructed block of a reconstructed frame is output based on the filtered block and the residual data.

23. The one or more non-transitory computer-readable media of claim 22, wherein: The interpolation filter information includes: A signal indicating that the interpolation filter information is the same as another reconstructed block.

24. One or more non-transitory computer-readable media according to claim 22 or 23, wherein: The interpolation filter information includes: A filter index residual signal for indicating a residual filter index value relative to a filter index of another reconstructed block.

25. One or more non-transitory computer-readable media according to claim 22 or 23, wherein: The interpolation filter information includes one or more of the following items: a sequence header signal for indicating whether multiple interpolation filter types are allowed for a given block of a sequence of frames, wherein the sequence of frames includes the reconstructed frame and further frames; a frame header signal for indicating whether multiple interpolation filter types are allowed for a given block of the reconstructed frame; and A block size signal is used to indicate one or more block sizes that allow multiple interpolation filter types and one or more other block sizes that do not allow multiple interpolation filter types.

Images