Context adaptive interpolation filter

Abstract

The present disclosure relates generally to video coding, in particular to methods and systems for deriving a context adaptive interpolation filter for generating prediction samples in inter prediction involving fractional motion vectors or block vectors. For example, an encoder and decoder may adaptively derive an interpolation filter using reconstructed samples. The reconstructed sample may belong to a template near the current block and a reference block associated with a motion vector or block vector. The derivation of the adaptive interpolation filter may be based on minimizing distortion between an interpolation template associated with the reference block and a template associated with the current block.

Description

Incorporation by reference

[0001] This application claims priority to U.S. Non - Provisional Patent Application No. 18 / 497,697, entitled "Context Adaptive Interpolation Filter", filed on October 30, 2023, which claims priority to U.S. Provisional Patent Application No. 63 / 520,295, entitled "Context Adaptive Interpolation Filter", filed on August 17, 2023. The entire contents of the above prior applications are hereby incorporated by reference in their entirety. Technical Field

[0002] The present disclosure generally relates to video coding, and particularly to methods and systems for deriving a context - adaptive interpolation filter for generating prediction samples in inter - frame prediction involving fractional motion vectors or block vectors. Background Art

[0003] Uncompressed digital video may contain a series of pictures and may have specific bit - rate requirements for storage, data processing, and transmission bandwidth in streaming applications. One purpose of video encoding and decoding can be to reduce redundancy in the uncompressed input video signal through various compression techniques. Summary of the Invention

[0004] The present disclosure generally relates to video coding, and particularly to methods and systems for deriving a context - adaptive interpolation filter for generating prediction samples in inter - frame prediction involving fractional motion vectors or block vectors. For example, an encoder and a decoder may adaptively derive an interpolation filter using reconstructed samples. The reconstructed samples may belong to a template near the current block and reference blocks associated with motion vectors or block vectors. The derivation of the adaptive interpolation filter may be based on minimizing the distortion between an interpolation template associated with a reference block and a template associated with the current block.

[0005] In an exemplary embodiment, a method for decoding a current block in a video bitstream is disclosed. The method may include receiving a vector (motion vector (MV) or block vector (BV)) from the video bitstream, the vector being used to identify a reference block of the current video block; determining a first set of reconstructed samples in a first neighboring template of the current block; determining a second set of reconstructed samples in a second neighboring template of the reference block; determining whether at least one adaptive interpolation filter is applied to the current video block; and in response to the application of the adaptive interpolation filter: deriving at least one adaptive interpolation filter based on the first set of reconstructed samples and the second set of reconstructed samples; and predicting samples in the current block by applying at least one adaptive interpolation filter to the reference block at least.

[0006] In the above exemplary embodiment, determining whether at least one adaptive interpolation filter is applied to the current block includes: receiving a syntax element from the video bitstream, the syntax element indicating an inter prediction mode of the current block; and determining that at least one adaptive interpolation filter is applied to the current video block only when the inter prediction mode of the current block is one of the translational inter prediction modes.

[0007] In any of the above exemplary embodiments, determining whether at least one adaptive interpolation filter is applied to the current block includes: receiving a high-level syntax flag at the sequence level, picture level, sub-picture level, slice level or tile level, and determining whether at least one adaptive interpolation filter is applied based on the high-level syntax flag.

[0008] In any of the above exemplary embodiments, the second neighboring template has the same shape and the same size as the first neighboring template, and the relative position of the second neighboring template with respect to the reference block is the same as the relative position of the first neighboring template with respect to the current block.

[0009] In any of the above exemplary embodiments, deriving at least one adaptive interpolation filter includes: deriving at least one adaptive interpolation filter by minimizing the distortion between the first set of reconstructed samples and the interpolation of the second set of reconstructed samples by at least one adaptive interpolation filter.

[0010] In any of the above exemplary embodiments, deriving at least one adaptive interpolation filter may include: identifying a plurality of predetermined interpolation filters; applying each of the plurality of predetermined interpolation filters to the second set of reconstructed samples; and selecting one of the plurality of predetermined interpolation filters that causes the least distortion compared to the first set of reconstructed samples when applied to the second set of reconstructed samples.

[0011] In any of the above exemplary embodiments, the distortion is quantified by the sum of absolute difference (SAD), the sum of square difference (SSD), the sum of square error (SSE), or the sum of absolute transform domain difference (SATD).

[0012] In any of the above exemplary embodiments, deriving at least one adaptive interpolation filter may include: identifying a plurality of predetermined interpolation filters; applying each of the plurality of predetermined interpolation filters to a second set of reconstructed samples; selecting a subset of the predetermined interpolation filters from the plurality of predetermined interpolation filters, wherein when the subset of the predetermined interpolation filters is applied to the second set of reconstructed samples, the distortion caused is less than that caused by other predetermined interpolation filters compared to the first set of reconstructed samples; sorting within the subset of the predetermined interpolation filters; receiving an index represented by a signal from the video bitstream; and sequentially selecting at least one adaptive interpolation filter from the subset of the predetermined interpolation filters according to the index represented by the signal.

[0013] In any of the above exemplary embodiments, the MV or BV points to a fractional pixel position; and the reference block includes reference samples at integer pixel positions closest to the fractional pixel position.

[0014] In any of the above exemplary embodiments, determining whether at least one adaptive interpolation filter is applied to a current block may include: determining whether the MV or BV includes an MV or BV value falling within a predetermined value set; and when the MV or BV value falls within the predetermined value set, determining that at least one adaptive interpolation filter is applied to the current block.

[0015] In any of the above exemplary embodiments, the predetermined value set includes fractional pixel values.

[0016] In any of the above exemplary embodiments, at least one adaptive interpolation filter includes a two-dimensional non-separable filter.

[0017] In any of the above exemplary embodiments, at least one adaptive interpolation filter includes separate one-dimensional interpolation filters.

[0018] In any of the above exemplary embodiments, the MV or BV points to a fractional pixel position in one or both of the vertical and horizontal directions, and at least one adaptive interpolation filter is only applied to the dimension with a fractional MV or BV.

[0019] In any of the above example embodiments, the method may further include: in response to determining that no adaptive interpolation filter is applied, applying a predetermined or explicitly signaled interpolation filter to a reference block to generate predicted samples of a current block.

[0020] In any of the above example embodiments, the number of filter taps of at least one adaptive interpolation filter is different from the number of taps of a predetermined or explicitly signaled interpolation filter.

[0021] In any of the above example embodiments, the number of filter taps of at least one adaptive interpolation filter is adaptively determined according to the size of the current block.

[0022] In any of the above example embodiments, the shapes and / or positions of a first neighboring template and a second neighboring template depend on encoded information available to both an encoder and a decoder, and the encoded information includes at least one of the following: a relative position of the current block within the current picture; a relative position of the current block within an associated maximum coding block; or encoded information associated with one or more neighboring blocks of the current block.

[0023] Aspects of the present disclosure also provide an electronic device or apparatus including a circuit or a processor configured to perform any of the above method implementations.

[0024] Aspects of the present disclosure also provide a non-transitory computer-readable medium storing instructions that, when executed by an electronic device, cause the electronic device to perform any of the above method implementations. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Other features, properties, and various advantages of the disclosed subject matter will become more apparent from the following detailed description and the accompanying drawings, in which:

[0026] Figure 1 A schematic diagram showing a simplified block diagram of a communication system (100) according to an example embodiment;

[0027] Figure 2 A schematic diagram showing a simplified block diagram of a communication system (200) according to an example embodiment;

[0028] Figure 3 A schematic diagram showing a simplified block diagram of a video decoder according to an example embodiment;

[0029] Figure 4 A schematic diagram showing a simplified block diagram of a video encoder according to an example embodiment;

[0030] Figure 5Shows a block diagram of a video encoder according to another exemplary embodiment;

[0031] Figure 6 Shows a block diagram of a video decoder according to another exemplary embodiment;

[0032] Figure 7 Shows a scheme for coding block partitioning according to an exemplary embodiment of the present disclosure;

[0033] Figure 8 Shows another scheme for coding block partitioning according to an exemplary embodiment of the present disclosure;

[0034] Figure 9 Shows another scheme for coding block partitioning according to an exemplary embodiment of the present disclosure;

[0035] Figure 10 Shows an example of a template of a block.

[0036] Figure 11 Shows an example template of a current block and a corresponding reference block.

[0037] Figure 12 Shows an example logic flow of a method for signaling an optical flow refinement flag and a composite inter prediction mode.

[0038] Figure 13 Shows a schematic diagram of a computer system according to an exemplary embodiment of the present disclosure. Detailed Description

[0039] Throughout the specification and claims, the terms may have nuanced meanings that are implied or implicit from the context beyond the explicitly stated meanings. The phrases "in one embodiment / implementation" or "in some embodiments / implementations" used herein do not necessarily refer to the same embodiment / implementation, and the phrases "in another embodiment / implementation" or "in other embodiments" used herein do not necessarily refer to different embodiments. For example, it is intended that the claimed subject matter include combinations of all or part of the exemplary embodiments / implementations.

[0040] In general, terms can be understood based on their usage in context. For example, terms used herein, such as "and", "or", or "and / or", can include various context-related meanings. Generally, "or" when used to relate a list, such as A, B, or C, is intended to mean A, B, and C (used here in an inclusive sense), as well as A, B, or C (used here in an exclusive sense). Additionally, terms such as "one or more", "at least one", "a", "an", or "the" used herein can be used in a singular or plural sense, at least in part depending on the context. Further, the terms "based on" or "determined by" can be understood as not necessarily intending to convey a set of exclusive factors, and instead can allow for the existence of other factors that may not be explicitly described, at least in part depending on the context.

[0041] Figure 1 is a simplified block diagram of a communication system (100) according to an embodiment disclosed in the present application. The communication system (100) includes a plurality of terminal devices, such as 110, 120, 130, and 140, which can communicate with each other via, for example, a network (150). In Figure 1 an embodiment, the first terminal device (110) and the second terminal device (120) can perform unidirectional data transmission. For example, the first terminal device (110) can encode video data in the form of one or more encoded bitstreams (such as a video picture stream captured by the terminal device (110)) for transmission via the network (150). The second terminal device (120) can receive the encoded video data from the network (150), decode the encoded video data to recover the video data, and display video pictures based on the recovered video data. Unidirectional data transmission can be implemented in applications such as media services.

[0042] In another embodiment, for example, during a video conferencing application, the third terminal device (130) and the fourth terminal device (140) can perform bidirectional transmission of encoded video data. For bidirectional transmission of data, in one example, each of the third terminal device (130) and the fourth terminal device (140) can encode video data (such as a video picture stream captured by the terminal device) for transmission to the other of the third terminal device (230) and the fourth terminal device (240), and can also receive encoded video data from the other terminal device to recover and display video pictures.

[0043] In Figure 1In an embodiment, the terminal device can be implemented as a server, a personal computer, and a smart phone, but the applicability of the basic principles disclosed in this application is not limited thereto. The embodiments disclosed in this application can be implemented in a desktop computer, a laptop computer, a tablet computer, a media player, a wearable computer, a dedicated video conferencing device, etc. The network (150) represents any number or type of network for transmitting encoded video data between terminal devices, including, for example, wired (wired) and / or wireless communication networks. The communication network (150) can exchange data in circuit switching, packet switching, and / or other types of channels. The network can include a telecommunications network, a local area network, a wide area network, and / or the Internet.

[0044] As an example, Figure 2 shows how a video encoder and a video decoder are placed in a streaming environment. The subject matter disclosed in this application is equally applicable to other video-supported applications, including, for example, video conferencing, digital TV broadcasting, gaming, virtual reality, storing compressed video on digital media including CDs, DVDs, memory sticks, etc.

[0045] As Figure 2 shown, a video streaming system can include an acquisition subsystem (213), and the acquisition subsystem can include a video source (201) such as a digital camera, and the video source creates an uncompressed video picture or image stream (202). In an embodiment, the video picture stream (202) includes samples recorded by the digital camera of the video source 201. Compared with the encoded video data (204) (or the encoded video bitstream), the video picture stream (202) is depicted as a thick line to emphasize the high data volume of the video picture stream. The video picture stream (202) can be processed by an electronic device (220), and the electronic device (220) includes a video encoder (203) coupled to the video source (201). The video encoder (203) can include hardware, software, or a combination of hardware and software to implement or carry out aspects of the subject matter disclosed in more detail below. Compared with the video picture stream (202), the encoded video data (204) (or the encoded video bitstream (204)) is depicted as a thin line to emphasize the lower data volume of the encoded video data (204) (or the encoded video bitstream (204)), which can be stored on a streaming server (205) for future use or directly stored in a downstream video device (not shown). One or more streaming client subsystems, such as Figure 2The client subsystems (206) and client subsystem (208) therein can access the streaming server (205) to retrieve copies (207) and (209) of the encoded video data (204). The client subsystem (206) can include, for example, a video decoder (210) in an electronic device (230). The video decoder (210) decodes the incoming copy (207) of the encoded video data and generates an uncompressed output video picture stream (211) that can be presented on a display (212) (such as a display screen) or another presentation device (not depicted).

[0046] Figure 3 A block diagram of a video decoder (310) of an electronic device (330) according to any embodiment disclosed in the present application is shown. The electronic device (330) can include a receiver (331) (such as a receiving circuit). The video decoder (310) can be used to replace Figure 2 the video decoder (210) in the example of

[0047] As Figure 3 shown, the receiver (331) can receive one or more encoded video sequences from a channel (301). To prevent network jitter and / or handle playback timing, a buffer memory (315) can be provided between the receiver (331) and an entropy decoder / parser (320) (hereinafter referred to as "parser (320)"). The parser (320) can reconstruct symbols (321) from the encoded video sequences. The categories of these symbols include information for managing the operation of the video decoder (310) and potential information for controlling a display device (312) (such as a display screen), etc. The parser (320) can parse / entropy decode the encoded video sequences. The parser (320) can extract subgroup parameter sets for at least one subgroup of pixels in the video decoder from the encoded video sequences. The subgroups can include Group of Pictures (GOP), pictures, tiles, slices, macroblocks, Coding Units (CUs), blocks, Transform Units (TUs), Prediction Units (PUs), and so on. The parser (320) can also extract information from the encoded video sequences, such as transform coefficients (such as Fourier transform coefficients), quantizer parameter values, motion vectors, and so on. The reconstruction of the symbols (321) can involve multiple different processing or functional units. The units involved and the way they are involved can be controlled by subgroup control information parsed by the parser (320) from the encoded video sequences.

[0048] The first unit may include a scaler / inverse transform unit (351). The scaler / inverse transform unit (351) may receive quantized transform coefficients as symbols (321) and control information from a parser (320), including indications of which type of inverse transform to use, block size, quantization factor / parameter, quantization scaling matrix, etc. The scaler / inverse transform unit (351) may output a block including sample values, which may be input into an aggregator (355).

[0049] In some cases, the output samples of the scaler / inverse transform unit (351) may belong to an intra-coded block; that is, a block that does not use predictive information from a previously reconstructed picture but may use predictive information from a previously reconstructed portion of the current picture. Such predictive information may be provided by an intra-picture prediction unit (352). In some cases, the intra-picture prediction unit (352) may generate a block having the same size and shape as the block being reconstructed using surrounding block information that has been reconstructed and stored in a current picture buffer (358). For example, the current picture buffer (358) buffers a partially reconstructed current picture and / or a fully reconstructed current picture. In some embodiments, the aggregator (355) adds the prediction information generated by the intra-prediction unit (352) to the output sample information provided by the scaler / inverse transform unit (351) on a per-sample basis.

[0050] In other cases, the output samples of the scaler / inverse transform unit (351) may belong to an inter-coded and potentially motion-compensated block. In this case, a motion compensation prediction unit (353) may access a reference image memory (357) based on a motion vector. After motion compensation of the extracted reference samples according to the symbol (321), these samples may be added by the aggregator (355) to the output of the scaler / inverse transform unit (351) (the output of unit 351 may be referred to as residual samples or a residual signal), thereby generating output sample information.

[0051] The output samples of the aggregator (355) may be employed by various loop filtering techniques in a loop filter unit (356) including several types of loop filters. The output of the loop filter unit (356) may be a sample stream, which may be output to a display device (312) and stored in a reference picture memory (357) for subsequent inter-picture prediction.

[0052] Figure 4 is a block diagram of a video encoder (403) according to an embodiment disclosed in the present application. The video encoder (403) is disposed in an electronic device (420). The electronic device (420) includes a transmitter (440) (e.g., a transmission circuit). The video encoder (403) may be used to replace Figure 4 the video encoder (403) in the embodiment.

[0053] A video encoder (403) may receive video samples from a video source (401). According to some example embodiments, the video encoder (403) may encode and compress pictures of a source video sequence into an encoded video sequence (443) in real time or under any other time constraints required by an application. Implementing an appropriate encoding speed is a function of a controller (450). In some embodiments, the controller (450) may be functionally coupled to and control other functional units as described below. Parameters set by the controller (450) may include rate control related parameters (picture skipping, quantizer, λ value of rate-distortion optimization techniques, etc.), picture size, group of pictures (GOP) layout, maximum motion vector search range, etc.

[0054] In some embodiments, the video encoder (403) operates in an encoding loop. The encoding loop may include a source encoder (430) and an (in - loop) decoder (433) embedded in the video encoder (403). The decoder (433) reconstructs symbols in a manner similar to a (remote) decoder to create sample data, even though the embedded decoder 433 processes the encoded video stream of the source encoder 430 without entropy coding (because in the video compression techniques contemplated in this application, any compression between symbols and the encoded video bitstream in entropy coding is lossless). At this point, it can be observed that any decoder technique, except parsing / entropy decoding which may only exist in a decoder, must also be able to exist in the corresponding encoder in a substantially identical functional form. For this reason, this application sometimes focuses on decoder operations related to the decoding part of the encoder. The description of encoder techniques can be simplified because encoder techniques are inverse to the decoder techniques described comprehensively. A more detailed description of the encoder is only needed in certain areas or aspects and is provided below.

[0055] During operation, in some example implementations, the source encoder (430) may perform motion - compensated predictive coding. Referring to one or more previously encoded pictures in the video sequence designated as "reference pictures", this motion - compensated predictive coding performs predictive coding on the input picture.

[0056] The local video decoder (433) may decode the encoded video data of pictures that may be designated as reference pictures. The local video decoder (433) replicates the decoding process that may be performed by a video decoder on a reference picture and may store the reconstructed reference picture in a reference picture cache (434). In this way, the video encoder (403) may locally store a copy of the reconstructed reference picture that has the same content as the reconstructed reference picture that will be obtained by a remote video decoder (in the absence of transmission errors).

[0057] The predictor (435) can perform a prediction search for the encoding engine (432). That is, for a new picture to be encoded, the predictor (435) can search in the reference picture memory (434) for sample data (as candidate reference pixel blocks) or some metadata that can be used as an appropriate prediction reference for the new picture, such as reference picture motion vectors, block shapes, etc.

[0058] The controller (450) can manage the encoding operations of the source encoder (430), including, for example, setting parameters and subgroup parameters for encoding video data.

[0059] The outputs of all the above functional units can be entropy encoded in the entropy encoder (445). The transmitter (440) can buffer the encoded video sequence created by the entropy encoder (445) to prepare for transmission through the communication channel (460), which can be a hardware / software link to a storage device that will store the encoded video data. The transmitter (440) can merge the encoded video data from the video encoder (403) with other data to be transmitted, such as encoded audio data and / or auxiliary data streams (sources not shown).

[0060] The controller (450) can manage the operations of the video encoder (403). During encoding, the controller (450) can assign a certain encoded picture type to each encoded picture, but this may affect the encoding techniques applicable to the corresponding picture. For example, a picture can usually be assigned to any one of the following picture types: intra picture (I picture), predictive picture (P picture), bi - predictive picture (B picture). The source picture can usually be spatially subdivided into multiple sample - encoding blocks, as described in further detail below.

[0061] Figure 5 is a schematic diagram of a video encoder (503) according to another exemplary embodiment disclosed in the present application. The video encoder (503) is used to receive the sample values in a processing block (such as a prediction block) within the current video picture in a sequence of video pictures, and encode the processing block into an encoded picture that is part of an encoded video sequence. The exemplary video encoder (503) can be used instead of Figure 4 the video encoder (403) in the example.

[0062] For example, the video encoder (503) receives a matrix of sampled values of the processing block. Next, the video encoder (503) determines, using, for example, rate - distortion optimization (RDO), whether to use the intra mode, the inter mode, or the bi - prediction mode to optimally encode the processing block.

[0063] In Figure 5 the embodiment of, the video encoder (503) includes asFigure 5 The inter-frame encoder (530), intra-frame encoder (522), residual calculator (523), switch (526), residual encoder (524), general controller (521), and entropy encoder (525) coupled together as shown in the example arrangement in

[0064] The inter-frame encoder (530) is configured to receive samples of a current block (e.g., a processing block), compare the block with one or more reference blocks in a reference picture (e.g., blocks in a previous picture and a subsequent picture arranged in display order), generate inter-frame prediction information (e.g., a redundancy information description according to an inter-frame coding technique, a motion vector, merge mode information), and calculate an inter-frame prediction result (e.g., a predicted block) based on the inter-frame prediction information using any suitable technique.

[0065] The intra-frame encoder (522) is configured to receive samples of a current block (e.g., a processing block), compare the block with the encoded blocks in the same picture, generate quantization coefficients after transformation, and in some cases also generate intra-frame prediction information (e.g., intra-frame prediction direction information according to one or more intra-frame coding techniques).

[0066] The general controller (521) is configured to determine general control data and control other components of the video encoder (503) based on the general control data. For example, the general controller determines a prediction mode of a block and provides a control signal to the switch (526) based on the prediction mode.

[0067] The residual calculator (523) can be used to calculate the difference (residual data) between the received block and the prediction result of a block selected from the intra-frame encoder (522) or the inter-frame encoder (530). The residual encoder (524) can be used to operate based on the residual data to encode the residual data to generate transform coefficients. The transform coefficients are then subjected to quantization processing to obtain quantized transform coefficients. In various embodiments, the video encoder (503) further includes a residual decoder (528). The residual decoder (528) is configured to perform an inverse transformation and generate decoded residual data. The entropy encoder (525) can be used to format the bitstream to produce an encoded block and perform entropy coding.

[0068] Figure 6 FIG. is a diagram of a video decoder (610) according to another embodiment disclosed in the present application. The video decoder (610) is configured to receive an encoded image as part of an encoded video sequence and decode the encoded image to generate a reconstructed picture. In an embodiment, the video decoder (610) can be used to replace Figure 4 the video decoder (410) in the embodiment.

[0069] In Figure 6 the embodiment, the video decoder (610) includes as Figure 6The entropy decoder (671), inter - frame decoder (680), residual decoder (673), reconstruction module (674) and intra - frame decoder (672) coupled together as shown.

[0070] The entropy decoder (671) can be used to reconstruct certain symbols from an encoded picture, where these symbols represent the syntax elements that make up the encoded picture. The inter - frame decoder (680) can be used to receive inter - frame prediction information and generate an inter - frame prediction result based on this inter - frame prediction information. The intra - frame decoder (672) can be used to receive intra - frame prediction information and generate a prediction result based on this intra - frame prediction information. The residual decoder (673) is used to perform inverse quantization to extract the de - quantized transform coefficients and process these de - quantized transform coefficients to convert the residual from the frequency domain to the spatial domain. The reconstruction module (674) can be used to combine the residual output by the residual decoder (673) with the prediction result (which can be output by the inter - frame prediction module or the intra - frame prediction module) in the spatial domain to form a reconstructed block, and this reconstructed block forms part of the reconstructed picture and part of the reconstructed video.

[0071] It should be noted that any suitable technology can be used to implement the video encoders (203), (403) and (503) and the video decoders (210), (310) and (610). In an embodiment, one or more integrated circuits can be used to implement the video encoders (203), (403) and (503) and the video decoders (210), (310) and (610). In another embodiment, one or more processors executing software instructions can be used to implement the video encoders (203), (403) and (503) and the video decoders (210), (310) and (610).

[0072] For block partitioning in encoding and decoding, a general partitioning can start from a base block and follow a predefined set of rules, a specific pattern, a partitioning tree, or any partitioning structure or scheme. The partitioning can be hierarchical and recursive. After partitioning the base block according to any of the example partitioning procedures or other procedures described below, or a combination thereof, a set of final partitions or coding blocks can be obtained. Each of these partitions can be at a different partitioning level in the partitioning hierarchy and can have various shapes. Each of these partitions can be referred to as a coding block (CB). For the various example partitioning implementations described further below, each resulting CB can be of any allowed size and partitioning level. Such partitions are called coding blocks because they can form units for which some basic encoding / decoding decisions can be made and the encoding / decoding parameters can be optimized, determined, and written to the encoded video bitstream. The highest or deepest level in the final partition represents the depth of the coding block partitioning structure tree. The coding blocks can be luma coding blocks or chroma coding blocks. The CB tree structure for each color can be referred to as a coding block tree (CBT). The coding blocks for all color channels can be collectively referred to as a coding unit (CU). The hierarchies for all color channels can be collectively referred to as a coding tree unit (CTU). The partitioning patterns or structures for the various color channels in a CTU can be the same or different.

[0073] In some implementations, the partitioning tree scheme or structure for the luma and chroma channels is not necessarily the same. In other words, the luma and chroma channels can have separate coding tree structures or patterns. Additionally, whether the luma and chroma channels use the same or different coding partitioning tree structures, and which coding partitioning tree structure is actually to be used, can depend on whether the slice being encoded is a P slice, a B slice, or an I slice. For example, for an I slice, the chroma channel and the luma channel can have separate coding partitioning tree structures or coding partitioning tree structure patterns, while for a P slice or a B slice, the luma and chroma channels can share the same coding partitioning tree scheme. When applying separate coding partitioning tree structures or patterns, the luma channel can be partitioned into CBs by one coding partitioning tree structure and the chroma channel can be partitioned into chroma CBs by another coding partitioning tree structure.

[0074] Figure 7 An example of a predefined 10-way partitioning structure / pattern is shown, which allows recursive partitioning to form a partitioning tree. The root block can start from a predefined level (e.g., starting from a base block at the 128x128 or 64x64 level). Figure 7The example partitioning structures therein include various 2:1 / 1:2 and 4:1 / 1:4 rectangular partitions. In some example embodiments, Figure 7 none of the rectangular partitions therein can be further subdivided. The coding tree depth can be further defined to indicate the depth of the split from the root node or root block. For example, the coding tree depth of the root node or root block can be set to 0, and when the root block is further split once according to Figure 7 , the coding tree depth increases by 1. In some embodiments, only the full square partitions in 710 are allowed to be recursively partitioned to the next level of the partition tree according to Figure 7 the pattern.

[0075] In some other example embodiments of coding block partitioning, a quadtree structure can be used. This quadtree splitting can be applied hierarchically and recursively to any square partition. Whether to further perform quadtree splitting on the base block, intermediate block or partition can be adjusted according to various local characteristics of the base block or intermediate block / partition.

[0076] In other examples, a ternary partitioning scheme can be used to partition the base block or any intermediate block, as Figure 8 shown. The ternary pattern can be implemented as a vertical split (as shown in FIG. 802) or a horizontal split (as shown in FIG. 804). Although Figure 8 the example split ratio therein is shown as 1:2:1, other ratios can also be predefined. In some embodiments, two or more different ratios can be predefined. In some embodiments, the width and height of the partitions of the example triple tree are always powers of 2 to avoid additional transformations.

[0077] The above partitioning schemes can be combined in any way at different partitioning levels. As an example, the above quadtree and binary partitioning schemes can be combined to divide the base block into a quadtree-binary-tree (QTBT) structure. In such a scheme, the base block or intermediate block / partition can be quadtreesplit or binary-split according to a set of predefined conditions (if specified). Figure 9A specific example is shown in which the basic block is first divided into four partitions by a quadtree, as shown by partitions 902, 904, 906, and 908. After that, each resulting partition is either divided into four further partitions by the quadtree at the next level (such as partition 908), or divided into two further partitions by binary splitting (horizontally or vertically, such as partition 902 or partition 906, both of which are symmetric), or not divided (such as partition 904). For square partitions, binary or quadtree splitting can be performed recursively, as shown by the overall example partitioning pattern of 910 and the corresponding tree structure / representation in 920, where solid lines represent quadtree splitting and dashed lines represent binary splitting. For each binary splitting node (non-leaf binary partition), a flag can be used to indicate whether the binary splitting is a horizontal split or a vertical split. As shown in 920 which is consistent with the partitioning structure of 910, the flag "0" can represent a horizontal binary split, and the flag "1" can represent a vertical binary split. For partitions that are split by a quadtree, there is no need to indicate the split type because a quadtree split always splits a block or partition both horizontally and vertically to produce 4 sub-blocks / partitions of equal size. In some embodiments, the flag "1" can represent a horizontal binary split, while the flag "0" can represent a vertical binary split.

[0078] In some example embodiments of QTBT, the quadtree and binary splitting rule sets can be represented by the following predefined parameters and their associated functions: – CTU size: The size of the root node of the quadtree (the size of the base block) – MinQTSize: The minimum allowed size of a quadtree leaf node – MaxBTSize: The maximum allowed size of a binary tree root node – MaxBTDepth: The maximum allowed depth of a binary tree – MinBTSize: The minimum allowed size of a binary tree leaf node

[0079] In some example embodiments of the QTBT partitioning structure, the CTU size can be set to 128×128 luma samples, and two corresponding 64×64 chroma sample blocks (when considering and using example chroma subsampling). MinQTSize can be set to 16×16, MaxBTSize can be set to 64×64, MinBTSize (for both width and height) can be set to 4×4, and MaxBTDepth can be set to 4. First, quadtree partitioning can be applied to the CTU to generate quadtree leaf nodes. The size of the quadtree leaf nodes can range from its minimum allowed size of 16×16 (i.e., MinQTSize) to 128×128 (i.e., the CTU size). If the node size is 128×128, it will not be first split by the binary tree because its size exceeds MaxBTSize (i.e., 64×64). Additionally, nodes not exceeding MaxBTSize can be partitioned by the binary tree. In Figure 9 the example of Figure 9 , the base block size is 128×128. According to the predefined rule set, the base block can only be split by the quadtree. The partitioning depth of the base block is 0. Each of the four resulting partitions is 64×64, which does not exceed MaxBTSize and can be further split by the quadtree or binary tree at level 1. This process continues. When the binary tree depth reaches MaxBTDepth (i.e., 4), further splitting is no longer considered. When the width of the binary tree node is equal to MinBTSize (i.e., 4), further horizontal splitting is no longer considered. Similarly, when the height of the binary tree node is equal to MinBTSize, further vertical splitting is no longer considered.

[0080] In some example embodiments, the above QTBT scheme can be used to support the flexibility of having the same QTBT structure or separate QTBT structures for luma and chroma. For example, for P slices and B slices, the luma and chroma CTBs in a CTU can share the same QTBT structure. However, for I slices, the luma CTB can be divided into CUs by one QTBT structure, while the chroma CTB can be divided into chroma CUs by another QTBT structure. This means that in I slices, the CU can refer to different color channels. For example, an I slice can consist of coded blocks of one luma component or coded blocks of two chroma components, while the CU in a P slice or B slice can consist of coded blocks of all three color components.

[0081] The above various CU partitioning schemes and the scheme of further dividing the CU into PBs (prediction blocks) can be combined in any way. The specific embodiments provided below are only non-limiting examples.

[0082] Inter-frame prediction can be implemented, for example, by a single-reference mode or a composite-reference mode. In some embodiments, a skip flag may first be included in the bitstream of the current block (or at a higher level) to indicate whether the current block is inter-frame encoded and will not be skipped. If the current block is inter-frame encoded, another flag may further be included in the bitstream as a signal to indicate whether the single-reference mode or the composite-reference mode is used to predict the current block. For the single-reference mode, one reference block may be used to generate a prediction block of the current block. For the composite-reference mode, two or more reference blocks may be used to generate a prediction block, for example, by weighted averaging. One or more reference frame indices and one or more corresponding motion vectors additionally used may be used to identify one or more reference blocks, where one or more motion vectors indicate one (or more) offsets in the position relative to the frame between one (or more) reference blocks and the current block, for example, offsets in terms of horizontal and vertical pixels. For example, in the single-reference mode, an inter-frame prediction block of the current block may be generated as a prediction block based on a single reference block identified by a motion vector in a reference frame; while in the composite-reference mode, the prediction block may be generated by weighted averaging of two reference blocks in two reference frames, and the two reference frames are indicated by two reference frame indices and two corresponding motion vectors. The motion vector may be encoded in various ways and included in the bitstream.

[0083] In some example embodiments, one or more reference picture lists may be formed based on information in a Reference Picture Set (RPS), and the reference picture list includes identification of short-term and long-term reference frames for inter-frame prediction. For example, a single picture reference list, denoted as L0 reference (or reference list 0), may be formed for uni-directional inter-frame prediction, while two picture reference lists may be formed for bi-directional inter-frame prediction, denoted as L0 (or reference list 0) and L1 (or reference list 1) respectively for each of the two prediction directions. The reference frames included in the L0 and L1 lists may be sorted in a variety of predetermined ways. The lengths of the L0 and L1 lists may be written into the video bitstream. Uni-directional inter-frame prediction may be a single-reference mode, or may also be a composite-reference mode when, in the composite prediction mode, multiple references used to generate a prediction block by weighted averaging are on the same side of the frame where the block to be predicted is located. Bi-directional inter-frame prediction can only be a composite mode because bi-directional inter-frame prediction involves at least two reference blocks.

[0084] In some other embodiments, the concept of inter-frame prediction may also be applied intra-frame. In other words, the reference block of the current block may be located in the same current frame. In this case, the position of the reference block relative to the current block may be indicated by a block vector (BV), similar to the motion vector (MV) in inter-frame prediction involving a reference frame. Both the BV or the MV may be referred to as a vector.

[0085] In some example embodiments, the MV or BV may point to a fractional position. Here, the fractional MV or BV generally refers to a motion vector or a block vector that includes fractional pixel values. In other words, the motion vector or block vector value can be a fraction of a pixel, an integer of a pixel, or a fraction of a pixel plus an integer value of a pixel. When the MV or BV is non-integer, the corresponding reference block (or prediction block) will be located between two or more integer reference blocks and will not be aligned with a block having known pixel values. Therefore, the reference block or prediction block corresponding to the fractional MV or BV needs to be derived rather than copied from existing reference samples. In the following disclosure, the MV or BV may be described as integer-pel and fractional-pel. The terms "integer pixel" or "fractional pixel" are interchangeable with the terms "full-pel" and "sub-pel", respectively, and have the same meaning.

[0086] In some example embodiments, this derivation or generation of prediction samples can be performed by an interpolation filter that operates on reference samples at integer positions. Specifically, interpolation can be performed on two or more integer reference blocks near the fractional reference block and containing existing reference samples to generate a fractional reference sample that serves as a prediction sample for the current block. For example, the characteristics of the interpolation filter can include the number of taps (i.e., the number of adjacent samples at integer pixel positions participating in the interpolation) and the filter coefficients.

[0087] This interpolation generation of reference samples at fractional pixel positions may need to be performed in the horizontal direction or the vertical direction or in both the horizontal and vertical directions. For example, when the BV or MV points to a prediction block located at an integer horizontal pixel coordinate and a fractional vertical pixel coordinate, the interpolation filter can be applied in the vertical direction, and no interpolation is required in the horizontal direction. When the BV or MV points to a prediction block located at an integer vertical pixel coordinate and a fractional horizontal pixel coordinate, the interpolation filter can be applied in the horizontal direction, and no interpolation is required in the vertical direction. When the BV or MV points to a prediction block located at fractional horizontal pixel coordinates and fractional vertical pixel coordinates, the interpolation filter can be applied along both the horizontal and vertical directions. Applying the interpolation filter in both the horizontal and vertical directions can be done either by sequentially using separable interpolation filters or by simultaneously using non-separable interpolation filters. Similarly, one-dimensional separable interpolation filters or two-dimensional non-separable interpolation filters can be designed to generate reference samples for inter-frame prediction.

[0088] In some example embodiments, such interpolation filters may be predefined for different pixel fractions of the MV or BV. For example, one or more interpolation filters with a predefined number of taps may be predefined for 1 / 2 pixel (1 / 2 pel). Similarly, one or more interpolation filters with a predefined number of taps may be predefined for 1 / 4 pixel (1 / 4 pel), and so on. These predefined interpolation filters may be grouped by pixel fraction and indexed within each fraction. The encoder may select an interpolation filter based on the fractional value of the MV or BV and write the index of the selected interpolation filter into the bitstream. The decoder may first extract and determine the fractional value of the MV or BV accordingly, and then determine the predefined interpolation filter to be used based on the fractional value and the index of the signal representation. In some embodiments, the specific interpolation filter for deriving the reference samples for the BV or MV with a specific pixel fraction may be fixed and predefined, so no index needs to be signaled.

[0089] The effectiveness of the interpolation filter is crucial for achieving a high degree of accuracy in inter-frame prediction. The predefined fixed interpolation filters may not be sufficient to adapt to the different statistics of dynamic images and video content for better coding efficiency. Therefore, other alternative methods may be needed: namely, using adaptive and dynamically generated interpolation filters to provide closer prediction blocks and prediction samples. In these alternative embodiments, it is also necessary to consider avoiding excessive signaling overhead.

[0090] In some other example embodiments, templates for the current block and the corresponding reference block may be defined and then used to derive the above-mentioned interpolation filter, which provides adaptability and efficient signaling of the interpolation filter, as described in further detail below.

[0091] The template of a block (whether it is the current block or the reference block) is defined as a set of available neighboring reconstructed samples of the block. Examples of templates may be: each template includes one or more rows of top reference samples, or one or more rows of left reference samples, or one or more rows of top and left reference samples (i.e., an inverted L shape), as Figure 10 shown. Specifically, in Figure 10 , three example template shapes are shown for block 1001, namely template 1002, template 1004, and template 1006, which respectively include one or more reconstructed left, top, and adjacent rows that include both left and top.

[0092] Templates can be defined for any block. In the case of inter-frame prediction, templates for the current block and the reference block can be defined and are shown as template 1102 and template 1104 in Figure 11 . In some example embodiments, for some purposes, the template shapes of the reference block and the current block may be defined to be the same. For example, they can both use the left template Figure 10The right template or the inverted L-shaped template shown in []. For some other purposes, the template shapes of the reference block and the current block may not need to be the same.

[0093] In some example embodiments for adaptively deriving interpolation filters in inter prediction, the reconstructed samples in the template of the current block and the template of the reference block of the current block can be used to adaptively derive one or more interpolation filters, which are used to generate the predicted sample values in the prediction block (especially at fractional positions). Then, the residual value of the current block is generated using the prediction block with the predicted samples for encoding. The way of deriving the interpolation filter can be defined such that the encoder and the decoder follow the same process. In this way, the explicit signaling of the interpolation filter in the bitstream can be reduced or avoided.

[0094] In some example embodiments, the embodiments for adaptively deriving interpolation filters in inter prediction described above and further below can be applied only to specific coding modes. For example, these methods can be applied only to the prediction mode using translational motion, that is, all samples in the current block use the same BV / MV. For example, for the general warped inter prediction mode, adaptive interpolation filter derivation may not be adopted. In such an embodiment, the decoder can respond to a syntax element in the bitstream indicating the inter prediction mode of the block to which the adaptive interpolation filter is applied, and then generate an interpolation filter from the templates of the current block and the reference block. Otherwise, the decoder will extract information about the interpolation filter from the signaling in the bitstream, or use the method in the prediction mode for inherent fractions (such as the warped mode).

[0095] In some example embodiments, whether to adopt the adaptive interpolation filter derivation method described above and further below can be written into the high-level syntax flags in the bitstream, including but not limited to the flags at the sequence level / picture level / sub-picture level / slice level / tile level. Therefore, the encoder can determine whether to adopt such adaptive interpolation filtering and include syntax elements in the bitstream to indicate whether this method is adopted at any one of the various signaling levels. The decoder will then receive this syntax element and determine whether to adopt adaptive interpolation filtering. If so, the decoder will derive the interpolation filter from the templates of the current block and the reference block. Otherwise, the decoder will extract information about the interpolation filter from the signaling in the bitstream.

[0096] The adaptive interpolation filter derivation method described above and in detail below is applicable to both inter and intra block-level prediction. In other words, the reference block can be in the same picture as the current block and located by BV. Or, the reference block can be in a different picture and located by MV, which indicates inter prediction.

[0097] In some example embodiments, deriving one or more interpolation filters based on the template of the current block and the template of the reference block (specified by the BV or MV of the current block) may involve an optimization process to derive the interpolation filter by finding an adaptive interpolation filter that, when applied to the template of the reference block, generates a reference template with the least difference from the template of the current block (presumably at the fractional pixel neighboring positions of the reference block). This optimization process may involve, for example, the least mean square method. Thus, an exemplary least mean square optimization procedure may be applied to derive the interpolation filter coefficients using the templates of the reference block and the current block, which can provide the least distortion between the template of the current block and the template interpolated from the integer template of the reference block. The interpolation filter may be initialized and then optimized in multiple iterations to minimize the above difference.

[0098] In the above embodiments, the MV or BV relative to the template of the current block may be used to locate the template of the reference block. The samples for interpolation in the integer positions near and around the fractional template position (when the BV or MV is fractional) may be determined by the filter shape. The above example embodiments assume that the interpolation filter derived based on the block template will be able to accurately represent, to some extent, the interpolation filter applicable near the samples of the reference block at fractional pixel positions.

[0099] In some example embodiments, in the above optimization process or other optimization processes for deriving an adaptive interpolation filter, the distortion between the interpolated reference template samples and the current template samples may be measured by predefined metrics, including but not limited to sum of absolute differences (SAD), sum of squared differences (SSD), sum of squared errors (SSE), sum of absolute transform domain differences (SATD).

[0100] In some example embodiments, when the BV or MV is fractional, it points to a fractional position. The integer positions near the fractional position pointed to by the BV and MV may be used as the reference block. Thus, the reference block may be associated with known reconstructed samples. The reference block may be determined relative to the fractional position using a predetermined rule. For example, the integer position of the reference block may be the closest to the fractional position. Another example is that the reference block may be the block at the next integer position starting from the fractional position along a certain predetermined direction. Finally, the reference block is interpolated by the interpolation filter to generate predicted samples.

[0101] In some example embodiments, a set of candidate interpolation filters may be predefined for adaptive selection, and the interpolation filter that minimizes the distortion between the current block template and the template obtained by interpolating from the template of the reference block may be selected as the interpolation filter for generating prediction samples. In such embodiments, the computational burden on the decoder may be alleviated without writing an interpolation filter selection signal to the bitstream, while still providing a degree of adaptability. In these example embodiments, the distortion may be measured, for example, by predefined metrics including but not limited to SAD, SSE, and SATD.

[0102] In some example embodiments, M interpolation filters may be predefined for adaptive selection. These predefined filters may be applied to the reference template to match with the current template, and based on the distortion of the template, N (where N is less than or equal to M) more matching interpolation filters may be selected from the M predefined interpolation filters. These N interpolation filters may be sorted according to the distortion (e.g., the smaller the distortion, the higher the ranking). For M equal to N, the interpolation filters only need to be reordered. An index may be explicitly written into the bitstream to indicate which one of the N interpolation filters is finally used for the current block. Thus, both the encoder and the decoder are aware of these M predefined interpolation filters and follow the same distortion calculation process to adaptively select and sort N interpolation filters from the M predefined interpolation filters. The encoder may further select an interpolation filter from the N sorted interpolation filters and write its index into the bitstream. The decoder may correspondingly extract the index from the bitstream and find the selected interpolation filter from the N interpolation filters through its own distortion calculation process, and then apply the selected interpolation filter to generate a prediction block with prediction samples. In these example embodiments, by signaling using a smaller index in the bitstream, the computational burden on the decoder may be reduced to a certain extent while still providing a degree of adaptability. These embodiments also enable the encoder to overcome a part of the inaccuracy that may exist when using an interpolation filter derived from a template to approximate the interpolation filter applied to the reference block by selecting from the N interpolation filters based on the best performance of the actual reference block compared to the current block. This process of selecting from the N interpolation filters may be performed by the encoder. Although the decoder cannot perform this selection (because the current block has not been reconstructed), the decoder does not need to perform this selection evaluation because the interpolation filter selected from the N interpolation filters is written into the bitstream. In these example embodiments, the distortion may be measured, for example, by predefined metrics including but not limited to SAD, SSE, and SATD.

[0103] In any of the above example embodiments and the embodiments described in detail below, when an interpolation filter is applied to generate a reference template (referred to as template_fractional) at a fractional position for comparison with the current template to determine distortion, the template of the reference block to which the interpolation filter is applied includes templates at multiple integer positions on the reference block, referred to as template_integer.

[0104] In some example embodiments, the derivation of the adaptive interpolation filter in the above or other embodiments further described below may be selectively applied. For example, adaptive interpolation filtering may be applied to certain MV or BV values, i.e., when the BV and / or MV points to one of a set of predefined integer and / or fractional positions, adaptive interpolation filtering may be applied. For BV or MV pointing to other values, a fixed or explicitly signaled interpolation filter may be applied. For example, for BV or MV values that work well for fixed or predefined interpolation filter statistics, the above adaptive interpolation filter derivation may not need to be forced. In this way, the decoder only needs to perform adaptive distortion calculation when the BV or MV belongs to the predefined value group, which provides a significant coding gain statistically. In such an example embodiment, the decoder is capable of determining whether to perform the adaptive derivation or selection of the interpolation filter, or use a fixed or explicitly signaled interpolation filter, based on the BV and MV values.

[0105] For example, the above adaptive interpolation filtering may be applied only when the BV and / or MV points to a fractional position rather than an integer position.

[0106] For another example, the above adaptive interpolation filtering may be applied only when the BV and / or MV points to a 1 / 2 pixel position.

[0107] For another example, the above adaptive interpolation filtering may be applied only when the BV and / or MV points to a 1 / 4 pixel position.

[0108] For another example, the above adaptive interpolation filtering may be applied only when the BV and / or MV points to a 1 / 8 pixel position.

[0109] For another example, the above adaptive interpolation filtering may be applied only when the BV and / or MV points to a 1 / 16 pixel position.

[0110] For another example, the above adaptive interpolation filtering may be applied only when the BV and / or MV points to a 1 / 32 pixel position.

[0111] Another example is that the above adaptive interpolation filtering may be applied only when the BV and / or MV points to any of the above fractional positions or smaller fractional positions.

[0112] Another example is that the above adaptive interpolation filtering may be applied only when the BV and / or MV points to any set of the above fractional positions.

[0113] Regarding the interpolation filter, in some example embodiments, when adaptive interpolation filtering as described above and in other embodiments further described below is employed, a one-dimensional or two-dimensional interpolation filter may be used. In other words, the input to the interpolation filter involves multiple rows and / or multiple columns of reference samples.

[0114] For example, when BV / MV points to a fractional horizontal coordinate and an integer vertical coordinate, a one-dimensional interpolation filter is adaptively derived (when adaptive interpolation is employed) and applied to the reference samples to generate predicted samples.

[0115] As another example, when BV / MV points to a fractional vertical coordinate and an integer horizontal coordinate, a one-dimensional interpolation filter is adaptively derived (when adaptive interpolation is employed) and applied to the reference samples to generate predicted samples.

[0116] As yet another example, when BV / MV points to a fractional vertical coordinate and a fractional horizontal coordinate, a two-dimensional interpolation filter is adaptively derived (when adaptive interpolation is employed) and applied to the reference samples to generate predicted samples. The two-dimensional interpolation filter can be a non-separable filter applied simultaneously in two dimensions or a separable filter applied sequentially.

[0117] In some example embodiments, when adaptive interpolation filtering as described above and in other embodiments further described below is employed, one (or more) horizontal interpolation filters and one (or more) vertical interpolation filters may be derived separately.

[0118] For example, when adaptive interpolation filtering as described above and in other embodiments further described below is employed and BV / MV points to a fractional vertical coordinate, vertical interpolation filtering is performed using the vertically interpolated filter derived separately as described above.

[0119] As another example, when adaptive interpolation filtering as described above and in other embodiments further described below is employed and BV / MV points to a fractional horizontal coordinate, horizontal interpolation filtering is performed using the horizontally interpolated filter derived separately as described above.

[0120] As yet another example, when a vertical interpolation filter is adaptively and separately derived, a top or left template may be used, or both the top and left templates may be used simultaneously, as Figure 10 shown as an example.

[0121] As yet another example, when a horizontal interpolation filter is adaptively and separately derived, a top or left template may be used, or both the top and left templates may be used simultaneously, also as Figure 10 shown as an example.

[0122] Regarding the shape of the interpolation filter, in some example embodiments, the number of filter taps of an interpolation filter using adaptively derived interpolation filter may be different from that of an interpolation filter when adaptive interpolation filter derivation is not applied (e.g., using a predefined or hard-coded fixed interpolation filter or an interpolation filter explicitly signaled). For example, for an adaptively derived filter, the number of filter taps may be further adaptive as it may depend on, for example, the size of the coding block.

[0123] In some example embodiments, the selection of a template for adaptive derivation of an interpolation filter may adaptively depend on the encoded information or any other information available to both the encoder and the decoder. This dependency may be predefined and known to both the encoder and the decoder so as to use a consistent template when deriving the adaptive interpolation filter on both sides. Such encoded information or other information may be known to both the encoder and the decoder. Such encoded information or other information may include, but is not limited to, the relative position of the block within the picture (e.g., whether the block is located at the top, right, bottom, and left boundaries of the picture), the relative position of the block within the associated largest coding block (e.g., whether the block is located at the top, right, bottom, and left boundaries of the associated largest coding block). Additionally, or alternatively, such encoded information and other information may include, but is not limited to, information related to neighboring blocks of the current block. Or, it may also include information related to up to T non-adjacent columns / rows of the current block to adaptively determine the selection of the template.

[0124] For example, the adaptive selection of the above template may involve selecting different template shapes, such as a left template, a top template, and an inverted L-shaped template (as Figure 10 shown), or the size of the template (e.g., the number of adjacent rows).

[0125] Figure 12 An example logic flow 1000 according to the above embodiments is shown. The logic flow 1200 starts at S1201. In S1210, a vector (motion vector (MV) or block vector (BV)) of a current video block is received from a video bitstream, and the vector identifies a reference block of the current block. In S1220, a first set of reconstructed samples in a first neighboring template of the current video block is determined. In S1230, a second set of reconstructed samples in a second neighboring template of the reference video block is determined. In S1240, it is determined whether to apply at least one adaptive interpolation filter to the current video block. In S1250, in response to determining that an adaptive interpolation filter is applied: at least one adaptive interpolation filter is derived based on the first set of reconstructed samples and the second set of reconstructed samples; and samples in the current block are predicted by applying at least the derived at least one adaptive interpolation filter to the reference block. The logic flow 1000 ends at S1299.

[0126] The above operations can be combined or arranged in any number or order as needed. Two or more steps and / or operations can be performed in parallel. The embodiments and implementations in the present disclosure can be used alone or in any combination in any order. In addition, each method (or embodiment), encoder, and decoder can be implemented by a processing circuit (such as one or more processors or one or more integrated circuits). In one example, one or more processors execute a program stored in a non-transitory computer-readable medium. The embodiments in the present disclosure can be applied to luminance blocks or chrominance blocks. The term "block" can be interpreted as a prediction block, a coding block, or a coding unit, i.e., a CU. The term "block" in this document can also be used to refer to a transform block. In the following items, when referring to the block size, it can refer to the width or height of the block, or the maximum value of the width and height, or the minimum value of the width and height, or the area size (width * height), or the aspect ratio of the block (width: height, or height: width).

[0127] The above techniques can be implemented as computer software that uses computer-readable instructions and is physically stored in one or more computer-readable media. For example, Figure 11 FIG. shows a computer system (1300) suitable for implementing certain embodiments of the disclosed subject matter.

[0128] The computer software can be encoded using any suitable machine code or computer language, and any suitable machine code or computer language can be subject to mechanisms such as assembly, compilation, linking, or the like to create code including instructions that can be directly executed by one or more computer central processing units (CPUs), graphics processing units (GPUs), etc., or executed through interpretation, microcode execution, etc.

[0129] The instructions can be executed on various types of computers or their components, such as including personal computers, tablet computers, servers, smart phones, gaming devices, Internet of Things devices, etc.

[0130] Figure 13 The components of the illustrated computer system (1300) are exemplary in nature and are not intended to impose any limitations on the scope of use or functionality of the computer software implementing the embodiments of the present disclosure. The configuration of the components should also not be construed as having any dependencies or requirements related to any one component or combination of components shown in the exemplary embodiments of the computer system (1300).

[0131] The computer system (1300) may include certain human-machine interface input devices. The input human-machine interface devices may include one or more of the following (only one of each is shown): keyboard (1301), mouse (1302), touchpad (1303), touch screen (1310), data glove (not shown), joystick (1305), microphone (1306), scanner (1307), camera (1308).

[0132] The computer system (1300) may also include certain human-machine interface output devices. Such human-machine interface output devices can stimulate the senses of one or more human users, for example, through tactile output, sound, light, and smell / taste. Such human-machine interface output devices may include tactile output devices (such as the tactile feedback of the touch screen (1310), data glove (not shown), or joystick (1305), but may also be tactile feedback devices that are not input devices), audio output devices (such as: speakers (1309), headphones (not depicted)), visual output devices (such as the screen (1310) including CRT screens, LCD screens, plasma screens, OLED screens, each screen having or not having touch screen input function, each screen having or not having tactile feedback function - some of these screens are capable of outputting two-dimensional visual output or output beyond three dimensions through devices such as stereoscopic image output, virtual reality glasses (not depicted), holographic displays, and smoke boxes (not depicted)), and printers (not depicted).

[0133] The computer system (1300) may also include human-accessible storage devices and their associated media, for example, including optical media such as CD / DVD ROM / RW (1320) with media such as CD / DVD (1321), thumb drives (1322), removable hard disk drives or solid state drives (1323), traditional magnetic media such as tapes and floppy disks (not shown), devices based on dedicated ROM / ASIC / PLD such as security dongles (not shown), etc.

[0134] Those skilled in the art should also understand that the term "computer-readable medium" used in connection with the currently disclosed subject matter does not cover transmission media, carrier waves, or other transient signals.

[0135] The computer system (1300) may further include an interface (1354) connected to one or more communication networks (1355). The network may be, for example, a wireless network, a wired network, or an optical network. The network may further be a local network, a wide area network, a metropolitan area network, a vehicle and industrial network, a real-time network, a delay-tolerant network, etc. Examples of networks include local area networks such as Ethernet, wireless LANs, cellular networks including GSM, 3G, 4G, 5G, LTE, etc., television wired or wireless wide area digital networks including cable television, satellite television, and terrestrial broadcast television, vehicle and industrial networks including CANBus, and so on.

[0136] The above-mentioned human-machine interface device, human-accessible storage device, and network interface may be attached to the kernel (1340) of the computer system (1300).

[0137] The kernel (1340) may include one or more central processing units (CPUs) (1341), a graphics processing unit (GPU) (1342), a dedicated programmable processing unit in the form of a Field Programmable Gate Area (FPGA) (1343), a hardware accelerator (1344) for certain tasks, a graphics adapter (1350), etc. These devices, as well as a read-only memory (ROM) (1345), a random access memory (1346), and an internal mass storage such as an internal non-user-accessible hard disk drive, SSD, etc. (1347), may be connected through a system bus (1348). In some computer systems, the system bus (1348) may be accessed in the form of one or more physical plugs to enable expansion through additional CPUs, GPUs, etc. Peripheral devices may be directly connected to the system bus (1348) of the kernel or connected to the system bus (1348) of the kernel through a peripheral bus (1349). For example, a screen (1310) may be connected to the graphics adapter (1350). The architecture of the peripheral bus includes PCI, USB, etc.

[0138] A computer-readable medium may have computer code thereon for performing various computer-implemented operations. The medium and the computer code may be media and computer code specifically designed and constructed for the purposes of this disclosure, or the medium and the computer code may be of the type well-known and available to those skilled in the field of computer software.

[0139] Although the present disclosure has described multiple exemplary embodiments, there are modifications, transformations, and various equivalent substitutions that fall within the scope of the present disclosure. Therefore, it should be understood that those skilled in the art will be able to design many systems and methods that, although not explicitly shown or described herein, embody the principles of the present disclosure and thus fall within the spirit and scope of the present disclosure.

Claims

1. A method for decoding a current block in a video bitstream, the method comprising: Receiving a vector from the video bitstream, the vector being used to identify a reference block of the current block; Determining a first set of reconstructed samples in a first neighboring template of the current block; Determining a second set of reconstructed samples in a second neighboring template of the reference block; Determining whether at least one adaptive interpolation filter is applied to the current block; And In response to applying the at least one adaptive interpolation filter: Deriving the at least one adaptive interpolation filter based on the first set of reconstructed samples and the second set of reconstructed samples; And Predicting samples in the current block by applying the derived at least one adaptive interpolation filter to the reference block at least.

2. The method according to claim 1, wherein: The vector points to a fractional pixel position; and The reference block includes reference samples at an integer pixel position closest to the fractional pixel position.

3. The method according to claim 1, wherein Determining whether the at least one adaptive interpolation filter is applied to the current block includes: Receiving a syntax element from the video bitstream, the syntax element indicating an inter prediction mode of the current block; and Determining that the at least one adaptive interpolation filter is applied to the current block only when the inter prediction mode of the current block is one of the translational inter prediction models.

4. The method according to claim 1, wherein Determining whether the at least one adaptive interpolation filter is applied to the current block includes: receiving a high-level syntax flag at sequence level, picture level, sub-picture level, slice level or tile level, and determining whether the at least one adaptive interpolation filter is applied based on the high-level syntax flag.

5. The method according to any one of claims 1 to 4, wherein The second neighboring template has the same shape and the same size as the first neighboring template, and the relative position of the second neighboring template with respect to the reference block is the same as the relative position of the first neighboring template with respect to the current block.

6. The method according to claim 5, wherein, Deriving the at least one adaptive interpolation filter includes: deriving the at least one adaptive interpolation filter by minimizing a distortion between the first set of reconstructed samples and an interpolation of the second set of reconstructed samples by the at least one adaptive interpolation filter.

7. The method according to claim 6, wherein The distortion is quantified by sum of absolute differences SAD, sum of squared differences SSD, sum of squared errors SSE or sum of absolute transform domain differences SATD.

8. The method according to claim 6, wherein, Deriving the at least one adaptive interpolation filter includes: Identifying a plurality of predetermined interpolation filters; Applying each of the plurality of predetermined interpolation filters to the second set of reconstructed samples; and Selecting one of the plurality of predetermined interpolation filters that causes the least distortion compared to the first set of reconstructed samples when applied to the second set of reconstructed samples.

9. The method according to claim 8, wherein The distortion is quantified by sum of absolute differences SAD, sum of squared differences SSD, sum of squared errors SSE or sum of absolute transform domain differences SATD.

10. The method according to claim 6, wherein, Deriving the at least one adaptive interpolation filter includes: Identifying a plurality of predetermined interpolation filters; Applying each of the plurality of predetermined interpolation filters to the second set of reconstructed samples; From the plurality of predetermined interpolation filters, select a subset of the predetermined interpolation filters, wherein, when the subset of the predetermined interpolation filters is applied to the second set of reconstructed samples, the distortion caused is less than that caused by other predetermined interpolation filters compared to the first set of reconstructed samples; Sort within the subset of the predetermined interpolation filters; Receive an index represented by a signal from the video bitstream; and Select the at least one adaptive interpolation filter in sequence from the subset of the predetermined interpolation filters according to the index represented by the signal.

11. The method according to any one of claims 1 to 4, wherein, Determining whether the at least one adaptive interpolation filter is applied to the current block includes: Determining whether the vector includes a vector value falling within a predetermined value set; and When the vector value falls within the predetermined value set, determining that the at least one adaptive interpolation filter is applied to the current block.

12. The method according to claim 11, wherein, The predetermined value set includes fractional pixel values.

13. The method according to any one of claims 1 to 4, wherein The at least one adaptive interpolation filter includes a two-dimensional non-separable filter or separate one-dimensional interpolation filters.

14. The method according to any one of claims 1 to 4, wherein The vector includes a motion vector or a block vector.

15. The method according to any one of claims 1 to 4, wherein The vector points to a fractional pixel position in one or both of the vertical and horizontal directions, and the at least one adaptive interpolation filter is only applied to the dimension with a fractional vector.

16. The method according to any one of claims 1 to 4, wherein The method further includes: in response to determining that the adaptive interpolation filter is not applied, applying a predetermined or explicitly signaled interpolation filter to the reference block to generate prediction samples of the current block.

17. The method according to claim 16, wherein, The number of filter taps of the at least one adaptive interpolation filter is different from the number of taps of the predetermined or explicitly signaled interpolation filter.

18. The method according to any one of claims 1 to 4, wherein Adaptive determine the number of filter taps of the at least one adaptive interpolation filter according to the size of the current block.

19. The method according to any one of claims 1 to 4, wherein The shapes and / or positions of the first adjacent template and the second adjacent template depend on the encoded information available to both the encoder and the decoder, and the encoded information includes at least one of the following: The relative position of the current block within the current picture; The relative position of the current block within the associated maximum coded block; or The encoded information associated with one or more adjacent blocks of the current block.

20. An electronic device, including a memory for storing instructions and a processor, the processor is configured to execute the instructions to perform the following operations: Receive from the video bitstream a vector identifying a reference block of the current block; Determine a first set of reconstructed samples in a first adjacent template of the current block; Determine a second set of reconstructed samples in a second adjacent template of the reference block; Determine whether to apply at least one adaptive interpolation filter to the current block; And In response to applying the at least one adaptive interpolation filter: Derive the at least one adaptive interpolation filter based on the first set of reconstructed samples and the second set of reconstructed samples; And Predict samples in the current block by at least applying the derived at least one adaptive interpolation filter to the reference block.

Images