System and method for switching interpolation filters - Patents.com

Abstract

To provide a system, a method and a device for adaptively switching interpolation filters during encoding of video data or decoding of a video bitstream.SOLUTION: A method for switching interpolation filters during a decoding process acquires an encoded video bitstream about a storage position and / or through a network, determines a reference frame that temporally precedes or follows the current frame for a coding unit in the current frame from the encoded video bitstream, and reconfigures a coding unit by using the reference frame and an interpolation filter. The interpolation filter is selected from a subset of interpolation filters from a set of the interpolation filters, and the subset of interpolation filters is determined on the basis of a coding level of the coding unit, a prediction mode for the coding unit, a result of motion estimation and a coding tool.SELECTED DRAWING: Figure 8

Description

[Technical field]

[0001]

[0001] This application relates to switching the interpolation filters to be used in the motion estimation (ME) and motion compensation (MC) processes in the context of advanced video codecs, including codes being developed in the Joint Exploration Model (JEM). For example, the proposed method improves the performance of inter-layer prediction by employing multiple interpolation filters in an adaptive manner. [Background technology]

[0002]

[0002] Video coding standards include ITU-T H.261, ISO / IEC MPEG-1 Visual, ITU-T H.262 or ISO / IEC MPEG-2 Visual, ITU-T H.263, ISO / IEC MPEG-4 Visual, and ITU-T H.264 (also known as ISO / IEC MPEG-4 AVC), including its Scalable Video Coding (SVC) and Multiview Video Coding (MVC) extensions.

[0003]

[0003] In addition, a new video coding standard, namely High Efficiency Video Coding (HEVC) or ITU-T H.265, including its range and screen content coding extensions, 3D Video Coding (3D-HEVC) Scalable Extensions (SHVC) and Multi-view Extensions (MV-HEVC), has been developed in recent years by the Joint Study Group on Video Coding (JCT-VC) and the Joint Study Group on 3D Video Coding Extensions Development (JCT-3V) with the ITU-T Video Coding Expert Group (VCEG) and the ISO / IEC Motion Picture Expert Group (MPEG).

[0004]

[0004] In 2016, MPEG and ITU-T VCEG established the joint exploration video team (JVET) to investigate new coding tools for the next generation video coding standard. The reference software is called the joint exploration model (JEM). Summary of the Invention

[0005]

[0005] Video compression techniques perform spatial and temporal prediction to reduce or remove redundancy inherent in the input video signal. To reduce temporal redundancy (i.e., visual similarity in adjacent frames), motion estimation is performed to track the movement of objects in the video. Motion vectors may indicate displacements in pixel distances. In some cases, motion vectors may have precision greater than integer pixel precision, such as 1 / 2 pixel, 1 / 4 pixel, or 1 / 16 pixel distances. Finer precision may enable a video coder to track the motion field more accurately and thus obtain better predictions.

[0006] According to at least one example, a method of encoding video data is provided that includes obtaining video data. The method further includes determining a subset of interpolation filters from a set of interpolation filters for a coding unit, where the subset of interpolation filters is determined based on information in the video data associated with the coding unit. The method further includes encoding the coding unit, where encoding the coding unit includes selecting an interpolation filter for motion estimation and motion compensation for the coding unit, where the interpolation filter is selected from the subset of interpolation filters. The method further includes generating an encoded video bitstream, where the encoded video bitstream includes the encoded coding unit.

[0007]

[0007] In another example, an apparatus is provided that includes a memory configured to store video data and a processor. The processor is configured to obtain the video data. The processor is configured to determine a subset of interpolation filters from a set of interpolation filters for a coding unit, where the subset of interpolation filters is determined based on information in the video data associated with the coding unit. The processor is configured to encode the coding unit, where the encoding process includes selecting an interpolation filter for motion estimation and motion compensation, where the interpolation filter is selected from the subset of interpolation filters. The processor is configured to generate an encoded video bitstream, where the encoded video bitstream includes the encoded coding unit.

[0008]

[0008] In another example, a computer-readable medium is provided having stored thereon instructions that, when executed by a processor, perform a method including obtaining video data. The method further includes determining a subset of interpolation filters from the set of interpolation filters for a coding unit, where the subset of interpolation filters is determined based on information in the video data associated with the coding unit. The method further includes encoding the coding unit, where encoding the coding unit includes selecting an interpolation filter for motion estimation and motion compensation for the coding unit, where the interpolation filter is selected from the set of interpolation filters. The method further includes generating an encoded video bitstream, where the encoded video bitstream includes the encoded coding unit.

[0009]

[0009] In another example, an apparatus is provided that includes means for obtaining video data. The apparatus further includes means for determining, for a coding unit, a subset of interpolation filters from a set of interpolation filters, where the subset of interpolation filters is determined based on information in the video data associated with the coding unit. The apparatus further includes means for encoding the coding unit, where encoding the coding unit includes selecting an interpolation filter for motion estimation and motion compensation for the coding unit, where the interpolation filter is selected from the subset of interpolation filters. The apparatus further includes means for generating an encoded video bitstream, where the encoded video bitstream includes the encoded coding unit.

[0010]

[0010] In some aspects, the methods, apparatus, and computer-readable media described above further comprise determining a subset of interpolation filters from the set of interpolation filters, wherein the subset of interpolation filters is determined based on a coding level of the coding unit, and wherein the interpolation filter is determined from the subset of interpolation filters.

[0011]

[0011] In some aspects, the methods, apparatus, and computer-readable media described above further comprise determining a subset of interpolation filters from the set of interpolation filters, wherein the subset of interpolation filters is determined based on a prediction mode for the coding unit, and wherein the interpolation filter is determined from the subset of interpolation filters.

[0012]

[0012] In some aspects, the methods, apparatus, and computer-readable media described above further comprise determining a subset of interpolation filters from the set of interpolation filters, wherein the subset of interpolation filters is determined based on motion information, and wherein the interpolation filter is determined from the subset of interpolation filters.

[0013]

[0013] In some aspects, the methods, apparatus, and computer-readable media described above further comprise determining a subset of interpolation filters from the set of interpolation filters, wherein the subset of interpolation filters is determined based on a coding tool used to code the coding unit, and wherein the interpolation filter is determined from the subset of interpolation filters.

[0014] In some aspects, the interpolation filters are selected from previously coded blocks. In some aspects, the interpolation filters are selected from neighboring blocks of the coding unit.

[0015]

[0015] In some aspects, the methods, apparatus, and computer-readable media described above further comprise merging the interpolation filter with a second interpolation filter, wherein the resulting merged interpolation filter is used for motion estimation.

[0016]

[0016] In some aspects, the interpolation filter is associated with a second interpolation filter, where the interpolation filter is for integer pixel positions and where the second interpolation filter is for fractional pixel positions.

[0017] In some aspects, the interpolation filter and the second interpolation filter have different cut-off frequencies.

[0018] In some aspects, the encoded video bitstream includes a value that identifies an interpolation filter.

[0019] In some aspects, the set of interpolation filters is not signaled along with the encoded video bitstream, where the identity of the interpolation filters may be implied from the encoded video bitstream.

[0020]

[0020] In some aspects, a device as described above may include a camera for capturing a picture.

[0021] According to at least one example, a method of decoding video data is provided that includes obtaining an encoded video bitstream. The method further includes determining a reference frame for a coding unit in a current frame from the encoded video bitstream. The method further includes determining a subset of interpolation filters from a set of interpolation filters, where the subset of interpolation filters is determined based on information in the encoded video bitstream associated with the coding unit. The method further includes determining an interpolation filter from the subset of interpolation filters. The method further includes using the reference frame and the interpolation filter to reconstruct the coding unit.

[0022]

[0022] In another example, an apparatus is provided that includes a memory configured to store video data and a processor. The processor is configured to obtain an encoded video bitstream. The processor is configured to determine a reference frame for a coding unit in a current frame from the encoded video bitstream. The processor is further configured to determine a subset of interpolation filters from a set of interpolation filters, where the subset of interpolation filters is determined based on information in the encoded video bitstream associated with the coding unit. The processor is configured to determine an interpolation filter from the subset of interpolation filters. The processor is configured to use the reference frame and the interpolation filters to reconstruct the coding unit.

[0023]

[0023] In another example, a computer-readable medium is provided having stored thereon instructions that, when executed by a processor, perform a method including obtaining an encoded video bitstream. The method further includes determining a reference frame for a coding unit in a current frame from the encoded video bitstream. The method further includes determining a subset of interpolation filters from a set of interpolation filters, where the subset of interpolation filters is determined based on information in the encoded video bitstream associated with the coding unit. The method further includes determining an interpolation filter from the set of interpolation filters. The method further includes using the reference frame and the interpolation filter to reconstruct the coding unit.

[0024]

[0024] In another example, an apparatus is provided that includes means for obtaining an encoded video bitstream. The apparatus further comprises means for determining a reference frame for a coding unit in a current frame from the encoded video bitstream. The apparatus further comprises means for determining a subset of interpolation filters from a set of interpolation filters, where the subset of interpolation filters is determined based on information in the encoded video bitstream associated with the coding unit. The apparatus further comprises means for determining an interpolation filter from the subset of interpolation filters. The apparatus further comprises means for using the reference frame and the interpolation filters to reconstruct the coding unit.

[0025]

[0025] In some aspects, the methods, apparatus, and computer-readable media described above further comprise determining a subset of interpolation filters from the set of interpolation filters, wherein the subset of interpolation filters is determined based on a coding level of the coding unit, and wherein the interpolation filter is determined from the subset of interpolation filters.

[0026]

[0026] In some aspects, the methods, apparatus, and computer-readable media described above further comprise determining a subset of interpolation filters from the set of interpolation filters, wherein the subset of interpolation filters is determined based on a prediction mode for the coding unit, and wherein the interpolation filter is determined from the subset of interpolation filters.

[0027]

[0027] In some aspects, the methods, apparatus, and computer-readable media described above further comprise determining a subset of interpolation filters from the set of interpolation filters, wherein the subset of interpolation filters is determined based on motion information, and wherein the interpolation filter is determined from the subset of interpolation filters.

[0028]

[0028] In some aspects, the methods, apparatus, and computer-readable media described above further comprise determining a subset of interpolation filters from the set of interpolation filters, wherein the subset of interpolation filters is determined based on a coding tool used to code the coding unit, and wherein the interpolation filter is determined from the subset of interpolation filters.

[0029] In some aspects, the interpolation filters are selected from a previously coded block. In some aspects, the interpolation filters are selected from a neighboring block.

[0030]

[0030] In some aspects, the methods, apparatus, and computer-readable media described above further comprise merging the interpolation filter with a second interpolation filter, wherein the resulting merged interpolation filter is used to reconstruct the prediction unit.

[0031] In some aspects, the interpolation filter is associated with a second interpolation filter, where the interpolation filter is for integer pixel positions and where the second interpolation filter is for fractional pixel positions. In some aspects, the interpolation filter and the second interpolation filter have different cutoff frequencies.

[0032] In some aspects, the encoded video bitstream includes a value that identifies an interpolation filter.

[0033]

[0033] In some aspects, the methods, apparatus, and computer-readable media described above further comprise deriving an identification of the interpolation filter using the encoded video bitstream.

[0034] In some aspects, the device as described above may include a camera for capturing pictures. In some aspects, the device may include a display for displaying video data. [Brief description of the drawings]

[0035]

[0035] Exemplary embodiments of the present invention are described in detail below with reference to the following drawings: [Figure 1]

[0036] FIG. 1 is a block diagram illustrating an example of an encoding device and a decoding device. [Diagram 2]

[0037] FIG. 2 illustrates an example of the encoding process for a current block from a current frame. [Diagram 3]

[0038] FIG. 3 illustrates an example of fractional pixel positions. [Figure 4]

[0039] FIG. 4 illustrates an example of the motion estimation step of the encoding process. [Diagram 5]

[0040] FIG. 5 illustrates an example of the motion estimation step of the decoding process. [Figure 6]

[0041] FIG. 6 is a graph illustrating the frequency responses of three example filters, F0, F1, and F2. [Figure 7]

[0042] FIG. 7 illustrates an example of a process for switching interpolation filters during the encoding process. [Figure 8]

[0043] FIG. 8 is an example of a process for switching interpolation filters in a decoding process. [Figure 9]

[0044] FIG. 9 is a block diagram illustrating an example encoding device. [Figure 10]

[0045] FIG. 10 is a block diagram illustrating an example video decoding device. Detailed Description

[0036]

[0046] Certain specific aspects and embodiments of the present disclosure are provided below. As will be apparent to one skilled in the art, some of these aspects and embodiments may be applied independently, and some of them may be applied in combination. In the following description, for the purpose of explanation, specific details are set forth in order to provide a thorough understanding of the embodiments of the present invention. However, it will be apparent that various embodiments may be practiced without these specific details. The drawings and description are not intended to be limiting.

[0037]

[0047] The following description provides exemplary embodiments only and is not intended to limit the scope, applicability, or configuration of the present disclosure. Rather, the following description of exemplary embodiments will provide those skilled in the art with an enabling description for implementing the exemplary embodiments. It should be understood that various changes can be made in the arrangement and function of elements without departing from the spirit and scope of the invention as set forth in the appended claims.

[0038]

[0048] Specific details are given in the following description to provide a thorough understanding of the embodiments. However, those skilled in the art will understand that the embodiments may be practiced without these specific details. For example, circuits, systems, networks, processes, and other components may be shown as components in block diagram form so as not to obscure the embodiments with unnecessary detail. In other instances, well-known circuits, processes, algorithms, structures, and techniques may be shown without unnecessary detail so as to avoid obscuring the embodiments.

[0039]

[0049] It should also be noted that the individual embodiments may be described as a process that is depicted as a flowchart, a flow diagram, a data flow diagram, a structure diagram, or a block diagram. Although a flowchart may describe operations as a sequential process, many of the operations may be performed in parallel or concurrently. Additionally, the order of operations may be rearranged. A process terminates when its operations are completed, but may have additional steps not included in the drawings. A process may correspond to a method, a function, a procedure, a subroutine, a subprogram, and the like. When a process corresponds to a function, its termination may correspond to a return of the function to a calling function or to the main function.

[0040]

[0050] The term "computer-readable medium" includes, but is not limited to, portable or non-portable storage devices, optical storage devices, and various other media capable of storing, containing, or transporting instruction(s) and / or data. Computer-readable media may include non-transitory media on which data may be stored and does not include carrier waves and / or transitory electronic signals propagating wirelessly or over wired connections. Examples of non-transitory media may include, but are not limited to, magnetic disks or tapes, optical storage media such as compact disks (CDs) or digital versatile disks (DVDs), flash memory, memory or memory devices. A computer-readable medium may store code and / or machine-executable instructions, which may represent a procedure, a function, a subprogram, a program, a routine, a subroutine, a module, a software package, a class, or any combination of instructions, data structures, or program statements. A code segment may be coupled to another code segment or a hardware circuit by passing and / or receiving information, data, arguments, parameters, or memory contents. Information, arguments, parameters, data, etc. can be passed, forwarded, or transmitted via any suitable means including memory sharing, message passing, token passing, network transmission, or the like.

[0041]

[0051] Furthermore, the embodiments may be implemented by hardware, software, firmware, middleware, microcode, hardware description languages, or any combination thereof. When implemented in software, firmware, middleware, or microcode, the program code or code segments (e.g., computer program product) to perform the necessary tasks may be stored in a computer-readable or machine-readable medium. Processor(s) may perform the necessary tasks.

[0042]

[0052] As more devices and systems provide consumers with the ability to consume digital video data, the need for efficient video coding techniques becomes more important. Video coding is necessary to reduce the storage and transmission requirements necessary to handle the large amounts of data present in digital video data. Various video coding techniques may be used to compress video data into a form that uses a lower bit rate while maintaining high video quality. As used herein, "coding" refers to "encoding" or "decoding."

[0043]

[0053] FIG. 1 is a block diagram illustrating an example of a video coding system 100 including an encoding device 104 and a decoding device 112. The encoding device 104 may be part of a source device, and the decoding device 112 may be part of a receiving device. The source device and / or the receiving device may include electronic devices such as a mobile or fixed telephone handset (e.g., a smartphone, a cellular telephone, or the like), a desktop computer, a laptop or notebook computer, a tablet computer, a set-top box, a television, a camera, a display device, a digital media player, a video game console, a video streaming device, an Internet Protocol (IP) camera, or any other suitable electronic device. In some examples, the source device and the receiving device may include one or more wireless transceivers for wireless communication. The coding techniques described herein are applicable to video coding in various multimedia applications, including streaming video transmission (e.g., over the Internet), television broadcasting or transmission, encoding digital video for storage on a data storage medium, decoding digital video stored on a data storage medium, or other applications. In some examples, system 100 may support one-way or two-way video transmission to support applications such as video conferencing, video streaming, video playback, video broadcasting, gaming, and / or video telephony.

[0044]

[0054] The encoding device 104 (or encoder) may be used to encode the video data using a video coding standard or protocol to generate an encoded video bitstream. Examples of video coding standards include ITU-T H.261, ISO / IEC MPEG-1 Visual, ITU-T H.262 or ISO / IEC MPEG-2 Visual, ITU-T H.263, ISO / IEC MPEG-4 Visual, ITU-T H.264 (also known as ISO / IEC MPEG-4 AVC) including its Scalable Video Coding (SVC) and Multiview Video Coding (MVC) extensions, and High Efficiency Video Coding (HEVC) or ITU-T H.265. There are various extensions to HEVC that deal with multi-layer video coding, including range and screen content coding extensions, 3D video coding (3D-HEVC) and multiview extensions (MV-HEVC) and scalable extensions (SHVC). HEVC and its extensions have been developed by the Joint Study Group on Video Coding (JCT-VC) and the Joint Study Group on 3D Video Coding Extensions Development (JCT-3V) with the ITU-T Video Coding Expert Group (VCEG) and the ISO / IEC Motion Picture Expert Group (MPEG). MPEG and ITU-T VCEG also established the joint exploration video team (JVET) to investigate new coding tools for the next generation of video coding standards. The reference software is called the joint exploration model (JEM).

[0045]

[0055] Many of the examples described herein provide examples using the JEM model, the HEVC standard, and / or extensions thereof. However, the techniques and systems described herein may also be applicable to other coding standards, such as AVC, MPEG, extensions thereof, or other suitable coding standards currently in existence or future coding standards. Thus, while the techniques and systems described herein may be described with reference to a particular video coding standard, those skilled in the art will understand that the description should not be construed as applying only to that particular standard.

[0046]

[0056] 1, a video source 102 may provide video data to an encoding device 104. The video source 102 may be part of a source device or may be part of a device other than the source device. The video source 102 may include a video capture device (e.g., a video camera, a camera phone, a video phone, or the like), a video archive containing stored video, a video server or content provider providing video data, a video feed interface receiving video from a video server or content provider, a computer graphics system for generating computer graphics video data, a combination of such sources, or any other suitable video source.

[0047]

[0057] The video data from the video source 102 may include one or more input pictures or frames. A video picture or frame is a still image of a scene. The encoder engine 106 (or encoder) of the encoding device 104 encodes the video data to generate an encoded video bitstream. In some examples, the encoded video bitstream (alternatively, a "video bitstream" or "bitstream") is a series of one or more coded video sequences. A coded video sequence (CVS) includes a series of AUs starting from an access unit (AU) with certain properties and having a random access point picture in a base layer, up to and not including a next AU with certain properties and having a random access point picture in a base layer. For example, certain characteristics of a random access point picture that initiates a CVS may include a RASL flag (e.g., NoRaslOutputFlag) equal to 1. Otherwise, a random access point picture (with a RASL flag equal to 0) does not initiate a CVS. An access unit (AU) contains one or more coded pictures and control information corresponding to the coded pictures that share the same output time. Coded slices of pictures are encapsulated at the bitstream level into data units called network abstraction layer (NAL) units. For example, a HEVC video bitstream may contain one or more CVSs that contain NAL units. Each of the NAL units has a NAL unit header. In one example, the header is 1 byte for H.264 / AVC (except for multi-layer extensions) and 2 bytes for HEVC.Syntax elements in a NAL unit header take up designated bits and are therefore visible to all kinds of systems and transport layers, such as transport streams, real-time transport (RTP) protocols, file formats, among others.

[0048]

[0058] Two classes of NAL units exist in the HEVC standard, including video coding layer (VCL) NAL units and non-VCL NAL units. A VCL NAL unit contains one slice or slice segment (described below) of coded picture data, and a non-VCL NAL unit contains control information related to one or more coded pictures. In some cases, a NAL unit may be referred to as a packet. An HEVC AU contains VCL NAL units that contain coded picture data and non-VCL NAL units (if any) that correspond to the coded picture data.

[0049]

[0059] A NAL unit may include a sequence of bits (e.g., an encoded video bitstream, a CVS of a bitstream, or the like) that forms a coded representation of video data, such as a coded representation of a picture in a video. The encoder engine 106 generates the coded representation of a picture by partitioning each picture into multiple slices. The slices are independent of other slices, such that information in a slice is coded without relying on data from other slices in the same picture. A slice includes one or more slice segments, including an independent slice segment and one or more dependent slice segments that depend on previous slice segments, if any. The slice is then partitioned into coding tree blocks (CTBs) of luma samples and chroma samples. The CTB of luma samples and one or more CTBs of chroma samples, together with syntax for the samples, are referred to as coding tree units (CTUs). The CTU is the basic processing unit for HEVC encoding. The CTU may be divided into multiple coding units (CUs) of variable size. The CU includes luma and chroma sample arrays, referred to as coding blocks (CBs).

[0050]

[0060] The luma and chroma CBs may be further divided into prediction blocks (PBs). A PB is a block of luma or chroma component samples that uses the same motion parameters for inter prediction or intra block copy prediction (when available or enabled for use). The luma PB and one or more chroma PBs, together with associated syntax, form a prediction unit (PU). In the case of inter prediction, a set of motion parameters (e.g., one or more motion vectors, reference indexes, or the like) is signaled in the bitstream for each PU and used for inter prediction of the luma PB and one or more chroma PBs. The motion parameters may also be referred to as motion information. The CB may also be partitioned into one or more transform blocks (TBs). A TB represents a square block of samples of a color component to which the same two-dimensional transform is applied to code the prediction residual signal. A transform unit (TU) represents a TB of luma and chroma samples and corresponding syntax elements.

[0051]

[0061] The size of a CU corresponds to the size of a coding mode and may be square in shape. For example, the size of a CU may be 8×8 samples, 16×16 samples, 32×32 samples, 64×64 samples, or any other suitable size up to the size of the corresponding CTU. The expression "N×N" is used herein to refer to the pixel dimensions of a video block in terms of vertical and horizontal dimensions (e.g., 8 pixels×8 pixels). The pixels in a block may be arranged in rows and columns. In some examples, a block may not have the same number of pixels horizontally as vertically. Syntax data associated with a CU may, for example, describe the partitioning of the CU into one or more PUs. The partitioning mode may differ between whether the CU is intra-prediction mode coded or inter-prediction mode coded. The PUs may be partitioned to be non-square in shape. Syntax data associated with a CU may also describe the partitioning of a CU, for example, by a CTU, into one or more TUs. The TUs may be square or non-square in shape.

[0052]

[0062] According to the HEVC standard, the transform may be performed using transform units (TUs). The TUs may be different for different CUs. The TUs may be sized based on the size of the PUs in a given CU. The TUs may be the same size or smaller than the PUs. In some examples, the residual samples corresponding to a CU may be subdivided into smaller units using a quad tree structure known as a residual quad tree (RQT). Leaf nodes of the RQT may correspond to the TUs. Pixel difference values ​​associated with the TUs may be transformed to create transform coefficients. The transform coefficients may then be quantized by the encoder engine 106.

[0053]

[0063] Once a picture of video data is partitioned into CUs, the encoder engine 106 predicts each PU using a prediction mode. The prediction unit or prediction block is then subtracted from the original video data to obtain a residual (described below). For each CU, the prediction mode may be signaled inside the bitstream using syntax data. The prediction mode may include intra prediction (or intra-picture prediction) or inter prediction (or inter-picture prediction). Intra prediction exploits the correlation between spatially adjacent samples in a picture. For example, using intra prediction, each PU is predicted from neighboring image data in the same picture, for example, using DC prediction to average the PU, plane prediction to fit a plane to the PU, directional prediction to extrapolate from neighboring data, or any other suitable type of prediction. Inter prediction uses temporal correlation between pictures to derive a motion compensated prediction for a block of image samples. For example, using inter prediction, each PU is predicted using motion compensated prediction from image data in one or more reference pictures (before or after the current picture in output order). The decision whether to code a picture area using inter-picture prediction or intra-picture prediction may be made, for example, at the CU level.

[0054]

[0064] In some examples, one or more slices of a picture are assigned a slice type. The slice types include I slice, P slice, and B slice. An I slice (intra frame, independently decodable) is a slice of a picture that is coded only by intra prediction, and is therefore independently decodable, since an I slice only requires data in a frame to predict any prediction unit or prediction block of the slice. A P slice (uni-directional predicted frame) is a slice of a picture that can be coded with intra prediction and with uni-directional inter prediction. Each prediction unit or prediction block in a P slice is coded with either intra prediction or inter prediction. When inter prediction is applied, a prediction unit or prediction block is predicted only by one reference picture, and therefore the reference samples are from only one reference area of ​​one frame. A B slice (bi-directional predicted frame) is a slice of a picture that can be coded with intra prediction and with inter prediction (e.g., either bi-prediction or uni-prediction). A prediction unit or prediction block of a B slice may be bidirectionally predicted from two reference pictures, where each picture contributes one reference region, and the sample sets of the two reference regions are weighted (e.g., with equal weights or with different weights) to create a prediction signal of the bidirectionally predicted block. As described above, the slices of a picture are coded independently. In some cases, a picture may be coded as just one slice.

[0055]

[0065] A PU may include data related to a prediction process (e.g., motion parameters or other suitable data). For example, when a PU is encoded using intra prediction, the PU may include data describing an intra prediction mode for the PU. As another example, when a PU is encoded using inter prediction, the PU may include data defining a motion vector for the PU. The data defining a motion vector for a PU may describe, for example, a horizontal component (Δx) of the motion vector, a vertical component (Δy) of the motion vector, a resolution for the motion vector (e.g., integer precision, quarter-pixel precision, or eighth-pixel precision), a reference picture to which the motion vector points, a reference index, a reference picture list for the motion vector (e.g., list 0, list 1, or list C), or any combination thereof.

[0056]

[0066] The encoding device 104 may then perform transformation and quantization. For example, following prediction, the encoder engine 106 may calculate a residual value corresponding to the PU. The residual value may comprise pixel difference values ​​between a current block of pixels being coded (PU) and a predictive block (e.g., a predicted version of the current block) used to predict the current block. For example, after generating a predictive block (e.g., issuing an inter-prediction or intra-prediction), the encoder engine 106 may generate a residual block by subtracting a predictive block created by a prediction unit from the current block. The residual block includes a set of pixel difference values ​​that quantify differences between pixel values ​​of the current block and pixel values ​​of the predictive block. In some examples, the residual block may be represented in a two-dimensional block format (e.g., a two-dimensional matrix or array of pixel values). In such examples, the residual block is a two-dimensional representation of pixel values.

[0057]

[0067] Any residual data that may remain after prediction is performed is transformed using a block transform, which may be based on a discrete cosine transform, a discrete sine transform, an integer transform, a wavelet transform, other suitable transform functions, or any combination thereof. In some cases, one or more block transforms (e.g., size 32×32, 16×16, 8×8, 4×4, or the like) may be applied to the residual data in each CU. In some examples, TUs may be used for the transform and quantization process implemented by the encoder engine 106. A given CU having one or more PUs may also include one or more TUs. As described in more detail below, the residual values ​​may be transformed into transform coefficients using a block transform, and then quantized and scanned using the TUs to create serialized transform coefficients for entropy coding.

[0058]

[0068] In some examples, following intra-prediction or inter-prediction coding using a PU of a CU, the encoder engine 106 may calculate residual data for the TUs of the CU. The PU may comprise pixel data in the spatial domain (or pixel domain). The TU may comprise coefficients in the transform domain after application of a block transform. As previously described, the residual data may correspond to pixel difference values ​​between pixels of the uncoded picture and predicted values ​​corresponding to the PU. The encoder engine 106 may form a TU including the residual data for the CU and then transform the TU to create transform coefficients for the CU.

[0059]

[0069] The encoder engine 106 may perform quantization of the transform coefficients. Quantization provides further compression by quantizing the transform coefficients to reduce the amount of data used to represent the coefficients. For example, quantization may reduce the bit depth associated with some or all of the coefficients. In one example, a coefficient having an n-bit value may be truncated to an m-bit value during quantization, where n is greater than m.

[0060]

[0070] Once quantization is performed, the coded video bitstream includes the quantized transform coefficients, prediction information (e.g., prediction modes, motion vectors, block vectors, or the like), partitioning information, and any other suitable data, such as other syntax data. Different elements of the coded video bitstream may then be entropy coded by the encoder engine 106. In some examples, the encoder engine 106 may scan the quantized transform coefficients utilizing a predefined scan order to create a serialized vector that may be entropy coded. In some examples, the encoder engine 106 may perform an adaptive scan. After scanning the quantized transform coefficients to form a vector (e.g., a one-dimensional vector), the encoder engine 106 may entropy code the vector. For example, the encoder engine 106 may use context-adaptive variable length coding, context-adaptive binary arithmetic coding, syntax-based context-adaptive binary arithmetic coding, probability interval partitioned entropy coding, or another suitable entropy coding technique.

[0061]

[0071] As previously mentioned, an HEVC bitstream includes a group of NAL units, including VCL NAL units and non-VCL NAL units. The VCL NAL units include coded picture data that form the coded video bitstream. For example, a sequence of bits that form the coded video bitstream are retransmitted in a VCL NAL unit. The non-VCL NAL units may include parameter sets that have high-level information related to the coded video bitstream, in addition to other information. For example, the parameter sets may include a video parameter set (VPS), a sequence parameter set (SPS), and a picture parameter set (PPS). Examples of purposes of parameter sets include providing bitrate efficiency, error resilience, and a system layer interface. Each slice references a single active PPS, SPS, and VPS to access information that the decoding device 112 may use to decode the slice. An identifier (ID) may be coded for each parameter set, including a VPS ID, an SPS ID, and a PPS ID. The SPS includes an SPS ID and a VPS ID. The PPS includes a PPS ID. Each slice header contains a PPS ID and an SPS ID. Using the IDs, the active parameter set can be identified for a given slice.

[0062]

[0072] The PPS contains information that applies to all slices in a given picture. Hence, all slices in a picture refer to the same PPS. Slices in different pictures may also refer to the same PPS. The SPS contains information that applies to all pictures in the same coded video sequence (CVS) or bitstream. As mentioned above, a coded video sequence is a series of access units (AUs) that have certain characteristics (as described above) and start with a random access point picture in the base layer (e.g., an instantaneous decode reference (IDR) picture or a broken link access (BLA) picture, or other suitable random access point picture) up to (and including) the next access unit (AU) that has certain characteristics and has a random access point picture in the base layer (either the end of the bitstream or). The information in the SPS may not change from picture to picture in the coded video sequence. Pictures in a coded video sequence may use the same SPS. The VPS contains information that applies to all layers in a coded video sequence or bitstream. The VPS includes a syntax structure with syntax elements that apply to the entire coded video sequence. In some examples, the VPS, SPS, or PPS may be transmitted in-band with the coded bitstream. In some examples, the VPS, SPS, or PPS may be transmitted out-of-band in a transmission separate from the NAL units that contain the coded video data.

[0063]

[0073] The video bitstream may also include supplemental enhancement information (SEI) messages. For example, SEI NAL units may be part of the video bitstream. In some cases, the SEI messages may include information that is not required by the decoding process. For example, the information in the SEI messages may not be essential for a decoder to decode a video picture of the bitstream, but the decoder may use this information to improve the display or processing of the picture (e.g., the decoded output). The information in the SEI messages may be embedded metadata. In one illustrative example, the information in the SEI messages may be used by a decoder-side entity to improve the viewability of the content. In some cases, a particular application standard may specify the presence of such SEI messages in the bitstream so that quality improvements can be brought to all devices that comply with the application standard (e.g., the carriage of frame packing SEI messages for frame-compatible plano-stereoscopic 3DTV video formats, where an SEI message is carried for each frame of video, the processing of recovery point SEI messages, the use of pan-scan rectangular SEI messages in DVB, in addition to many other examples).

[0064]

[0074] An output 110 of the encoding device 104 may send the NAL units constituting the encoded video data to a decoding device 112 of a receiving device via a communication link 120. An input 114 of the decoding device 112 may receive the NAL units. The communication link 120 may include channels provided by a wireless network, a wired network, or a combination of wired and wireless networks. The wireless network may include any wireless interface or a combination of multiple wireless interfaces, and may include any suitable wireless network (e.g., the Internet or other wide area network, a packet-based network, WiFi, radio frequency (RF), UWB, WiFi-Direct, cellular, Long Term Evolution (LTE), WiMax, or the like). The wired network may include any wired interface (e.g., fiber, Ethernet, powerline Ethernet, Ethernet over coaxial cable, Digital Signal Line (DSL), or the like). Wired and / or wireless networks may be implemented using a variety of equipment, such as base stations, routers, access points, bridges, gateways, switches, or the like. The encoded video data may be modulated according to a communication standard, such as a wireless communication protocol, and transmitted to a receiving device.

[0065]

[0075] In some examples, encoding device 104 may store the encoded video data in storage device 108. Output 110 may take the encoded video data from encoder engine 106 or from storage device 108. Storage device 108 may include any of a variety of distributed or locally accessed data storage media. For example, storage device 108 may include a hard drive, a storage disk, flash memory, volatile or non-volatile memory, or any other suitable digital storage medium for storing the encoded video data.

[0066]

[0076] An input 114 of the decoding device 112 may receive the encoded video bitstream data and provide the video bitstream data to the decoder engine 116 or to a storage device 118 for later use by the decoder engine 116. The decoder engine 116 may decode the encoded video bitstream data by entropy decoding (e.g., using an entropy decoder) and extracting elements of one or more coded video sequences that make up the encoded video data. The decoder engine 116 may then rescale the encoded video bitstream data and perform an inverse transform thereon. The residual data is then passed to a prediction stage of the decoder engine 116. The decoder engine 116 then predicts a block of pixels (e.g., a PU). In some examples, the prediction is added to the output of the inverse transform (the residual data).

[0067]

[0077] The decoding device 112 may output the decoded video to a video destination device, which may include a display or other output device for displaying the decoded video data to a content consumer. In some aspects, the video destination device 122 may be part of a receiving device that includes the decoding device 112. In some aspects, the video destination device 122 may be part of a separate device other than the receiving device.

[0068]

[0078] In some examples, the video encoding device 104 and / or the video decoding device 112 may be integrated with an audio encoding device and an audio decoding device, respectively. The video encoding device 104 and / or the video decoding device 112 may also include other hardware or software necessary to implement the coding techniques described above, such as one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), discrete logic, software, hardware, firmware, or any combination thereof. The video encoding device 104 and the video decoding device 112 may be integrated as part of a combined encoder / decoder (codec) in the respective devices. An example of specific details of the encoding device 104 is described below with reference to FIG. 9. An example of specific details of the decoding device 112 is described below with reference to FIG. 10.

[0069]

[0079] Extensions to the HEVC standard include a multiview video coding extension, called MV-HEVC, and a scalable video coding extension, called SHVC. MV-HEVC and SHVC extensions share the concept of layered coding, with different layers being included in the coded video bitstream. Each layer in a coded video sequence is addressed by a unique layer identifier (ID). A layer ID may be present in the header of a NAL unit to identify the layer with which the NAL unit is associated. In MV-HEVC, different layers typically represent different views of the same scene in a video bitstream. In SHVC, different scalable layers are provided that represent the video bitstream at different spatial resolutions (or picture resolutions) or with different reconstruction fidelity. The scalable layers may include a base layer (with layer ID=0) and one or more enhancement layers (with layer ID=1, 2, ... n). The base layer may conform to the HEVC first version profile and represent the lowest available layer in the bitstream. An enhancement layer has increased spatial resolution, temporal resolution or frame rate, and / or reconstruction fidelity (or quality) compared to a base layer. Enhancement layers may be hierarchically organized and may (or may not) depend on lower layers. In some examples, different layers may be coded using a single-standard codec (e.g., all layers are coded using HEVC, SHVC, or other coding standard). In some examples, different layers may be coded using a multi-standard codec. For example, a base layer may be coded using AVC, while one or more enhancement layers may be coded using SHVC and / or MV-HEVC extensions to the HEVC standard.

[0070]

[0080] Video compression techniques perform spatial and temporal prediction to reduce or remove redundancy inherent in the input video signal. To reduce temporal redundancy (i.e., visual similarity in adjacent frames), motion estimation is performed to track the movement of objects in the video. Motion estimation is the process of identifying the best matching (e.g., most similar) block of pixels in one reference frame (or in two reference frames in case of bi-prediction) for a given block of pixels from a current frame. One or more reference frames may be identified for the current block using one or more reference indexes. A block of pixels in a reference frame may be referred to as a candidate block, predictor, or prediction block. The size of the block may vary. The offset between the current block from the current frame and the candidate block from the reference frame may be characterized by a motion vector, which may indicate the degree of displacement and the direction of the displacement.

[0071]

[0081] A motion vector may indicate a displacement in terms of a distance in pixels. In some cases, the motion vector may have a precision greater than integer pixel precision, such as ½ pixel, ¼ pixel, or 1 / 16 pixel distance. Finer precision may enable a video coder to track motion fields more accurately and thus obtain better predictions.

[0072]

[0082] When a video coder is reconstructing a frame and encounters a motion vector with a fractional pixel value, the video coder may perform interpolation to generate the fractional pixel value. For example, and as described further below, the video coder may apply an interpolation filter to the available pixels to create interpolated pixels for half-pixel, quarter-pixel, or some other fractional pixel positions.

[0073]

[0083] Many video codecs use fixed interpolation filters, in which both the encoder and decoder can be provided with the filter coefficients in advance, and these coefficients do not need to be provided in the encoded bitstream.

[0074]

[0084] However, motion estimation and compensation can be optimized by using optimal interpolation filters rather than fixed filters. The best filter for a given situation tends to be highly localized and content-dependent. Therefore, using optimal filters may significantly increase the signaling overhead and / or increase the complexity of the encoder in that filter coefficients need to be transmitted from the encoder to the decoder.

[0075]

[0085] In various implementations, systems and methods are provided in which different interpolation filters may be applied to different coding units and signaled with minimal additional increase to the size of the coded bitstream. For example, in the case of HEVC, a particular set of filter candidates may be available when a coding unit is at, among other things, the sequence level, the picture level, the coding tree unit level, the coding unit level, the transform unit level, or the prediction unit level. In addition, in the case of JEM, a particular set of filter candidates may be available at, among other things, the quadtree plus binary tree (QTBT) level, where a coding unit may also be referred to as a block. In these and other examples, a video coder may switch between multiple interpolation filters while encoding a video or decoding a bitstream.

[0076]

[0086] In various implementations, a set of N filters may be defined, each having different characteristics (e.g., filter length, cutoff frequency, transition band, or amount of ripple, among other characteristics). Subsets of the set of N filters may be defined for blocks of different coding levels. Using various criteria, the encoder may select a filter from the subset of filters. The selected filter may then be signaled or indicated, explicitly or implicitly, to the decoder. The subset of filters applied at a given level may differ based on certain criteria, such as the hit rate of the filter in previously coded blocks, among other criteria. In some implementations, filter selection for integer pixel positions or chroma components may also be signaled, implicitly or implicitly, using similar mechanisms.

[0077]

[0087] 2 illustrates an example of an encoding process 200 for a current block 236 from a current frame. The current block 236 may also be referred to as a coding unit. The current block 236 may be smaller than the entire current frame.

[0078]

[0088] In this example, the encoding process 200 first performs motion estimation 242 on the current block 236. The motion estimation 242 may include identifying one or more reference frames from among the available reference frames 264. The reference frames may include a frame that occurs before, temporally, or after the current frame. Performing the motion estimation 242 may include searching a search area in the reference frame(s) for a region that “best” matches the current block 236. The best match may be determined, for example, by subtracting the candidate region from the current block 236 to determine a residual energy. In this example, the candidate region with the lowest residual energy may be chosen as the best match.

[0079]

[0089] Motion estimation 242 further includes creating a motion vector 243, which estimates the degree and direction of motion for the current block 236 relative to a reference frame(s). In most cases, one or more reference indexes are associated with the motion vector, where the reference index identifies the reference frame(s) used to determine the motion vector. In some cases, after motion estimation, the best motion vector may be selected, for example, using a rate-distortion model.

[0080]

[0090] Motion compensation 244 may include generating a prediction block 245 by performing motion compensation using a best motion vector. The prediction block 245 is a block selected from a reference from using the best motion vector. The prediction block 245 may be subtracted from the current block 236 to create a residual block 251. A block transform 252 may be applied to the residual block 251. The resulting transform coefficients may undergo quantization 254 to create quantized coefficients 255. Entropy coding 256 may be applied to the quantized coefficients 255 to further reduce the bit rate and create the encoded bitstream 210.

[0081]

[0091] The quantized coefficients 255 may also be input to an inverse quantization 258 and then to an inverse transform 260. The inverse quantization 258 and inverse transform 260 are the inverse of the block transform 252 and quantization 254 steps and create a reconstructed residual block 261. The reconstructed residual block 261 may be added to the predicted block 245 to create a reconstructed block 253. The reconstructed block 253 is approximately the same as the current block 236. The reconstructed video block 253 may be included in a reference frame that will be used to code additional blocks.

[0082]

[0092] The exemplary process 200 primarily describes temporal motion estimation. In other examples, encoding processes such as H.264 / AVC and HEVC encoding processes may also perform spatial prediction to create intra-coded blocks. The encoding process is described in more detail with reference to FIG.

[0083]

[0093] As previously described, the best matching reference block for a current block may be at a full pixel offset or a fractional pixel offset from the current block. When the motion information indicates that the best match is at a fractional pixel position, the video coder may reconstruct the block by interpolating pixels at the fractional position from pixels in the reference block.

[0084]

[0094] Figure 3 illustrates an example of a fractional pixel location. In the example of Figure 3, a block 300 from a reference frame is illustrated that is 3 pixels wide and 3 pixels high. Integer pixel locations are illustrated using shaded blocks with capital letters (e.g., A 0,0 , A 1,0 , A 0,1 , A -1,0 , A 0,-1 , A 1,1 , A 1,-1 , A -1,1 , and A -1,-1 ). Quarter pixel locations, also referred to herein as sub-pixel locations, are illustrated using lower case letters (e.g., a 0,0 , b 0,0 , c 0,0 , etc.) In this example, there are 15 sub-pixel positions per pixel position, labeled a through r (note that the "l" and "o" have been omitted for clarity).

[0085]

[0095] Various methods can be used to derive samples for sub-pixel locations. For example, for half pixel location b 0,0 The sample at position A 0,0 and A 1,0As another example, position A -1,0 , A 1,0 , A 0,0 , and A from the adjacent block 2,0 The pixel at position b 0,0 can be averaged to create samples for

[0086]

[0096] In HEVC, position a 0,0 , b 0,0 , and c 0,0 Horizontal sub-pixel samples for, as well as position d 0,0 , h 0,0 , and n 0,0 The vertical sub-pixel samples for position e can be derived by applying an 8-tap filter to the nearest integer position samples. 0,0 , i 0,0 , p 0,0 , f 0,0 , j 0,0 , q 0,0 , g 0,0 , k 0,0 , and r 0,0 The sample for position a 0,i , b 0,i and c 0,i where i=-3...4. An example of an 8-tap luma interpolation filter is illustrated in Table 1.

[0087] [Table 1]

[0088]

[0097] Table 1 contains eight coefficients per phase shift, one for each tap. Each phase shift corresponds to a quarter pixel position. For example, phase shift 1 corresponds to sub-pixel position a 0,0 and d 0,0 and phase shift 2 corresponds to sub-pixel position b 0,0 and h 0,0, and so on. In other examples, a filter with longer taps (e.g., with more coefficients) may be applied. A filter with longer taps may have different characteristics than an 8-tap filter and may require more complex calculations when applied.

[0089]

[0098] In other video codecs, including the codec being developed in JEM, 1 / 16 sub-pixel motion vector resolution may be enabled. At the 1 / 16 sub-pixel positions, a filter with 16 phases may be used for interpolation. An example filter with 16 phases is provided in Table 2. However, in JEM, an 8-tap filter with fixed coefficients is used.

[0090] [Table 2]

[0091]

[0099] 2, content-adaptive binary arithmetic coding (CABAC) is one system that may be used in entropy coding 256 to encode and decode syntax elements in the bitstream. CABAC may achieve good compression performance by using the element's context to select a probability model for each syntax element; by adapting probability estimates based on local statistics; and by using arithmetic coding instead of variable length coding.

[0092]

[0100] CABAC uses binary arithmetic coding, which means that only binary decisions (1 or 0) are coded. Non-binary-valued symbols (e.g., transform coefficients or motion vectors, or any symbol with more than one possible value) are "binarized" or converted to a binary code before arithmetic coding. Then, for each bit of the binarized symbol, a context model is selected. The context model is a probability model for one or more bins of the binarized symbol. The context model is chosen from available models based on statistics of the most recently coded data symbols. The context model stores the probability that each bin is a "1" or a "0". The arithmetic coder then codes each bin according to the selected probability model. The selected context model is then updated based on the actual coded value (e.g., if the bin value was a "1", the frequency count of "1" is increased).

[0093]

[0101] As an example, three candidate context models named skip_flag_C[0], skip_flag_C[1], and skip_flag_C[2] may be used to code the syntax element cu_skip_flag. To choose the appropriate context from the three candidates, a selection value x may be calculated as follows: x=(cu_skip_flag[xNbL][yNbL] && availableL)+(cu_skip_flag[xNbA][yNbA] && availableA).

[0102] For the above equation, the context model to use may be determined as follows:

[0103] (x0,y0) specifies the location of the top-left luma sample of the current luma block relative to the top-left sample of the current picture.

[0094]

[0104] The position (xNbL, yNbL) may be set to (x0-1, y0), and the variable availableL may indicate the availability of the block located immediately to the left of the current block.

[0095]

[0105] The position (xNbA, yNbA) may be set equal to (x0, y0-1), and the variable availableA may specify the availability of the coding block located directly above the current block.

[0096]

[0106] Cu_skip_flag[xNbL][yNbL] and cu_skip_flag[xNbA][yNbA] may represent cu_skip_flag for the first block (block L) and the second block (block A), respectively.

[0097]

[0107] As mentioned above, the codecs implemented by HEVC and JEM use fixed interpolation filters when sub-pixel motion vectors are enabled. The interpolation filters may not be optimal for any given situation. For example, a filter with a higher cutoff frequency may be more appropriate when the coding unit has more detail, so that detail may be preserved when the interpolation is performed. As another example, a filter with a lower cutoff frequency may be more appropriate when the processing unit has less detail, so that the coding unit may be coded with fewer bits.

[0098]

[0108] Motion estimation and motion compensation may be optimized by using optimal interpolation filters rather than fixed filters. The best filter for a given situation tends to be highly localized and content-dependent. Therefore, using optimal filters may significantly increase the signaling overhead and / or increase the complexity of the encoder in that filter coefficients need to be transmitted from the encoder to the decoder.

[0099]

[0109] In various implementations, a video coder may be configured to switch between multiple available interpolation filters when performing motion estimation and motion compensation for different prediction units. In some cases, inter-layer prediction may be improved by selecting an interpolation filter in an adaptive manner. In various implementations, a set of N filters may be defined, each having characteristics such as filter length, cutoff frequency, transition band, amount of ripple, and / or combinations of characteristics.

[0100]

[0110] f400. Also illustrated in FIG. 4 is a set of filters 410, which represents all filters defined for use by the codec. As described above, in the motion estimation 444 step, a reference block from a reference frame 464 is determined. The reference block is the portion of the reference frame 464 that best matches (e.g., is most similar to) the coding unit 402 in terms of appearance. In this example, the coding unit 402 may be a sequence, a picture, a slice, a coding tree unit, a coding unit, a transform unit, a prediction unit, or some other block of video data. Using the reference block, the motion estimation 444 may determine motion information 406 for the coding unit 402. The motion information 406 may include, among other things, an index that may be used to identify the reference frame and one or more motion vectors.

[0101]

[0111] In the example of FIG. 4, the motion estimation 444 step may also determine a filter selection criterion 414. The filter selection criterion 414 may be used to select a subset of filters 412 (also referred to herein as filter candidates) that are applicable to a particular coding unit 402. For example, in some implementations, the filter selection criterion 414 may be a coding level of the coding unit 402 (e.g., a sequence level, a picture level, a slice level, a coding tree unit level, a coding unit level, a transform unit level, a prediction unit level, etc.). In this example, the subset of filters 412 may be selected based on the coding level of the coding unit 402. For example, the particular filters in the subset and the number of filters in the subset may vary across different processing levels.

[0102]

[0112] In various implementations, the motion estimation 444 may select an optimal interpolation filter from among the subset of filters 412. For example, the motion estimation 444 may use rate-distortion optimization to determine which of the filters in the subset of filters 412 provides the best compression. Various other methods may be used to select the optimal filter.

[0103]

[0113] In various implementations, the selected filter may be identified using a filter index 408. The filter index 408 may identify a filter chosen from among a subset of filters 412. For example, the possible indexes may be 0, 1, or 2 (for F1, F3, and F9, respectively, in the illustrated example), and when filter F1 is selected, the filter index 408 may be "0." In some cases, the filter index 408 may be included in the output bitstream 420 along with the motion information 406. In these implementations, the filter index 408 may be said to be explicitly signaled in the bitstream 420.

[0104]

[0114] In some cases, the filter index 408 may be implicitly signaled, i.e., the filter index 408 is not included in the bitstream 420, but instead may be derived by the decoder when decoding the bitstream 420. For example, in some cases, the filter selection criteria 414 results in a subset of filters 412 that includes only one filter. As another example, in some cases, the subset of filters 412 may include multiple filters, but the motion estimation 444 determines to use a default filter (e.g., for temporal merge candidates, filter F1 is always selected). In other examples, one or more factors, such as the coding level, prediction mode, and / or coding tool being used, may be used by the decoder to derive the appropriate filter.

[0105]

[0115] In some implementations, the motion estimation 444 may select a filter based on statistics derived from previously coded blocks. For example, in previously coded blocks, some filters from the subset of filters 412 may have never been used due to having poor performance. In this example, the motion estimation 444 may eliminate poorly performing filters when selecting a filter for the current particular coding unit 402. Conversely, when a particular filter has been used very frequently in previously coded blocks, the motion estimation 444 may prioritize this filter when selecting a filter for the particular coding unit 402.

[0106]

[0116] In some implementations, motion estimation 444 may select a filter from a neighboring block, such as a spatially neighboring one or a temporally neighboring one. For example, in some cases, coding unit 402 may copy a motion vector from a neighboring block. In this example, coding unit 402 may also borrow a filter index from the same neighboring block. As another example, when several neighboring blocks use the same filter, motion estimation 444 may determine that coding unit 402 should use the same filter.

[0107]

[0117] In some implementations, the filter selection criteria 414 may be based on various factors. For example, the filters in the subset of filters 412 may be selected based on the prediction mode (e.g., inter prediction, intra prediction, etc.) used for the current block, the resolution or different of the motion vectors, the coding tool (e.g., bidirectional optical flow) being used, and / or the pixels in the overlapped block motion compensation. As another example, the filters in the subset of filters 412 may be selected based on the size and / or shape of the current block, the resolution of the current frame, the distance between the current frame and a reference frame, and / or the quantization parameter (QP) value for the reference picture.

[0108]

[0118] As an example, when a coding unit 402 is a large set of data, such as a frame or slice, or when the coding unit 402 has little detail, the filters in the subset of filters 412 may be filters with low cutoffs. Filters with low cutoff frequencies tend to smooth out detail, so when the coding unit 402 has little detail, no data is lost by using filters with low cutoff frequencies. As another example, when a coding unit 402 is a small set of data, such as a coding unit, or when the coding unit 402 has a high degree of detail, the subset of filters 412 may include filters with high cutoff frequencies. Filters with high cutoff frequencies may preserve high frequency components in the coding unit 402, thus preserving detail.

[0109]

[0119] In some implementations, additional filter candidates may be generated by merging filters from the subset of filters 412. For example, given two filter candidates, an even phase shift from the first candidate filter may be merged with an odd phase shift from the second candidate to generate a third candidate. As another example, a fourth candidate may be constructed by combining an odd phase shift from the first candidate with an even phase shift from the second candidate.

[0110]

[0120] In some implementations, multiple interpolation filters may be combined. For example, an interpolation filter for integer samples may be counted into an interpolation filter for fractional samples. In this example, the filter for integer samples may have a higher cutoff frequency than that of the filter with the smallest non-zero phase shift from the filter for fractional samples. Alternatively, the filter for integer samples may have a lower cutoff frequency than that of the filter for fractional samples.

[0111]

[0121] In some implementations, a coding tool is also associated with a filter. For example, bidirectional optical flow may use a gradient filter, and frame rate up-conversion may use a refinement filter. In some implementations, a tool-specific filter may be generated according to each filter candidate in the subset of filters 412. For example, given three interpolation filter candidates, three gradient filters may be derived for bidirectional optical flow. In these implementations, by deriving the tool-specific filters, the tool-specific filters do not need to be described in the output bitstream 420.

[0112]

[0122] In some examples, when a filter is applied to both integer and fractional pixel positions, the filter for the integer pixel positions may be applied first, and then the filtered integer samples may be used as inputs for further interpolation to derive the fractional pixel samples.

[0113]

[0123] In some examples, when a filter is applied to both integer and fractional pixel locations, the filter for the integer pixel locations may only be applicable when the associated motion vector has integer precision, in these examples, the fractional pixel samples may be derived using unfiltered pixels at the integer locations as input.

[0114]

[0124] FIG. 5 illustrates an example of a motion estimation 544 step of the decoding process 500. Also illustrated in FIG. 5 is a set of filters 510, which represents all filters defined for use by the codec. In the decoder process 500, the motion estimation 544 may be used to reconstruct or predict the prediction unit 502. For the prediction unit 502 in the current frame, the encoded bitstream 520 may include motion information 506. The motion information 506 may include, for example, an index to a reference frame 564 and one or more motion vectors. Using the motion information 506, the motion estimation 544 may identify a reference block in the reference frame 564 and predict or generate the prediction unit 502.

[0115]

[0125] In some implementations, the motion estimation 544 may also determine a filter selection criterion 514. The filter selection criterion 514 may be used to select a subset of filters 512 that are applicable to the prediction unit 502. For example, the filter selection criterion 514 may be based on the coding level, prediction mode, and / or coding tool being used of the prediction unit 502. The particular filters in the subset and the number of filters in the subset may change as the bitstream 520 is decoded and different conditions are encountered.

[0116]

[0126] In some cases, the bitstream 520 includes an explicit filter index 508 that may be used to select a filter from the subset of filters 512. In various implementations, the filter index 508 may be associated with the motion information 506. When the filter index 508 is included in the bitstream 520, the filter index 508 is said to be explicitly signaled. The filter index 508 may identify a filter from among the subset of filters 512 to be used for the interpolation. For example, the possible indexes may be 0, 1, or 2 (for F1, F3, and F9, respectively, in the illustrated example), and when filter F1 is selected, the filter index 508 may be "0."

[0117]

[0127] In some cases, the decoder process 500 may derive the filter index 508. For example, in some cases, the filter selection criteria 514 results in a subset of filters 512 that includes only one filter. As another example, in some cases, the subset of filters 512 may include multiple filters, but the motion estimation 544 determines to use a default filter (e.g., for a temporal merge candidate, filter F1 is always selected). In other examples, one or more factors, such as the coding level, prediction mode, and / or coding tool being used, may be used by the decoder to derive the appropriate filter. Since the decoder should use the same filter as determined by the encoder, in cases where the decoder cannot deterministically derive the identity of the filter, the identity of the filter will be signaled in the bitstream 520.

[0118]

[0128] Various methods may be used to determine the interpolation filter to use when the filter index is implicit. In some examples, the interpolation filter may be determined by other blocks coded before the current block. For example, a block that meets certain conditions may determine the filter by inferring or inheriting the filter chosen for the other block. For example, when the current motion vector is derived from a previously coded block (e.g., as is the case with spatial merge candidates, affine merge candidates, and other candidates), the interpolation filter chosen for the block from which the motion vector is derived may be used for the current block. In this example, explicit signaling of the filter index may not be needed because once the decoder determines the motion vector, the decoder may derive the filter for the current block without requiring any additional information.

[0119]

[0129] In some examples, the interpolation filter for a particular block may always or sometimes choose a predefined default filter. In these examples, filter inheritance or searching for filters from another block is not necessary. In some cases, once determined, the default filter is applied at all coding levels. In some examples, a subset of filters may be biased towards the selection of a particular filter. For example, the filters may be tested in sequential order, and a particular filter may be set as the first candidate for blocks that derive motion vectors from frame rate up-conversion, temporal motion vector prediction (TMVP), advanced motion vector prediction (AMVP), or zero motion vector merging candidates. In some examples, the default filter may change across frames and / or slices. For example, the filter that was most frequently chosen in the previous picture(s) and / or slice(s) may be set as the default filter for all blocks belonging to the current picture or slice. Explicit signaling of the filter index may not be required, since the decoder may mimic the same procedure without any side information from the encoder.

[0120]

[0130] The filter index may be explicitly included in the bitstream if necessary otherwise. In some examples, a syntax element that may be called "interpolation_filter_index" may be signaled by the encoder to the decoder to indicate the interpolation filter chosen for blocks for which a filter cannot be chosen implicitly. By signaling, it is meant that the interpolation_filter_index may be included in the bitstream to be read by the decoder.

[0121]

[0131] In some examples, the value of interpolation_filter_index may be binarized using a binarization code, such as a fixed-length code, an exponential-Golomb code, a unary code, or a shortened unary code. For example, when there are three filters in the subset of filters, a shortened unary code may be used to indicate the interpolation_filter_index. In this example, two flags (e.g., FLAG0 and FLAG1) may be used to code three possible indexes 0, 1, and 2 for the three filters in the subset. For example, the three indexes may be coded as (0,N / A), (1,0), and (1,1), respectively. In this example, only one flag (FLAG0) may be used to indicate interpolation_filter_index=0.

[0122]

[0132] In some examples, a certain context model may be associated with the interpolation_filter_index, which may be used when encoding the interpolation_filter_index. For example, X context models may be associated with the interpolation_filter_index, e.g., named as interpolation_filter_index_C[0], interpolation_filter_index_C[1], ..., interpolation_filter_index_C[X-1]. In this example, among these context models, a context model with index x, interpolation_filter_index_C[x], may be selected as a context for coding the interpolation_filter_index for the current block.

[0123]

[0133] In some examples, when the interpolation_filter_index may be represented using two flags (e.g., FLAG0 and FLAG1), there may be a certain number of context models associated with each flag. For example, there may be Y context models associated with FLAG0, named as interpolation_filter_index_C[0], interpolation_filter_index_C[1], ..., interpolation_filter_index_C[Y-1]. In this example, when FLAG0 is set, a context model with index x, interpolation_filter_index_C[x], may be chosen as the context for coding the interpolation_filter_index of the current block. In this example, FLAG1 may not be set or may not be important. As another example, there may be Z context models associated with FLAG1, named as interpolation_filter_index_C[0], interpolation_filter_index_C[1], ..., interpolation_filter_index_C[Z-1]. In this example, when FLAG1 is set, the context model with index x, interpolation_filter_index_C[x], is chosen as the context for coding the interpolation_filter_index of the current block. In this example, FLAG0 may not be set or may not be significant.

[0124]

[0134] In the above examples, x may be selected based on various criteria. For example, x may be based on the size of the current block, the shape of the current block, the resolution of the current frame, the inter-prediction direction, the motion vector(s) for the current block, the motion vector difference for the current block, the reference frame for the current block, the interpolation_filter_index(s) of the neighboring block relative to the current block, some other criteria, or a combination of criteria. In various examples, a function may be used to generate a value of x, where the function may receive as input one or more of the above example criteria. In some examples, different functions may be used for different frame resolutions.

[0125]

[0135] In some examples, when x is based on the size of the current block, a threshold P may be used to determine the value of x. For example, when the size of the current block is less than P, x may be 0, otherwise x may be 1. The threshold P may be an integer. The size of the current block may be calculated as M×N, where M and N are the width and height of the current block in pixels.

[0126]

[0136] In some examples, when x is based on the size of the current block, the value of x is calculated using the integer array {a K}, where the array is sorted in ascending order. For example, K-1 <=S K Then, x may be equal to K, where S is the size of the current block. In this example, S may be the number of pixels in the current block, calculated as M×N, where M and N are the width and height of the current block in pixels.

[0127]

[0137] ​In some examples, when x is based on the shape of the current block, the value of x can be determined from the relationship between the height and width of the block. For example, given the height M and width N for the current block, when M < N, x can be equal to 0; when M = N, x can be equal to 1; and when M > N, x can be equal to 2.

[0128]

[0138] In some examples, when x is based on the prediction direction of the current block, the value of x can be determined from the prediction direction. For example, when the current block is coded using single inter prediction, x can be equal to 0; when the current block is coded using dual inter prediction, x can be equal to 1. As another example, the value of x can be based on the value InerDir, where when the current block is coded using single inter prediction from reference list 0, InterDir is 1; when the current block is coded using single inter prediction from reference list 1, InterDir is 2; and when the current block is coded using dual inter prediction, InterDir is 3. In this example, x can be equal to InterDir - 1.

[0129]

[0139] In some examples, when x is based on the interpolation_filter_index of a neighboring block, the value of x may be determined from the interpolation_filter_index of one or more neighboring blocks. For example, the value of x may be determined by a function f(left_idx, top_idx), where left_idx is the interpolation_filter_index of the left neighboring block and top_idx is the interpolation_filter_index of the upper neighboring block. The function f may be any function. In some examples, different functions may be used for different frame resolutions. Alternatively or additionally, the value of x may be equal to a+b, where a is equal to 1 when the interpolation_filter_index of the left neighboring block is available and greater than zero (otherwise a is equal to 0), and b is equal to 1 when the interpolation_filter_index of the upper neighboring block is available and greater than zero (otherwise b is equal to 0).

[0130]

[0140] In some examples, a combination of factors may be considered when determining the value of x. For example, the size of the current block may be used to select the interpolation_filter_index value from the neighboring blocks. For example, x may be equal to the result of f(left_idx, top_idx) when the size of the current picture is less than a threshold P, where P is an integer, and the size of the current block is calculated M×N, where M and N are the width and height of the current block in pixels. When the size of the current block is greater than or equal to P, x may be equal to the output of the function g(left_idx, top_idx).

[0131]

[0141] An example application of switching interpolation filters is now described. As mentioned above, the best filter may be chosen for each block from among N filters that may have different characteristics. The best filter may be chosen, for example, using rate-distortion optimization, or some other selection mechanism.

[0132]

[0142] To reduce the encoder complexity, the best filter can either be inherited from an already coded block, reused from a previous decision, or fixed without evaluating the performance of all available filters. To reduce the signaling overhead, a fixed set of filters is determined in advance and the encoder may signal the filter index to the decoder instead of the filter coefficients.

[0133]

[0143] The following example uses HM16.6-based JEM3.0 to demonstrate the application of interpolation filter switching. JEM is a project of the International Telecommunication Union Video Coding Experts Group (ITU-VCEG) and the International Organization for Standardization / International Electrotechnical Commission Motion Picture Experts Group (ISO / IEC MPEG). JEM is a reference software that provides a platform for researching and potentially standardizing video coding techniques with better compression capabilities than HEVC.

[0134]

[0144] In the following example, the coding unit is a block defined in the QTBT structure implemented on JEM3.0. In other examples, the coding unit may be a larger structure, such as a collection of QTBT blocks, a CTU, a slice, a picture, a sequence, or another suitable coding unit. Note that this example only shows an interpolation filter for the luma component, and the switching of the interpolation filters may be extended to integer samples as well as chroma components.

[0135]

[0145] In this example, the candidate set of interpolation filters includes three filters with different characteristics: one 12-tap filter (F0) and two 8-tap filters (F1 and F2). Filter F0 is illustrated in Table 3, filter F1 is illustrated in Table 4, and filter F2 is illustrated in Table 5. In this example, the set of filter candidates does not change across different pictures and / or slices. Since JEM3.0 supports motion vector resolution up to 1 / 16, each filter candidate is configured with 16 different phase shifts, as shown in the table below.

[0136] [Table 3]

[0137] [Table 4]

[0138] [Table 5]

[0139]

[0146] FIG. 6 is a graph illustrating the frequency response of three example filters, F0, F1, and F2. Each curve corresponds to eight identical phase shifts for each filter. As illustrated by this example, filter F0 has a high cutoff frequency, while filter F2 has a low cutoff frequency. Filter F1 has a fairly neutral cutoff frequency.

[0140]

[0147] Continuing with the example, for a given coding unit, one of filters F0, F1, and F2 may be chosen. The selected filter may be signaled implicitly or explicitly.

[0141]

[0148] In this example, implicit signaling of filter indices is enabled for merge mode because the decoder can mimic the same procedure that the encoder performs to determine merge motion vector candidates without any side information from the encoder. For example, the interpolation filter for a coding unit can be determined by either inheriting the filter index from the block from which the current motion is derived or by selecting a default filter, which in this example is filter F0.

[0142]

[0149] Continuing with this example, filter inheritance may be used when the current motion vector is derived from a spatial merge candidate, a combined merge candidate when the two candidates being combined use the same filter, and an affine merge candidate. In each of these cases, the content of the block from which it is inherited may influence the selection of the filter. Also, in this example, when the current motion vector is derived from a frame rate up-conversion merge candidate or a temporal merge candidate (e.g., TMVP, AMVP, STMVP, and zero motion vector) candidate, a default filter, F0, is selected. In each of these cases, a filter with a high cutoff frequency may be desirable to preserve details in the block.

[0143]

[0150] Continuing with this example, explicit signaling of filter indexes is enabled for AMVP mode. In JEM3.0, two additional rate-distortion optimizations are included in the encoder, which can be used to choose the best filter from three example filter candidates. The additional optimization process can be simplified by reusing integer motion vectors or by skipping the additional process by reusing the filter chosen in a particular configuration. As an example, in the case of an illumination compensation (IC) tool, the filter chosen when illumination compensation is off can be reused in the case when illumination compensation is on. This can reduce the encoding time.

[0144]

[0151] Continuing with this example, when explicit signaling of filter indices is enabled, the interpolation_filter_index may be binarized using "0" for F0, "10" for F1, and "11" for F2. Two flags, FLAG0 and FLAG1, may be used to code the interpolation_filter_index as follows:

[0152] FLAG0

[0153] if(FLAG0)

[0154] FLAG1

[0155] Continuing with the above example, two contexts may be provided for each flag: (FLAG0_ctx0, FLAG0_ctx1) for FLAG0, and (FLAG1_ctx0, FLAG1_ctx1) for FLAG1. As an example, when the current block is coded using uni-prediction, FLAG0_ctx0 may be used to code FLAG0 and FLAG1_ctx0 may be used to code FLAG1. As another example, when the current block is coded using bi-prediction, FLAG0_ctx1 may be used to code FLAG0 and FLAG1_ctx1 may be used to code FLAG1.

[0145]

[0156] In another example of encoding two interpolation candidates, F0 and F2, the interpolation_filter_index may be binarized as "0" for F0 and "1" for F1. In this example, one flag may be used to code the interpolation_filter_index. Each flag may have an associated context, FLAG0_ctx0 and FLAG0_ctx1. When the size of the current block is less than 256 (for the purposes of this example), FLAG0_ctx0 may be used to code FLAG0. Otherwise, when the size is 256 or greater, FLAG0_ctx1 may be used to code FLAG0. The size of the current block is calculated as w×h, where w and h represent the width and height of the current block in pixels, respectively.

[0146]

[0157] 7 illustrates an example of a process 700 for switching interpolation filters during an encoding process. At 702, the process 700 includes acquiring video data. The video data may be acquired from a video capture device such as a camera. Alternatively or additionally, the video data may be read from a stored location.

[0147]

[0158] At 704, the process includes encoding the coding unit, where the encoding process includes selecting an interpolation filter for motion estimation and motion compensation for the coding unit, where the interpolation filter is selected from a set of interpolation filters. In some implementations, the interpolation filter is selected from a subset of interpolation filters selected from the set of interpolation filters. In some implementations, the subset of interpolation filters is determined based on a coding level of the coding unit. In some implementations, the subset is determined based on a prediction mode of the coding unit. In some implementations, the subset is determined based on a result of motion estimation. In some implementations, the subset is determined based on a coding tool used to code the coding unit.

[0148]

[0159] In some implementations, the interpolation filters are selected from a previously coded block, hi some implementations, the interpolation filters are selected from neighboring blocks.

[0149]

[0160] In some implementations, an interpolation filter is merged with another interpolation filter, and the resulting merged interpolation filter is used for motion estimation.

[0150]

[0161] In some implementations, the interpolation filter is coupled to a second interpolation filter. In these implementations, the interpolation filter may be for integer pixel positions and the second interpolation filter may be for fractional pixel positions. In some implementations, the interpolation filter and the second interpolation filter may have different cutoff frequencies.

[0151]

[0162] At 706, the process 700 includes generating an encoded video bitstream, where the video bitstream includes the encoded coding units. In some implementations, the encoded video bitstream may include an explicit value identifying the selected interpolation filter, such as a filter index. In some implementations, the value identifying the interpolation filter may be implicit from the encoded video bitstream.

[0152]

[0163] 8 is an example of a process 800 for switching interpolation filters in a decoding process. At 802, the process 800 includes obtaining an encoded video bitstream. The bitstream may be obtained, for example, from a storage location and / or over a network.

[0153]

[0164] At 804, the process 800 includes determining a reference frame for a coding unit in the current frame from the encoded video bitstream. The reference frame may temporally precede or follow the current frame. The reference frame may be the same frame as the current frame.

[0154]

[0165] At 806, process 800 includes determining an interpolation filter from a set of interpolation filters. In some implementations, the interpolation filter may be selected from a subset of interpolation filters from the set of interpolation filters. In some implementations, the subset of interpolation filters may be determined based on a coding level of the coding unit. In some implementations, the subset may be determined based on a prediction mode for the coding unit. In some implementations, the subset may be determined based on results of motion estimation. In some implementations, the subset may be determined based on a coding tool used to code the coding unit.

[0155]

[0166] In some implementations, the interpolation filters may be selected from a previously coded block, hi some implementations, the interpolation filters may be selected from a neighboring block.

[0156]

[0167] In some implementations, the interpolation filter may be merged with a second interpolation filter, and the resulting merged interpolation filter may be used for motion estimation.

[0157]

[0168] In some implementations, the interpolation filter may be coupled to a second interpolation filter. In these implementations, the interpolation filter may be for integer pixel positions and the second interpolation filter may be for fractional pixel positions. In some implementations, the interpolation filter and the second interpolation filter may have different cutoff frequencies.

[0158]

[0169] In some implementations, the encoded video bitstream includes an explicit value, such as a filter index, that identifies the selected interpolation filter. In some implementations, the process includes deriving the identity of the interpolation filter using data from the bitstream.

[0159]

[0170] At 808, the process 800 includes using the reference frames determined at 804 and the interpolation filters determined at 806 to reconstruct the coding unit.

[0160]

[0171] The coding techniques described herein may be implemented in an example video encoding and decoding system (e.g., system 100). In some examples, the system includes a source device that provides encoded video data to be decoded at a later time by a destination device. In particular, the source device provides the video data to the destination device via a computer-readable medium. The source and destination devices may comprise any of a wide range of devices, including desktop computers, notebook (i.e., laptop) computers, tablet computers, set-top boxes, telephone handsets such as so-called "smart" phones, so-called "smart" pads, televisions, cameras, display devices, digital media players, video game consoles, video streaming devices, or the like. In some cases, the source and destination devices may be equipped for wireless communication.

[0161]

[0172] The destination device may receive the encoded video data to be decoded via a computer-readable medium. The computer-readable medium may comprise any type of medium or device capable of moving encoded video data from a source device to a destination device. In one example, the computer-readable medium may comprise a communication medium that allows the source device to transmit the encoded video data directly to the destination device in real time. The encoded video data may be modulated according to a communication standard, such as a wireless communication protocol, and transmitted to the destination device. The communication medium may comprise any wireless or wired communication medium, such as a radio frequency (RF) spectrum or one or more physical transmission lines. The communication medium may form part of a packet-based network, such as a local area network, a wide area network, or a global network like the Internet. The communication medium may include routers, switches, base stations, or any other equipment that may be useful in facilitating communication from a source device to a destination device.

[0162]

[0173] In some examples, the encoded data may be output from an output interface to a storage device. Similarly, the encoded data may be accessed from a storage device by an input interface. The storage device may include any of a variety of distributed or locally accessed data storage media, such as a hard drive, a Blu-ray disc, a DVD, a CD-ROM, a flash memory, a volatile or non-volatile memory, or any other suitable digital storage medium for storing encoded video data. In further examples, the storage device may correspond to a file server or another intermediate storage device that may store the encoded video generated by the source device. The destination device may access the stored video data from the storage device via streaming or download. The file server may be any type of server capable of storing the encoded video data and transmitting the encoded video data to the destination device. Exemplary file servers include a web server (e.g., for a website), an FTP server, a network-attached storage (NAS) device, or a local disk drive. The destination device may access the encoded video data through any standard data connection, including an Internet connection. This may include a wireless channel (e.g., a Wi-Fi connection), a wired connection (e.g., DSL, cable modem, etc.), or a combination of both suitable for accessing the encoded video data stored on a file server. The transmission of the encoded video data from the storage device may be a streaming transmission, a download transmission, or a combination thereof.

[0163]

[0174] The techniques of this disclosure are not necessarily limited to wireless applications or settings. The techniques may be applied to video coding to support any of a variety of multimedia applications, such as over-the-air television broadcasting, cable television transmission, satellite television transmission, Internet streaming video transmission such as Dynamic Adaptive Streaming over HTTP (DASH), digital video encoded on a data storage medium, decoding of digital video stored on a data storage medium, or other applications. In some examples, the system may be configured to support one-way or two-way video transmission to support applications such as video streaming, video playback, video broadcasting, and / or video telephony.

[0164]

[0175] In one example, the source device includes a video source, a video encoder, and an output interface. The destination device may include an input interface, a video decoder, and a display device. The video encoder of the source device may be configured to apply the techniques disclosed herein. In other examples, the source device and the destination device may include other components or arrangements. For example, the source device may receive video data from an external video source, such as an external camera. Similarly, the destination device may interface with an external display device rather than including an integrated display device.

[0165]

[0176] The above exemplary system is merely an example. The techniques for processing video data in parallel may be performed by any digital video encoding and / or decoding device. Generally, the techniques of this disclosure are performed by a video encoding device, but the techniques may also be performed by a video encoder / decoder, typically referred to as a "CODEC." Additionally, the techniques of this disclosure may also be performed by a video preprocessor. The source device and destination device are merely examples of such coding devices, where the source device generates coded video data for transmission to the destination device. In some examples, the source and destination devices may operate in a substantially symmetrical manner, such that each of the devices includes video encoding and decoding components. Thus, the exemplary system may support one-way or two-way video transmission between video devices, for example, video streaming, video playback, video broadcasting, or video telephony.

[0166]

[0177] The video source may include a video capture device such as a video camera, a video archive containing previously captured video, and / or a video feed interface for receiving video from a video content provider. As a further alternative, the video source may generate computer graphics-based data as the source video, or a combination of computer-generated video, archived video, and live video. In some cases, when the video source is a video camera, the source device and destination device may form a so-called camera phone or video phone. However, as mentioned above, the techniques described in this disclosure may be applicable to video coding in general and may be applied to wireless and / or wired applications. In each case, the captured, pre-captured, or computer-generated video may be encoded by a video encoder. The encoded video information may then be output by an output interface onto a computer-readable medium.

[0167]

[0178] As mentioned, computer-readable media may include transitory media, such as wireless broadcast or wired network transmissions, or storage media (i.e., non-transitory storage media), such as hard disks, flash drives, compact discs, digital video discs, Blu-ray discs, or other computer-readable media. In some examples, a network server (not shown) may provide the encoded video data to a destination device and receive the encoded video data from a source device, e.g., via a network transmission. Similarly, a computing device of a media manufacturing facility, such as a disc stamping facility, may receive the encoded video data from a source device and manufacture discs including the encoded video data. Thus, computer-readable media may be understood to include one or more computer-readable media of various forms in various examples.

[0168]

[0179] The input interface of the destination device receives information from the computer-readable medium. The information of the computer-readable medium may include syntax information defined by the video encoder, including syntax elements that describe the processing and / or characteristics of blocks and other coded units, e.g., group of pictures (GOPs), which is also used by the video decoder. The display device may display the decoded video data to a user and may comprise any of a variety of display devices, such as a cathode ray tube (CRT), a liquid crystal display (LCD), a plasma display, an organic light emitting diode (OLED) display, or another type of display device. Various embodiments of the present invention have been described.

[0169]

[0180] Specific details of the encoding device 104 and the decoding device 112 are shown in FIG. 9 and FIG. 10, respectively. FIG. 9 is a block diagram illustrating an example encoding device 104 that may implement one or more of the techniques described in this disclosure. The encoding device 104 may generate, for example, a syntax structure described herein (e.g., a syntax structure for a VPS, SPS, PPS, or other syntax elements). The encoding device 104 may perform intra-prediction and inter-predictive coding of video blocks within a video slice. As previously mentioned, intra-coding relies at least in part on spatial prediction to reduce or remove spatial redundancy within a given video frame or picture. Inter-coding relies at least in part on temporal prediction to reduce or remove temporal redundancy within adjacent or surrounding frames of a video sequence. Intra-mode (I-mode) may refer to any of several spatial-based compression modes. An inter-mode, such as unidirectional prediction (P mode) or bi-prediction (B mode), may refer to any of several temporal-based compression modes.

[0170]

[0181] Encoding device 104 includes partitioning unit 35, prediction processing unit 41, filter unit 63, picture memory 64, adder 50, transform processing unit 52, quantization unit 54, and entropy coding unit 56. Prediction processing unit 41 includes motion estimation unit 42, motion compensation unit 44, and intra-prediction processing unit 46. For video block reconstruction, encoding device 104 also includes inverse quantization unit 58, inverse transform processing unit 60, and adder 62. Filter unit 63 is intended to represent one or more loop filters, such as a deblocking filter, an adaptive loop filter (ALF), and a sample adaptive offset (SAO) filter. Although filter unit 63 is illustrated in FIG. 9 as an in-loop filter, in other configurations filter unit 63 may be implemented as a post-loop filter. Post-processing device 57 may perform additional processing on the encoded video data generated by encoding device 104. Techniques of this disclosure may be implemented by encoding device 104 in some instances. However, in other cases, one or more of the techniques of this disclosure may be implemented by post-processing device 57.

[0171]

[0182] As shown in FIG. 9, encoding device 104 receives video data, and partitioning unit 35 partitions the data into video blocks. Partitioning may also include video block partitioning, e.g., by a quadtree structure of CUs and LCUs, as well as partitioning into slices, slice segments, tiles, or other larger units. Encoding device 104 generally illustrates components that encode video blocks within a video slice to be encoded. A slice may be divided into a plurality of video blocks (and possibly into a set of video blocks called tiles). Prediction processing unit 41 may select one of a plurality of possible coding modes, such as one of a plurality of intra-predictive coding modes or one of a plurality of inter-predictive coding modes, for a current video block based on an error result (e.g., a coding rate and a level of distortion, or the like). Prediction processing unit 41 may provide the resulting intra- or inter-coded block to adder 50 to generate residual block data, and to adder 62 to reconstruct the coded block for use as a reference picture.

[0172]

[0183] Intra-prediction processing unit 46 within prediction processing unit 41 may perform intra-predictive coding of the current video block relative to one or more neighboring blocks in the same frame or slice as the current block to be coded to provide spatial compression. Motion estimation unit 42 and motion compensation unit 44 within prediction processing unit 41 perform inter-predictive coding of the current video block relative to one or more predictive blocks in one or more reference pictures to provide temporal compression.

[0173]

[0184] Motion estimation unit 42 may be configured to determine an inter-prediction mode for a video slice according to a predetermined pattern for the video sequence. The predetermined pattern may designate a video slice in the sequence as a P slice, a B slice, or a GPB slice. Motion estimation unit 42 and motion compensation unit 44 may be highly integrated, but are illustrated separately for conceptual purposes. Motion estimation performed by motion estimation unit 42 is the process of generating motion vectors, which estimate motion with respect to a video block. A motion vector may indicate, for example, the displacement of a prediction unit (PU) of a video block in a current video frame or picture relative to a predictive block in a reference picture.

[0174]

[0185] The predictive block is a block found to closely match the PU of the video block to be coded in terms of pixel differences, which may be determined by sum of absolute difference (SAD), sum of square difference (SSD), or other difference metric. In some examples, encoding device 104 may calculate values ​​for sub-integer pixel locations of reference pictures stored in picture memory 64. For example, encoding device 104 may interpolate values ​​for quarter-pixel locations, eighth-pixel locations, or other fractional pixel locations of the reference pictures. Thus, motion estimation unit 42 may perform motion searches for full-pixel and fractional pixel locations and output motion vectors with fractional pixel accuracy.

[0175]

[0186] Motion estimation unit 42 calculates a motion vector for a PU of a video block in an inter-coded slice by comparing the position of the PU with the position of a predictive block of a reference picture. The reference picture may be selected from a first reference picture list (List 0) or a second reference picture list (List 1), each of which identifies one or more reference pictures stored in picture memory 64. Motion estimation unit 42 sends the calculated motion vector to entropy encoding unit 56 and motion compensation unit 44.

[0176]

[0187] The motion compensation performed by motion compensation unit 44 may involve fetching or generating a predictive block based on a motion vector determined by motion estimation, possibly performing interpolation to sub-pixel precision. Upon receiving a motion vector for a PU of a current video block, motion compensation unit 44 may locate the predictive block to which the motion vector points in a reference picture list. Encoding device 104 forms a residual video block by subtracting pixel values ​​of the predictive block from pixel values ​​of the current video block being coded and forming pixel difference values. The pixel difference values ​​form residual data for the block and may include both luma and chroma difference components. Adder 50 represents one or more components that perform this subtraction operation. Motion compensation unit 44 may also generate syntax elements associated with the video block and the video slice for use by decoding device 112 in decoding the video block of the video slice.

[0177]

[0188] Intra-prediction processing unit 46 may intra-predict the current block as an alternative to inter-prediction performed by motion estimation unit 42 and motion compensation unit 44, as described above. Specifically, intra-prediction processing unit 46 may determine an intra-prediction mode to use to encode the current block. In some examples, intra-prediction processing unit 46 may encode the current block using various intra-prediction modes, e.g., during separate encoding passes, and intra-prediction processing unit 46 may select an appropriate intra-prediction mode to use from the tested modes. For example, intra-prediction processing unit 46 may calculate a rate-distortion value using a rate-distortion analysis for the various tested intra-prediction modes and select the intra-prediction mode with the best rate-distortion characteristics among the tested modes. The rate-distortion analysis generally determines the amount of distortion (or error) between the encoded block and the original uncoded block that was coded to create the encoded block, as well as the bitrate (i.e., number of bits) used to create the encoded block. Intra-prediction processing unit 46 may calculate a ratio from the rates and distortions for the various coded blocks to determine which intra-prediction mode exhibits the best rate-distortion value for the block.

[0178]

[0189] In either case, after selecting an intra-prediction mode for the block, intra-prediction processing unit 46 may provide information indicating the selected intra-prediction mode for the block to entropy coding unit 56. Entropy coding unit 56 may encode the information indicating the selected intra-prediction mode. Encoding device 104 may include in the transmitted bitstream configuration data a definition of the coding contexts for the various blocks, as well as an indication of the most probable intra-prediction mode, an intra-prediction mode index table, and a modified intra-prediction mode index table to use for each of the contexts. The bitstream configuration data may include multiple intra-prediction mode index tables and multiple modified intra-prediction mode index tables (also referred to as codeword mapping tables).

[0179]

[0190] After prediction processing unit 41 generates a predictive block for a current video block via either inter- or intra-prediction, encoding device 104 forms a residual video block by subtracting the predictive block from the current video block. The residual video data in the residual block may be included in one or more TUs and applied to transform processing unit 52. Transform processing unit 52 converts the residual video data to residual transform coefficients using a transform, such as a discrete cosine transform (DCT) or a conceptually similar transform. Transform processing unit 52 may convert the residual video data from the pixel domain to a transform domain, such as the frequency domain.

[0180]

[0191] Transform processing unit 52 may send the resulting transform coefficients to quantization unit 54. Quantization unit 54 quantizes the transform coefficients to further reduce the bit rate. The quantization process may reduce the bit depth associated with some or all of the coefficients. The degree of quantization may be modified by adjusting a quantization parameter. In some examples, quantization unit 54 may then perform a scan of a matrix including the quantized transform coefficients. Alternatively, entropy coding unit 56 may perform the scan.

[0181]

[0192] Following quantization, entropy encoding unit 56 entropy encodes the quantized transform coefficients. For example, entropy encoding unit 56 may perform context-adaptive variable length coding (CAVLC), context-adaptive binary arithmetic coding (CABAC), syntax-based context-adaptive binary arithmetic coding (SBAC), probability interval partitioned entropy (PIPE) coding, or another entropy encoding technique. Following entropy encoding by entropy encoding unit 56, the encoded bitstream may be transmitted to decoding device 112 or archived for later transmission or retrieval by decoding device 112. Entropy encoding unit 56 may also entropy encode motion vectors and other syntax elements for the current video slice being coded.

[0182]

[0193] Inverse quantization unit 58 and inverse transform processing unit 60 apply inverse quantization and inverse transform, respectively, to reconstruct the residual block in the pixel domain for subsequent use as a reference block of a reference picture. Motion compensation unit 44 may calculate a reference block by adding the residual block to a predictive block of one of the reference pictures in the reference picture list. Motion compensation unit 44 may also apply one or more interpolation filters to the reconstructed residual block to calculate sub-integer pixel values ​​for use during motion estimation. Adder 62 adds the reconstructed residual block to a motion compensated predictive block created by motion compensation unit 44 to create a reference block for storage in picture memory 64. The reference block may be used by motion estimation unit 42 and motion compensation unit 44 as a reference block for inter predicting blocks in subsequent video frames or pictures.

[0183]

[0194] In this manner, encoding device 104 of FIG. 9 represents an example of a video encoder configured to generate syntax for an encoded video bitstream. Encoding device 104 may generate VPS, SPS, and PPS parameter sets, for example, as described above. Encoding device 104 may perform any of the techniques described herein, including the processes described above. Although the techniques of this disclosure have been generally described with respect to encoding device 104, as noted above, some of the techniques of this disclosure may also be implemented by post-processing device 57.

[0184]

[0195] 10 is a block diagram illustrating an example decoding device 112. The decoding device 112 includes an entropy decoding unit 80, a prediction processing unit 81, an inverse quantization unit 86, an inverse transform processing unit 88, an adder 90, a filter unit 91, and a picture memory 92. The prediction processing unit 81 includes a motion compensation unit 82 and an intra-prediction processing unit 84. The decoding device 112 may, in some examples, perform a decoding path that is generally inverse to the encoding path described with respect to the encoding device 104 from FIG.

[0185]

[0196] During the decoding process, the decode device 112 receives an encoded video bitstream representing video blocks of encoded video slices and associated syntax elements sent by the encoding device 104. In some embodiments, the decode device 112 may receive the encoded video bitstream from the encoding device 104. In some embodiments, the decode device 112 may receive the encoded video bitstream from a network entity 79, such as a server, a media aware network element (MANE), a video editor / splicer, or other such device configured to implement one or more of the techniques described above. The network entity 79 may or may not include the encoding device 104. Some of the techniques described in this disclosure may be implemented by the network entity 79 before the network entity 79 sends the encoded video bitstream to the decode device 112. In some video decoding systems, the network entity 79 and the decode device 112 may be part of separate devices, while in other cases, the functions described with respect to the network entity 79 may be performed by the same device that comprises the decode device 112.

[0186]

[0197] Entropy decoding unit 80 of decoding device 112 entropy decodes the bitstream to generate quantized coefficients, motion vectors, and other syntax elements. Entropy decoding unit 80 forwards the motion vectors and other syntax elements to prediction processing unit 81. Decoding device 112 may receive syntax elements at the video slice level and / or the video block level. Entropy decoding unit 80 may process and parse both fixed-length and variable-length syntax elements, such as VPS, SPS, and PPS, or in multiple parameter sets.

[0187]

[0198] When a video slice is coded as an intra-coded (I) slice, intra-prediction processing unit 84 of prediction processing unit 81 may generate predictive data for video blocks of the current video slice based on the signaled intra-prediction mode and data from previously decoded blocks of the current frame or picture. When a video frame is coded as an inter-coded (i.e., B, P or GPB) slice, motion compensation unit 82 of prediction processing unit 81 creates predictive blocks for video blocks of the current video slice based on motion vectors and other syntax elements received from entropy decoding unit 80. The predictive blocks may be created from one of the reference pictures in the reference picture list. Decoding device 112 may construct the reference frame lists, i.e., List 0 and List 1, using a default construction technique based on the reference pictures stored in picture memory 92.

[0188]

[0199] Motion compensation unit 82 may determine prediction information for video blocks of the current video slice by parsing the motion vectors and other syntax elements, and use the prediction information to create predictive blocks for the current video block being decoded. For example, motion compensation unit 82 may use one or more syntax elements in the parameter set to determine a prediction mode (e.g., intra or inter prediction) used to code the video blocks of the video slice, an inter-predictive slice type (e.g., a B slice, a P slice, or a GPB slice), configuration information for one or more reference picture lists for the slice, a motion vector for each inter-coded video block of the slice, an inter-prediction status for each inter-coded video block of the slice, and other information for decoding video blocks in the current video slice.

[0189]

[0200] Motion compensation unit 82 may also perform interpolation based on an interpolation filter. Motion compensation unit 82 may use an interpolation filter, such as that used by encoding device 104 during encoding of the video block, to calculate interpolated values ​​for sub-integer pixels of the reference block. In this case, motion compensation unit 82 may determine the interpolation filter used by encoding device 104 from the received syntax element and may use the interpolation filter to create the predictive block.

[0190]

[0201] Inverse quantization unit 86 inverse quantizes, or dequantizes, the quantized transform coefficients provided in the bitstream and decoded by entropy decoding unit 80. The inverse quantization process may involve the use of quantization parameters calculated by encoding device 104 for each video block in a video slice to determine the degree of quantization, and similarly, the degree of inverse quantization to be applied. Inverse transform processing unit 88 applies an inverse transform (e.g., an inverse DCT or other suitable inverse transform), an inverse integer transform, or a conceptually similar inverse transform process, to the transform coefficients to create residual blocks in the pixel domain.

[0191]

[0202] After motion compensation unit 82 generates a predictive block for a current video block based on the motion vector and other syntax elements, decoding device 112 forms a decoded video block by adding a residual block from inverse transform processing unit 88 with a corresponding predictive block generated by motion compensation unit 82. Adder 90 represents one or more components that perform this addition operation. If desired, a loop filter (either during or after the coding loop) may also be used to smooth pixel transitions or otherwise improve video quality. Filter unit 91 is intended to represent one or more loop filters, such as a deblocking filter, an adaptive loop filter (ALF), and a sample adaptive offset (SAO) filter. Although filter unit 91 is illustrated in FIG. 10 as an in-loop filter, in other configurations filter unit 91 may be implemented as a post-loop filter. The decoded video blocks in a given frame or picture are then stored in a picture memory 92, which stores reference pictures used for subsequent motion compensation. Picture memory 92 also stores decoded video for later presentation on a display device, such as video destination device 122 shown in FIG.

[0192]

[0203] In the foregoing description, aspects of the present application have been described with reference to specific embodiments thereof, but those skilled in the art will recognize that the present invention is not limited thereto. Thus, while exemplary embodiments of the present application have been described in detail herein, it should be understood that the inventive concepts may be embodied and used in various other ways, and the appended claims are intended to be construed to include such variations, except as limited by the prior art. The various features and aspects of the invention described above may be used individually or together. Moreover, the embodiments may be utilized in any number of environments and applications beyond those described herein without departing from the broader spirit and scope of the present specification. Thus, the specification and drawings should be regarded as illustrative, and not restrictive. For purposes of illustration, the methods have been described in a particular order. It should be understood that in alternative embodiments, the methods may be performed in an order different from that described.

[0193]

[0204] Where a component is described as being "configured to" perform a particular operation, such configuration may be achieved, for example, by designing electronic circuitry or other hardware to perform the operation, by programming a programmable electronic circuit (e.g., a microprocessor or other suitable electronic circuitry) to perform the operation, or any combination thereof.

[0194]

[0205] The various exemplary logic blocks, modules, circuits, and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, firmware, or a combination thereof. To clearly illustrate this interchangeability of hardware and software, various exemplary components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the particular application and design constraints imposed on the overall system. Those skilled in the art may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the invention.

[0195]

[0206] The techniques described herein may also be implemented in electronic hardware, computer software, firmware, or any combination thereof. Such techniques may be implemented in any of a variety of devices, such as a general purpose computer, a wireless communication device handset, or an integrated circuit device having multiple uses, including applications in wireless communication device handsets and other devices. Any features described as modules or components may be implemented together in an integrated logic device, or separately as discrete but interoperable logic devices. When implemented in software, the techniques may be realized, at least in part, by a computer-readable data storage medium comprising program code including instructions that, when executed, perform one or more of the methods described above. The computer-readable data storage medium may form part of a computer program product, which may include packaging materials. The computer-readable medium may comprise memory or data storage media, such as random access memory (RAM), such as synchronous dynamic random access memory (SDRAM), read only memory (ROM), non-volatile random access memory (NVRAM), electrically erasable programmable read only memory (EEPROM), FLASH memory, magnetic or optical data storage media, and the like. The techniques may additionally or alternatively be realized, at least in part, by a computer-readable communications medium, such as a propagated signal or wave, that can be accessed, read, and / or executed by a computer and that carries or communicates program code in the form of data structures or instructions.

[0196]

[0207] The program code may be executed by a processor, which may include one or more processors, such as one or more digital signal processors (DSPs), general-purpose microprocessors, application specific integrated circuits (ASICs), field programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuits. Such a processor may be configured to perform any of the techniques described in this disclosure. A general-purpose processor may be a microprocessor, but alternatively, the processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration. Thus, as used herein, the term "processor" may refer to any of the foregoing structures, any combination of the foregoing structures, or any other structure or apparatus suitable for implementing the techniques described herein. Additionally, in some aspects, the functionality described herein may be provided within dedicated software or hardware modules configured for encoding and decoding, or may be incorporated into a composite video encoder-decoder (CODEC). The invention as described in the claims of the original application is set forth below. [C1] 1. A method for encoding video data, comprising the steps of: obtaining the video data; determining a subset of interpolation filters from a set of interpolation filters for a coding unit, where the subset of interpolation filters is determined based on information in the video data associated with the coding unit; encoding the coding unit, where encoding the coding unit includes selecting an interpolation filter for motion estimation and motion compensation for the coding unit, where the interpolation filter is selected from a subset of the interpolation filters; generating an encoded video bitstream, wherein the encoded video bitstream includes the encoded coding unit; A method for providing the above. [C2] The method of claim 1, wherein the subset of interpolation filters is determined based on a coding level of the coding unit. [C3] The method of C1, wherein the subset of interpolation filters is determined based on a prediction mode for the coding unit. [C4] The method of C1, wherein the subset of interpolation filters is determined based on motion information associated with the coding unit. [C5] The method of claim 1, wherein the subset of interpolation filters is determined based on a coding tool used to code the coding unit. [C6] The method of claim 1, wherein the subset of interpolation filters is determined based on a size of the coding unit. [C7] The method of C1, wherein the interpolation filters are selected from previously coded blocks. [C8] The method of C1, wherein the interpolation filters are selected from neighboring blocks of the coding unit. [C9] merging the interpolation filter with a second interpolation filter, where the resulting merged interpolation filter is used for the motion estimation and the motion compensation. The method of claim C1, further comprising: [C10] The method of claim 1, wherein the interpolation filter is associated with a second interpolation filter, the interpolation filter being for integer pixel positions and the second interpolation filter being for fractional pixel positions. [C11] The method of C10, wherein the interpolation filter and the second interpolation filter have different cutoff frequencies. [C12] The method of C1, wherein the encoded video bitstream includes a value that identifies the interpolation filter. [C13] The method of claim 1, wherein the set of interpolation filters is not signaled along with the encoded video bitstream, and identities of the interpolation filters may be implied from the encoded video bitstream. [C14] 1. An apparatus comprising: a memory configured to store video data; obtaining the video data; determining a subset of interpolation filters from a set of interpolation filters for a coding unit, where the subset of interpolation filters is determined based on information in the video data associated with the coding unit; encoding the coding unit, wherein the encoding process includes selecting an interpolation filter for motion estimation and motion compensation, wherein the interpolation filter is selected from a subset of the interpolation filters; generating an encoded video bitstream, wherein the encoded video bitstream includes the encoded coding unit; a processor configured to: An apparatus comprising: [C15] The apparatus of C14, wherein the apparatus comprises a mobile device having a camera for capturing pictures. [C16] A computer-readable medium having instructions stored thereon that, when executed by a processor, cause the processor to: Obtaining video data; and determining a subset of interpolation filters from a set of interpolation filters for a coding unit, where the subset of interpolation filters is determined based on information in the video data associated with the coding unit; encoding the coding unit, where encoding the coding unit includes selecting an interpolation filter for motion estimation and motion compensation, where the interpolation filter is selected from a subset of the interpolation filters; generating an encoded video bitstream, wherein the encoded video bitstream includes the encoded coding unit; 23. A computer-readable medium for causing a computer to perform operations including: [C17] 1. An apparatus for encoding video data, comprising: means for acquiring video data; means for determining a subset of interpolation filters from a set of interpolation filters for a coding unit, wherein the subset of interpolation filters is determined based on information in the video data associated with the coding unit; means for encoding the coding unit, wherein the means for encoding the coding unit includes selecting an interpolation filter for motion estimation and motion compensation, wherein the interpolation filter is selected from a set of interpolation filters; means for generating an encoded video bitstream, wherein the encoded video bitstream includes the encoded coding unit; An apparatus comprising: [C18] 1. A method for decoding video data, comprising the steps of: Obtaining an encoded video bitstream; determining a reference frame for a coding unit in a current frame from the encoded video bitstream; determining a subset of interpolation filters from a set of interpolation filters, where the subset of interpolation filters is determined based on information in the encoded video bitstream associated with the coding unit; determining an interpolation filter from the subset of interpolation filters; using the reference frames and the interpolation filters to reconstruct the coding unit; A method for providing the above. [C19] The method of C18, wherein the subset of interpolation filters is determined based on a coding level of the coding unit. [C20] The method of C18, wherein the subset of interpolation filters is determined based on a prediction mode for the coding unit. [C21] The method of C18, wherein the subset of interpolation filters is determined based on motion information associated with the coding unit. [C22] The method of C18, wherein the subset of interpolation filters is determined based on a coding tool used to code the coding unit. [C23] The method of C18, wherein the subset of interpolation filters is determined based on a size of the coding unit. [C24] The method of C18, wherein the interpolation filters are selected from previously coded blocks. [C25] The method of C18, wherein the interpolation filters are selected from neighboring blocks. [C26] merging the interpolation filter with a second interpolation filter, where the resulting merged interpolation filter is used to reconstruct the coding unit. The method of C18, further comprising: [C27] The method of C18, wherein the interpolation filter is associated with a second interpolation filter, the interpolation filter being for integer pixel positions and the second interpolation filter being for fractional pixel positions. [C28] The method of C27, wherein the interpolation filter and the second interpolation filter have different cutoff frequencies. [C29] The method of C18, wherein the encoded video bitstream includes a value that identifies the interpolation filter. [C30] deriving an identity of the interpolation filter using the encoded video bitstream. The method of C18, further comprising: [C31] 1. An apparatus comprising: a memory configured to store the encoded video data; Obtaining an encoded video bitstream; determining a reference frame for a coding unit in a current frame from the encoded video bitstream; determining a subset of interpolation filters from a set of interpolation filters, where the subset of interpolation filters is determined based on information in the encoded video bitstream associated with the coding unit; determining an interpolation filter from the subset of interpolation filters; using the reference frames and the interpolation filters to reconstruct the coding unit; a processor configured to: An apparatus comprising: [C32] a display for displaying said video data The apparatus of C31, further comprising: [C33] The apparatus of C31, wherein the apparatus comprises a mobile device having a camera for capturing pictures. [C34] A computer-readable medium having instructions stored thereon that, when executed by a processor, cause the processor to: Obtaining an encoded video bitstream; determining a reference frame for a coding unit in a current frame from the encoded video bitstream; determining a subset of interpolation filters from a set of interpolation filters, where the subset of interpolation filters is determined based on information in the encoded video bitstream associated with the coding unit; determining an interpolation filter from the subset of interpolation filters; using the reference frames and the interpolation filters to reconstruct the coding unit; 23. A computer-readable medium for causing a computer to perform operations including: [C35] 1. An apparatus for decoding video data, comprising: means for obtaining an encoded video bitstream; means for determining a reference frame for a coding unit in a current frame from the encoded video bitstream; means for determining a subset of interpolation filters from a set of interpolation filters, wherein the subset of interpolation filters is determined based on information in the encoded video bitstream associated with the coding unit; means for determining an interpolation filter from the subset of interpolation filters; means for using the reference frames and the interpolation filter to reconstruct the coding unit; An apparatus comprising:

Claims

1. 1. A method for encoding video data, comprising the steps of: obtaining the video data; encoding, for a coding unit, a syntax element indicating a subset of interpolation filters from a set of interpolation filters, comprising determining a context model for encoding the syntax element based on motion information including reference frames to be used for encoding the coding unit; encoding the coding unit, where encoding the coding unit includes selecting an interpolation filter for motion estimation and motion compensation for the coding unit, where the interpolation filter is selected from a subset of the interpolation filters; generating an encoded video bitstream, wherein the encoded video bitstream includes the encoded coding unit; A method for providing the above.

2. The method of claim 1 , wherein the context model is further determined based on interpolation filters of neighboring blocks of the coding unit.

3. The method of claim 1 , wherein the context model is determined further based on a prediction direction of the coding unit.

4. The method of claim 1 , wherein the context model is determined based on whether a first or second reference list contains the reference frame.

5. An apparatus comprising: a memory configured to store video data; obtaining the video data; encoding, for a coding unit, a syntax element indicating a subset of interpolation filters from a set of interpolation filters, where, for encoding the syntax element, the processor is further configured to determine a context model based on motion information including reference frames to be used for encoding the coding unit; encoding the coding unit, where encoding the coding unit includes selecting an interpolation filter for motion estimation and motion compensation, where the interpolation filter is selected from a subset of the interpolation filters; generating an encoded video bitstream, wherein the encoded video bitstream includes the encoded coding unit; a processor configured to: An apparatus comprising:

6. The apparatus of claim 5 , wherein the apparatus comprises a mobile device having a camera for capturing pictures.

7. The apparatus of claim 5 , wherein the processor is further configured to determine the context model based on interpolation filters of neighboring blocks of the coding unit.

8. The apparatus of claim 5 , wherein the processor is further configured to determine the context model based on a prediction direction of the coding unit.

9. The apparatus of claim 5 , wherein the processor is further configured to determine the context model based on whether a first or second reference list includes the reference frame.

10. 1. An apparatus for encoding video data, comprising: means for acquiring video data; means for encoding, for a coding unit, a syntax element indicating a subset of an interpolation filter from a set of interpolation filters, the encoding comprising means for determining a context model for encoding the syntax element based on motion information including a reference frame to be used for encoding the coding unit; means for encoding the coding unit, wherein the means for encoding the coding unit includes selecting an interpolation filter for motion estimation and motion compensation, wherein the interpolation filter is selected from a set of interpolation filters; means for generating an encoded video bitstream, wherein the encoded video bitstream includes the encoded coding unit; An apparatus comprising:

11. 1. A method for decoding video data, comprising the steps of: Obtaining an encoded video bitstream; determining a reference frame for a coding unit in a current frame from the encoded video bitstream; decoding a syntax element indicating a subset of interpolation filters from a set of interpolation filters; the decoding comprising: determining a context model for decoding the syntax element based on the reference frame; determining the subset of interpolation filters from the set of interpolation filters based on the syntax elements; determining an interpolation filter from the subset of interpolation filters; using the reference frames and the interpolation filters to reconstruct the coding unit; A method for providing the above.

12. 12. The method of claim 11, wherein the interpolation filter is associated with a second interpolation filter, the interpolation filter being for integer pixel positions and the second interpolation filter being for fractional pixel positions.

13. The method of claim 11 , wherein the context model is further determined based on interpolation filters of neighboring blocks of the coding unit.

14. The method of claim 11 , wherein the context model is determined further based on a prediction direction of the coding unit.

15. The method of claim 11 , wherein the context model is determined based on whether a first or second reference list contains the reference frame.

16. An apparatus comprising: a memory configured to store the encoded video data; Obtaining an encoded video bitstream; determining a reference frame for a coding unit in a current frame from the encoded video bitstream; decoding a syntax element indicating a subset of interpolation filters from a set of interpolation filters; the decoding comprising: determining a context model for decoding the syntax element based on the reference frame; determining the subset of interpolation filters from the set of interpolation filters based on the syntax elements; determining an interpolation filter from the subset of interpolation filters; using the reference frames and the interpolation filters to reconstruct the coding unit; a processor configured to: An apparatus comprising:

17. a display for displaying said video data 20. The apparatus of claim 16, further comprising:

18. The apparatus of claim 16 , wherein the apparatus comprises a mobile device having a camera for capturing pictures.

19. The apparatus of claim 16 , wherein the processor is further configured to determine the context model based on interpolation filters of neighboring blocks of the coding unit.

20. The apparatus of claim 16 , wherein the processor is further configured to determine the context model based on a prediction direction of the coding unit.

21. The apparatus of claim 16 , wherein the processor is further configured to determine the context model based on whether a first or second reference list includes the reference frame.

22. 1. An apparatus for decoding video data, comprising: means for obtaining an encoded video bitstream; means for determining a reference frame for a coding unit in a current frame from the encoded video bitstream; means for decoding a syntax element indicating a subset of interpolation filters from a set of interpolation filters; the decoding comprising determining a context model for decoding the syntax element based on the reference frame; means for determining the subset of interpolation filters from the set of interpolation filters based on the syntax elements; means for determining an interpolation filter from the subset of interpolation filters; means for using the reference frames and the interpolation filter to reconstruct the coding unit; An apparatus comprising:

Images