Intra prediction using enhanced interpolation filters

Abstract

Techniques for processing video data using an enhanced interpolation filter for intra prediction are described herein. For example, a device may determine an intra prediction mode for predicting a block of video data. The device may determine a type of smoothing filter to use for the block of video data, where the type of smoothing filter is determined at least in part based on comparing at least one of a width of the block of video data and a height of the block of video data to a first threshold. The device may further perform intra prediction on the block of video data using the determined type of smoothing filter and the intra prediction mode.

Description

[Technical Field]

[0001] TECHNICAL FIELD This application relates to video coding (e.g., including encoding and / or decoding of video data). For example, aspects of this application relate to systems and techniques for performing intra prediction using enhanced interpolation filters. [Background technology]

[0002]

[0003] Many devices and systems enable video data to be processed and output for consumption. Digital video data comprises a large amount of data to meet the demands of consumers and video providers. For example, consumers of video data desire the highest quality video, having high fidelity, resolution, frame rates, etc. As a result, the large amount of video data required to meet these demands places a strain on communication networks and devices that process and store the video data.

[0003] Various video coding techniques may be used to compress video data. Video coding is performed according to one or more video coding standards. For example, video coding standards include, among others, Versatile Video Coding (VVC), High Efficiency Video Coding (HEVC), Advanced Video Coding (AVC), MPEG-2 Part 2 coding (MPEG stands for Moving Picture Experts Group), as well as proprietary video codecs / formats such as AOMedia Video 1 (AV1) developed by the Alliance for Open Media. Video coding generally utilizes prediction methods (e.g., inter-prediction, intra-prediction, etc.) that exploit redundancies present in video images or sequences. The goal of video coding techniques is to compress video data into a form that uses a lower bitrate while avoiding or minimizing degradation of video quality. As ever-evolving video services become available, encoding techniques with better coding efficiency are needed. [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] B. Bross, J. Chen, S. Liu, "Versatile Video Coding (Draft 10)", 19th JVET Meeting,Teleconference, July 2020, JVET-S2001 [Non-patent document 2] J.Chen, Y.Ye, S.Kim, "Algorithm description for Versatile Video Coding and Test Model 10(VTM10)", 19th JVET Meeting, Teleconference, July 2020, JVET-S2002 [Non-patent document 3] X. Zhao, V. Seregin, M. Karczewicz, "Six tap intra interpolation filter", 4th JVET Meeting, Chengdu, China, October 2016, JVET-D0119 Summary of the Invention [Means for solving the problem]

[0005] In some examples, systems and techniques are described for performing intra prediction using enhanced interpolation filters that may apply various types and degrees of smoothing based on information such as block size, intra prediction mode, etc. According to at least one illustrative example, a method for processing video data is provided. The method includes determining an intra prediction mode for predicting a block of video data, determining a type of smoothing filter to use for the block of video data, where the type of smoothing filter is determined at least in part based on comparing at least one of a width of the block of video data and a height of the block of video data to a first threshold, and performing intra prediction on the block of video data using the determined type of smoothing filter and the intra prediction mode.

[0006] In another example, an apparatus is provided for processing video data, the apparatus including at least one memory (e.g., configured to store data such as virtual content data, one or more images, etc.) and at least one processor (e.g., implemented in circuitry) coupled to the at least one memory. The one or more processors are configured to and capable of: determining an intra-prediction mode for predicting a block of video data; determining a type of smoothing filter to use for the block of video data, where the type of smoothing filter is determined at least in part based on comparing at least one of a width of the block of video data and a height of the block of video data to a first threshold; and performing intra-prediction on the block of video data using the determined type of smoothing filter and the intra-prediction mode.

[0007] In another example, a non-transitory computer-readable medium is provided having stored thereon instructions that, when executed by one or more processors, cause the one or more processors to determine an intra-prediction mode for a block of video data; determine a type of smoothing filter to use for the block of video data, where the type of smoothing filter is determined at least in part based on comparing at least one of a width of the block of video data and a height of the block of video data to a first threshold; and perform intra-prediction for the block of video data using the determined type of smoothing filter and the intra-prediction mode.

[0008] In another example, an apparatus for processing video data is provided that includes means for determining an intra-prediction mode for predicting a block of video data, means for determining a type of smoothing filter to use on the block of video data, where the type of smoothing filter is determined at least in part based on comparing at least one of a width of the block of video data and a height of the block of video data to a first threshold, and means for performing intra-prediction on the block of video data using the determined type of smoothing filter and the intra-prediction mode.

[0009] In some aspects, the process, apparatus, and computer-readable medium may further include using a first smoothing interpolation filter as the determined type of smoothing filter based at least in part on a determination that at least one of the block width and the block height is greater than a first threshold, and determining reference pixels for intra-prediction of the block of video data using the first smoothing interpolation filter.

[0010] In some aspects, the first smoothing interpolation filter includes a 6-tap Gaussian filter.

[0011] In some aspects, the process, apparatus, and computer-readable medium may further include using a second smoothing interpolation filter as the determined type of smoothing filter based at least in part on determining that at least one of the block width and the block height is less than or equal to the first threshold, and determining reference pixels for intra-prediction of the block of video data using the second smoothing interpolation filter.

[0012] In some aspects, the second smoothing interpolation filter includes a 4-tap Gaussian filter.

[0013] In some aspects, the process, apparatus, and computer-readable medium may further include determining a minimum offset between the angular direction of the intra-prediction mode and one of the vertical intra-prediction mode and the horizontal intra-prediction mode, and determining a type of smoothing filter to use for the block of video data based on comparing the determined minimum offset to a second threshold.

[0014] In some aspects, the process, apparatus, and computer-readable medium may further include determining a low-pass filter as the type of smoothing filter based at least in part on determining that the determined minimum offset is greater than a second threshold and determining that the intra-prediction mode is an integer-angled mode associated with an integer-valued reference pixel position.

[0015] In some aspects, the low pass filter performs reference pixel smoothing without interpolation, and the low pass filter comprises a ┌┘ filter.

[0016] In some aspects, the process, apparatus, and computer-readable medium may further include determining a Gaussian filter as the type of smoothing filter based at least in part on determining that the determined minimum offset is greater than a second threshold and determining that the intra-prediction mode is a fractional angle mode associated with a fractional reference pixel position.

[0017] In some aspects, the Gaussian filter performs smooth interpolation without reference pixel smoothing.

[0018] In some aspects, the Gaussian filter includes a 6-tap Gaussian filter based on a determination that at least one of the block width and the block height is greater than a first threshold.

[0019] In some aspects, the Gaussian filter includes a 4-tap Gaussian filter based on determining that at least one of the block width and the block height is less than or equal to a first threshold.

[0020] In some aspects, the process, apparatus, and computer-readable medium may further include using an interpolation filter as the determined type of smoothing filter based at least in part on a determination that the determined minimum offset is less than or equal to a second threshold, where the interpolation filter is a 4-tap cubic filter, and performing intra prediction on the block of video data using the interpolation filter without applying reference pixel smoothing.

[0021] In some aspects, the process, apparatus, and computer-readable medium may further include determining a low-pass filter as the type of smoothing filter based at least in part on determining that the intra-prediction mode is an integer angle mode and determining that the determined minimum offset is greater than a second threshold.

[0022] In some aspects, the process, apparatus, and computer-readable medium may further include performing reference pixel smoothing using a large-tap low-pass filter based at least in part on determining that at least one of the block width and the block height is greater than a first threshold, wherein the large-tap low-pass filter applies a higher degree of reference pixel smoothing than the small-tap low-pass filter.

[0023] In some aspects, the process, apparatus, and computer-readable medium may further include performing reference pixel smoothing using a small tap low pass filter based at least in part on determining that at least one of the block width and the block height is less than or equal to a first threshold, the small tap low pass filter applying a lesser degree of reference pixel smoothing than the large tap low pass filter.

[0024] In some aspects, the process, apparatus, and computer-readable medium may further include determining that the intra-prediction mode is an intra-angle mode based at least in part on comparing the slope of the intra-prediction mode with one or more pixel positions determined from the block width and the block height.

[0025] In some aspects, the process, apparatus, and computer-readable medium may further include determining that an offset between the angular direction of the intra-prediction mode and the vertical intra-prediction mode or the horizontal intra-prediction mode is less than a second threshold, and performing intra-prediction on the block of video data using a cubic interpolation filter based on determining that the offset between the angular direction of the intra-prediction mode and the vertical intra-prediction mode or the horizontal intra-prediction mode is less than the second threshold.

[0026] In some aspects, the process, apparatus, and computer-readable medium may further include performing reference line extension using a weak interpolation filter, the weak interpolation filter being used to perform the reference line extension prior to using a cubic interpolation filter to perform intra prediction, the cubic interpolation filter having a higher cutoff frequency than the weak interpolation filter and applying a higher degree of smoothing than the weak interpolation filter.

[0027] In some aspects, the weak interpolation filter includes a 4-tap sinc-based interpolation filter and a 6-bit 4-tap interpolation filter.

[0028] In some aspects, the type of smoothing filter is signaled in the video bitstream.

[0029] In some aspects, the type of smoothing filter is signaled for an individual set of a prediction block, a coding block, a coding tree unit (CTU), a slice, or a set of sequences.

[0030] In some aspects, the process, apparatus, and computer-readable medium may further include determining the type of smoothing filter based on at least one of the width and height of the block without using information explicitly signaled in the video bitstream.

[0031] In some aspects, the process, apparatus, and computer-readable medium may further include determining a block of residual data related to the block of video data, and decoding the block of video data using the block of residual data and the predictive block determined based on performing intra prediction on the block of video data.

[0032] In some aspects, the process, apparatus, and computer-readable medium may further include generating an encoded video bitstream that includes information related to the block of video data.

[0033] In some aspects, the process, apparatus, and computer-readable medium may further include storing (eg, in at least one memory of the apparatus) the encoded video bitstream.

[0034] In some aspects, the process, apparatus, and computer-readable medium may further include transmitting (eg, using a transmitter of the apparatus) the encoded video bitstream.

[0035] In some aspects, each of the devices described above may be or may be part of a mobile device (e.g., a mobile phone or so-called “smartphone,” a tablet computer, or other type of mobile device), a network-connected wearable device, an extended reality device (e.g., a virtual reality (VR) device, an augmented reality (AR) device, or a mixed reality (MR) device), a personal computer, a laptop computer, a server computer (e.g., a video server or other server device), a television, a vehicle (or a vehicular computing device or system), a camera (e.g., a digital camera, an Internet Protocol (IP) camera, etc.), a multi-camera system, a robotics device or system, an aerial device or system, or other device. In some aspects, each of the devices may include at least one camera for capturing one or more images or video frames. For example, each of the devices may include one camera (e.g., an RGB camera) or multiple cameras for capturing one or more images and / or one or more videos, including video frames. In some aspects, each of the devices may include a display for displaying one or more images, videos, notifications, or other displayable data. In some aspects, each of the devices may include a transmitter configured to transmit one or more video frames and / or syntax data to at least one device over a transmission medium. In some aspects, each of the devices may include one or more sensors.

[0036] This Summary is not intended to identify key or essential features of the claimed subject matter, nor is it intended to be used alone to determine the scope of the claimed subject matter, which subject matter should be understood by reference to the entire specification of this patent, any or all drawings, and appropriate portions of each claim.

[0037] The above, together with other features and embodiments, will become more apparent with reference to the following specification, claims, and accompanying drawings.

[0038] Exemplary embodiments of the present application are described in detail below with reference to the following figures: [Brief explanation of the drawings]

[0039] [Figure 1] 1 is a block diagram illustrating an example of an encoding device and a decoding device according to some examples. [Figure 2A] FIG. 10 illustrates an example of an angular prediction mode, according to some examples. [Figure 2B] FIG. 1 illustrates an example of directional intra-prediction modes in versatile video coding (VVC), according to some examples. [Figure 3] FIG. 1 illustrates an example of a mode dependent intra smoothing (MDIS) process, according to some examples. [Figure 4] FIG. 10 illustrates an example of reference line extension, according to some examples. [Figure 5] FIG. 10 illustrates an example of switchable Gaussian filtering based on one or more of block size and intra-prediction mode, according to some examples. [Figure 6] 1 is a flowchart illustrating an example of a process for performing intra prediction using an extended smoothing filter, according to some examples. [Figure 7] 1 is a block diagram illustrating an exemplary video encoding device according to some examples. [Figure 8] 1 is a block diagram illustrating an exemplary video decoding device according to some examples. DETAILED DESCRIPTION OF THE INVENTION

[0040]

[0023] Several aspects and embodiments of the present disclosure are provided below. As will be apparent to those skilled in the art, some of these aspects and embodiments may be applied independently, and some of them may be applied in combination. In the following description, for purposes of explanation, specific details are set forth to provide a thorough understanding of the embodiments of the present application. However, it will be apparent that various embodiments may be practiced without these specific details. The figures and description are not intended to be limiting.

[0041] The following description provides exemplary embodiments only and is not intended to limit the scope, applicability, or configuration of the present disclosure. Rather, the following description of exemplary embodiments provides those skilled in the art with an enabling description for implementing the exemplary embodiments. It should be understood that various changes may be made in the function and arrangement of elements without departing from the spirit and scope of the present application, as set forth in the appended claims.

[0042] Digital video data can include large amounts of data, especially as the demand for high-quality video data continues to grow. For example, video data consumers desire increasingly higher quality video with high fidelity, resolution, frame rates, etc. However, the large amounts of video data required to meet such demands can place a significant strain on communication networks and devices that process and store the video data.

[0043] Video coding devices implement video compression techniques to efficiently encode and decode video data. Video compression techniques may include applying different prediction modes, including spatial prediction (e.g., intra-frame or intra-prediction), temporal prediction (e.g., inter-frame or inter-prediction), inter-layer prediction (across different layers of video data), and / or other prediction techniques to reduce or remove redundancy inherent in video sequences. A video encoder may partition each picture of an original video sequence into rectangular regions called video blocks or coding units (described in more detail below). These video blocks may be coded using particular prediction modes.

[0044] A video block may be divided into one or more groups of smaller blocks in one or more ways. A block may include a coding tree block, a prediction block, a transform block, and / or other appropriate block. In general, references to a "block" may refer to such a video block (e.g., a coding tree block, a coding block, a prediction block, a transform block, or other appropriate block or sub-block, as would be understood by one of ordinary skill in the art) unless otherwise specified. Furthermore, each of these blocks may also be referred to interchangeably herein as a "unit" (e.g., a coding tree unit (CTU), a coding unit, a prediction unit (PU), a transform unit (TU), etc.). In some cases, a unit may refer to a coding logical unit that is coded in a bitstream, and a block may refer to a portion of a video frame buffer that a process targets.

[0045] In an inter-prediction mode, a video encoder may search for a block similar to a block being coded in a frame (or picture) located at another temporal location, called a reference frame or picture. The video encoder may limit this search to a certain spatial displacement from the block to be coded. A two-dimensional (2D) motion vector, including a horizontal displacement component and a vertical displacement component, may be used to locate the best match. In an intra-prediction mode, the video encoder may use spatial prediction techniques to form a predicted block based on data from previously coded neighboring blocks in the same picture.

[0046] The video encoder may determine a prediction error. For example, the prediction may be determined as the difference between pixel values in the block being coded and the predicted block. The prediction error is sometimes referred to as a residual. The video encoder may also apply a transform (e.g., a discrete cosine transform (DCT) or other appropriate transform) to the prediction error to generate transform coefficients. After the transform, the video encoder may quantize the transform coefficients. The quantized transform coefficients and motion vectors may be represented using syntax elements and, together with control information, form a coded representation of the video sequence. In some instances, the video encoder may entropy code the quantized transform coefficients and / or syntax elements, thereby further reducing the number of bits required for their representation.

[0047] After entropy decoding and de-quantizing the received bitstream, the video decoder may use the syntax elements and control information discussed above to construct prediction data (e.g., a prediction block) for decoding the current frame. For example, the video decoder may add the predicted block and the compressed prediction error. The video decoder may determine the compressed prediction error by weighting the transform basis functions using the quantization coefficients. The difference between the reconstructed frame and the original frame is called the reconstruction error.

[0048] Video coding may be performed according to a particular video coding standard. Examples of video coding standards include, but are not limited to, ITU-T H.261, ISO / IEC MPEG-1 Visual, ITU-T H.262 or ISO / IEC MPEG-2 Visual, ITU-T H.263, ISO / IEC MPEG-4 Visual, its Scalable Video Coding (SVC) and Multiview Video Coding (MVC) extensions, Advanced Video Coding (AVC) or ITU-T H.264, its Scope and Screen Content Coding, High Efficiency Video Coding (HEVC) or ITU-T H.265, including 3D Video Coding (3D-HEVC), Multiview (MV-HEVC), and Scalable (SHVC) extensions, Versatile Video Coding (VVC) or ITU-T H.266 and its extensions, VP9, Alliance of Open Media (AOMedia) Video 1 (AV1), Essential Video Coding (EVC), among others.

[0049] As described above, a video encoder may partition each picture of an original video sequence into one or more smaller blocks or rectangular regions, which may then be coded using, for example, intra-prediction (or intra-frame prediction) to remove spatial redundancy inherent in the original video sequence. When a block is coded in an intra-prediction mode, a predictive block is formed based on previously coded and reconstructed blocks, which are available at both the video encoder and the video decoder to form a prediction reference. For example, spatial prediction of pixel values within a current block (e.g., currently being coded or currently being decoded) may be determined using pixel values of neighboring, previously coded blocks. These pixel values are used as reference pixels. The reference pixels may be organized into one or more reference pixel lines and / or reference pixel groups. In some examples, intra-prediction may be applied to both the luma and chroma components of a block.

[0050] Several different intra-prediction modes may be utilized to provide different spatial prediction techniques for forming reference or predicted blocks that are predicted based on data from previously coded neighboring blocks within the same picture (e.g., from reference pixels). The intra-prediction modes may include planar mode and DC mode and / or directional intra-prediction mode (also referred to as "normal intra-prediction mode"). In some examples, a single planar intra-prediction mode and a single DC intra-prediction mode may be used, along with multiple directional intra-prediction modes. The intra-prediction modes describe different variations or approaches for calculating pixel values within an area being coded based on reference pixel values. In an illustrative example, the HEVC standard provides 33 directional intra-prediction modes. In another illustrative example, VVC and / or VVC Test Model 5 (VTM5) extend the HEVC directional intra-prediction modes to provide a total of 93 directional intra-prediction modes.

[0051] In a video decoder, the selection of an intra-prediction mode for each coded block (e.g., the selection of an intra-prediction mode made by a video encoder when generating a coded block) may be determined (e.g., derived) by the decoder or signaled to the video decoder. For example, in some cases, the intra-prediction modes between neighboring blocks may be correlated (e.g., if two adjacent, previously coded blocks were predicted using intra-prediction mode 2, the best intra-prediction mode for the current block may also be intra-prediction mode 2). In some examples, for each current block, the video encoder and video decoder may calculate a most probable intra-prediction mode. The video encoder may also signal the intra-prediction mode to the video decoder (e.g., using a flag, a mode parameter, a mode selector, etc.).

[0052] As previously mentioned, the current VVC standard provides 93 directional intra-prediction modes. Each intra-prediction mode is associated with a different angle direction so that the intra-prediction modes are unique and non-overlapping. Directional intra-prediction modes can be classified as either integer angle modes or fractional (non-integer angle) modes. For a given block of video data, integer angle intra-prediction modes have reference pixels at integer positions; for example, integer angle intra-prediction modes have gradients that pass through the positions of reference pixels located around the currently coded block. In comparison, fractional intra-prediction modes do not have reference pixels at integer positions, but instead have gradients that pass through a point somewhere between two adjacent reference pixels (e.g., the gradient of a pixel at fractional position i+f (i: integer part, f: fractional part) passes through pixel i and pixel i+1).

[0053] According to the VVC standard, one or more smoothing filters and / or operations may be applied to reference pixels based on the intra-prediction mode. By smoothing or filtering the reference pixels, more accurate intra-prediction results may be obtained as the intra-prediction results are calculated from the smoothed reference pixels. In some examples, reference pixel smoothing may be performed for both fractional and integer (e.g., integer gradient) intra-prediction modes. In addition to smoothing filters for reference pixel smoothing, the VVC standard also specifies the utilization of one or more interpolation filters. In some examples, smoothing may be performed by directly smoothing the reference pixels. In some examples, the smoothing operation may be combined with or performed in conjunction with an interpolation operation (e.g., by applying a smoothing interpolation filter).

[0054] For example, an interpolation filter may be used to perform interpolation for fractional intra-prediction modes. Fractional intra-prediction modes are non-integer value gradients and are therefore associated with fractional reference pixel positions (e.g., at positions between neighboring reference pixels). Intra-prediction for fractional intra-prediction modes may therefore interpolate between values of neighboring reference pixels to calculate interpolated values for fractional reference pixel positions. In some scenarios, most of the directional intra-prediction modes may be fractional (e.g., non-integer) modes. For example, in the VVC standard, intra-prediction modes -14, -12, -10, -6, 2, 18, 34, 50, 66, 72, 76, 78, and 80 may be integer intra-prediction modes (also referred to as "integer gradient modes"), and the remaining modes of the 93 directional intra-prediction modes are fractional intra-prediction modes.

[0055] The VVC standard specifies the use of a fixed degree of smoothing for all block sizes. For example, according to the VVC standard, a coding device (e.g., a video encoding device and / or a video decoding device) may use a 4-tap Gaussian interpolation filter and / or a

[0121] low-pass filter for all block sizes. In some cases, using a fixed degree of smoothing for all block sizes (e.g., a 4-tap Gaussian interpolation filter and / or a

[0121] low-pass filter for all block sizes) may result in reduced intra prediction performance. For example, larger block sizes (e.g., blocks having a width and / or height of 16 or more samples) may benefit from a higher degree of smoothing than smaller block sizes (e.g., blocks having a width and / or height of less than 16 samples). Because the block partitioning scheme in VVC allows different block sizes based on different inputs, parameters, and other analysis factors, large and small block sizes may be encountered when performing intra prediction according to the VVC standard. In some cases, larger block sizes may be associated with portions of the original video sequence image that already contain relatively smooth edges and / or a relatively low number of features. Small block sizes may be associated with portions of the original video sequence images that contain a relatively large number of features, orientations, and so on.

[0056] Because the creation of larger block sizes is often associated with the presence of relatively smooth video data within the blocks, intra prediction of larger block sizes may, in some instances, benefit from applying a higher degree of smoothing, while intra prediction of smaller block sizes may benefit from applying a lower degree of smoothing.

[0057] As described in more detail herein, systems, apparatuses, methods, and computer-readable media (collectively “systems and techniques”) for providing improved intra prediction are described herein. For example, as described in more detail herein, the systems and techniques may perform intra prediction using multiple smoothing and / or interpolation filters, each with a different degree of smoothing and / or filtering. According to some aspects, the systems and techniques may include selecting one or more smoothing filters and interpolation filters (and associated types of smoothing and / or associated degrees of smoothing) based on the size of the block currently being coded. For example, one or more of the block width and block height may be compared to a predetermined threshold, with smaller blocks (e.g., blocks having widths and / or heights less than the threshold) receiving a different degree or smoothing than larger blocks (e.g., blocks having widths and / or heights greater than the threshold).

[0058] In some examples, smoothing and / or interpolation filters may additionally or alternatively be selected based on the intra-prediction mode being used for a picture or a portion of a picture (e.g., a block, a slice, etc.). The relationship between a particular intra-prediction mode and a smoothing or interpolation filter may be determined in advance and / or in real time (e.g., as a picture, a block, a slice, etc. is encoded or decoded). In an illustrative example, the intra-prediction mode of a currently coded block may be compared with the vertical and horizontal intra-prediction modes to determine a minimum distance (e.g., an angular distance or offset) between the current block intra-prediction mode and one of the vertical and horizontal intra-prediction modes. The minimum distance may be compared with a predetermined threshold (defined in the VVC standard in some examples) to determine whether smoothing and / or filtering should be applied to the currently coded block. In some examples, variable smoothing of reference pixels with block-level switching as described herein may provide enhanced intra-prediction, as described in more depth below.

[0059] Further details regarding the systems and techniques are described with respect to the figures.

[0060] 1 is a block diagram illustrating an example of a system 100 including an encoding device 104 and a decoding device 112. The encoding device 104 may be part of a source device, and the decoding device 112 may be part of a receiving device. The source device and / or receiving device may include an electronic device such as a mobile or fixed telephone handset (e.g., a smartphone, a cellular telephone, etc.), a desktop computer, a laptop or notebook computer, a tablet computer, a set-top box, a television, a camera, a display device, a digital media player, a video gaming console, a video streaming device, an Internet Protocol (IP) camera, or any other suitable electronic device. In some examples, the source device and receiving device may include one or more wireless transceivers for wireless communication. The coding techniques described herein are applicable to video coding in various multimedia applications, including streaming video transmission (e.g., over the Internet), television broadcasting or transmission, encoding digital video for storage on a data storage medium, decoding digital video stored on a data storage medium, or other applications. As used herein, the term coding may refer to encoding and / or decoding. In some examples, the system 100 may support one-way or two-way video transmission to support applications such as video conferencing, video streaming, video playback, video broadcasting, gaming, and / or video telephony.

[0061] Encoding device 104 (or encoder) may be used to encode video data using a video coding standard, format, codec, or protocol to generate an encoded video bitstream. Examples of video coding standards and formats / codecs include ITU-T H.261, ISO / IEC MPEG-1 Visual, ITU-T H.262 or ISO / IEC MPEG-2 Visual, ITU-T H.263, ISO / IEC MPEG-4 Visual, ITU-T H.264 (also known as ISO / IEC MPEG-4 AVC), including its Scalable Video Coding (SVC) and Multiview Video Coding (MVC) extensions, High Efficiency Video Coding (HEVC) or ITU-T H.265, and Versatile Video Coding (VVC) or ITU-T H.266. There are various extensions to HEVC dealing with multi-layer video coding, including range and screen content coding extensions, 3D video coding (3D-HEVC), and multiview extensions (MV-HEVC) and scalable extensions (SHVC). HEVC and its extensions are being developed by the Joint Collaboration Team on Video Coding (JCT-VC), as well as the Joint Collaboration Team on 3D Video Coding Extension Development (JCT-3V) of the ITU-T Video Coding Experts Group (VCEG) and the ISO / IEC Motion Picture Experts Group (MPEG). VP9, AOMedia Video 1 (AV1) developed by the Alliance for Open Media (AOMedia), and Essential Video Coding (EVC) are other video coding standards to which the techniques described herein may be applied.

[0062] VVC, the latest video coding standard, was developed by the Joint Video Experts Group (JEVT) of ITU-T and ISO / IEC to achieve, at least in part, substantial compression capabilities beyond HEVC for a wide range of applications. The VVC specification was finalized and published by both ITU-T and ISO / IEC in July 2020. The VVC specification specifies normative bitstream and picture formats, high-level syntax (HLS) and coding unit-level syntax, parsing processes, decoding processes, etc. VVC also specifies profile / tier / level (PTL) restrictions, byte stream formats, hypothetical reference decoders, and supplemental enhancement information (SEI) in annexes.

[0063] The systems and techniques described herein may be applied to any existing video codec (e.g., VVC, HEVC, AVC, or other suitable existing video codec) and / or may be efficient coding tools for any video coding standard being developed and / or future video coding standards. For example, the examples described herein may be performed using a video codec such as VVC, HEVC, AVC, and / or extensions thereof. However, the techniques and systems described herein may also be applicable to other coding standards, codecs, or formats, such as MPEG, JPEG (or other coding standards for still images), VP9, AV1, extensions thereof, or other suitable coding standards that are already available or not yet available or developed. For example, in some examples, encoding device 104 and / or decoding device 112 may operate according to a proprietary video codec / format, such as AV1, an extension of AVI, and / or a successor version of AV1 (e.g., AV2), or other proprietary format or industry standard. Thus, those skilled in the art will appreciate that while the techniques and systems described herein may be described with respect to a particular video coding standard, the description should not be construed as applying only to that particular standard.

[0064] 1, video source 102 may provide video data to encoding device 104. Video source 102 may be part of a source device or part of a device other than the source device. Video source 102 may include a video capture device (e.g., a video camera, a camera phone, a video phone, etc.), a video archive containing stored video, a video server or content provider providing video data, a video feed interface receiving video from a video server or content provider, a computer graphics system for generating computer graphics video data, a combination of such sources, or any other suitable video source.

[0065] Video data from the video source 102 may include one or more input pictures or frames. A picture or frame is a still image that is, in some cases, part of a video. In some examples, data from the video source 102 may be a still image that is not part of a video. In HEVC, VVC, and other video coding specifications, a video sequence may include a series of pictures. A picture may include three sample arrays, denoted SL, SCb, and SCr. SL is a two-dimensional array of luma samples, SCb is a two-dimensional array of Cb chrominance samples, and SCr is a two-dimensional array of Cr chrominance samples. Chrominance samples are sometimes referred to herein as “chroma” samples. A pixel refers to all three components (luma and chroma samples) for a given location within the picture's array. In other cases, a picture may be monochrome and include only an array of luma samples, in which case the terms pixel and sample may be used interchangeably. For example, the techniques described herein refer to individual samples for purposes of illustration, but the same techniques may be applied to pixels (e.g., all three sample components for a given location in an array of pictures). For example, the techniques described herein refer to pixels (e.g., all three sample components for a given location in an array of pictures), but the same techniques may be applied to individual samples.

[0066] The encoder engine 106 (or encoder) of the encoding device 104 encodes video data to generate a coded video bitstream. In some examples, a coded video bitstream (or "video bitstream" or "bitstream") is a series of one or more coded video sequences. A coded video sequence (CVS) includes a series of AUs, starting with an access unit (AU) having a random access point picture with some characteristics in the base layer, up to and including the next AU having a random access point picture with some characteristics in the base layer. For example, some characteristics of a random access point picture that starts a CVS may include a RASL flag equal to 1 (e.g., NoRaslOutputFlag). Otherwise, a random access point picture (having a RASL flag equal to 0) does not start a CVS. An access unit (AU) includes one or more coded pictures and control information corresponding to coded pictures that share the same output time. At the bitstream level, coded slices of a picture are encapsulated in data units called network abstraction layer (NAL) units. For example, an HEVC video bitstream may include one or more CVSs that contain NAL units. Each NAL unit has a NAL unit header. In one example, the header is 1 byte for H.264 / AVC (excluding multi-layer extensions) and 2 bytes for HEVC. Syntax elements in the NAL unit header take designated bits and are therefore recognizable to all kinds of systems, and in particular to transport layers, such as transport streams, real-time transport (RTP) protocols, file formats, etc.

[0067] Two classes of NAL units exist in the HEVC standard, including video coding layer (VCL) NAL units and non-VCL NAL units. A VCL NAL unit contains one slice or slice segment (described below) of coded picture data, while a non-VCL NAL unit contains control information related to one or more coded pictures. In some cases, NAL units are referred to as packets. An HEVC AU contains VCL NAL units that contain coded picture data and non-VCL NAL units that correspond to the coded picture data (if any).

[0068] An NAL unit may include a sequence of bits that form a coded representation of video data (e.g., a coded video bitstream, CVS of a bitstream, etc.), such as a coded representation of a picture in a video. The encoder engine 106 generates coded representations of pictures by partitioning each picture into multiple slices. Slices are independent of other slices, such that information in a slice is coded without depending on data from other slices in the same picture. A slice includes one or more slice segments, including an independent slice segment and, if present, one or more dependent slice segments that depend on previous slice segments. A slice is partitioned into coding tree blocks (CTBs) of luma samples and chroma samples. A CTB of luma samples and one or more CTBs of chroma samples, along with syntax for the samples, are called a coding tree unit (CTU). A CTU is sometimes called a "treeblock" or "largest coding unit" (LCU). A CTU is the basic processing unit for HEVC encoding. A CTU may be split into multiple coding units (CUs) of various sizes. A CU contains a luma sample array and a chroma sample array called a coding block (CB).

[0069] The luma CB and the chroma CB may be further split into prediction blocks (PBs). A PB is a block of luma or chroma component samples that uses the same motion parameters for inter prediction or intra block copy prediction (when available or enabled for use). A luma PB and one or more chroma PBs, together with associated syntax, form a prediction unit (PU). For inter prediction, a set of motion parameters (e.g., one or more motion vectors, reference indexes, etc.) is signaled in the bitstream for each PU and used for inter prediction of the luma PB and one or more chroma PBs. The motion parameters are sometimes referred to as motion information. A CB may also be partitioned into one or more transform blocks (TBs). A TB represents a square block of color component samples to which a residual transform (e.g., possibly the same two-dimensional transform) is applied to code the prediction residual signal. A transform unit (TU) represents a TB of luma samples and chroma samples and corresponding syntax elements.

[0070] The size of a CU corresponds to the size of a coding mode and may be square in shape. For example, the size of a CU may be 8x8 samples, 16x16 samples, 32x32 samples, 64x64 samples, or any other appropriate size up to the size of the corresponding CTU. The phrase "NxN" is used herein to refer to the pixel dimensions of a video block in terms of vertical and horizontal dimensions (e.g., 8 pixels x 8 pixels). The pixels in a block may be arranged in rows and columns. In some examples, a block may not have the same number of pixels in the horizontal direction as in the vertical direction. Syntax data associated with a CU may, for example, describe the partitioning of the CU into one or more PUs. The partitioning mode may differ between whether the CU is intra-prediction mode coded or inter-prediction mode coded. The PU may be partitioned to be non-square in shape. Syntax data associated with a CU may also, for example, describe the partitioning of a CU into one or more TUs according to a CTU. The TUs may be square or non-square in shape.

[0071] According to the HEVC standard, transforms may be performed using transform units (TUs). TUs may be different for different CUs. TUs may be sized based on the size of the PUs within a given CU. TUs may be the same size or smaller than the PUs. In some examples, residual samples corresponding to a CU may be subdivided into smaller units using a quadtree structure known as a residual quadtree (RQT). Leaf nodes of the RQT may correspond to TUs. Pixel difference values associated with the TUs may be transformed to generate transform coefficients. The transform coefficients may be quantized by the encoder engine 106.

[0072] Once a picture of video data is partitioned into CUs, the encoder engine 106 predicts each PU using a prediction mode. The prediction unit or prediction block is subtracted from the original video data to obtain a residual (described below). For each CU, the prediction mode may be signaled within the bitstream using syntax data. The prediction mode may include intra prediction (or intra-picture prediction) or inter prediction (or inter-picture prediction). Intra prediction exploits the correlation between spatially neighboring samples within a picture. For example, using intra prediction, each PU is predicted from neighboring image data in the same picture using, for example, DC prediction to find the average value of the PU, planar prediction to fit a flat surface to the PU, directional prediction to extrapolate from neighboring data, or any other suitable type of prediction. Inter prediction uses temporal correlation between pictures to derive motion-compensated predictions for blocks of image samples. For example, using inter prediction, each PU is predicted using motion-compensated prediction from image data in one or more reference pictures (before or after the current picture in output order). The decision to code a picture area using inter-picture prediction or intra-picture prediction may be made, for example, at the CU level.

[0073] The encoder engine 106 and the decoder engine 116 (described in more detail below) may be configured to operate according to VVC. According to some VVCs, a video coder (such as the encoder engine 106 and / or the decoder engine 116) partitions a picture into multiple coding tree units (CTUs). (A CTB for luma samples and one or more CTBs for chroma samples, together with the syntax for the samples, are referred to as a CTU.) The video coder can partition the CTUs according to a tree structure, such as a quad-tree binary tree (QTBT) structure or a multi-type tree (MTT) structure. The QTBT structure eliminates the concept of multiple partition types, such as the distinction between CUs, PUs, and TUs in HEVC. The QTBT structure includes two levels, including a first level partitioned according to quad-tree partitioning and a second level partitioned according to binary tree partitioning. The root node of the QTBT structure corresponds to a CTU. The leaf nodes of the binary tree correspond to coding units (CUs).

[0074] In the MTT partitioning structure, blocks may be partitioned using quadtree partitioning, binary tree partitioning, and one or more types of triple tree partitioning. Triple tree partitioning is a partition in which a block is split into three sub-blocks. In some examples, triple tree partitioning divides a block into three sub-blocks without splitting the original block through the center. Partition types in MTT (e.g., quadtree, binary tree, and triple tree) can be symmetric or asymmetric.

[0075] When operating according to the AV1 codec, encoding device 104 and decoding device 112 may be configured to code video data in blocks. In AV1, the largest coding block that can be processed is called a superblock. In AV1, a superblock can be either 128x128 luma samples or 64x64 luma samples. However, in successor video coding formats (e.g., AV2), a superblock can be defined by a different (e.g., larger) luma sample size. In some examples, a superblock is the top level of a block quadtree. Encoding device 104 may further partition the superblock into smaller coding blocks. Encoding device 104 may partition the superblock and other coding blocks into smaller blocks using rectangular or non-rectangular partitions. Non-rectangular blocks may include N / 2xN, NxN / 2, N / 4xN, and NxN / 4 blocks. Encoding device 104 and decoding device 112 may perform separate prediction and transform processes for each of the coding blocks.

[0076] AV1 also defines tiles of video data. A tile is a rectangular array of superblocks that may be coded independently of other tiles. That is, encoding device 104 and decoding device 112 may encode and decode coding blocks within a tile, respectively, without using video data from other tiles. However, encoding device 104 and decoding device 112 may perform filtering across tile boundaries. Tiles may be uniform or non-uniform in size. Tile-based coding may enable parallel processing and / or multithreading for encoder and decoder implementations.

[0077] In some examples, the encoding device 104 and the decoding device 112 may use a single QTBT structure or MTT structure to represent each of the luminance and chrominance components, while in other examples, the video coder may use two or more QTBT structures or MTT structures, such as one QTBT structure or MTT structure for the luminance component and another QTBT structure or MTT structure for both chrominance components (or two QTBT structures and / or MTT structures for each chrominance component).

[0078] The encoding device 104 and the decoding device 112 may be configured to use quadtree partitioning per HEVC, QTBT partitioning, MTT partitioning, or other partitioning structures.

[0079] In some examples, one or more slices of a picture are assigned a slice type. Slice types include I slice, P slice, and B slice. An I slice (independently decodable intra frame) is a slice of a picture that is coded only by intra prediction and is therefore independently decodable because it only requires data within the frame to predict any prediction unit or prediction block of the slice. A P slice (unidirectionally predicted frame) is a slice of a picture that can be coded using both intra prediction and unidirectional inter prediction. Each prediction unit or prediction block in a P slice is coded using either intra prediction or inter prediction. When inter prediction is applied, a prediction unit or prediction block is predicted by only one reference picture, and therefore the reference samples are from only one reference region of a frame. A B slice (bidirectionally predicted frame) is a slice of a picture that can be coded using both intra prediction and inter prediction (e.g., either bi-predictive or uni-predictive). A prediction unit or prediction block of a B slice may be bidirectionally predicted from two reference pictures, where each picture contributes to one reference region, and the sample sets of the two reference regions are weighted (for example, with equal weights or with different weights) to generate a prediction signal of the bidirectionally predicted block. As mentioned above, the slices of a picture are coded independently. In some cases, a picture may be coded as just one slice.

[0080] As mentioned above, intra-picture prediction exploits the correlation between spatially neighboring samples within a picture. There are multiple intra-prediction modes (also referred to as "intra modes"). In some examples, intra-prediction of luma blocks includes 35 modes, including planar mode, DC mode, and 33 angular modes (e.g., diagonal intra-prediction mode and angular modes adjacent to the diagonal intra-prediction mode). The encoding device 104 and / or decoding device 112 may select a prediction mode for each block that minimizes the residual between the predictive block and the block to be coded (e.g., based on the sum of absolute error (SAE), sum of absolute differences (SAD), sum of absolute transform error (SATD), or other measure of similarity). For example, the SAE may be calculated by utilizing the absolute difference between each pixel (or sample) in the block to be coded and the corresponding pixel (or sample) in the predictive block being used for comparison. The pixel (or sample) differences may be summed to create a measure of block similarity, such as an L1-norm difference image, Manhattan distance between two image blocks, or other calculation. Using SAE as an example, the SAE for prediction using each of the intra-prediction modes indicates the magnitude of the prediction error. The intra-prediction mode with the best match for the actual current block is given by the intra-prediction mode that provides the smallest SAE.

[0081] The 35 modes of intra prediction are indexed as shown in Table 1 below. In other examples, more intra modes may be defined, including prediction angles that may not already be represented by the 33 angle modes. In other examples, the prediction angles associated with the angle modes may differ from those used in HEVC.

[0082] [Table 1]

[0083] To perform planar prediction for an N × N block, for each sample p located at (x, y), xy For , the predicted sample value can be calculated by applying a bilinear filter to four specific neighboring reconstructed samples (used as reference samples for intra prediction). The four reference samples are the top right reconstructed sample TR, the bottom left reconstructed sample BL, and the sample in the same column as the current sample (r x,-1 ) and row (r -1,y ) The planar mode can be formulated as follows: p xy =((N-x1)*·L+(N-y1)*·T+x1*·R+y1*·B) / (2*N)

[0084] where x1=x+1, y1=y+1, R=TR, and B=BL.

[0085] In DC mode, the predicted block is filled by the average value of neighboring reconstructed samples. In general, both planar and DC modes are applied to model smoothly changing and unchanging image regions.

[0086] For angular intra-prediction modes in HEVC, which have 33 different prediction directions, the intra-prediction process may be described as follows: For each given angular intra-prediction mode, the intra-prediction direction may be identified accordingly, e.g., intra-mode 18 corresponds to a purely horizontal prediction direction, and intra-mode 26 corresponds to a purely vertical prediction direction. The angular prediction modes are shown in the example diagram 200a of FIG. 2A. In some codecs, a different number of intra-prediction modes may be used. For example, in addition to planar mode and DC mode, 93 angular modes may be defined, where mode 2 indicates a −135° prediction direction, mode 34 indicates a −45° prediction direction, and mode 66 indicates a 45° prediction direction. In some codecs (e.g., VVC), angles greater than (less than) −135° and greater than 45° may also be defined, which may be referred to as wide-angle intra-modes. Although the description herein relates to intra-mode design in HEVC (i.e., having 35 modes), the disclosed techniques may also be applied to more intra-modes (e.g., intra-modes defined by VVC or other codecs).

[0087] The coordinates (x, y) of each sample of the prediction block are projected along a particular intra-prediction direction (e.g., one of the angular intra-prediction modes). For example, assuming a particular intra-prediction direction, the coordinates (x, y) of a sample of the prediction block are first projected to the row / column of a neighboring reconstructed sample along the intra-prediction direction. If (x, y) is projected to a fractional position α between two neighboring reconstructed samples L and R, the predicted value of (x, y) may be calculated using a two-tap bilinear interpolation filter formulated as follows: p xy =(1-a) L + a R

[0088] To avoid floating point operations, in HEVC the above calculation can be approximated using integer arithmetic as follows: p xy =((32-a')·L+a'·R+16)>>5

[0089] where a' is an integer equal to 32*a.

[0090] In some examples, before intra prediction, the neighboring reference samples are filtered using a 2-tap bilinear or 3-tap (1,2,1) / 4 filter, which may be referred to as intra-reference smoothing or mode-dependent intra smoothing (MDIS). When performing intra prediction, given an intra-prediction mode index (predModeIntra) and a block size (nTbS), it determines whether a reference smoothing process is performed and which smoothing filter is used. The intra-prediction mode index is an index that indicates the intra-prediction mode.

[0091] Inter-picture prediction uses temporal correlation between pictures to derive motion-compensated predictions for blocks of image samples. Using a translational motion model, the position of a block in a previously decoded picture (reference picture) is indicated by a motion vector (Δx, Δy), where Δx specifies the horizontal displacement of the reference block relative to the position of the current block and Δy specifies its vertical displacement. In some cases, the motion vector (Δx, Δy) can be integer sample precision (also called integer precision), in which case the motion vector points to the integer pel grid (or integer pixel sampling grid) of the reference frame. In some cases, the motion vector (Δx, Δy) can be fractional sample precision (also called fractional pel precision or non-integer precision) to more accurately capture the motion of the underlying object without being restricted to the integer pel grid of the reference frame. The precision of the motion vector is represented by the quantization level of the motion vector. For example, the quantization level can be integer precision (e.g., 1 pixel) or fractional pel precision (e.g., 1 / 4 pixel, 1 / 2 pixel, or other sub-pixel value). When the corresponding motion vector has fractional sample accuracy, interpolation is applied to the reference picture to derive a prediction signal. For example, available samples at integer positions may be filtered (e.g., using one or more interpolation filters) to estimate values at fractional positions. A previously decoded reference picture is indicated by a reference index (refIdx) to a reference picture list. The motion vector and the reference index may be referred to as motion parameters. Two types of inter-picture prediction may be performed, including uni-prediction and bi-prediction.

[0092] With inter-prediction using bi-prediction (also called bidirectional inter-prediction), two sets of motion parameters (Δx0, y0, refIdx0 and Δx1, y1, refIdx1) are used to generate two motion-compensated predictions (from the same reference picture or possibly from different reference pictures). For example, with bi-prediction, each prediction block uses two motion-compensated prediction signals to generate B prediction units. To obtain a final motion-compensated prediction, the two motion-compensated predictions are combined. For example, the two motion-compensated predictions may be combined by averaging. In another example, weighted prediction may be used, in which case a different weight may be applied to each motion-compensated prediction. Reference pictures that may be used in bi-prediction are stored in two separate lists, denoted as List 0 and List 1. The motion parameters may be derived in the encoding device 104 using a motion estimation process.

[0093] When using inter prediction using uni-prediction (also called unidirectional inter prediction), a set of motion parameters (Δx0, y0, refIdx0) is used to generate a motion-compensated prediction from a reference picture. For example, when using uni-prediction, each prediction block uses at most one motion-compensated prediction signal to generate P prediction units.

[0094] A PU may include data related to the prediction process (e.g., motion parameters or other suitable data). For example, when a PU is encoded using intra prediction, the PU may include data describing the intra prediction mode of the PU. As another example, when a PU is encoded using inter prediction, the PU may include data defining a motion vector of the PU. The data defining the motion vector of the PU may describe, for example, the horizontal component (Δx) of the motion vector, the vertical component (Δy) of the motion vector, the resolution of the motion vector (e.g., integer precision, ¼ pixel precision, or ⅛ pixel precision), the reference picture to which the motion vector points, the reference index, the reference picture list (e.g., list 0, list 1, or list C) of the motion vector, or any combination thereof.

[0095] AV1 includes two general techniques for encoding and decoding coding blocks of video data. The two general techniques are intra-prediction (e.g., intra-frame prediction or spatial prediction) and inter-prediction (e.g., inter-frame prediction or temporal prediction). In the context of AV1, when predicting a block of a current frame of video data using an intra-prediction mode, encoding device 104 and decoding device 112 do not use video data from other frames of the video data. For most intra-prediction modes, video encoding device 104 encodes the block of the current frame based on the difference between sample values in the current block and predicted values generated from reference samples in the same frame. Video encoding device 104 determines the predicted values generated from the reference samples based on the intra-prediction mode.

[0096] After performing prediction using intra prediction and / or inter prediction, the encoding device 104 may perform transform and quantization. For example, following prediction, the encoder engine 106 may calculate a residual value corresponding to the PU. The residual value may include pixel difference values between the current block of pixels being coded (PU) and a predictive block (e.g., a predicted version of the current block) used to predict the current block. For example, after generating a predictive block (e.g., issuing inter prediction or intra prediction), the encoder engine 106 may generate a residual block by subtracting the predictive block generated by the prediction unit from the current block. The residual block includes a set of pixel difference values that quantify differences between pixel values of the current block and pixel values of the predictive block. In some examples, the residual block may be represented in a two-dimensional block format (e.g., a two-dimensional matrix or array of pixel values). In such examples, the residual block is a two-dimensional representation of pixel values.

[0097] Any residual data that may remain after prediction is performed is transformed using a block transform, which may be based on a discrete cosine transform, a discrete sine transform, an integer transform, a wavelet transform, other suitable transform functions, or any combination thereof. In some cases, one or more block transforms (e.g., of size 32x32, 16x16, 8x8, 4x4, or other suitable sizes) may be applied to the residual data in each CU. In some examples, TUs may be used for the transform and quantization processes implemented by the encoder engine 106. A given CU having one or more PUs may also include one or more TUs. As described in further detail below, residual values may be transformed into transform coefficients using a block transform, and may be quantized and scanned using the TUs to generate serialized transform coefficients for entropy coding.

[0098] In some examples, following intra-predictive coding or inter-predictive coding using a PU of a CU, encoder engine 106 may calculate residual data for the TUs of the CU. The PU may include pixel data in the spatial domain (or pixel domain). The TUs may comprise coefficients in the transform domain after applying a block transform. As mentioned above, the residual data may correspond to pixel difference values between pixels of the uncoded picture and predicted values corresponding to the PU. Encoder engine 106 may form TUs including the residual data for the CU and then transform the TUs to generate transform coefficients for the CU.

[0099] The encoder engine 106 may perform quantization of the transform coefficients. Quantization achieves further compression by quantizing the transform coefficients to reduce the amount of data used to represent the coefficients. For example, quantization may reduce the bit depth associated with some or all of the coefficients. In one example, a coefficient having an n-bit value may be truncated to an m-bit value during quantization, where n is greater than m.

[0100] Once quantization is performed, the coded video bitstream includes any other suitable data, such as quantized transform coefficients, prediction information (e.g., prediction modes, motion vectors, block vectors, etc.), partition information, and other syntax data. Different elements of the coded video bitstream may be entropy coded by the encoder engine 106. In some examples, the encoder engine 106 may scan the quantized transform coefficients utilizing a predefined scan order to generate serialized vectors that can be entropy coded. In some examples, the encoder engine 106 may perform adaptive scanning. After scanning the quantized transform coefficients to form vectors (e.g., one-dimensional vectors), the encoder engine 106 may entropy code the vectors. For example, the encoder engine 106 may use context-adaptive variable length coding, context-adaptive binary arithmetic coding, syntax-based context-adaptive binary arithmetic coding, probability interval partition entropy coding, or another suitable entropy coding technique.

[0101] The output 110 of the encoding device 104 may send the NAL units that make up the coded video bitstream data to a decoding device 112 of a receiving device via a communication link 120. The input 114 of the decoding device 112 may receive the NAL units. The communication link 120 may include channels provided by a wireless network, a wired network, or a combination of wired and wireless networks. The wireless network may include any wireless interface or combination of wireless interfaces, including any suitable wireless network (e.g., the Internet or other wide area network, a packet-based network, WiFi™, radio frequency (RF), UWB, WiFi-Direct, cellular, Long Term Evolution (LTE), WiMax™, etc.). The wired network may include any wired interface (e.g., fiber, Ethernet, powerline Ethernet, Ethernet over coaxial cable, digital signal line (DSL), etc.). The wired and / or wireless networks may be implemented using a variety of equipment, such as base stations, routers, access points, bridges, gateways, switches, etc. The encoded video bitstream data may be modulated according to a communication standard, such as a wireless communication protocol, and transmitted to a receiving device.

[0102] In some examples, encoding device 104 may store the encoded video bitstream data in storage 108. Output unit 110 may retrieve the encoded video bitstream data from encoder engine 106 or from storage 108. Storage 108 may include any of a variety of distributed or locally accessed data storage media. For example, storage 108 may include a hard drive, a storage disk, flash memory, volatile or non-volatile memory, or any other suitable digital storage medium for storing encoded video data. Storage 108 may also include a decoded picture buffer (DPB) for storing reference pictures for use in inter-prediction. In further examples, storage 108 may correspond to a file server or another intermediate storage device that may store encoded video generated by a source device. In such cases, receiving devices, including decoding device 112, can access the stored video data from the storage device via streaming or download. The file server may be any type of server capable of storing encoded video data and transmitting the encoded video data to receiving devices. Exemplary file servers include web servers (e.g., for websites), FTP servers, network-attached storage (NAS) devices, or local disk drives. The receiving device may access the encoded video data through any standard data connection, including an Internet connection, and may include a wireless channel (e.g., a Wi-Fi connection), a wired connection (e.g., DSL, cable modem, etc.), or a combination of both, that is suitable for accessing encoded video data stored on a file server. The transmission of the encoded video data from storage 108 may be a streaming transmission, a download transmission, or a combination thereof.

[0103] The input 114 of the decoding device 112 may receive the encoded video bitstream data and provide the video bitstream data to the decoder engine 116 or to the storage 118 for later use by the decoder engine 116. For example, the storage 118 may include a DPB for storing reference pictures for use in inter-prediction. A receiving device including the decoding device 112 may receive the encoded video data to be decoded via the storage 108. The encoded video data may be modulated according to a communication standard, such as a wireless communication protocol, and transmitted to the receiving device. The communication medium for the transmitted encoded video data may comprise any wireless or wired communication medium, such as a radio frequency (RF) spectrum or one or more physical transmission lines. The communication medium may form part of a packet-based network, such as a local area network, a wide area network, or a global network such as the Internet. The communication medium may include routers, switches, base stations, or any other equipment that may be useful for facilitating communication from a source device to a receiving device.

[0104] The decoder engine 116 may decode the encoded video bitstream data by entropy decoding and extracting (e.g., using an entropy decoder) elements of one or more coded video sequences that make up the encoded video data. The decoder engine 116 may rescale and perform an inverse transform on the encoded video bitstream data. The residual data is passed to a prediction stage of the decoder engine 116. The decoder engine 116 predicts blocks of pixels (e.g., PUs). In some examples, the prediction is added to the output of the inverse transform (the residual data).

[0105] Decoding device 112 may output the decoded video to video destination device 122, which may include a display or other output device for displaying the decoded video data to a content consumer. In some aspects, video destination device 122 may be part of a receiving device that includes decoding device 112. In some aspects, video destination device 122 may be part of a separate device other than the receiving device.

[0106] In some examples, the video encoding device 104 and / or the video decoding device 112 may be integrated with an audio encoding device and an audio decoding device, respectively. The video encoding device 104 and / or the video decoding device 112 may also include other hardware or software necessary to implement the coding techniques described above, such as one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), discrete logic, software, hardware, firmware, or any combination thereof. The video encoding device 104 and the video decoding device 112 may be integrated as part of a combined encoder / decoder (codec) in the respective devices. An example of specific details of the encoding device 104 is described below with reference to FIG. 8. An example of specific details of the decoding device 112 is described below with reference to FIG. 9.

[0107] The exemplary system shown in FIG. 1 is one illustrative example that may be used herein. Techniques for processing video data using the techniques described herein may be performed by any digital video encoding and / or decoding device. Generally, the techniques of this disclosure are performed by a video encoding device or a video decoding device, but the techniques may also be performed by a composite video encoder / decoder, commonly referred to as a "codec." Additionally, the techniques of this disclosure may also be performed by a video preprocessor. The source device and receiving device are merely examples of such coding devices, in which the source device generates coded video data for transmission to the receiving device. In some examples, the source device and receiving device may operate substantially symmetrically, such that each of the devices includes video encoding and decoding components. Thus, the exemplary system may support one-way or two-way video transmission between video devices, for example, for video streaming, video playback, video broadcasting, or video telephony.

[0108] Extensions to the HEVC standard include a multiview video coding extension called MV-HEVC and a scalable video coding extension called SHVC. MV-HEVC and SHVC extensions share the concept of layered coding, in which different layers are included in the coded video bitstream. Each layer in a coded video sequence is addressed by a unique layer identifier (ID). The layer ID may be present in the header of an NAL unit to identify the layer with which the NAL unit is associated. In MV-HEVC, different layers can represent different views of the same scene in the video bitstream. In SHVC, different scalable layers are provided that represent the video bitstream at different spatial resolutions (i.e., picture resolutions) or with different reconstruction fidelity. The scalable layers may include a base layer (with layer ID = 0) and one or more enhancement layers (with layer ID = 1, 2, ... n). The base layer may conform to the first version profile of HEVC and represent the lowest available layer in the bitstream. Enhancement layers have increased spatial resolution, temporal resolution or frame rate, and / or reconstruction fidelity (i.e., quality) compared to the base layer. Enhancement layers are organized hierarchically and may (or may not) depend on lower layers. In some examples, different layers may be coded using a single-standard codec (e.g., all layers are coded using HEVC, SHVC, or other coding standard). In some examples, different layers may be coded using a multi-standard codec. For example, the base layer may be coded using AVC, and one or more enhancement layers may be coded using SHVC and / or MV-HEVC extensions to the HEVC standard.

[0109] Generally, a layer includes a set of VCL NAL units and a corresponding set of non-VCL NAL units. NAL units are assigned specific layer ID values. Layers may be hierarchical, in the sense that a layer may depend on lower layers. A layer set refers to a set of layers represented in a bitstream that is self-contained, meaning that a layer in a layer set can depend on other layers in the layer set in the decoding process but does not depend on any other layers for decoding. Thus, layers in a layer set can form an independent bitstream that can represent video content. A set of layers in a layer set can be obtained from another bitstream by the operation of a sub-bitstream extraction process. A layer set may correspond to a set of layers to be decoded when a decoder wishes to operate according to certain parameters.

[0110] As previously described, an HEVC bitstream includes a group of NAL units, including VCL NAL units and non-VCL NAL units. VCL NAL units include coded picture data that form a coded video bitstream. For example, a sequence of bits that form a coded video bitstream resides in a VCL NAL unit. Non-VCL NAL units may include, in addition to other information, parameter sets that have high-level information about the coded video bitstream. For example, parameter sets may include a video parameter set (VPS), a sequence parameter set (SPS), and a picture parameter set (PPS). Example purposes of parameter sets include providing bitrate efficiency, error resiliency, and a system layer interface. Each slice references a single active PPS, SPS, and VPS to access information that decoding device 112 can use to decode the slice. An identifier (ID) may be coded for each parameter set and includes a VPS ID, an SPS ID, and a PPS ID. An SPS includes an SPS ID and a VPS ID. A PPS includes a PPS ID and an SPS ID. Each slice header contains a PPS ID, which can be used to identify the active parameter set for a given slice.

[0111] A PPS contains information that applies to all slices in a given picture. In some examples, all slices in a picture reference the same PPS. Slices in different pictures may also reference the same PPS. An SPS contains information that applies to all pictures in the same coded video sequence (CVS) or bitstream. As previously described, a coded video sequence is a series of access units (AUs) starting with a random access point picture (e.g., an instantaneous decoding reference (IDR) picture or a broken link access (BLA) picture, or other suitable random access point picture) with certain characteristics (described above) in the base layer until just before the next access unit (AU) that has a random access point picture with certain characteristics in the base layer (or the end of the bitstream). Information in an SPS may not change from picture to picture in a coded video sequence. Pictures in a coded video sequence may use the same SPS. A VPS contains information that applies to all layers in a coded video sequence or bitstream. A VPS contains a syntax structure with syntax elements that apply to the entire coded video sequence. In some embodiments, the VPS, SPS, or PPS may be transmitted in-band with the encoded bitstream. In some embodiments, the VPS, SPS, or PPS may be transmitted out-of-band in a separate transmission from the NAL units containing the coded video data.

[0112] This disclosure may generally refer to “signaling” certain information, such as syntax elements. The term “signaling” may generally refer to communication of values for syntax elements and / or other data used to decode encoded video data. For example, video encoding device 104 may signal values for syntax elements in a bitstream. Generally, signaling refers to generating values in the bitstream. As mentioned above, video source 102 may transport the bitstream to video destination device 122 substantially in real time or not in real time, such as may occur when storing syntax elements in storage 108 for later retrieval by video destination device 122.

[0113] A video bitstream may also include supplemental enhancement information (SEI) messages. For example, an SEI NAL unit may be part of the video bitstream. In some cases, an SEI message may include information that is not required by the decoding process. For example, the information in an SEI message may not be essential for a decoder to decode video pictures of the bitstream, but the decoder may use the information to improve the display or processing of the pictures (e.g., the decoded output). The information in an SEI message may be embedded metadata. In an illustrative example, the information in an SEI message may be used by a decoder-side entity to improve the viewability of the content. In some instances, some application standards may require the presence of such SEI messages in the bitstream so that quality improvements can be provided to all devices that comply with the application standard (e.g., the conveyance of frame packing SEI messages for frame-compatible planar stereoscopic 3DTV video formats in which SEI messages are conveyed for every frame of the video, the processing of recovery point SEI messages, the use of pan-scan rectangular SEI messages in DVB, among many other examples).

[0114] As described above, encoding device 104 may encode one or more blocks or rectangular regions of a picture of the original video sequence using intra-prediction and / or intra-frame prediction to remove spatial redundancy. Decoding device 112 may decode the encoded blocks by using the same intra-prediction mode used by encoding device 104. Intra-prediction modes describe different variations or approaches for calculating pixel values within the area being coded based on reference pixel values. In the VVC standard, one or more smoothing filters and interpolation filters may be selected based on the intra-prediction mode and then applied to the reference pixels and / or intra-prediction of the current block. In this approach, the same selection of smoothing or interpolation filters used for intra-prediction is provided for all block sizes, e.g., a fixed degree of smoothing is applied to all possible block sizes. Different directional intra-prediction modes are provided in the VVC standard.

[0115] FIG. 2B shows an example diagram 200b of directional intra-prediction modes (also referred to as "angular intra-prediction modes") in VVC. In some examples, planar and DC modes remain the same in VVC as they are in HEVC. As shown, intra-prediction modes with even indices between 2 and 66 may be equivalent to 33 HEVC intra-prediction modes, and the remaining intra-prediction modes in FIG. 2B represent newly added intra-prediction modes in VVC. As an illustrative example, to better capture arbitrary edge directions presented in natural video, the number of directional intra-prediction modes in VTM5 (VVC Test Model 5) has been increased from 33 HEVC directions to a total of 93 directions. Intra-prediction modes are described in more detail in B. Bross, J. Chen, and S. Liu, "Versatile Video Coding (Draft 10)," 19th JVET Meeting, Teleconference, July 2020, JVET-S2001, which is incorporated by reference in its entirety for all purposes. In some examples, the denser directional intra-prediction modes introduced in the VVC standard may be applied to all block sizes and to both luma and chrominance intra-prediction. In some cases, these directional intra-prediction modes may be used in combination with either multiple reference lines (MLR) and / or intra-sub-partition modes (ISP). Further details are described in J. Chen, Y. Ye, S. Kim, "Algorithm description for Versatile Video Coding and Test Model 10 (VTM10)," 19th JVET Meeting, Teleconference, July 2020, JVET-S2002, which is incorporated herein by reference in its entirety for all purposes.

[0116] In some examples, mode-dependent intra smoothing (MDIS) may be utilized to smooth an intra-predicted signal by applying a smoothing filter and / or a type of smoothing based on the intra-prediction mode of a currently coded block. FIG. 3 is a flow diagram illustrating an example of an MDIS process 300 that may be used for intra-prediction. In an illustrative example, the example MDIS process of FIG. 3 may be the same as the MDIS process of the VVC standard. The example MDIS process 300 may be used to select a particular interpolation filter and / or a particular smoothing filter to be used in intra-prediction for a currently coded block. As described in more detail below, in some examples, the selection of the interpolation and / or smoothing filter may be based at least in part on the intra-prediction mode of the currently coded block.

[0117] The example MDIS process 300 may begin at operation 302 by determining whether the intra-prediction mode of the currently coded block is a horizontal intra-prediction mode or a vertical intra-prediction mode. With reference to the directional intra-prediction modes shown in FIG. 2B , the horizontal intra-prediction mode is shown as mode 18, and the vertical intra-prediction mode is shown as mode 50. In response to a determination at operation 302 that the intra-prediction mode is either a horizontal mode or a vertical mode (e.g., a “Yes” output of 302), the example MDIS process may proceed to operation 304. As shown, operation 304 terminates the MDIS process without performing reference pixel smoothing or applying an interpolation filter. In some examples, no smoothing or interpolation may be performed for the horizontal and vertical intra-prediction modes because the reference pixel values for these two modes may be directly copied in determining the predicted pixel values of the current block.

[0118] If the intra-prediction mode is not a horizontal mode or a vertical mode (e.g., a “No” output of operation 302), the exemplary MDIS process may proceed to determining whether smoothing is required for the current block. As shown, the determination of whether smoothing should be performed on the current block may be performed in operation 306 based at least in part on the intra-prediction mode of the current block. For example, the intra-prediction mode may be used to calculate a minimum distance minDistVerHor, where minDistVerHor is the minimum of {|intra-prediction mode number-vertical intra-prediction mode number|, |intra-prediction mode number-horizontal intra-prediction mode number|}. The minimum distance minDistVerHor may also be referred to as a minimum angular offset and / or a minimum angular distance. In an illustrative example, the number of vertical intra-prediction modes may be 50, and the number of horizontal intra-prediction modes may be 18. Thus, if the number of intra prediction modes of the current block is 30, the minimum angle offset may be calculated as min{|30-50|, |30-18|}=min{20,12}=12.

[0119] In operation 306, the minimum angular offset minDistVerHor may then be compared to a threshold value intraHorVerDistThres[nTbS], which in some examples may be a predetermined threshold value provided by the VVC standard, for example, determined by providing the current transform block size nTbS as an index to a lookup function or lookup table intraHorVerDistThres. As shown in FIG. 3, if the minimum angular offset minDistVerHor is less than or equal to the threshold value intraHorVerDistThres[nTbS], operation 306 may determine that no smoothing is needed for the current block, e.g., a “No” output of 306.

[0120] If smoothing is not required, the exemplary MDIS process can proceed from operation 306 to operation 307, which is shown here as applying an interpolation filter without any reference pixel smoothing. In some examples, the interpolation filter applied by operation 307 may be a cubic interpolation filter, such as the 4-tap (6-bit) cubic interpolation filter shown in FIG. 3. Because operation 306 determined that direct reference pixel smoothing is not required, operation 307 may apply only a 4-tap cubic interpolation filter, and no reference pixel smoothing is performed, e.g., because the minimum angular offset indicated in operation 306 that the intra-prediction mode was within a threshold distance from either the horizontal mode or the vertical mode.

[0121] If operation 306 determines that the minimum angular offset minDistVerHor is greater than the threshold intraHorVerDistThres[nTbS], operation 306 may determine that smoothing is required for the current block, e.g., a "Yes" output. In response to determining that smoothing is required, the intra-prediction mode for the current block may be further analyzed in a subsequent operation 308.

[0122] In some examples, operation 308 may analyze the intra-prediction mode for the current block to determine whether it is an integer gradient intra-prediction mode or a fractional gradient intra-prediction mode (also referred to as an "integer angle mode" and a "fractional angle mode," respectively). As previously mentioned, integer angle modes are associated with specific integer-valued reference pixel positions of the current block, while fractional angle modes are not associated with specific integer-valued reference pixel positions. Instead, fractional angle modes are associated with some intermediate (e.g., fractional) positions between adjacent integer-valued reference pixel positions.

[0123] Based on operation 308 determining that the intra-prediction mode for the current block is an integer angle mode (e.g., a "Yes" output of 308), operation 308 can proceed to operation 309. As shown, in some cases, it is determined that interpolation is not necessary for the integer angle mode, so operation 309 may perform reference pixel smoothing but does not perform interpolation. For example, integer angle intra-prediction modes can use reference pixel values directly, so only reference pixel smoothing is performed. In some examples, the reference pixel smoothing of operation 309 may be performed by applying a low-pass filter, such as a

[0121] filter, which calculates the average of the sum of twice the reference pixel value plus the values of the immediately neighboring left and right (or above and below) reference pixel locations.

[0124] Based on operation 308 determining that the intra-prediction mode for the current block is a fractional angle mode (e.g., a non-integer angle mode, a "No" output of 308), in some cases, a subsequent operation 310 may calculate interpolated values for fractional reference pixel positions associated with the intra-prediction mode. For example, operation 310 may calculate interpolated fractional reference pixel position values, which may be calculated based on one or more reference pixel values obtained from one or more neighboring integer-valued reference pixel positions. Recall that operation 306 previously determined that smoothing should be performed on the intra-prediction of the current block (e.g., because operation 306 determined that the minimum angle offset minDistVerHor > the threshold value intraHorVerDistThres[nTbS]), a "No" output of operation 308 may correspond to a scenario in which both smoothing and interpolation are applied to the current block.

[0125] In some examples, as shown in FIG. 3, the smoothing operation and the interpolation operation may be performed in a single combined step, for example, by applying a smoothing interpolation filter. In an illustrative example, the smoothing interpolation filter may be provided as a Gaussian interpolation filter that smooths the generated intra-predicted signal and simultaneously interpolates the fractional reference pixel position values. The smoothing interpolation filter, such as the Gaussian interpolation filter described above, may apply smoothing without performing direct reference pixel smoothing. In some examples, the smoothing interpolation filter may include a 4-tap (6-bit) Gaussian interpolation filter, as shown in operation 310.

[0126] In the context of the example MDIS process 300 of Figure 3, it should be noted that the MDIS process (and the VVC standard) do not use variable degrees of smoothing based on block size or other characteristics. In some examples, the systems and techniques described herein may provide variable degrees of smoothing and / or interpolation based at least in part on factors including, but not limited to, the intra-prediction mode of the current block, the size of the current block, the width of the current block, the height of the current block, etc.

[0127] In some examples, video coding techniques may include using a directional intra-prediction mode together with one or more of a main reference line extension (MRL) and / or an intra sub-partition mode (ISP) to perform intra prediction. In an illustrative example, the intra prediction may include using one or more side reference pixels to extend the main reference pixel line for intra prediction.

[0128] FIG. 4 shows an example diagram 400 of reference line extension using one or more side reference pixels. Shown for a currently coded block 405 is an overline of reference pixels 410 including a series of calculated reference line extension pixels 420. Also shown is a set of left reference pixels 430. For intra prediction in vertical modes (e.g., intra prediction modes >= 34, so as not to be mixed with certain vertical intra prediction modes 50), one or more pixels from the left reference pixels 430 of the currently coded block 405 may be used to extend the overline of reference pixels 410, e.g., by generating or possibly calculating values for the reference line extension pixels 420. The calculation of the reference line extension pixels 420 may be used to extend the length of the overline of reference pixels 410 so that it extends beyond the left edge of the current block 405, as shown in FIG. 4.

[0129] In the current VVC standard, the top line of reference pixels 410 can be expanded by identifying a nearest neighbor in the left reference pixel 430, and the value of the identified nearest neighbor is set equal to the value of at least one of the reference line expansion pixels 420. In an illustrative example, FIG. 4 shows a point P (e.g., indicated by 423) located in the reference line expansion pixel 420 of the top line of reference pixels 410. The top reference pixel line 410 is expanded based on the left reference pixel 430. In the current VVC standard, the reference line expansion process proceeds by determining which of the left reference pixels 430 is the nearest neighbor to the expanded reference line pixel P 423, and then setting the value of the expanded reference line pixel P equal to the value of the identified nearest neighbor in the left reference pixel 430. As shown in FIG. 4, the nearest neighbor in the column of the left reference pixel 430 is denoted as X1, and the pixel value at the X1 location is therefore used to create the expanded reference line pixel P (e.g., 423). The top line of reference pixels 410 may be extended to a desired length using this methodology, after which intra prediction is performed using the extended reference line formed by the original top line of reference pixels 410 and the reference line extended pixels 420. In some examples, a similar process may be applied to intra prediction in horizontal modes (e.g., intra prediction modes < 34, so as not to be mixed with certain horizontal intra prediction modes 18), where the values of the identified nearest neighboring pixels in the top reference line are projected to extend the left line of reference pixels.

[0130] Various improvements to the VVC intra prediction process have been proposed in JVET-D0119, as described in X. Zhao, V. Seregin, and M. Karczewicz, "Six tap intra interpolation filter," 4th JVET Meeting, Chengdu, China, October 2016, JVET-D0119, which is incorporated herein by reference in its entirety for all purposes. For example, JVET-D0119 proposes improving the intra prediction process by introducing two methods: (1) using a 6-tap (8-bit) cubic interpolation instead of a 4-tap (6-bit) cubic interpolation as described above to perform the exemplary MDIS process of FIG. 3, and (2) using the same 4-tap (6-bit) cubic interpolation (again, as described above with respect to the exemplary MDIS process of FIG. 3) to perform the exemplary reference line extension described with respect to FIG. 4, instead of projecting nearest neighbor pixel values.

[0131] As mentioned above, in some instances, larger block sizes may benefit from applying a higher degree of smoothing during intra prediction. However, VVC uses a fixed degree of smoothing (e.g., 4-tap Gaussian interpolation or ∇ filtering) for all block sizes, which, in light of the above observations, may result in inefficient or less efficient intra prediction. With regard to JVET-D0119 discussed above, the use of 4-tap cubic interpolation to extend one or more lines of reference pixels (e.g., the top and / or left reference pixel lines) may be problematic because its use may result in over-smoothing when intra prediction is performed using the extended portion of the extended reference line, thereby introducing inaccuracy and / or inefficiency into the overall intra prediction process.

[0132] For example, over-smoothing may occur in such a situation because the extended pixels of the extended reference line are subject to at least two separate interpolation operations, each of which results in some degree of smoothing and edge degradation. The first interpolation operation is a 4-tap cubic interpolation to determine extended top / left reference pixel line values based on nearest neighbor values from left / top reference pixels, respectively. The interpolated reference pixel values of the extended reference pixel line may then participate in a second interpolation operation during intra prediction for the current block, such as the interpolation operation described with respect to the illustrated MDIS process of FIG. 3. For example, the interpolated reference pixel values of the extended reference pixel line may be utilized in one or more of 4-tap cubic interpolation, 4-tap Gaussian smoothing interpolation, and / or low-pass reference pixel smoothing, each of which may result in over-smoothing in the overall intra prediction process.

[0133] As previously mentioned, systems and techniques for performing intra prediction using one or more advanced interpolation filters are described herein. The systems and techniques may be performed by the encoding device 104, the decoding device 112, by both the encoding device 104 and the decoding device 112, and / or by other devices. The aspects described herein may be applied independently and / or in a combined manner. In some examples, the systems and techniques described herein may be used to perform one or more intra prediction modes (e.g., for filtering used during or in conjunction with application of the intra prediction modes).

[0134] In some examples, the systems and techniques described herein may provide variable degrees of reference pixel smoothing using block-level switching. For example, multiple smoothing filters and / or Gaussian interpolation filters (also referred to as "Gaussian smoothing interpolation filters"), each with a different degree of smoothing, may be used to smooth reference pixels during intra-prediction. In some cases, the selection of the determined smoothing filter and / or the determined interpolation filter may be explicitly signaled at different coding levels, for example, per prediction block, per coding block, per CTU, per slice, and / or per sequence (e.g., within an SPS) level. In some examples, the selection of the determined smoothing and / or interpolation filter may be implicitly determined using decoding information, including, but not limited to, block size, prediction mode, QP, and / or CU-level mode flags (MRL, ISP, etc.), in which case explicit signaling of the filter selection is not required. For example, in some examples, the encoding device 104 and / or the decoding device 112 may implicitly determine or select smoothing filters and / or interpolation filters to use in intra prediction based on a determination that the currently coded block has a certain size, has a width and / or height greater than a threshold, has a width and / or height less than a threshold, etc.

[0135] In one illustrative example, the processing of fractional angle (e.g., non-integer angle) intra-prediction modes can be extended from the approach described in the VVC standard to include selecting from a first Gaussian smoothing interpolation filter that applies a higher degree of smoothing and at least a second Gaussian smoothing interpolation filter that applies a lower degree of smoothing. As discussed above with respect to Figure 3, the approach used by the VVC standard utilizes the same 4-tap Gaussian smoothing interpolation filter for all fractional angle intra-prediction modes, regardless of the size of the block currently being coded.

[0136] FIG. 5 is an illustrative diagram illustrating one example of a process 500 for performing switchable smoothing and / or interpolation to apply variable degrees of intra-prediction smoothing based on at least the intra-prediction mode of a current block and the size of the current block. In the context of the example discussed immediately above, the disclosed systems and techniques for intra-prediction using an extended interpolation filter may include selecting a first filter comprising a 6-tap Gaussian smoothing filter or a second filter comprising a 4-tap Gaussian smoothing interpolation filter for a fractional-angle intra-prediction mode. The 6-tap Gaussian smoothing interpolation filter may apply a higher degree of smoothing than the 4-tap Gaussian smoothing interpolation filter. In some examples, the 4-tap Gaussian smoothing interpolation filter of FIG. 5 may be the same as or similar to the 4-tap Gaussian smoothing interpolation filter described with respect to the example VVC MDIS process 300 of FIG. 3. In some examples, the filtering, interpolation, and / or smoothing degree selection process may be implicit depending on the block size of the currently coded block, as seen in FIG. 5.

[0137] In some examples, the variable smoothing filtering and interpolation process for reference pixels with block-level switching shown in FIG. 5 may be the same as or similar to the example VVC MDIS process of FIG. 3 , except for operation 510 (e.g., comparing one or more of the width of the currently coded block and the height of the currently coded block to at least a first threshold T) and subsequent operations 512 (e.g., selecting and applying a 6-tap Gaussian smoothing interpolation filter with a relatively high degree of smoothing in response to exceeding the first threshold T) and 514 (e.g., selecting and applying a 4-tap Gaussian smoothing interpolation filter with a relatively low degree of smoothing in response to not exceeding the first threshold T).

[0138] At operation 502, the process may determine whether the intra-prediction mode for the currently decoded block is a horizontal intra-prediction mode (e.g., mode 18) or a vertical intra-prediction mode (e.g., mode 50). If the intra-prediction mode is either a horizontal mode or a vertical mode, the process determines at block 504 not to perform reference pixel smoothing (referred to as "reference pixel smoothing" in FIG. 5) and not to perform interpolation filtering, as previously described with respect to the example MDIS process of FIG. 3. The process may then proceed to process the currently coded block and perform intra prediction without applying reference pixel smoothing or interpolation filtering.

[0139] At operation 506, the process may determine whether the minimum angular offset minDistVerHor is greater than a threshold value intraHorVerDistThres[nTbS]. In some cases, one or more of minDistVerHor and / or intraHorVerDistThres[nTbS] may be the same as or similar to the corresponding variable values discussed above with respect to the example MDIS process of FIG. 3. In an illustrative example, the angular offset variable minDistVerHor may be set equal to Min(Abs(predModeIntra-50), Abs(predModeIntra-18)), where predModeIntra indicates the intra-prediction mode number, 50 indicates the vertical intra-prediction mode number, and 18 indicates the horizontal intra-prediction mode number. In some cases, predModeIntra may be set equal to IntraPredModeY[xCb][yCb] or IntraPredModeC[xCb][yCb]. In some examples, the threshold variable intraHorVerDistThres[nTbS] may be given as specified in Table 2 below for different values of the currently coded transform block size nTbS.

[0140] [Table 2]

[0141] In some examples, if operation 506 determines that the angular offset minDistVerHor is less than or equal to the value of the threshold variable intraHorVerDistThres[nTbS] (e.g., minDistVerHor≦intraHorVerDistThres[nTbs]), the process may decide not to perform reference pixel smoothing and may further decide to apply a 4-tap cubic interpolation filter for intra prediction of the currently coded block in operation 507. For example, the process may apply a 4-tap cubic filter to predict or interpolate one or more reference pixels without performing any reference pixel smoothing.

[0142] If operation 506 determines that the angle offset minDistVerHor is greater than the threshold intraHorVerDistThres[nTbS] (e.g., minDistVerHor>intraHorVerDistThres[nTbS]), the process may then determine, in operation 508, whether an integer angle mode is present among the intra prediction modes of the block currently being coded, as previously described with respect to the exemplary MDIS process of FIG.

[0143] In one example, when operation 508 determines that integer angle mode is present among the intra-prediction modes of the currently coded block, the process may determine in operation 509 to perform reference pixel smoothing using a

[0121] low-pass filter and not perform interpolation filtering. The process may then terminate in operation 509 after performing reference pixel smoothing to smooth the reference pixels using a

[0121] filter. No interpolation is performed, and the smoothed reference pixels are directly copied for intra-prediction for the currently coded block.

[0144] In one example, when operation 508 determines that a fractional (e.g., non-integer) angle mode is present among the intra-prediction modes of the currently coded block, the process may proceed to operation 510, where it may determine whether the width of the block is greater than or equal to a threshold T and / or whether the height of the block is greater than or equal to a threshold T. In some examples, operation 510 may include determining which of the block width and the block height is greater than or equal to the threshold T. In some examples, the value of the threshold T may be a predetermined value, e.g., 16, 32, 64, or one or more other predetermined values.

[0145] If operation 510 determines that the block width and block height are greater than or equal to threshold T (e.g., height ≧ T && width ≧ T), the process may then determine not to perform reference pixel smoothing and terminate by applying a 6-tap Gaussian smoothing interpolation filter to the intra prediction of the currently coded block at operation 512. For example, the process may apply a 6-tap Gaussian smoothing interpolation filter to predict one or more pixels of the current block without any reference pixel smoothing.

[0146] If the width or height of the block is not greater than the threshold T (e.g., height < T and / or width < T), the process may determine not to perform reference pixel smoothing in operation 514 and end by applying a 4-tap Gaussian smoothing interpolation filter. For example, the process may apply a 4-tap (6-bit) Gaussian smoothing interpolation filter to predict one or more pixels of the currently coded block without any reference pixel smoothing. As described above, for example, operation 514 may be triggered in response to operation 510 determining that the currently coded block has a relatively small size. Thus, the 4-tap Gaussian smoothing interpolation filter of operation 514 may apply a lower degree of smoothing than the 6-tap Gaussian smoothing interpolation filter of operation 512. Similarly, recalling that the 6-tap Gaussian smoothing interpolation filter applies a higher degree of smoothing and that a larger block size can benefit more from a greater smoothing compared to a smaller block size, the 6-tap Gaussian smoothing interpolation filter of operation 512 may be triggered based in part in response to operation 510 determining that the currently coded block has a relatively large block size.

[0147] In some cases, the exemplary 6-tap Gaussian smoothing interpolation filter applied in operation 514 may be derived using the convolution of one or more different phases of the [1 4 6 4 1] low-pass filter and the bilinear filter.

[0148] In one illustrative example, such as with respect to a scenario in which operation 508 determines that the intra-prediction mode of the currently coded block is an integer angle mode, operation 509 shown in FIG. 5 may be expanded to include a selection between a larger tap smoothing filter (e.g., a [1 4 6 4 1] low-pass filter, not shown) or a smaller tap

[0121] low-pass filter currently shown as being applied in connection with operation 509. In some examples, the selection criteria for selecting between the larger tap [1 4 6 4 1] filter or the smaller tap

[0121] filter may be performed in the same or similar manner as the selection criteria implemented in operation 510. For example, one or more of the width of the currently coded block and the height of the currently coded block may be compared to at least one threshold, with larger blocks (e.g., determined to be greater than or equal to the threshold) causing the larger tap [1 4 6 4 1] filter to be applied to the intra prediction, and smaller blocks (e.g., determined to be less than the threshold) causing the smaller tap

[0121] filter to be applied to the intra prediction. In some cases, one or more of the same or similar explicit and / or explicit selection processes described with respect to operation 510 as being based on factors such as block size may be used in such instances where the integer angle reference pixel smoothing of operation 509 is extended to select from different tap filters and / or degrees based on the block size currently being coded.

[0149] In some examples, the systems and techniques described herein may perform weak filtering interpolation for reference line extension, e.g., to avoid or minimize the over-smoothing problem discussed above as potentially occurring when reference line extension is based on 4-tap cubic interpolation, followed by another interpolation during intra prediction. For example, instead of using 4-tap cubic filtering to interpolate values for reference line extension pixels (e.g., interpolated based on nearest neighboring pixel values of a vertical pixel reference), a weaker filter-based interpolation may be used to reduce or mitigate possible over-smoothing problems that may arise in some cases in the context of extending reference lines. By utilizing weaker interpolation to determine values for reference line extension pixels, the rest of the intra prediction process and its associated interpolation and smoothing operations described herein may remain the same without the aforementioned over-smoothing problems.

[0150] In one illustrative example, a 4-tap sync-based interpolation (e.g., with appropriate windowing) may be used to provide weak interpolation for calculating interpolated values for reference line extended pixels. In some examples, the 4-tap sync-based interpolation may be weaker than cubic interpolation, such as a 4-tap cubic interpolation (e.g., with a higher cutoff frequency). In one illustrative example, the weak interpolation for reference line extended pixels may be provided as a 6-bit 4-tap weak filter, an example of which is provided below (note that the coefficient at position (32-i) / 32 is a mirror version of i / 32): {0,64,0,0}, / / 0 / 32 position {-1,64,1,0}, / / 1 / 32 position {-3,65,3,-1}, / / 2 / 32 position {-3,63,5,-1}, / / 3 / 32 position {-4,63,6,-1}, / / 4 / 32 position {-5,62,9,-2}, / / 5 / 32 position {-5,60,11,-2}, / / 6 / 32 position {-5,58,13,-2}, / / 7 / 32 position {-6,57,16,-3}, / / 8 / 32 position {-6,55,18,-3}, / / 9 / 32 position {-7,54,21,-4}, / / 10 / 32 position {-7,52,23,-4}, / / 11 / 32 position {-6,48,26,-4}, / / 12 / 32 position {-7,47,29,-5}, / / 13 / 32 position {-6,43,32,-5}, / / 14 / 32 position {-6,41,34,-5}, / / 15 / 32 position {-5,37,37,-5}, / / 16 / 32 position

[0151] These systems and techniques enable prediction (e.g., intra-prediction) to be performed using an enhanced interpolation filter. The systems and techniques described herein may, in some examples, provide advantages over other techniques that utilize multiple interpolation filters. For example, in some cases, multiple interpolation filters with, for example, different interpolation filter taps may be applied within one block, slice, tile, and / or picture. In one example, the interpolation filter type and interpolation filter tap (length) may depend on the block height and / or width, block shape (width-to-height ratio), block area size, intra-prediction mode, and / or neighboring decoded information, including, but not limited to, reconstructed sample values and intra-prediction mode. In such cases, when intra-prediction is a vertical-shaped angular intra-prediction mode and the width is 8 or less or other sizes, a 6-tap sixth-order interpolation filter is used, and in some cases, a 4-tap Gaussian interpolation filter is used. When intra-prediction is a horizontal-shaped intra-prediction mode and the width is 8 or less or other sizes, a 6-tap sixth-order interpolation filter is used, and in some cases, a 4-tap Gaussian interpolation filter is used. In one example using the systems and techniques described herein, if the width and height of the coding block are greater than or equal to a threshold T, a 6-tap Gaussian filter is used (and no pixel smoothing is applied), and in some cases a 4-tap Gaussian filter is used (and no pixel smoothing is applied).

[0152] 6 is a flow chart illustrating an example of a process 600 for processing image and / or video data. At block 602, the process 600 may include determining an intra-prediction mode for predicting a block of video data.

[0153] At block 604, process 600 may include determining a type of smoothing filter to use for the block of video data. For example, process 600 may determine the type of smoothing filter based at least in part on comparing at least one of a width of the block of video data and a height of the block of video data to a first threshold. In some aspects, the type of smoothing filter is signaled in the video bitstream. In some cases, the type of smoothing filter is signaled for an individual set of a predictive block, a coding block, a coding tree unit (CTU), a slice, or a set of sequences. At block 606, process 600 may include performing intra prediction on the block of video data using the determined type of smoothing filter and the intra prediction mode.

[0154] In some examples, process 600 may include using a first smoothing interpolation filter as the determined type of smoothing filter based at least in part on determining that the width of the block, the height of the block, or the width and height of the block are greater than a first threshold. In one illustrative example, the first smoothing interpolation filter includes a 6-tap Gaussian filter. In such examples, process 600 may further include using the first smoothing interpolation filter to determine reference pixels for intra-prediction of the block of video data.

[0155] In some examples, process 600 may include using a second smoothing interpolation filter as the determined type of smoothing filter based at least in part on determining that the width of the block, the height of the block, or the width and height of the block are less than or equal to (e.g., less than) a first threshold. In one illustrative example, the second smoothing interpolation filter includes a 4-tap Gaussian filter. In such examples, process 600 may further include using the second smoothing interpolation filter to determine reference pixels for intra-prediction of the block of video data.

[0156] In some cases, process 600 may include determining a minimum offset between the angular direction of the intra-prediction mode and one of the vertical intra-prediction mode and the horizontal intra-prediction mode. Process 600 may further include determining a type of smoothing filter to use for the block of video data based on comparing the determined minimum offset with a second threshold. In one example, process 600 may include determining a low-pass filter as the type of smoothing filter based at least in part on determining that the determined minimum offset is greater than the second threshold and determining that the intra-prediction mode is an integer angle mode associated with integer-valued reference pixel positions. In one illustrative example, the low-pass filter includes a

[0121] filter and performs reference pixel smoothing without interpolation.

[0157] In another example, the process 600 may include determining a Gaussian filter as a type of smoothing filter based at least in part on a determination that the determined minimum offset is greater than a second threshold and a determination that the intra-prediction mode is a fractional angle mode associated with a fractional-value reference pixel position. In some cases, the Gaussian filter performs smooth interpolation without reference pixel smoothing. In one illustrative example, the Gaussian filter includes a 6-tap Gaussian filter based on a determination that at least one of the block width and the block height is greater than a first threshold. In another illustrative example, the Gaussian filter includes a 4-tap Gaussian filter based on a determination that at least one of the block width and the block height is less than or equal to the first threshold.

[0158] In some aspects, process 600 may include using an interpolation filter as the determined type of smoothing filter based at least in part on a determination that the determined minimum offset is less than or equal to (e.g., less than) a second threshold. In one illustrative example, the interpolation filter includes a 4-tap cubic filter. Process 600 may further include performing intra prediction on the block of video data using the interpolation filter without applying reference pixel smoothing.

[0159] In some examples, process 600 may include determining a low-pass filter as the type of smoothing filter based at least in part on determining that the intra-prediction mode is an integer angle mode and determining that the determined minimum offset is greater than a second threshold. In some cases, process 600 may include performing reference pixel smoothing using a large-tap low-pass filter based at least in part on determining that the block width, the block height, or the block width and height are greater than a first threshold. The large-tap low-pass filter applies a higher degree of reference pixel smoothing than the small-tap low-pass filter. In some cases, process 600 may include performing reference pixel smoothing using a small-tap low-pass filter based at least in part on determining that the block width, the block height, or the block width and height are less than or equal to (e.g., less than) the first threshold. The small-tap low-pass filter applies a lower degree of reference pixel smoothing than the large-tap low-pass filter.

[0160] In some cases, process 600 may include determining that the intra prediction mode is an integer angle mode based at least in part on comparing the slope of the intra prediction mode with one or more pixel positions determined from the block width and the block height.

[0161] In some aspects, process 600 may include determining that an offset between the angular direction of the intra-prediction mode and the vertical intra-prediction mode or the horizontal intra-prediction mode is less than a second threshold. Process 600 may further include performing intra prediction on the block of video data using a cubic interpolation filter based on determining that the offset between the angular direction of the intra-prediction mode and the vertical intra-prediction mode or the horizontal intra-prediction mode is less than the second threshold.

[0162] In some examples, the process 600 may include performing reference line extension using a weak interpolation filter. In some cases, a weak interpolation filter is used to perform reference line extension prior to using a cubic interpolation filter to perform intra prediction. In some cases, the cubic interpolation filter has a higher cutoff frequency than the weak interpolation filter and applies a higher degree of smoothing than the weak interpolation filter. In some aspects, the weak interpolation filter includes a 4-tap sync-based interpolation filter and a 6-bit 4-tap interpolation filter.

[0163] In some aspects, process 600 may include determining the type of smoothing filter based on the block width, the block height, or the block width and height without using information explicitly signaled in the video bitstream.

[0164] In some cases, process 600 may be performed by a decoding device (e.g., decoding device 112 of FIGS. 1 and 8). For example, process 600 may further include determining a block of residual data for the block of video data. Process 600 may include decoding the block of video data using the block of residual data and the predictive block determined based on performing intra prediction on the block of video data.

[0165] In some cases, process 600 may be performed by an encoding device (e.g., encoding device 104 of FIGS. 1 and 7). For example, process 600 may include generating an encoded video bitstream that includes information related to blocks of video data. In some examples, process 600 may include storing the encoded video bitstream (e.g., in at least one memory of the device). In some examples, process 600 may include transmitting the encoded video bitstream (e.g., using a transmitter of the device).

[0166] In some implementations, the processes (or methods) described herein may be performed by a computing device or apparatus, such as the system 100 shown in FIG. 1. For example, the processes may be performed by the encoding device 104 shown in FIGS. 1 and 8, by another video source device or video transmission device, by the decoding device 112 shown in FIGS. 1 and 9, and / or by another client-side device, such as a player device, a display, or any other client-side device. In some cases, the computing device or apparatus may include a processor, microprocessor, microcomputer, or other component of a device configured to perform steps of the processes described herein. In some examples, the computing device or apparatus may include a camera configured to capture video data (e.g., a video sequence) including video frames. In some examples, the camera or other capture device that captures the video data is separate from the computing device, in which case the computing device receives or acquires the captured video data. The computing device may further include a network interface configured to communicate the video data. The network interface may be configured to communicate Internet Protocol (IP)-based data or other types of data. In some examples, the computing device or apparatus may include a display for displaying output video content, such as sample pictures of a video bitstream.

[0167] Processes are described in terms of logical flow diagrams, where the operations represent sequences of operations that may be implemented in hardware, computer instructions, or a combination thereof. In the context of computer instructions, the operations represent computer-executable instructions stored on one or more computer-readable storage media that, when executed by one or more processors, perform the described operations. Generally, computer-executable instructions include routines, programs, objects, components, data structures, etc. that perform particular functions or implement particular data types. The order in which the operations are described is not intended to be construed as a limitation, and any number of the described operations may be combined in any order and / or in parallel to implement a process.

[0168] Additionally, the process may run under the control of one or more computer systems configured with executable instructions and may be implemented as code (e.g., executable instructions, one or more computer programs, or one or more applications) that collectively execute on one or more processors, by hardware, or a combination thereof. As mentioned above, the code may be stored in a computer-readable or machine-readable storage medium, for example, in the form of a computer program comprising a plurality of instructions executable by one or more processors. The computer-readable or machine-readable storage medium may be non-transitory.

[0169] The coding techniques described herein may be implemented in an exemplary video encoding and decoding system (e.g., system 100). In some examples, the system includes a source device that provides encoded video data to be subsequently decoded by a destination device. Specifically, the source device provides the video data to the destination device via a computer-readable medium. The source device and destination device may comprise any of a wide range of devices, including desktop computers, notebook (i.e., laptop) computers, tablet computers, set-top boxes, telephone handsets such as so-called "smart" phones, so-called "smart" pads, televisions, cameras, display devices, digital media players, video gaming consoles, video streaming devices, etc. In some cases, the source device and destination device may be equipped for wireless communication.

[0170] The destination device may receive the encoded video data to be decoded via a computer-readable medium. The computer-readable medium may comprise any type of medium or device capable of moving encoded video data from a source device to a destination device. In one example, the computer-readable medium may comprise a communication medium for enabling the source device to transmit the encoded video data directly to the destination device in real time. The encoded video data may be modulated according to a communication standard, such as a wireless communication protocol, and transmitted to the destination device. The communication medium may comprise any wireless or wired communication medium, such as a radio frequency (RF) spectrum or one or more physical transmission lines. The communication medium may form part of a packet-based network, such as a local area network, a wide area network, or a global network such as the Internet. The communication medium may include routers, switches, base stations, or any other equipment that may be useful for facilitating communication from the source device to the destination device.

[0171] In some examples, the encoded data may be output from the output interface to a storage device. Similarly, the encoded data may be accessed from the storage device by the input interface. The storage device may include any of a variety of distributed or locally accessed data storage media, such as a hard drive, a Blu-ray disc, a DVD, a CD-ROM, flash memory, volatile or non-volatile memory, or any other suitable digital storage medium for storing encoded video data. In further examples, the storage device may correspond to a file server or another intermediate storage device that may store the encoded video generated by the source device. The destination device may access the stored video data from the storage device via streaming or download. The file server may be any type of server capable of storing encoded video data and transmitting the encoded video data to the destination device. Exemplary file servers include a web server (e.g., for a website), an FTP server, a network-attached storage (NAS) device, or a local disk drive. The destination device may access the encoded video data through any standard data connection, including an Internet connection. This may include a wireless channel (e.g., a Wi-Fi connection), a wired connection (e.g., DSL, cable modem, etc.), or a combination of both, suitable for accessing the encoded video data stored on the file server. The transmission of the encoded video data from the storage device may be a streaming transmission, a download transmission, or a combination thereof.

[0172] The techniques of this disclosure are not necessarily limited to wireless applications or settings. The techniques may be applied to video coding supporting any of a variety of multimedia applications, such as over-the-air television broadcast, cable television transmission, satellite television transmission, Internet streaming video transmission such as Dynamic Adaptive Streaming over HTTP (DASH), digital video encoded on a data storage medium, decoding of digital video stored on a data storage medium, or other applications. In some examples, a system may be configured to support one-way or two-way video transmission to support applications such as video streaming, video playback, video broadcasting, and / or video telephony.

[0173] In one example, the source device includes a video source, a video encoder, and an output interface. The destination device may include an input interface, a video decoder, and a display device. The video encoder of the source device may be configured to apply the techniques disclosed herein. In other examples, the source device and the destination device may include other components or arrangements. For example, the source device may receive video data from an external video source, such as an external camera. Similarly, the destination device may interface with an external display device rather than including an integrated display device.

[0174] The above exemplary system is merely an example. The techniques for processing video data in parallel may be performed by any digital video encoding and / or decoding device. Generally, the techniques of this disclosure are performed by a video encoding device, but the techniques may also be performed by a video encoder / decoder, typically referred to as a "codec." Additionally, the techniques of this disclosure may also be performed by a video preprocessor. The source device and destination device are merely examples of such coding devices, in which the source device generates coded video data for transmission to the destination device. In some examples, the source device and destination device may operate substantially symmetrically, such that each of the devices includes video encoding and decoding components. Thus, the exemplary system may support one-way or two-way video transmission between video devices, for example, for video streaming, video playback, video broadcasting, or video telephony.

[0175] The video source may include a video capture device such as a video camera, a video archive containing previously captured video, and / or a video feed interface for receiving video from a video content provider. As a further alternative, the video source may generate computer graphics-based data as the source video, or a combination of live video, archived video, and computer-generated video. In some cases, when the video source is a video camera, the source device and destination device may form a so-called camera phone or video phone. However, as mentioned above, the techniques described in this disclosure may be applicable to video coding generally and may be applied to wireless and / or wired applications. In each case, the captured, pre-captured, or computer-generated video may be encoded by a video encoder. The encoded video information may then be output onto a computer-readable medium by an output interface.

[0176] As mentioned, computer-readable media may include transitory media such as wireless broadcast or wired network transmissions, or storage media (i.e., non-transitory storage media) such as hard disks, flash drives, compact discs, digital video discs, Blu-ray discs, or other computer-readable media. In some examples, a network server (not shown) may receive encoded video data from a source device and provide the encoded video data to a destination device, e.g., via a network transmission. Similarly, a computing device at a media production facility, such as a disc stamping facility, may receive encoded video data from a source device and manufacture discs including the encoded video data. Thus, computer-readable media may be understood to include one or more computer-readable media of various forms in various examples.

[0177] An input interface of the destination device receives information from a computer-readable medium. The information of the computer-readable medium may include syntax information defined by a video encoder, including syntax elements that describe characteristics and / or processing of blocks and other coded units, e.g., groups of pictures (GOPs), which syntax information is also used by a video decoder. The display device displays the decoded video data to a user and may comprise any of a variety of display devices, such as a cathode ray tube (CRT), a liquid crystal display (LCD), a plasma display, an organic light-emitting diode (OLED) display, or another type of display device. Various embodiments of the present application have been described.

[0178] Specific details of encoding device 104 and decoding device 112 are shown in Figures 8 and 9, respectively. Figure 8 is a block diagram illustrating an example encoding device 104 that may implement one or more of the techniques described in this disclosure. Encoding device 104 may, for example, generate syntax structures described herein (e.g., syntax structures for VPS, SPS, PPS, or other syntax elements). Encoding device 104 may perform intra-prediction and inter-predictive coding of video blocks within video slices. As previously described, intra-coding relies at least in part on spatial prediction to reduce or remove spatial redundancy within a given video frame or picture. Inter-coding relies at least in part on temporal prediction to reduce or remove temporal redundancy within adjacent or surrounding frames of a video sequence. Intra-mode (I-mode) may refer to any of several spatial-based compression modes. Inter-modes, such as unidirectional prediction (P-mode) or bidirectional prediction (B-mode), may refer to any of several temporal-based compression modes.

[0179] Encoding device 104 includes partitioning unit 35, prediction processing unit 41, filter unit 63, picture memory 64, adder 50, transform processing unit 52, quantization unit 54, and entropy coding unit 56. Prediction processing unit 41 includes motion estimation unit 42, motion compensation unit 44, and intra-prediction processing unit 46. For video block reconstruction, encoding device 104 also includes inverse quantization unit 58, inverse transform processing unit 60, and adder 62. Filter unit 63 is intended to represent one or more loop filters, such as a deblocking filter, an adaptive loop filter (ALF), and a sample adaptive offset (SAO) filter. Although filter unit 63 is shown in FIG. 8 as being an in-loop filter, in other configurations, filter unit 63 may be implemented as a post-loop filter. Post-processing device 57 may perform additional processing on the encoded video data generated by encoding device 104. Techniques of this disclosure may be implemented by encoding device 104 in some instances. However, in other cases, one or more of the techniques of this disclosure may be implemented by post-processing device 57.

[0180] As shown in FIG. 8, encoding device 104 receives video data, and partitioning unit 35 partitions the data into video blocks. Partitioning may also include partitioning into slices, slice segments, tiles, or other larger units, as well as video block partitioning according to, for example, a quadtree structure of LCUs and CUs. Encoding device 104 generally refers to the components that encode video blocks within a video slice to be coded. A slice may be divided into multiple video blocks (and possibly into sets of video blocks called tiles). Prediction processing unit 41 may select one of multiple possible coding modes, such as one of multiple intra-predictive coding modes or one of multiple inter-predictive coding modes, for a current video block based on error results (e.g., coding rate, level of distortion, etc.). Prediction processing unit 41 may provide the resulting intra- or inter-coded block to summer 50 to generate residual block data and to summer 62 to reconstruct a coding block for use as a reference picture.

[0181] Intra-prediction processing unit 46 within prediction processing unit 41 may perform intra-predictive coding of the current video block relative to one or more neighboring blocks in the same frame or slice as the current block to be coded to provide spatial compression. Motion estimation unit 42 and motion compensation unit 44 within prediction processing unit 41 perform inter-predictive coding of the current video block relative to one or more predictive blocks in one or more reference pictures to provide temporal compression.

[0182] Motion estimation unit 42 may be configured to determine inter-prediction modes for video slices according to a predetermined pattern for the video sequence. The predetermined pattern may designate video slices in the sequence as P slices, B slices, or GPB slices. Motion estimation unit 42 and motion compensation unit 44 may be highly integrated but are shown separately for conceptual purposes. Motion estimation performed by motion estimation unit 42 is the process of generating motion vectors that estimate the motion of video blocks. A motion vector may, for example, indicate the displacement of a prediction unit (PU) of a video block in a current video frame or picture relative to a predictive block in a reference picture.

[0183] The predictive block is a block found to closely match the PU of the video block to be coded, in terms of pixel differences, which may be determined by sum of absolute differences (SAD), sum of squared differences (SSD), or other difference metrics. In some examples, encoding device 104 may calculate values for sub-integer pixel locations of reference pictures stored in picture memory 64. For example, encoding device 104 may interpolate values for quarter-pixel locations, eighth-pixel locations, or other fractional pixel locations of the reference pictures. Thus, motion estimation unit 42 may perform motion searches for full-pixel locations and fractional pixel locations and output motion vectors with fractional pixel locations.

[0184] Motion estimation unit 42 calculates a motion vector for a PU of a video block in an inter-coded slice by comparing the position of the PU with the position of a predictive block of a reference picture. The reference pictures may be selected from a first reference picture list (List 0) or a second reference picture list (List 1), each of which identifies one or more reference pictures stored in picture memory 64. Motion estimation unit 42 sends the calculated motion vector to entropy coding unit 56 and motion compensation unit 44.

[0185] Motion compensation performed by motion compensation unit 44 may involve fetching or generating a predictive block based on a motion vector determined by motion estimation, possibly performing interpolation to sub-pixel precision. Upon receiving the motion vector of the PU of the current video block, motion compensation unit 44 may locate the predictive block to which the motion vector points within a reference picture list. Encoding device 104 forms a residual video block by subtracting pixel values of the predictive block from pixel values of the current video block being coded to form pixel difference values. The pixel difference values form residual data for the block and may include both luma and chroma difference components. Adder 50 represents one or more components that perform this subtraction operation. Motion compensation unit 44 may also generate syntax elements associated with the video block and video slice for use by decoding device 112 in decoding the video block of the video slice.

[0186] The intra-prediction processing unit 46 may intra-predict the current block as an alternative to the inter-prediction performed by the motion estimation unit 42 and the motion compensation unit 44, as described above. Specifically, the intra-prediction processing unit 46 may determine an intra-prediction mode to use to encode the current block. In some examples, the intra-prediction processing unit 46 may encode the current block using various intra-prediction modes, e.g., during separate encoding passes, and the intra-prediction processing unit 46 may select an appropriate intra-prediction mode to use from the tested modes. For example, the intra-prediction processing unit 46 may calculate rate-distortion values for the various tested intra-prediction modes using a rate-distortion analysis and select the intra-prediction mode with the best rate-distortion characteristics from among the tested modes. The rate-distortion analysis generally determines the amount of distortion (or error) between the coded block and the original uncoded block that was coded to generate the coded block, as well as the bitrate (i.e., number of bits) used to generate the coded block. Intra-prediction processing unit 46 may calculate ratios from the distortions and rates for various coding blocks to determine which intra-prediction mode exhibits the best rate-distortion value for the block.

[0187] In either case, after selecting an intra-prediction mode for the block, intra-prediction processing unit 46 may provide information indicating the selected intra-prediction mode for the block to entropy coding unit 56. Entropy coding unit 56 may encode the information indicating the selected intra-prediction mode. Encoding device 104 may include definitions of coding contexts for the various blocks, as well as indications of the most probable intra-prediction mode, intra-prediction mode index table, and modified intra-prediction mode index table to use for each of the contexts, in the transmitted bitstream configuration data. The bitstream configuration data may include multiple intra-prediction mode index tables and multiple modified intra-prediction mode index tables (also referred to as codeword mapping tables).

[0188] After prediction processing unit 41 generates a predictive block for a current video block via either inter-prediction or intra-prediction, encoding device 104 forms a residual video block by subtracting the predictive block from the current video block. The residual video data in the residual block may be included in one or more TUs and applied to transform processing unit 52. Transform processing unit 52 converts the residual video data into residual transform coefficients using a transform, such as a discrete cosine transform (DCT) or a conceptually similar transform. Transform processing unit 52 may convert the residual video data from the pixel domain to a transform domain, such as the frequency domain.

[0189] Transform processing unit 52 may send the resulting transform coefficients to quantization unit 54. Quantization unit 54 quantizes the transform coefficients to further reduce the bit rate. The quantization process may reduce the bit depth associated with some or all of the coefficients. The degree of quantization may be modified by adjusting a quantization parameter. In some examples, quantization unit 54 may then perform a scan of a matrix containing the quantized transform coefficients. Alternatively, entropy coding unit 56 may perform the scan.

[0190] Following quantization, entropy coding unit 56 entropy codes the quantized transform coefficients. For example, entropy coding unit 56 may perform context-adaptive variable length coding (CAVLC), context-adaptive binary arithmetic coding (CABAC), syntax-based context-adaptive binary arithmetic coding (SBAC), probability interval partitioned entropy (PIPE) coding, or another entropy coding technique. Following entropy coding by entropy coding unit 56, the coded bitstream may be transmitted to decoding device 112 or archived for later transmission or retrieval by decoding device 112. Entropy coding unit 56 may also entropy code motion vectors and other syntax elements of the current video slice being coded.

[0191] Inverse quantization unit 58 and inverse transform processing unit 60 apply inverse quantization and inverse transform, respectively, to reconstruct the residual block in the pixel domain for later use as a reference block of a reference picture. Motion compensation unit 44 may calculate a reference block by adding the residual block to a predictive block of one of the reference pictures in the reference picture list. Motion compensation unit 44 may also apply one or more interpolation filters to the reconstructed residual block to calculate sub-integer pixel values for use in motion estimation. Adder 62 adds the reconstructed residual block to the motion-compensated predictive block generated by motion compensation unit 44 to generate a reference block for storage in picture memory 64. The reference block may be used by motion estimation unit 42 and motion compensation unit 44 as a reference block for inter-predicting blocks in subsequent video frames or pictures.

[0192] 8 represents an example of a video encoder configured to perform the techniques described herein. For example, encoding device 104 may perform any of the techniques described herein, including the processes described herein. In some cases, some of the techniques of this disclosure may also be implemented by post-processing device 57.

[0193] 9 is a block diagram illustrating an exemplary decoding device 112. Decoding device 112 includes entropy decoding unit 80, prediction processing unit 81, inverse quantization unit 86, inverse transform processing unit 88, summer 90, filter unit 91, and picture memory 92. Prediction processing unit 81 includes motion compensation unit 82 and intra-prediction processing unit 84. Decoding device 112 may, in some examples, perform a decoding path that is generally inverse to the encoding path described with respect to encoding device 104 from FIG.

[0194] During the decoding process, decode device 112 receives an encoded video bitstream representing video blocks and associated syntax elements of encoded video slices sent by encoding device 104. In some embodiments, decode device 112 may receive the encoded video bitstream from encoding device 104. In some embodiments, decode device 112 may receive the encoded video bitstream from a network entity 79, such as a server, a media-aware network element (MANE), a video editor / splitter, or other such device configured to implement one or more of the techniques described above. Network entity 79 may or may not include encoding device 104. Some of the techniques described in this disclosure may be implemented by network entity 79 before network entity 79 sends the encoded video bitstream to decode device 112. In some video decoding systems, network entity 79 and decode device 112 may be part of separate devices, while in other cases, the functions described with respect to network entity 79 may be performed by the same device that comprises decode device 112.

[0195] Entropy decoding unit 80 of decoding device 112 entropy decodes the bitstream to generate quantized coefficients, motion vectors, and other syntax elements. Entropy decoding unit 80 forwards the motion vectors and other syntax elements to prediction processing unit 81. Decoding device 112 may receive syntax elements at the video slice level and / or the video block level. Entropy decoding unit 80 may process and parse both fixed-length and variable-length syntax elements in one or more parameter sets, such as VPS, SPS, and PPS.

[0196] When a video slice is coded as an intra-coded (I) slice, intra-prediction processing unit 84 of prediction processing unit 81 may generate predictive data for video blocks of the current video slice based on the signaled intra-prediction mode and data from previously decoded blocks of the current frame or picture. When a video frame is coded as an inter-coded (i.e., B, P, or GPB) slice, motion compensation unit 82 of prediction processing unit 81 generates predictive blocks for video blocks of the current video slice based on motion vectors and other syntax elements received from entropy decoding unit 80. The predictive blocks may be generated from one of the reference pictures in the reference picture list. Decoding device 112 may construct the reference frame lists, i.e., List 0 and List 1, using a default construction technique based on the reference pictures stored in picture memory 92.

[0197] Motion compensation unit 82 determines prediction information for video blocks of the current video slice by parsing motion vectors and other syntax elements, and uses the prediction information to generate predictive blocks for the current video block being decoded. For example, motion compensation unit 82 may use one or more syntax elements in a parameter set to determine a prediction mode (e.g., intra prediction or inter prediction) used to code the video blocks of the video slice, an inter-prediction slice type (e.g., B slice, P slice, or GPB slice), construction information about one or more reference picture lists for the slice, a motion vector for each inter-coded video block of the slice, an inter-prediction status for each inter-coded video block of the slice, and other information for decoding video blocks in the current video slice.

[0198] Motion compensation unit 82 may also perform interpolation based on an interpolation filter. Motion compensation unit 82 may calculate interpolated values for sub-integer pixels of the reference block using an interpolation filter such as that used by encoding device 104 during encoding of the video block. In this case, motion compensation unit 82 may determine the interpolation filter used by encoding device 104 from a received syntax element and may use that interpolation filter to generate the predictive block.

[0199] Inverse quantization unit 86 inverse quantizes, or dequantizes, the quantized transform coefficients provided in the bitstream and decoded by entropy decoding unit 80. The inverse quantization process may involve determining the degree of quantization using a quantization parameter calculated by encoding device 104 for each video block in a video slice, as well as determining the degree of inverse quantization to apply. Inverse transform processing unit 88 applies an inverse transform (e.g., an inverse DCT or other suitable inverse transform), an inverse integer transform, or a conceptually similar inverse transform process to the transform coefficients to generate residual blocks in the pixel domain.

[0200] After motion compensation unit 82 generates a predictive block for a current video block based on the motion vector and other syntax elements, decoding device 112 forms a decoded video block by adding a residual block from inverse transform processing unit 88 with the corresponding predictive block generated by motion compensation unit 82. Adder 90 represents one or more components that perform this addition operation. If desired, a loop filter (either in the coding loop or after the coding loop) may also be used to smooth pixel transitions or otherwise improve video quality. Filter unit 91 is intended to represent one or more loop filters, such as a deblocking filter, an adaptive loop filter (ALF), and a sample adaptive offset (SAO) filter. Although filter unit 91 is shown in FIG. 9 as being an in-loop filter, in other configurations, filter unit 91 may be implemented as a post-loop filter. The decoded video blocks in a given frame or picture are then stored in picture memory 92, which stores reference pictures used for subsequent motion compensation. Picture memory 92 also stores decoded video for later presentation on a display device, such as video destination device 122 shown in FIG.

[0201] 9 represents an example of a video decoder configured to perform the techniques described herein. For example, decoding device 112 may perform any of the techniques described herein, including the processes described herein.

[0202] As used herein, the term “computer-readable medium” includes, but is not limited to, portable or non-portable storage devices, optical storage devices, and various other media capable of storing, containing, or transporting instructions and / or data. Computer-readable media may include non-transitory media that can store data and do not include carrier waves and / or transitory electronic signals propagating wirelessly or over wired connections. Examples of non-transitory media may include, but are not limited to, magnetic disks or tapes, optical storage media such as compact discs (CDs) or digital versatile discs (DVDs), flash memory, memory, or memory devices. A computer-readable medium may store code and / or machine-executable instructions, which may represent a procedure, a function, a subprogram, a program, a routine, a subroutine, a module, a software package, a class, or any combination of instructions, data structures, or program statements. A code segment may be coupled to another code segment or a hardware circuit by passing and / or receiving information, data, arguments, parameters, or memory contents. Information, arguments, parameters, data, etc. may be passed, forwarded, or transmitted via any suitable means including memory sharing, message passing, token passing, network transmission, etc.

[0203] In some embodiments, computer-readable storage devices, media, and memories may include cables or wireless signals containing bitstreams, etc. However, when referred to, non-transitory computer-readable storage media specifically excludes media such as energy, carrier signals, electromagnetic waves, and signals themselves.

[0204] Specific details are provided in the above description to provide a thorough understanding of the embodiments and examples provided herein. However, it will be understood by those skilled in the art that the embodiments may be practiced without these specific details. For clarity of explanation, in some instances, the technology may be presented as including individual functional blocks, including functional blocks comprising devices, device components, and method steps or routines embodied in software or a combination of hardware and software. Additional components other than those shown in the figures and / or described herein may be used. For example, circuits, systems, networks, processes, or other components may be shown as components in block diagram form to avoid obscuring the embodiments in unnecessary detail. In other instances, well-known circuits, processes, algorithms, structures, and techniques may be shown without unnecessary detail to avoid obscuring the embodiments.

[0205] Individual embodiments may be described above as a process or method that is depicted as a flowchart, a flow diagram, a data flow diagram, a structure diagram, or a block diagram. While a flowchart may describe operations as a sequential process, many of the operations may be performed in parallel or simultaneously. In addition, the order of operations may be rearranged. A process terminates when its operations are completed, but may have additional steps not included in the diagram. A process may correspond to a method, a function, a procedure, a subroutine, a subprogram, etc. When a process corresponds to a function, its termination may correspond to a return of the function to the calling function or the main function.

[0206] The processes and methods according to the examples described above may be implemented using computer-executable instructions stored or otherwise available from a computer-readable medium. Such instructions may include, for example, instructions and data that cause a general-purpose computer, special-purpose computer, or processing device to perform a particular function or group of functions, or otherwise configure a general-purpose computer, special-purpose computer, or processing device to perform a particular function or group of functions. Portions of the computer resources used may be accessible over a network. The computer-executable instructions may be, for example, binary, intermediate format instructions such as assembly language, firmware, source code, etc. Examples of computer-readable media that may be used to store instructions, information used, and / or information created during methods according to the described examples include magnetic or optical disks, flash memory, USB devices with non-volatile memory, network-attached storage devices, etc.

[0207] Devices implementing processes and methods according to these disclosures may include hardware, software, firmware, middleware, microcode, hardware description languages, or any combination thereof, and may take any of a variety of form factors. When implemented in software, firmware, middleware, or microcode, program code or code segments (e.g., a computer program product) to perform the necessary tasks may be stored on a computer-readable or machine-readable medium. A processor may perform the necessary tasks. Typical examples of form factors include laptops, smartphones, mobile phones, tablet devices or other small form factor personal computers, personal digital assistants, rack-mounted devices, standalone devices, etc. The functionality described herein may also be embodied in peripheral devices or add-in cards. Such functionality may also be implemented on a circuit board, in different chips, or different processes running within a single device, as further examples.

[0208] The instructions, media for carrying such instructions, computing resources for executing the instructions, and other structures for supporting such computing resources are exemplary means for providing the functionality described in this disclosure.

[0209] While aspects of the present application have been described in the foregoing description with reference to specific embodiments thereof, those skilled in the art will recognize that the present application is not limited thereto. While exemplary embodiments of the present application have been described in detail herein, it should be understood that the concepts of the present invention may be embodied and employed in various other ways, and that the appended claims are intended to be construed to include such variations except insofar as limited by the prior art. Various features and aspects of the applications described above may be used individually or together. Furthermore, the embodiments may be utilized in any number of environments and applications other than those described herein without departing from the broader spirit and scope of the present specification. Accordingly, the specification and drawings should be regarded as illustrative and not restrictive. For illustrative purposes, methods have been described in a particular order. It should be appreciated that in alternative embodiments, methods may be performed in an order different from that described.

[0210] Those skilled in the art will appreciate that the less than ("<") and greater than (">") symbols or terms used herein may be replaced with the less than or equal to ("≦") and greater than or equal to ("≧") symbols, respectively, without departing from the scope of this description.

[0211] When a component is described as being "configured to" perform some operation, such configuration may be achieved, for example, by designing electronic circuitry or other hardware to perform the operation, by programming programmable electronic circuitry (e.g., a microprocessor or other suitable electronic circuitry) to perform the operation, or any combination thereof.

[0212] The phrase "coupled to" refers to any component that is physically connected, either directly or indirectly, to another component and / or that is in communication, either directly or indirectly, with another component (e.g., connected to the other component via a wired or wireless connection and / or other suitable communication interface).

[0213] Claim language or other language reciting "at least one of" a set and / or "one or more" of a set indicates that one element of the set or multiple elements of the set (in any combination) satisfies the claim. For example, claim language reciting "at least one of A and B" means A, B, or A and B. As another example, claim language reciting "at least one of A, B, and C" means A, B, C, or A and B, or A and C, or B and C, or A, B, and C. The language "at least one of" a set and / or "one or more" of a set does not limit the set to the items listed in the set. For example, claim language reciting "at least one of A and B" can mean A, B, or A and B, and can additionally include items not listed in the set of A and B.

[0214] The various illustrative logic blocks, modules, circuits, and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, firmware, or a combination thereof. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the particular application and design constraints imposed on the overall system. Those skilled in the art may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

[0215] The techniques described herein may also be implemented in electronic hardware, computer software, firmware, or any combination thereof. Such techniques may be implemented in any of a variety of devices, such as a general-purpose computer, a wireless communication device handset, or an integrated circuit device having multiple uses, including applications in wireless communication device handsets and other devices. Any features described as modules or components may be implemented together in an integrated logic device or separately as discrete but interoperable logic devices. If implemented in software, the techniques may be realized at least in part by a computer-readable data storage medium comprising program code including instructions that, when executed, perform one or more of the methods described above. The computer-readable data storage medium may form part of a computer program product, which may include packaging materials. The computer-readable medium may comprise a memory or data storage medium, such as a random access memory (RAM) such as synchronous dynamic random access memory (SDRAM), a read-only memory (ROM), a nonvolatile random access memory (NVRAM), an electrically erasable programmable read-only memory (EEPROM), a flash memory, a magnetic or optical data storage medium, etc. The techniques may additionally or alternatively be realized at least in part by a computer-readable communication medium, such as a propagated signal or wave, that carries or communicates program code in the form of instructions or data structures and that can be accessed, read, and / or executed by a computer.

[0216] The program code may be executed by a processor, which may include one or more processors, such as one or more digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), field-programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuitry. Such a processor may be configured to perform any of the techniques described in this disclosure. A general-purpose processor may be a microprocessor, but alternatively, the processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of computing devices, e.g., a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration. Accordingly, the term “processor,” as used herein, may refer to any of the above structures, any combination of the above structures, or any other structure or apparatus suitable for implementing the techniques described herein. Additionally, in some aspects, the functionality described herein may be provided in dedicated software or hardware modules configured for encoding and decoding, or may be incorporated into a composite video encoder-decoder (codec).

[0217] Illustrative examples of the present disclosure include the following:

[0218] Aspect 1: A method for processing video data, comprising steps of obtaining a block of video data, processing the block using an intra-prediction mode, and determining a type of interpolation filter to use for the block based on at least one of a width and a height of the block.

[0219] Aspect 2: The method of aspect 1, further comprising: determining that at least one of the width of the base block and the height of the block is greater than a threshold; determining a first type of interpolation filter to use for the block; and determining reference pixels for the block using the first type of interpolation filter.

[0220] Aspect 3: The method of aspect 1, wherein the first type interpolation filter comprises a 6-tap Gaussian filter.

[0221] Aspect 4: The method of aspect 1, further comprising: determining that at least one of the width of the base block and the height of the block is less than or equal to a threshold; determining a second type of interpolation filter to use for the block; and determining reference pixels for the block using the second type of interpolation filter.

[0222] Example 5: The method of example 4, wherein the second type interpolation filter comprises a 4-tap Gaussian filter.

[0223] Aspect 6: The method of any one of aspects 1 to 5, wherein the type of interpolation filter is explicitly signaled in the video bitstream.

[0224] Example 7: The method of example 6, wherein the type of interpolation filter is explicitly signaled per prediction block, coding block, coding tree unit (CTU), slice, or sequence.

[0225] Aspect 8: The method of any one of aspects 1 to 5, further comprising determining the type of interpolation filter based on at least one of the width and height of the block without using information explicitly signaled in the video bitstream.

[0226] Aspect 9: An apparatus comprising a memory configured to store video data and a processor, wherein the processor is configured to obtain a block of the video data, process the block using an intra-prediction mode, and determine a type of interpolation filter to use for the block based on at least one of a width and a height of the block.

[0227] Aspect 10: The apparatus of aspect 9, wherein the processor is configured to: determine that at least one of the width of the base block and the height of the block is greater than a threshold; determine a first type of interpolation filter to use for the block; and determine reference pixels for the block using the first type of interpolation filter.

[0228] Example 11: The apparatus of example 9, wherein the first type interpolation filter comprises a 6-tap Gaussian filter.

[0229] Aspect 12: The apparatus of aspect 9, wherein the processor is configured to: determine that at least one of the width of the base block and the height of the block is less than or equal to a threshold; determine a second type of interpolation filter to use for the block; and determine reference pixels for the block using the second type of interpolation filter.

[0230] Example 13: The apparatus of example 12, wherein the second type interpolation filter is a 4-tap Gaussian filter.

[0231] Example 14: The apparatus of any one of Examples 9 to 13, wherein the type of interpolation filter is explicitly signaled in the video bitstream.

[0232] Example 15: The apparatus of example 14, wherein the type of interpolation filter is explicitly signaled per prediction block, coding block, coding tree unit (CTU), slice, or sequence.

[0233] Aspect 16: The apparatus of any one of aspects 9 to 13, wherein the processor is configured to determine the type of interpolation filter based on at least one of a width and a height of the block without using information explicitly signaled in the video bitstream.

[0234] Embodiment 17: The apparatus of any one of embodiments 9 to 16, wherein the apparatus includes an encoder.

[0235] Example 18: The apparatus of any one of Examples 9 to 17, wherein the apparatus includes a decoder.

[0236] Embodiment 19: The apparatus of any one of embodiments 9 to 18, wherein the apparatus is a mobile device.

[0237] Aspect 20: The apparatus of any of aspects 9 to 19, wherein the apparatus is an augmented reality device.

[0238] Example 21: The device of any one of Examples 9 to 20, further comprising a display configured to display video data.

[0239] Example 22: The apparatus of any one of Examples 9 to 21, further comprising a camera configured to capture one or more pictures.

[0240] Aspect 23: A computer-readable medium having stored thereon instructions that, when executed by a processor, perform the method of any of aspects 1 to 22.

[0241] Aspect 24: An apparatus comprising means for performing the operations of any of aspects 1 to 22.

[0242] Aspect 25: A method for processing video data, comprising steps of obtaining a block of video data, processing the block using an intra-prediction mode, and determining a type of smoothing filter to use for the block based on at least one of a width and a height of the block.

[0243] Aspect 26: The method of aspect 25, further comprising a step of determining whether the angle of the intra prediction mode is an integer angle, wherein the step of determining the type of smoothing filter is further based on a determination that the angle of the intra prediction mode is an integer angle.

[0244] Aspect 27: The method of any one of aspects 25 or 26, further comprising: determining that at least one of the width of the base block and the height of the block is greater than a threshold; determining a first type of smoothing filter to use for the block; and processing at least one predicted pixel for the block using the first type of smoothing filter.

[0245] Example 28: The method of example 27, wherein the first type smoothing filter comprises a [1 4 6 4 1] filter.

[0246] Aspect 29: The method of any one of aspects 25 or 26, further comprising: determining that at least one of the width of the base block and the height of the block is less than or equal to a threshold; determining a second type of smoothing filter to use for the block; and processing at least one predicted pixel for the block using the second type of smoothing filter.

[0247] Aspect 30: The method of aspect 29, wherein the second type smoothing filter comprises a

[0121] filter.

[0248] Example 31: The method of any one of Examples 25 to 30, wherein the type of smoothing filter is explicitly signaled in the video bitstream.

[0249] Example 32: The method of example 31, wherein the type of interpolation filter is explicitly signaled per prediction block, coding block, coding tree unit (CTU), slice, or sequence.

[0250] Aspect 33: The method of any one of aspects 25 to 30, further comprising determining the type of smoothing filter based on at least one of the width and height of the block without using information explicitly signaled in the video bitstream.

[0251] Aspect 34: An apparatus comprising a memory configured to store video data and a processor, wherein the processor is configured to obtain a block of the video data, process the block using an intra-prediction mode, and determine a type of smoothing filter to use for the block based on at least one of a width and a height of the block.

[0252] Aspect 35: The apparatus of aspect 34, wherein the processor is configured to determine whether the angle of the intra prediction mode is an integer angle, and wherein determining the type of smoothing filter is further based on determining that the angle of the intra prediction mode is an integer angle.

[0253] Aspect 36: The apparatus of any one of aspects 34 or 35, wherein the processor is configured to: determine that at least one of the width of the base block and the height of the block is greater than a threshold; determine a first type of smoothing filter to use for the block; and process at least one predicted pixel for the block using the first type of smoothing filter.

[0254] Example 37: The apparatus of example 36, wherein the first type smoothing filter comprises a [1 4 6 4 1] filter.

[0255] Embodiment 38: The apparatus of any one of embodiments 34 or 35, wherein the processor is configured to determine that at least one of the width of the base block and the height of the block is less than or equal to a threshold, determine a second type of smoothing filter to use for the block, and process at least one predicted pixel for the block using the second type of smoothing filter.

[0256] Aspect 39: The apparatus of aspect 38, wherein the second type smoothing filter comprises a

[0121] filter.

[0257] Example 40: The apparatus of any one of examples 34 to 39, wherein the type of smoothing filter is explicitly signaled in the video bitstream.

[0258] Example 41: The apparatus of example 40, wherein the type of interpolation filter is explicitly signaled per prediction block, coding block, coding tree unit (CTU), slice, or sequence.

[0259] Aspect 42: The apparatus of any one of aspects 34 to 39, wherein the processor is configured to determine the type of smoothing filter based on at least one of a width and a height of the block without using information explicitly signaled in the video bitstream.

[0260] Embodiment 43: The apparatus of any one of embodiments 34 to 42, wherein the apparatus includes an encoder.

[0261] Embodiment 44: The device of any one of embodiments 34 to 43, wherein the device includes a decoder.

[0262] Embodiment 45: The apparatus of any one of embodiments 34 to 44, wherein the apparatus is a mobile device.

[0263] Aspect 46: The device of any one of aspects 34 to 45, wherein the device is an augmented reality device.

[0264] Embodiment 47: The device of any one of embodiments 34 to 46, further comprising a display configured to display video data.

[0265] Embodiment 48: The apparatus of any one of embodiments 34 to 47, further comprising a camera configured to capture one or more pictures.

[0266] Aspect 49: A computer-readable medium having stored thereon instructions that, when executed by a processor, perform the method of any of aspects 25 to 48.

[0267] Embodiment 50: An apparatus comprising means for performing the operations of any of embodiments 25 to 48.

[0268] Aspect 51: A computer-readable medium having stored thereon instructions that, when executed by a processor, perform the method of any of aspects 1 to 22 and aspects 25 to 48.

[0269] Embodiment 52: An apparatus comprising means for performing the operations of any of embodiments 1 to 22 and embodiments 25 to 48.

[0270] Aspect 53: An apparatus for processing video data, comprising at least one memory and at least one processor coupled to the at least one memory, wherein the at least one processor is configured to: determine an intra-prediction mode for predicting a block of video data; determine a type of smoothing filter to use for the block of video data, wherein the type of smoothing filter is determined at least in part based on comparing at least one of a width of the block of video data and a height of the block of video data to a first threshold; and perform intra-prediction on the block of video data using the determined type of smoothing filter and the intra-prediction mode.

[0271] Aspect 54: The apparatus of aspect 53, wherein at least one processor is configured to use a first smoothing interpolation filter as the determined type of smoothing filter based at least in part on a determination that at least one of the block width and the block height is greater than a first threshold, and to use the first smoothing interpolation filter to determine reference pixels for intra prediction of the block of video data.

[0272] Embodiment 55: The apparatus of any of embodiments 53 to 54, wherein the first smoothing interpolation filter comprises a 6-tap Gaussian filter.

[0273] Aspect 56: The apparatus of aspect 55, wherein at least one processor is configured to use a second smoothing interpolation filter as the determined type of smoothing filter based at least in part on a determination that at least one of the block width and the block height is less than or equal to a first threshold, and to use the second smoothing interpolation filter to determine reference pixels for intra prediction of the block of video data.

[0274] Example 57: The apparatus of example 56, wherein the second smoothing interpolation filter comprises a 4-tap Gaussian filter.

[0275] Aspect 58: The device of any of aspects 53 to 57, wherein at least one processor is configured to determine a minimum offset between the angular direction of the intra-prediction mode and one of the vertical intra-prediction mode and the horizontal intra-prediction mode, and determine a type of smoothing filter to use for the block of video data based on comparing the determined minimum offset with a second threshold.

[0276] Aspect 59: The apparatus of aspect 58, wherein at least one processor is configured to determine a low-pass filter as the type of smoothing filter based at least in part on a determination that the determined minimum offset is greater than a second threshold and a determination that the intra prediction mode is an integer angle mode associated with integer-valued reference pixel positions.

[0277] Aspect 60: The apparatus of aspect 59, wherein the low-pass filter performs reference pixel smoothing without interpolation, and the low-pass filter is a

[0121] filter.

[0278] Aspect 61: The apparatus of aspect 58, wherein at least one processor is configured to determine a Gaussian filter as the type of smoothing filter based at least in part on a determination that the determined minimum offset is greater than a second threshold and a determination that the intra prediction mode is a fractional angle mode associated with a fractional value reference pixel position.

[0279] Aspect 62: The apparatus of aspect 61, wherein the Gaussian filter performs smooth interpolation without reference pixel smoothing.

[0280]

[0063] Aspect 63: The apparatus of aspect 61, wherein the Gaussian filter comprises a 6-tap Gaussian filter based on a determination that at least one of the block width and the block height is greater than a first threshold.

[0281]

[0047] Aspect 64: The apparatus of aspect 61, wherein the Gaussian filter comprises a 4-tap Gaussian filter based on a determination that at least one of the block width and the block height is not greater than a first threshold.

[0282] Aspect 65: The apparatus of aspect 58, wherein at least one processor is configured to use an interpolation filter as the determined type of smoothing filter based at least in part on a determination that the determined minimum offset is less than or equal to a second threshold, the interpolation filter including a 4-tap cubic filter; and to perform intra prediction on the block of video data using the interpolation filter without applying reference pixel smoothing.

[0283] Aspect 66: The apparatus of aspect 58, wherein at least one processor is configured to determine a low-pass filter as the type of smoothing filter based at least in part on determining that the intra prediction mode is an integer angle mode and determining that the determined minimum offset is greater than a second threshold.

[0284] Aspect 67: The apparatus of aspect 67, wherein at least one processor is configured to perform reference pixel smoothing using a large tap low pass filter based at least in part on a determination that at least one of the block width and the block height is greater than a first threshold, wherein the large tap low pass filter applies a higher degree of reference pixel smoothing than the small tap low pass filter.

[0285] Aspect 68: The apparatus of aspect 67, wherein at least one processor is configured to perform reference pixel smoothing using a small tap low pass filter based at least in part on a determination that at least one of the block width and the block height is less than or equal to a first threshold, wherein the small tap low pass filter applies a lower degree of reference pixel smoothing than the large tap low pass filter.

[0286] Aspect 69: The device of any of aspects 53 to 68, wherein at least one processor is configured to determine that the intra prediction mode is an integer angle mode based at least in part on comparing the slope of the intra prediction mode to one or more pixel positions determined from the block width and the block height.

[0287] Embodiment 70: The device of any of embodiments 53 to 69, wherein at least one processor is configured to determine that an offset between the angular direction of the intra-prediction mode and the vertical intra-prediction mode or the horizontal intra-prediction mode is less than a second threshold, and to perform intra-prediction on the block of video data using a cubic interpolation filter based on determining that the offset between the angular direction of the intra-prediction mode and the vertical intra-prediction mode or the horizontal intra-prediction mode is less than the second threshold.

[0288] Aspect 71: The apparatus of aspect 70, wherein at least one processor is configured to perform reference line extension using a weak interpolation filter, the weak interpolation filter being used to perform the reference line extension prior to using a cubic interpolation filter to perform intra prediction, the cubic interpolation filter having a higher cutoff frequency than the weak interpolation filter and applying a higher degree of smoothing than the weak interpolation filter.

[0289] Example 72: The apparatus of example 71, wherein the weak interpolation filter includes a 4-tap sync-based interpolation filter and a 6-bit 4-tap interpolation filter.

[0290]

[0072] Aspect 73: The apparatus of any of aspects 53 to 72, wherein the type of smoothing filter is signaled in the video bitstream.

[0291]

[0041] Aspect 74: The apparatus of any of aspects 53 to 73, wherein the type of smoothing filter is signaled for an individual set of a prediction block, a coding block, a coding tree unit (CTU), a slice, or a set of sequences.

[0292] Aspect 75: The apparatus of any of aspects 53 to 74, wherein at least one processor is configured to determine the type of smoothing filter based on at least one of a width and a height of the block without using information explicitly signaled in the video bitstream.

[0293] Aspect 76: The apparatus of any of aspects 53 to 75, wherein at least one processor is configured to determine a block of residual data related to the block of video data, and decode the block of video data using the block of residual data and the predictive block determined based on performing intra prediction on the block of video data.

[0294]

[0071] Aspect 77: The apparatus of any of aspects 53 to 75, wherein at least one processor is configured to generate an encoded video bitstream that includes information related to blocks of video data.

[0295] Example 78: The apparatus of example 77, further comprising storing the encoded video bitstream in at least one memory.

[0296]

[0072] Aspect 79: The apparatus of any of aspects 77 to 78, further comprising a transmitter configured to transmit the encoded video bitstream.

[0297] Aspect 80: A method for processing video data, comprising: determining an intra-prediction mode for predicting a block of video data; determining a type of smoothing filter to use for the block of video data, wherein the type of smoothing filter is determined at least in part based on comparing at least one of a width of the block of video data and a height of the block of video data to a first threshold; and performing intra-prediction on the block of video data using the determined type of smoothing filter and the intra-prediction mode.

[0298] Aspect 81: The method of aspect 80, further comprising: using a first smoothing interpolation filter as the determined type of smoothing filter based at least in part on a determination that at least one of the block width and the block height is greater than a first threshold; and using the first smoothing interpolation filter to determine reference pixels for intra prediction of the block of video data.

[0299] Embodiment 82: The method of embodiment 81, wherein the first smoothing interpolation filter comprises a 6-tap Gaussian filter.

[0300] Aspect 83: The method of any of aspects 80 to 82, further comprising: using a second smoothing interpolation filter as the determined type of smoothing filter based at least in part on a determination that at least one of the block width and the block height is less than or equal to a first threshold; and using the second smoothing interpolation filter to determine reference pixels for intra prediction of the block of video data.

[0301] Embodiment 84: The method of embodiment 83, wherein the second smoothing interpolation filter comprises a 4-tap Gaussian filter.

[0302] Aspect 85: Any of the methods of aspects 80 to 84, further comprising: determining a minimum offset between the angular direction of the intra-prediction mode and one of the vertical intra-prediction mode and the horizontal intra-prediction mode; and determining a type of smoothing filter to use for the block of video data based on comparing the determined minimum offset with a second threshold.

[0303] Aspect 86: The method of aspect 85, further comprising a step of determining a low-pass filter as the type of smoothing filter based at least in part on a determination that the determined minimum offset is greater than a second threshold and a determination that the intra prediction mode is an integer angle mode associated with integer-valued reference pixel positions.

[0304] Aspect 87: The method of aspect 86, wherein the low-pass filter performs reference pixel smoothing without interpolation, and the low-pass filter includes a

[0121] filter.

[0305] Aspect 88: The method of aspect 85, further comprising a step of determining a Gaussian filter as the type of smoothing filter based at least in part on a determination that the determined minimum offset is greater than a second threshold and a determination that the intra prediction mode is a fractional angle mode associated with a fractional value reference pixel position.

[0306] Aspect 89: The method of aspect 88, wherein the Gaussian filter performs smooth interpolation without reference pixel smoothing.

[0307] Embodiment 90: The method of embodiment 88, wherein the Gaussian filter comprises a 6-tap Gaussian filter based on a determination that at least one of the block width and the block height is greater than a first threshold.

[0308] Embodiment 91: The method of embodiment 88, wherein the Gaussian filter comprises a 4-tap Gaussian filter based on a determination that at least one of the block width and the block height is less than or equal to a first threshold.

[0309] Aspect 92: The method of aspect 85, further comprising: using an interpolation filter as the determined type of smoothing filter based at least in part on a determination that the determined minimum offset is less than or equal to a second threshold, the interpolation filter comprising a 4-tap cubic filter; and performing intra prediction on the block of video data using the interpolation filter without applying reference pixel smoothing.

[0310] Aspect 93: The method of aspect 85, further comprising a step of determining a low-pass filter as the type of smoothing filter based at least in part on determining that the intra-prediction mode is an integer angle mode and determining that the determined minimum offset between the intra-prediction mode and the horizontal mode or the vertical mode is greater than a second threshold.

[0311] Aspect 94: The method of aspect 93, further comprising applying reference pixel smoothing using a large tap low pass filter based at least in part on a determination that at least one of the block width and the block height is greater than a first threshold, wherein the large tap low pass filter applies a greater degree of reference pixel smoothing than the small tap low pass filter.

[0312] Aspect 95: The method of aspect 93, further comprising applying reference pixel smoothing using a small tap low pass filter based at least in part on a determination that at least one of the block width and the block height is less than or equal to a first threshold, wherein the small tap low pass filter applies a lower degree of reference pixel smoothing than the large tap low pass filter.

[0313] Aspect 96: The method of any of aspects 80 to 95, further comprising determining that the intra prediction mode is an integer angle mode based at least in part on comparing the slope of the intra prediction mode with one or more pixel positions determined from the block width and the block height.

[0314] Aspect 97: Any of the methods of aspects 80 to 96, further comprising: determining that an offset between the angular direction of the intra-prediction mode and the vertical intra-prediction mode or the horizontal intra-prediction mode is less than a second threshold; and performing intra-prediction on the block of video data using a cubic interpolation filter based on determining that the determined offset is less than the second threshold.

[0315] Aspect 98: The method of aspect 97, further comprising the step of performing reference line extension using a weak interpolation filter, wherein the weak interpolation filter is used to perform the reference line extension prior to using a cubic interpolation filter to perform intra prediction, and the cubic interpolation filter has a higher cutoff frequency than the weak interpolation filter and applies a higher degree of smoothing than the weak interpolation filter.

[0316] Embodiment 99: The method of embodiment 98, wherein the weak interpolation filter includes a 4-tap sync-based interpolation filter and a 6-bit 4-tap interpolation filter.

[0317] Embodiment 100: The method of any of embodiments 80 to 99, wherein the type of smoothing filter is signaled in the video bitstream.

[0318] Embodiment 101: The method of any of embodiments 80 to 100, wherein the type of smoothing filter is signaled for an individual set of a prediction block, a coding block, a coding tree unit (CTU), a slice, or a set of sequences.

[0319] Aspect 102: The method of any of aspects 80 to 101, further comprising determining the type of smoothing filter based on at least one of the width and height of the block without using information explicitly signaled in the video bitstream.

[0320] Aspect 103: The method of any of aspects 80 to 102, further comprising: determining a block of residual data related to the block of video data; and decoding the block of video data using the block of residual data and the predicted block determined based on the step of performing intra prediction on the block of video data.

[0321]

[0047] Aspect 104: The method of any of aspects 80 to 102, further comprising generating an encoded video bitstream that includes information related to the block of video data.

[0322] Embodiment 105: The method of embodiment 104, further comprising the step of storing the encoded video bitstream.

[0323]

[0047] Aspect 106: The method of any of aspects 104 or 105, further comprising transmitting the encoded video bitstream.

[0324] Aspect 107: Aspect 43: A computer-readable medium having stored thereon instructions that, when executed by a processor, perform the method of any of aspects 53 to 106.

[0325] Embodiment 108: An apparatus comprising means for performing the operations of any of embodiments 53 to 106. [Explanation of symbols]

[0326] 2 Mode 18 Intramode, mode 26 Intramode 34 Mode 35 division units 41 Prediction Processing Unit 42 Motion Estimation Unit 44 Motion Compensation Unit 46 intra prediction processing units 50 modes, vertical intra prediction modes, adder 52 Conversion Processing Unit 54 quantization units 56 Entropy Coding Units 57 After-treatment Devices 58 Inverse Quantization Unit 60 Inverse Transformation Processing Unit 62 Adder 63 Filter unit 64 picture memory 66 Mode 79 Network Entities 80 Entropy Decoding Unit 81 Prediction Processing Unit 82 Motion Compensation Unit 84 Intra Prediction Processing Units 86 Inverse Quantization Unit 88 Inverse Transformation Processing Unit 90 Adder 91 Filter unit 92 Picture Memory 100 systems 102 Video Sources 104 Encoding device, video encoding device 106 Encoder Engine 108 Storage 110 Output section 112 Decoding Device, Video Decoding Device 114 Input section 116 Decoder Engine 118 Storage 120 Communication Links 122 Video Destination Device Figure 200a Figure 200b 300 MDIS Process 400 Figures 405 Currently coded block, current block 410 Reference pixel top line, upper reference pixel line 420 Reference Line Extension Pixels 423 Reference line pixel P 430 left reference pixels 500 processes 600 processes

Claims

1. 1. An apparatus for processing video data, comprising: At least one memory; and at least one processor coupled to the at least one memory, the at least one processor comprising: determining an intra-prediction mode for predicting a block of video data; selecting, for a non-integer angle intra-prediction mode, between a first Gaussian smoothing interpolation filter or a second Gaussian smoothing interpolation filter for use on the block of video data, the first Gaussian smoothing interpolation filter applying a higher degree of smoothing than the second Gaussian smoothing interpolation filter when interpolating fractional reference pixel locations, the selection being based at least in part on comparing at least one of a width of the block of video data and a height of the block of video data to a first threshold; performing intra prediction for the block of video data using the selected smoothing interpolation filter and the intra prediction mode for the non-integer angle intra prediction mode; An apparatus configured to:

2. For the non-integer angle intra prediction modes, the at least one processor: using the first Gaussian smoothing interpolation filter as the selected smoothing interpolation filter based at least in part on a determination that at least one of the width of the block and the height of the block is greater than the first threshold; determining reference pixels for intra prediction of the block of video data using the first Gaussian smoothing interpolation filter; The apparatus of claim 1 , configured to:

3. The apparatus of claim 2 , wherein the first Gaussian smoothing interpolation filter comprises a 6-tap Gaussian filter.

4. For the non-integer angle intra prediction modes, the at least one processor: using a second Gaussian smoothing interpolation filter as the selected smoothing interpolation filter based at least in part on determining that at least one of the width of the block and the height of the block is less than or equal to the first threshold; determining reference pixels for intra prediction of the block of video data using the second Gaussian smoothing interpolation filter; The apparatus of claim 1 , configured to:

5. The apparatus of claim 4 , wherein the second Gaussian smoothing interpolation filter comprises a 4-tap Gaussian filter.

6. the at least one processor: determining a block of residual data for said block of video data; Decoding the block of video data using the block of residual data and a prediction block determined based on performing the intra prediction on the block of video data. The apparatus of claim 1 , configured to:

7. the at least one processor: generating an encoded video bitstream including information relating to said blocks of video data; The apparatus of claim 1 , configured to:

8. storing said encoded video bitstream in said at least one memory; The apparatus of claim 7, further comprising:

9. a transmitter configured to transmit the encoded video bitstream; The apparatus of claim 7, further comprising:

10. 1. A method for processing video data, comprising the steps of: determining an intra-prediction mode for predicting a block of video data; selecting, for a non-integer angle intra-prediction mode, between a first Gaussian smoothing interpolation filter or a second Gaussian smoothing interpolation filter to use on the block of video data, the first Gaussian smoothing interpolation filter applying a higher degree of smoothing than the second Gaussian smoothing interpolation filter when interpolating fractional reference pixel locations, the selection being based at least in part on comparing at least one of a width of the block of video data and a height of the block of video data to a first threshold; for the non-integer angle intra-prediction mode, performing intra prediction on the block of video data using the selected smoothing interpolation filter and the intra-prediction mode; A method comprising:

11. For the non-integer angle intra prediction modes, using the first Gaussian smoothing interpolation filter as the selected smoothing interpolation filter based at least in part on a determination that at least one of the width of the block and the height of the block is greater than the first threshold; determining reference pixels for intra prediction of the block of video data using the first Gaussian smoothing interpolation filter; Further comprising: The method of claim 10 , wherein the first Gaussian smoothing interpolation filter comprises a 6-tap Gaussian filter.

12. For the non-integer angle intra prediction modes, using the second Gaussian smoothing interpolation filter as the selected smoothing interpolation filter based at least in part on a determination that at least one of the width of the block and the height of the block is less than or equal to the first threshold; determining reference pixels for intra prediction of the block of video data using the second Gaussian smoothing interpolation filter; Further comprising: The method of claim 10 , wherein the second Gaussian smoothing interpolation filter comprises a 4-tap Gaussian filter.

13. determining a block of residual data for said block of video data; decoding the block of video data using the block of residual data and a prediction block determined based on performing the intra prediction on the block of video data; 11. The method of claim 10, further comprising:

14. generating an encoded video bitstream including information relating to said blocks of video data.

11. The method of claim 10, further comprising:

Images