Intra prediction using extended interpolation filters

Abstract

Techniques for processing video data using an enhanced interpolation filter for intra prediction are described herein. For example, a device may determine an intra prediction mode for predicting a block of video data. The device may determine a type of smoothing filter to use for the block of video data, where the type of smoothing filter is determined at least in part based on comparing at least one of a width of the block of video data and a height of the block of video data to a first threshold. The device may further perform intra prediction on the block of video data using the determined type of smoothing filter and the intra prediction mode.

Description

Technical Field

[0001] This application relates to video coding (including, for example, encoding and / or decoding of video data). For example, aspects of this application relate to systems and techniques for performing intra prediction using an extended interpolation filter.

Background Art

[0002] Many devices and systems enable video data to be processed and output for consumption. Digital video data includes a large amount of data to meet the needs of consumers and video providers. For example, consumers of video data desire the highest quality video with high fidelity, resolution, frame rate, etc. As a result, the large amount of video data required to meet these needs places a burden on communication networks and devices that process and store video data.

[0003] To compress video data, various video coding techniques can be used. Video coding is performed according to one or more video coding standards. For example, video coding standards include, among others, Versatile Video Coding (VVC), High Efficiency Video Coding (HEVC), Advanced Video Coding (AVC), MPEG-2 Part 2 coding (MPEG represents Moving Picture Experts Group), and proprietary video codec / formats such as AOMedia Video 1 (AV1) developed by the Alliance for Open Media. Video coding generally utilizes prediction methods (such as inter prediction, intra prediction, etc.) that exploit the redundancy present in a video image or sequence. The purpose of video coding techniques is to compress video data in a form that uses a lower bitrate while avoiding or minimizing degradation of video quality. As evolving video services become available, encoding techniques with better coding efficiency are needed. [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] B. Bross, J. Chen, S. Liu, "Versatile Video Coding (Draft 10)," 19th JVET Meeting, Teleconference, July 2020, JVET-S2001 [Non-Patent Document 2] J.Chen, Y.Ye, S.Kim, "Algorithm description for Versatile Video Coding and Test Model 10(VTM10)", 19th JVET Meeting, Teleconference, July 2020, JVET-S2002 [Non-Patent Document 3] X. Zhao, V. Seregin, M. Karczewicz, "Six tap intra interpolation filter", 4th JVET Meeting, Chengdu, China, October 2016, JVET-D0119 [Overview of the project] [Means for solving the problem]

[0005] Several examples describe systems and techniques for performing intra-prediction using enhanced interpolation filters, which can apply various types and degrees of smoothing based on information such as block size and intra-prediction mode. According to at least one exemplary example, a method for processing video data is provided. This method includes the steps of: determining an intra-prediction mode for predicting blocks of video data; determining a type of smoothing filter to use on blocks of video data, wherein the type of smoothing filter is determined at least in part on comparing at least one of the width and height of blocks of video data to a first threshold; and performing intra-prediction on blocks of video data using the determined type of smoothing filter and the intra-prediction mode.

[0006] In another example, a device for processing video data is provided, comprising at least one memory (for example, configured to store data such as virtual content data, one or more images) and at least one processor (for example, implemented in circuitry) coupled to at least one memory. The one or more processors are configured and capable of: determining an intra-prediction mode for predicting blocks of video data; determining a type of smoothing filter to use on blocks of video data, the type of smoothing filter being determined at least in part on comparing at least one of the width and height of the blocks of video data to a first threshold; and performing an intra-prediction on blocks of video data using the determined type of smoothing filter and the intra-prediction mode.

[0007] In another example, there is a non-temporary computer-readable medium storing instructions thereon, wherein when an instruction is executed by one or more processors, the medium causes one or more processors to determine an intra-prediction mode for a block of video data, and to determine a type of smoothing filter to use for the block of video data, wherein the type of smoothing filter is determined at least in part on comparing at least one of the width and height of the block of video data to a first threshold, and to perform an intra-prediction for the block of video data using the determined type of smoothing filter and the intra-prediction mode.

[0008] In another example, a device for processing video data is provided. This device includes means for determining an intra-prediction mode for predicting blocks of video data; means for determining a type of smoothing filter to be used on blocks of video data, wherein the type of smoothing filter is determined at least in part on comparing at least one of the width and height of blocks of video data with a first threshold; and means for performing an intra-prediction on blocks of video data using the determined type of smoothing filter and the intra-prediction mode.

[0009] In some embodiments, the process, apparatus, and computer-readable medium may further include using a first smoothing interpolation filter as a determined type of smoothing filter, at least in part on the determination that at least one of the block width and block height is greater than a first threshold, and using the first smoothing interpolation filter to determine a reference pixel for intra-prediction of a block of video data.

[0010] In some embodiments, the first smoothing interpolation filter includes a 6-tap Gaussian filter.

[0011] In some embodiments, the process, apparatus, and computer-readable medium may further include using a second smoothing interpolation filter as the determined type of smoothing filter, at least in part on the determination that at least one of the block width and block height is less than or equal to a first threshold, and using the second smoothing interpolation filter to determine a reference pixel for intra-prediction of a block of video data.

[0012] In some embodiments, the second smoothing interpolation filter includes a 4-tap Gaussian filter.

[0013] In some embodiments, the process, apparatus, and computer-readable medium may further include determining a minimum offset between the angular direction of an intra-prediction mode and one of a vertical intra-prediction mode and a horizontal intra-prediction mode, and determining the type of smoothing filter to use for a block of video data, based on comparing the determined minimum offset with a second threshold.

[0014] In some embodiments, the process, apparatus, and computer-readable medium may further include determining a low-pass filter as the type of smoothing filter, at least in part, based on the determination that the determined minimum offset is greater than a second threshold and the determination that the intra-prediction mode is an integer-angled mode associated with an integer-valued reference pixel position.

[0015] In some embodiments, the low-pass filter performs reference pixel smoothing without interpolation, and the low-pass filter includes a

[0121] filter.

[0016] In some embodiments, the process, apparatus, and computer-readable medium may further include determining a Gaussian filter as the type of smoothing filter, at least in part, based on the determination that the determined minimum offset is greater than a second threshold and the determination that the intra-prediction mode is a fractional angle mode associated with the reference pixel position of the fractional value.

[0017] In some embodiments, the Gaussian filter performs smoothing interpolation without smoothing the reference pixels.

[0018] In some embodiments, the Gaussian filter includes a 6-tap Gaussian filter based on the determination that at least one of the block width and block height is greater than a first threshold.

[0019] In some embodiments, the Gaussian filter includes a 4-tap Gaussian filter based on the determination that at least one of the block width and block height is less than or equal to a first threshold.

[0020] In some embodiments, a process, apparatus, and computer-readable medium may use an interpolation filter as the determined type of smoothing filter, at least in part on the determination that the determined minimum offset is less than or equal to a second threshold, wherein the interpolation filter is a 4-tap cubic filter, and further may include using the interpolation filter to perform intra-prediction on blocks of video data without applying reference pixel smoothing.

[0021] In some embodiments, the process, apparatus, and computer-readable medium may further include determining a low-pass filter as the type of smoothing filter, at least in part, based on the determination that the intra-prediction mode is an integer-angle mode and the determination that the determined minimum offset is greater than a second threshold.

[0022] In some embodiments, the process, apparatus, and computer-readable medium perform reference pixel smoothing using a large-tap low-pass filter based at least in part on a determination that at least one of the block width and block height is greater than a first threshold, and the large-tap low-pass filter may further include applying a higher degree of reference pixel smoothing than a small-tap low-pass filter.

[0023] In some embodiments, the process, apparatus, and computer-readable medium may further include performing reference pixel smoothing using a small-tap low-pass filter based at least in part on a determination that at least one of the block width and block height is less than or equal to a first threshold, and the small-tap low-pass filter applies a lower degree of reference pixel smoothing than the large-tap low-pass filter.

[0024] In some embodiments, the process, apparatus, and computer-readable medium may further include determining that the intra prediction mode is an intra angle mode based at least in part on comparing the slope of the intra prediction mode with one or more pixel positions determined from the block width and block height.

[0025] In some embodiments, the process, apparatus, and computer-readable medium may further include determining that an offset between an angular direction of the intra prediction mode and a vertical intra prediction mode or a horizontal intra prediction mode is less than a second threshold, and based on determining that the offset between the angular direction of the intra prediction mode and the vertical intra prediction mode or the horizontal intra prediction mode is less than the second threshold, performing intra prediction on a block of video data using a cubic interpolation filter.

[0026] In some embodiments, the process, apparatus, and computer-readable medium may further include performing a reference line extension using a weak interpolation filter, the weak interpolation filter being used to perform the reference line extension prior to the use of a cubic interpolation filter for performing intra-prediction, the cubic interpolation filter having a higher cutoff frequency than the weak interpolation filter and applying a higher degree of smoothing than the weak interpolation filter.

[0027] In some embodiments, weak interpolation filters include 4-tap sinc-based interpolation filters and 6-bit 4-tap interpolation filters.

[0028] In some embodiments, the type of smoothing filter is signaled within the video bitstream.

[0029] In some embodiments, the type of smoothing filter is signaled to individual sets of prediction blocks, coding blocks, coding tree units (CTUs), slices, or sequences.

[0030] In some embodiments, the process, apparatus, and computer-readable medium may further include determining the type of smoothing filter based on at least one of the width and height of a block, without using information explicitly signaled in the video bitstream.

[0031] In some embodiments, the process, apparatus, and computer-readable medium may further include determining blocks of residual data with respect to blocks of video data, and decoding blocks of video data using blocks of residual data and predicted blocks determined based on performing intra-predictions on blocks of video data.

[0032] In some embodiments, the process, apparatus, and computer-readable medium may further include generating an encoded video bitstream containing information relating to blocks of video data.

[0033] In some embodiments, the process, apparatus, and computer-readable medium may further include storing the encoded video bitstream (for example, in at least one memory of the apparatus).

[0034] In some embodiments, the process, apparatus, and computer-readable medium may further include transmitting an encoded video bitstream (for example, using a transmitter in the apparatus).

[0035] In some embodiments, each of the devices described above may be a mobile device (e.g., a mobile phone or so-called “smartphone”, a tablet computer, or other type of mobile device), a networked wearable device, an extended reality device (e.g., a virtual reality (VR) device, an augmented reality (AR) device, or a mixed reality (MR) device), a personal computer, a laptop computer, a server computer (e.g., a video server or other server device), a television, a vehicle (or a computing device or system in a vehicle), a camera (e.g., a digital camera, an Internet Protocol (IP) camera, etc.), a multi-camera system, a robotics device or system, an aerospace device or system, or any other device, or a part thereof. In some embodiments, each of the devices may include at least one camera for capturing one or more images or video frames. For example, each of the devices may include one camera (e.g., an RGB camera) or more cameras for capturing one or more images and / or one or more videos, including video frames. In some embodiments, each of the devices may include a display for displaying one or more images, videos, notifications, or other displayable data. In some embodiments, each of the devices may include a transmitter configured to transmit one or more video frames and / or syntax data to at least one device via a transmission medium. In some embodiments, each of the devices may include one or more sensors.

[0036] This summary is not intended to identify the main or essential features of the claimed subject matter, nor is it intended to be used alone to determine the scope of the claimed subject matter. The subject matter should be understood by referring to the entire specification of this patent, any or all of the drawings, and the appropriate portions of each claim.

[0037] The above, along with other features and embodiments, will become clearer with reference to the following specification, claims, and accompanying drawings.

[0038] Exemplary embodiments of this application will be described in detail below with reference to the following figures. [Brief explanation of the drawing]

[0039] [Figure 1] This is a collection of examples of encoding and decoding devices, with several examples. [Figure 2A] This figure shows an example of an angle prediction mode based on several examples. [Figure 2B] This figure shows an example of a directional intra-predictive mode in versatile video coding (VVC), illustrated with several examples. [Figure 3] This figure shows an example of a mode-dependent intra smoothing (MDIS) process using several examples. [Figure 4] This figure shows an example of reference line extension using several examples. [Figure 5] This figure shows an example of switchable Gaussian filtering based on one or more of the block size and intra prediction modes, with several examples. [Figure 6] This flowchart shows an example of the process for performing intra-prediction using an extended smoothing filter, with several examples. [Figure 7] This is a diagram illustrating video encoding devices with several examples. [Figure 8] The Academy illustrates video decoding devices with several examples. [Modes for carrying out the invention]

[0040] Several aspects and embodiments of this disclosure are provided below. As will be apparent to those skilled in the art, some of these aspects and embodiments may be applied independently, and some of them may be applied in combination. Specific details are provided below for illustrative purposes to provide a complete understanding of the embodiments of this application. However, it will be apparent that various embodiments may be practiced without these specific details. The figures and description are not intended to be limiting.

[0041] The following description provides only exemplary embodiments and is not intended to limit the scope, applicability, or configuration of the disclosure. Rather, the following description of exemplary embodiments provides a description that enables the implementation of exemplary embodiments for those skilled in the art. It should be understood that various modifications may be made to the function and configuration of the elements without departing from the spirit and scope of the application as set forth in the appended claims.

[0042] Digital video data can contain massive amounts of data, especially given the ever-increasing demand for high-quality video data. For example, video data consumers desire increasingly higher quality video with high fidelity, resolution, and frame rates. However, the large amounts of video data required to meet such demands can place a significant burden on communication networks, as well as on devices that process and store the video data.

[0043] Video coding devices implement video compression techniques to efficiently encode and decode video data. Video compression techniques may include applying different prediction modes, including spatial prediction (e.g., intra-frame prediction or intra-prediction), temporal prediction (e.g., inter-frame prediction or inter-prediction), inter-layer prediction (across different layers of video data), and / or other prediction techniques to reduce or eliminate redundancy inherent in the video sequence. A video encoder may divide each picture of the original video sequence into rectangular regions called video blocks or coding units (described in more detail below). These video blocks may be encoded using a specific prediction mode.

[0044] A video block can be divided in one or more ways into one or more groups of smaller blocks. A block may include coding tree blocks, prediction blocks, transformation blocks, and / or other suitable blocks. Generally, a “block” may refer to such a video block (e.g., a coding tree block, coding block, prediction block, transformation block, or other suitable blocks or subblocks as will be understood by those skilled in the art) unless otherwise specified. Furthermore, each of these blocks may also be interchangeably referred to herein as a “unit” (e.g., a coding tree unit (CTU), a coding unit, a prediction unit (PU), a transformation unit (TU), etc.). In some cases, a unit may represent a coding logic unit encoded in a bitstream, and a block may represent a portion of a video frame buffer that a process targets.

[0045] In interprediction mode, the video encoder can search for a block similar to the one being encoded in a frame (or picture) located at a different time location, called a reference frame or reference picture. The video encoder may restrict this search to a certain spatial displacement from the block to be encoded. The best match location may be determined using a two-dimensional (2D) motion vector that includes horizontal and vertical displacement components. In intraprediction mode, the video encoder may use spatial prediction techniques to form a predicted block based on data from previously encoded neighboring blocks within the same picture.

[0046] A video encoder can determine a prediction error. For example, the prediction may be determined as the difference between the pixel values ​​in the encoded block and the predicted block. The prediction error is sometimes called the residual. The video encoder can also apply a transformation (e.g., a discrete cosine transform (DCT) or another suitable transformation) to the prediction error to generate transformation coefficients. After the transformation, the video encoder may quantize the transformation coefficients. The quantized transformation coefficients and motion vectors can be represented using syntax elements and, together with control information, form a coded representation of the video sequence. In some cases, the video encoder may entropy encode the quantized transformation coefficients and / or syntax elements, thereby further reducing the number of bits required for their representation.

[0047] After entropy decoding and dequantizing the received bitstream, the video decoder can use the syntax elements and control information discussed above to construct prediction data (e.g., prediction blocks) for decoding the current frame. For example, the video decoder may add the predicted blocks to the compressed prediction error. The video decoder may determine the compressed prediction error by weighting the transformation basis function using quantization coefficients. The difference between the reconstructed frame and the original frame is called the reconstruction error.

[0048] Video coding may be performed in accordance with a specific video coding standard. Examples of video coding standards include, but are not limited to, ITU-T H.261, ISO / IEC MPEG-1 Visual, ITU-T H.262 or ISO / IEC MPEG-2 Visual, ITU-T H.263, ISO / IEC MPEG-4 Visual, Advanced Video Coding (AVC) or ITU-T H.264, its Scope and Screen Content Coding, High Efficiency Video Coding (HEVC) or ITU-T H.265, Multipurpose Video Coding (VVC) or ITU-T H.266 and its extensions, VP9, ​​Alliance of Open Media (AOMedia) Video 1 (AV1), and Essential Video Coding (EVC).

[0049] As described above, a video encoder can divide each picture in the original video sequence into one or more smaller blocks or rectangular regions, which can then be encoded, for example, using intra-prediction (or intra-frame prediction) to remove spatial redundancy inherent in the original video sequence. When blocks are encoded in intra-prediction mode, the predicted blocks are formed based on previously encoded and reconstructed blocks, and these blocks are available to both the video encoder and video decoder to form prediction references. For example, the spatial prediction of pixel values ​​within the current block (e.g., currently being encoded or decoded) can be determined using the pixel values ​​of adjacent, previously encoded blocks. These pixel values ​​are used as reference pixels. Reference pixels can be organized into one or more reference pixel lines and / or reference pixel groups. In some examples, intra-prediction can be applied to both the lumen and chroma components of a block.

[0050] Several different intra-prediction modes may be available to provide different spatial prediction techniques for forming a predicted reference or predicted block based on data from previously encoded neighboring blocks within the same picture (for example, from a reference pixel). Intra-prediction modes may include planar modes, DC modes, and / or directional intra-prediction modes (also called “normal intra-prediction modes”). In some examples, a single planar intra-prediction and a single DC intra-prediction mode may be used, along with multiple directional intra-prediction modes. An intra-prediction mode describes a different variant or technique for calculating a pixel value in an area being coded based on a reference pixel value. In an exemplary example, the HEVC standard provides 33 directional intra-prediction modes. In another exemplary example, VVC and / or VVC Test Model 5 (VTM5) extend the HEVC directional intra-prediction modes to provide a total of 93 directional intra-prediction modes.

[0051] In a video decoder, the selection of an intra-prediction mode for each encoded block (e.g., the selection of an intra-prediction mode made by the video encoder when generating the encoded block) may be determined by the decoder (e.g., derived) or signaled to the video decoder. For example, in some cases, intra-prediction modes between neighboring blocks may be correlated (e.g., if two adjacent, previously encoded blocks were predicted using intra-prediction mode 2, the best intra-prediction mode for the current block may also be intra-prediction mode 2). In some examples, for each current block, the video encoder and video decoder may calculate the most probable intra-prediction mode. The video encoder may also signal the intra-prediction mode to the video decoder (e.g., using flags, mode parameters, mode selectors, etc.).

[0052] The current VVC standard provides 93 directional intra-prediction modes, as previously mentioned. Each intra-prediction mode is associated with a different angular direction such that each intra-prediction mode is unique and non-overlapping. Directional intra-prediction modes can be classified as either integer-angle modes or fractional (non-integer-angle) modes. For a given block of video data, an integer-angle intra-prediction mode has reference pixels at integer positions; for example, an integer-angle intra-prediction mode has a slope that passes through the positions of reference pixels located around the block currently being coded. In contrast, a fractional intra-prediction mode does not have reference pixels at integer positions, but instead has a slope that passes through a point somewhere between two adjacent reference pixels (for example, a slope of a pixel at fractional position i+f (i: integer part, f: fractional part) passes through pixels i and i+1).

[0053] According to the VVC standard, one or more smoothing filters and / or operations may be applied to a reference pixel based on the intra-prediction mode. By smoothing or filtering the reference pixel, more accurate intra-prediction results may be obtained as the intra-prediction results are calculated from the smoothed reference pixel. In some examples, reference pixel smoothing may be performed for both fractional intra-prediction modes and integer (e.g., integer slope) intra-prediction modes. In addition to smoothing filters for reference pixel smoothing, the VVC standard also specifies the use of one or more interpolation filters. In some examples, smoothing may be performed by directly smoothing the reference pixel. In some examples, the smoothing operation may be combined with or performed together with an interpolation operation (e.g., by applying a smoothing interpolation filter).

[0054] For example, an interpolation filter may be used to perform interpolation for fractional intra-prediction modes. Fractional intra-prediction modes are non-integer value slopes and are therefore associated with fractional reference pixel positions (e.g., positions between adjacent reference pixels). Intra-predictions for fractional intra-prediction modes can therefore be interpolated between adjacent reference pixel values ​​to compute an interpolated value for a fractional reference pixel position. In some scenarios, the majority of directional intra-prediction modes may be fractional (e.g., non-integer) modes. For example, in the VVC standard, intra-prediction modes -14, -12, -10, -6, 2, 18, 34, 50, 66, 72, 76, 78, and 80 may be integer intra-prediction modes (also called "integer slope modes"), while the remaining modes of the 93 directional intra-prediction modes are fractional intra-prediction modes.

[0055] The VVC standard specifies the use of a fixed degree of smoothing for all block sizes. For example, according to the VVC standard, coding devices (e.g., video coding devices and / or video decoding devices) may use a 4-tap Gaussian interpolation filter and / or a

[0121] low-pass filter for all block sizes. In some cases, the use of a fixed degree of smoothing for all block sizes (e.g., a 4-tap Gaussian interpolation filter and / or a

[0121] low-pass filter for all block sizes) may result in a reduction in intra-predictive performance. For example, larger block sizes (e.g., blocks with a width and / or height of 16 or more samples) may benefit from a higher degree of smoothing than smaller block sizes (e.g., blocks with a width and / or height of fewer than 16 samples). Because the block partitioning scheme in VVC allows for different block sizes based on different inputs, parameters, and other analytical factors, large and small block sizes may be encountered when performing intra-predictive analysis according to the VVC standard. In some cases, larger block sizes may be associated with portions of the original video sequence image that already contain relatively smooth edges and / or a relatively low number of features. Smaller block sizes can be associated with portions of the original video sequence image that contain a relatively large number of features, directions, etc.

[0056] Since creating larger block sizes is often associated with the presence of relatively smooth video data within the blocks, intra-predictions with larger block sizes may, in some cases, benefit from applying a higher degree of smoothing, while intra-predictions with smaller block sizes may benefit from applying a lower degree of smoothing.

[0057] This specification describes systems, apparatus, methods, and computer-readable media (collectively referred to as "Systems and Techniques") for providing improved intra-prediction, as will be described in more detail herein. For example, as will be described in more detail herein, Systems and Techniques may perform intra-prediction using multiple smoothing and / or interpolation filters, each using a different degree of smoothing and / or filtering. According to some embodiments, Systems and Techniques may include selecting one or more smoothing and interpolation filters (as well as associated types of smoothing and / or associated degrees of smoothing) based on the size of the block currently being coded. For example, one or more of the block width and / or block height may be compared to a predetermined threshold, and smaller blocks (e.g., blocks with a width and / or height less than the threshold) receive a different degree or smoothing than larger blocks (e.g., blocks with a width and / or height greater than the threshold).

[0058] In some examples, smoothing and / or interpolation filters may be selected, either additionally or as an alternative, based on the intra-prediction mode used for a picture or a portion of a picture (e.g., a block, slice, etc.). The relationship between a particular intra-prediction mode and the smoothing or interpolation filter may be predetermined and / or determined in real time (e.g., as the picture, block, slice, etc., is encoded or decoded). In an exemplary example, the intra-prediction mode of the block currently being coded may be compared to the vertical and horizontal intra-prediction modes to determine the minimum distance (e.g., angular distance or offset) between the current block intra-prediction mode and one of the vertical and horizontal intra-prediction modes. The minimum distance may be compared to a predetermined threshold (defined in some examples by the VVC standard) to determine whether smoothing and / or filtering should be applied to the block currently being coded. In some examples, variable smoothing of a reference pixel with block-level switching, as described herein, may provide enhanced intra-prediction, as will be described in more detail below.

[0059] Further details regarding the system and techniques will be explained with reference to the drawings.

[0060] Figure 1 is a block diagram showing an example of a system 100 including an encoding device 104 and a decoding device 112. The encoding device 104 may be part of a source device, and the decoding device 112 may be part of a receiving device. The source device and / or receiving device may include electronic devices such as mobile or fixed telephone handsets (e.g., smartphones, cellular phones, etc.), desktop computers, laptop or notebook computers, tablet computers, set-top boxes, televisions, cameras, display devices, digital media players, video gaming consoles, video streaming devices, Internet Protocol (IP) cameras, or any other suitable electronic devices. In some examples, the source device and receiving device may include one or more wireless transceivers for wireless communication. The coding techniques described herein are applicable to video coding in a variety of multimedia applications, including (e.g., streaming video transmission over the Internet), television broadcasting or transmission, encoding of digital video for storage on a data storage medium, decoding of digital video stored on a data storage medium, or other applications. The term coding as used herein may refer to encoding and / or decoding. In some cases, system 100 can support one-way or two-way video transmission to support applications such as video conferencing, video streaming, video playback, video broadcasting, games, and / or video telephony.

[0061] The encoding device 104 (or encoder) may be used to encode video data using a video coding standard, format, codec, or protocol for generating an encoded video bitstream. Examples of video coding standards and formats / codecs include ITU-T H.261, ISO / IEC MPEG-1 Visual, ITU-T H.262 or ISO / IEC MPEG-2 Visual, ITU-T H.263, ISO / IEC MPEG-4 Visual, including its Scalable Video Coding (SVC) and Multiview Video Coding (MVC) extensions, ITU-T H.264 (also known as ISO / IEC MPEG-4 AVC), High Efficiency Video Coding (HEVC) or ITU-T H.265, and Multipurpose Video Coding (VVC) or ITU-T H.266. Various extensions exist to HEVC trade with multi-layer video coding, including range and screen content coding extensions, 3D video coding (3D-HEVC), and multi-view extensions (MV-HEVC) and scalable extensions (SHVC). HEVC and its extensions are developed by the Joint Collaboration Team on Video Coding (JCT-VC), the Joint Collaboration Team on 3D Video Coding Extension Development (JCT-3V) of the ITU-T Video Coding Expert Group (VCEG), and the ISO / IEC Motion Picture Expert Group (MPEG). VP9, ​​AOMedia Video 1 (AV1) developed by the Alliance for Open Media (AOMedia), and Essential Video Coding (EVC) are other video coding standards to which the techniques described herein may be applied.

[0062] VVC, the latest video coding standard, was developed by the ITU-T and ISO / IEC Joint Video Expert Group (JEVT) to achieve at least partially superior compression capabilities compared to HEVC for a wide range of applications. The VVC specification was finalized in July 2020 and published by both ITU-T and ISO / IEC. The VVC specification specifies normative bitstream and picture formats, high-level syntax (HLS) and coding unit-level syntax, parsing processes, decoding processes, and more. VVC also specifies profile / tier / level (PTL) regulations, bytestream format, virtual reference decoder, and supplemental enhancement information (SEI) in its appendices.

[0063] The systems and techniques described herein may be applied to any existing video codec (e.g., VVC, HEVC, AVC, or other suitable existing video codecs) and / or may be efficient coding tools for any video coding standards currently under development and / or future video coding standards. For example, the examples described herein may be performed using video codecs such as VVC, HEVC, AVC, and / or their extensions. However, the techniques and systems described herein may also be applicable to other coding standards, codecs, or formats such as MPEG, JPEG (or other coding standards for still images), VP9, ​​AV1, their extensions, or other suitable coding standards that are already available or are not yet available or under development. For example, in some examples, the encoding device 104 and / or the decoding device 112 may operate according to a proprietary video codec / format, such as AV1, extensions of AVI, and / or successor versions of AV1 (e.g., AV2), or other proprietary formats or industry standards. Therefore, while the techniques and systems described herein may be described in relation to specific video coding standards, those skilled in the art will understand that the descriptions should not be interpreted as applying only to those specific standards.

[0064] Referring to Figure 1, the video source 102 may provide video data to the encoding device 104. The video source 102 may be part of a source device or part of a device other than a source device. The video source 102 may include a video capture device (e.g., a video camera, camera phone, videophone, etc.), a video archive containing stored video, a video server or content provider that provides video data, a video feed interface that receives video from a video server or content provider, a computer graphics system for generating computer graphics video data, a combination of such sources, or any other suitable video source.

[0065] The video data from video source 102 may include one or more input pictures or input frames. A picture or frame is a still image that is possibly part of the video. In some examples, the data from video source 102 may be a still image that is not part of the video. In HEVC, VVC, and other video coding specifications, a video sequence may include a series of pictures. A picture may include three sample arrays denoted as SL, SCb, and SCr. SL is a two-dimensional array of lumar samples, SCb is a two-dimensional array of Cb chrominance samples, and SCr is a two-dimensional array of Cr chrominance samples. Chrominance samples are sometimes referred to herein as “chroma” samples. A pixel refers to all three components (lumar samples and chroma samples) for a given location within the array of pictures. In other cases, a picture may be monochrome and may include only an array of lumar samples, in which case the terms pixel and sample may be used interchangeably. With regard to the exemplary technique described herein, referring to individual samples for illustrative purposes, the same technique may be applied to pixels (for example, all three sample components for a given location in an array of pictures). With regard to the exemplary technique described herein, referring to pixels (for example, all three sample components for a given location in an array of pictures) for illustrative purposes, the same technique may be applied to individual samples.

[0066] The encoder engine 106 (or encoder) of the encoding device 104 encodes video data to produce an encoded video bitstream. In some examples, the encoded video bitstream (or “video bitstream” or “bitstream”) is a series of one or more coded video sequences. A coded video sequence (CVS) includes a series of AUs, starting with an access unit (AU) having a random access point picture with certain characteristics in the base layer, up to immediately before the next AU having a random access point picture with certain characteristics in the base layer. For example, some characteristics of the random access point picture that initiates the CVS may include a RASL flag equal to 1 (e.g., NoRaslOutputFlag). Otherwise, the random access point picture (with a RASL flag equal to 0) does not initiate the CVS. An access unit (AU) includes one or more coded pictures and control information corresponding to coded pictures that share the same output time. Coded slices of pictures are encapsulated at the bitstream level in data units called network abstraction layer (NAL) units. For example, an HEVC video bitstream may contain one or more CVSs, each containing a NAL unit. Each NAL unit has a NAL unit header. In one example, the header is 1 byte for H.264 / AVC (except for multi-layer extensions) and 2 bytes for HEVC. The syntax elements within the NAL unit header take the specified bits and are therefore recognizable to all types of systems, and especially to transport layers such as transport streams, real-time transport (RTP) protocols, and file formats.

[0067] The HEVC standard has two classes of NAL units: video coding layer (VCL) NAL units and non-VCL NAL units. A VCL NAL unit contains one slice or slice segment (described below) of coded picture data, while a non-VCL NAL unit contains control information related to one or more coded pictures. In some cases, NAL units are referred to as packets. An HEVC AU contains a VCL NAL unit containing coded picture data and (if any) a non-VCL NAL unit corresponding to the coded picture data.

[0068] A NAL unit may contain a sequence of bits that form a coded representation of video data (e.g., an encoded video bitstream, a CVS of a bitstream, etc.), such as a coded representation of a picture in a video. The encoder engine 106 generates a coded representation of a picture by dividing each picture into multiple slices. Slices are independent of each other so that the information in a slice is coded without depending on data from other slices within the same picture. A slice contains one or more slice segments, including an independent slice segment and, if present, one or more dependent slice segments that depend on the previous slice segment. A slice is divided into coding tree blocks (CTBs) of lumar samples and chromar samples. One or more CTBs of lumar samples and one or more CTBs of chromar samples, along with the syntax for the sample, are called coding tree units (CTUs). A CTU is sometimes called a “tree block” or “largest coding unit” (LCU). A CTU is the basic processing unit for HEVC coding. A CTU can be split into multiple coding units (CUs) of varying sizes. The CU includes a lumens sample array and a chromatic sample array, which are called coding blocks (CBs).

[0069] The lumar CB and chroma CB can be further split into prediction blocks (PBs). A PB is a block of samples of the lumar or chroma component that uses the same motion parameters for inter-prediction or intra-block copy prediction (when available or enabled for use). The lumar PB and one or more chroma PBs, along with their associated syntax, form a prediction unit (PU). For inter-prediction, a set of motion parameters (e.g., one or more motion vectors, reference indices, etc.) is signaled in the bitstream for each PU and used for inter-prediction of the lumar PB and one or more chroma PBs. The motion parameters are sometimes called motion information. The CB can also be divided into one or more transformation blocks (TBs). A TB represents a square block of samples of the color component to which a residual transformation (e.g., possibly the same 2D transformation) is applied to code the prediction residual signal. A transformation unit (TU) represents the TBs of the lumar and chroma samples, as well as the corresponding syntax elements.

[0070] The size of a CU corresponds to the size of the coding mode and can be square in shape. For example, the size of a CU can be 8×8 samples, 16×16 samples, 32×32 samples, 64×64 samples, or any other suitable size up to the size of the corresponding CTU. The phrase "N×N" is used herein to refer to the pixel dimensions of a video block, converted to vertical and horizontal dimensions (e.g., 8 pixels × 8 pixels). Pixels within a block can be arranged in rows and columns. In some examples, a block may not have the same number of pixels horizontally as vertically. Syntax data associated with a CU may, for example, describe the division of the CU into one or more PUs. The division mode may differ between whether the CU is intra-predictive mode coded or inter-predictive mode coded. PUs may be divided such that their shape is non-square. Syntax data associated with a CU may also describe the division of the CU into one or more TUs following a CTU. TUs may be square or non-square in shape.

[0071] According to the HEVC standard, the conversion can be performed using conversion units (TUs). TUs may differ for different CUs. TUs may be sized based on the size of PUs within a given CU. TUs may be the same size as PUs, or smaller. In some examples, residual samples corresponding to CUs may be subdivided into smaller units using a quadtree structure known as a residual quadtree (RQT). Leaf nodes in the RQT may correspond to TUs. Pixel difference values ​​associated with TUs may be transformed to generate conversion coefficients. These conversion coefficients may be quantized by the encoder engine 106.

[0072] Once the video data picture is divided into CUs, the encoder engine 106 predicts each PU using a prediction mode. The prediction unit or prediction block is subtracted from the original video data to obtain a residual (described below). For each CU, the prediction mode may be signaled within the bitstream using syntax data. The prediction mode may include intra-prediction (or intra-picture prediction) or inter-prediction (or inter-picture prediction). Intra-prediction utilizes correlations between spatially neighboring samples within a picture. For example, with intra-prediction, each PU is predicted from neighboring image data within the same picture using, for example, a DC prediction to find the mean of the PU, a plane prediction to fit a flat surface to the PU, a direction prediction to extrapolate from neighboring data, or any other suitable type of prediction. Inter-prediction uses temporal correlations between pictures to derive motion-compensated predictions for blocks of image samples. For example, with inter-prediction, each PU is predicted using motion-compensated predictions from image data in one or more reference pictures (before or after the current picture in output order). The decision of whether to code a picture area using interpicture prediction or intrapicture prediction may be made, for example, at the CU level.

[0073] The encoder engine 106 and decoder engine 116 (described in more detail below) may be configured to operate according to VVC. According to some VVCs, the video coder (such as the encoder engine 106 and / or decoder engine 116) divides a picture into multiple coding tree units (CTUs) (a CTB for a lumen sample and one or more CTBs for a chroma sample, along with the syntax for the sample, are called CTUs). The video coder may divide the CTUs according to a tree structure such as a quad-binary tree (QTBT) structure or a multi-type tree (MTT) structure. The QTBT structure eliminates the concept of multiple division types, such as the distinction between CU, PU, ​​and TU in HEVC. The QTBT structure includes two levels, a first level divided according to a quad-tree division and a second level divided according to a binary tree division. The root node of the QTBT structure corresponds to a CTU. The leaf nodes of the binary tree correspond to coding units (CUs).

[0074] In an MTT partitioned structure, blocks can be partitioned using quadtree partitions, binary tree partitions, and one or more types of triple-tree partitions. A triple-tree partition is a partition in which a block is split into three subblocks. In some examples, a triple-tree partition divides a block into three subblocks without splitting the original block through a central point. The partition types in an MTT (e.g., quadtree, binary tree, and triple-tree) can be symmetric or asymmetric.

[0075] When operating according to the AV1 codec, the encoding device 104 and decoding device 112 may be configured to code video data in blocks. In AV1, the largest coding block that can be processed is called the superblock. In AV1, the superblock can be either 128×128 lumasamps or 64×64 lumasamps. However, in successor video coding formats (e.g., AV2), the superblock may be defined by a different (e.g., larger) lumasamp size. In some examples, the superblock is the top level of a block quadtree. The encoding device 104 may further subdivide the superblock into smaller coding blocks. The encoding device 104 may subdivide the superblock and other coding blocks into smaller blocks using rectangular or non-rectangular partitions. Non-rectangular blocks may include N / 2×N, N×N / 2, N / 4×N, and N×N / 4 blocks. The encoding device 104 and decoding device 112 may perform separate predictions and transform the process for each coding block.

[0076] AV1 also defines tiles of video data. A tile is a rectangular array of superblocks that can be coded independently of other tiles. That is, the encoding device 104 and decoding device 112 can encode and decode coding blocks within a tile, respectively, without using video data from other tiles. However, the encoding device 104 and decoding device 112 can perform filtering across tile boundaries. Tiles may be uniform or non-uniform in terms of size. Tile-based coding can enable parallel processing and / or multithreading for encoder and decoder implementations.

[0077] In some examples, the encoding device 104 and decoding device 112 may use a single QTBT or MTT structure to represent each of the luminance and chrominance components, but in other examples, the video coder may use two or more QTBT or MTT structures, such as one QTBT or MTT structure for the luminance component and another QTBT or MTT structure for both chrominance components (or two QTBT and / or MTT structures for each chrominance component).

[0078] The encoding device 104 and the decoding device 112 may be configured to use a quadtree partition, QTBT partition, MTT partition, or other partitioning structure per HEVC.

[0079] In some examples, one or more slices of a picture are assigned a slice type. Slice types include I-slices, P-slices, and B-slices. An I-slice (independently decodeable intra-frame) is a slice of a picture that is coded using intra-prediction only, and is therefore independently decodeable because an I-slice only requires data in the frame to predict any prediction unit or prediction block in the slice. A P-slice (unidirectional prediction frame) is a slice of a picture that can be coded using intra-prediction and unidirectional inter-prediction. Each prediction unit or prediction block in a P-slice is coded using either intra-prediction or inter-prediction. When inter-prediction is applied, the prediction unit or prediction block is predicted by only one reference picture, and therefore the reference sample comes from only one reference region in a frame. A B-slice (bidirectional prediction frame) is a slice of a picture that can be coded using intra-prediction and inter-prediction (e.g., either bidirectional or unidirectional). A prediction unit or prediction block of a B-slice can be predicted bidirectionally from two reference pictures, where each picture contributes to one reference region, and the sample sets of the two reference regions are weighted (e.g., with equal or different weights) to generate the prediction signal for the bidirectional prediction block. As described above, slices of a single picture are coded independently. In some cases, a picture may be coded as just one slice.

[0080] As described above, intra-picture prediction utilizes correlations between spatially neighboring samples within a picture. There are multiple intra-prediction modes (also called “intra-modes”). In some examples, intra-prediction of a lumen block includes 35 modes, including a planar mode, a DC mode, and 33 angular modes (e.g., a diagonal intra-prediction mode and angular modes adjacent to the diagonal intra-prediction mode). The encoding device 104 and / or decoding device 112 may select a prediction mode for each block that minimizes the residual between the predicted block and the block to be encoded (e.g., based on absolute error sum (SAE), absolute difference sum (SAD), absolute transformation error sum (SATD), or other measures of similarity). For example, SAE may be calculated by utilizing the absolute difference between each pixel (or sample) in the block to be encoded and the corresponding pixel (or sample) in the prediction block used for comparison. The differences of pixels (or samples) can be added together to create a measure of block similarity, such as a difference image of the L1 norm, the Manhattan distance between two image blocks, or other calculations. Using SAE as an example, the SAE for a prediction using each intra-prediction mode indicates the magnitude of the prediction error. The intra-prediction mode that has the best match for the actual current block is given by the intra-prediction mode that gives the minimum SAE.

[0081] The 35 intra-prediction modes are indexed as shown in Table 1 below. In other examples, more intra-modes may be defined, including predicted angles that may not yet be represented by the 33 angular modes. In other examples, the predicted angles associated with the angular modes may differ from those used in HEVC.

[0082] [Table 1]

[0083] To perform plane prediction for an N×N block, for each sample p located at (x, y), xy For this, the predicted sample value can be calculated by applying a bilinear filter to four specific neighboring reconstructed samples (used as reference samples for intra-prediction). The four reference samples are the reconstructed sample TR in the upper right, the reconstructed sample BL in the lower left, and the current sample in the same column (r x,-1 ) and row (r -1,y This includes two reconstructed samples located at ). The planar mode can be formulated as follows: p xy =((N-x1)*·L+(N-y1)*·T+x1*·R+y1*·B) / (2*N)

[0084] In the equation, x1 = x + 1, y1 = y + 1, R = TR, and B = BL.

[0085] In DC mode, prediction blocks are filled with the mean value of neighboring reconstructed samples. Generally, both planar and DC modes are applied to model smoothly changing and invariant regions of the image.

[0086] In the case of an angular intra-prediction mode in HEVC, which has 33 different prediction directions, the intra-prediction process can be described as follows: For each given angular intra-prediction mode, the intra-prediction direction can be identified accordingly; for example, intra-mode 18 corresponds to a purely horizontal prediction direction, and intra-mode 26 corresponds to a purely vertical prediction direction. The angular prediction modes are illustrated in the illustrative Figure 200a in Figure 2A. Some codecs may use a different number of intra-prediction modes. For example, in addition to the planar and DC modes, 93 angular modes can be defined, where mode 2 indicates a prediction direction of -135°, mode 34 indicates a prediction direction of -45°, and mode 66 indicates a prediction direction of 45°. Some codecs (e.g., VVC) may also define angles greater than -135° (less than -135°) and greater than 45° (greater than 45°), which are sometimes called wide-angle intra-modes. The description herein relates to intramode design in HEVC (i.e., having 35 modes), but the techniques disclosed may be applied to more intramodes (e.g., intramodes defined by VVC or other codecs).

[0087] The coordinates (x,y) of each sample in the prediction block are projected along a specific intra-prediction direction (e.g., one of the angular intra-prediction modes). For example, assuming a specific intra-prediction direction, the coordinates (x,y) of a sample in the prediction block are first projected along the intra-prediction direction to the row / column of the neighboring reconstructed samples. If (x,y) is projected to a fractional position α between two neighboring reconstructed samples L and R, the predicted value of (x,y) can be calculated using a two-tap bilinear interpolation filter formulated as follows: p xy = (1-a)·L+a·R

[0088] To avoid floating-point arithmetic, in HEVC, the above calculation can be approximated using integer arithmetic as follows: p xy =((32-a')·L+a'·R+16)>>5

[0089] In the formula, a' is an integer equal to 32*a.

[0090] In some examples, before intra-prediction, neighbor reference samples are filtered using a 2-tap bilinear or 3-tap (1,2,1) / 4 filter, which is sometimes called intra-reference smoothing or mode-dependent intra-smoothing (MDIS). When performing intra-prediction, assuming an intra-prediction mode index (predModeIntra) and block size (nTbS), it is determined whether a reference smoothing process is performed and which smoothing filter is used. The intra-prediction mode index is an index that indicates the intra-prediction mode.

[0091] Interpicture prediction uses time correlation between pictures to derive motion-compensated predictions for blocks of image samples. Using a translational motion model, the position of a block in a previously decoded picture (reference picture) is indicated by a motion vector (Δx, Δy), where Δx specifies the horizontal displacement of the reference block relative to the current block position, and Δy specifies its vertical displacement. In some cases, the motion vector (Δx, Δy) can be integer sample precision (also called integer precision), in which case the motion vector points to an integer Pell grid (or integer pixel sampling grid) of the reference frame. In some cases, the motion vector (Δx, Δy) can be fractional sample precision (also called fractional Pell precision or non-integer precision) to more accurately capture the motion of the underlying object without being limited to an integer Pell grid of the reference frame. The precision of the motion vector is represented by the quantization level of the motion vector. For example, the quantization level can be integer precision (e.g., 1 pixel) or fractional Pell precision (e.g., 1 / 4 pixel, 1 / 2 pixel, or other subpixel values). When the corresponding motion vector has fractional sample precision, interpolation is applied to the reference picture to derive the predicted signal. For example, to estimate values ​​at fractional positions, available samples at integer positions may be filtered (e.g., using one or more interpolation filters). Previously decoded reference pictures are indicated by their reference index (refIdx) in the reference picture list. The motion vector and reference index are sometimes called motion parameters. Two types of interpicture prediction can be performed, including single and bi-prediction.

[0092] Interpretation using biprediction (also called bidirectional interprediction) uses two sets of motion parameters (Δx0, y0, refIdx0 and Δx1, y1, refIdx1) to generate two motion-compensated predictions (from the same reference picture or possibly different reference pictures). For example, with biprediction, each prediction block uses two motion-compensated prediction signals to generate B prediction units. The two motion-compensated predictions are combined to obtain the final motion-compensated prediction. For example, the two motion-compensated predictions may be combined by averaging. In another example, weighted prediction may be used, in which case different weights may be added to each motion-compensated prediction. The reference pictures that may be used in biprediction are stored in two separate lists, shown as List 0 and List 1. The motion parameters may be derived in the encoding device 104 using a motion estimation process.

[0093] Interpretation using single prediction (also called unidirectional interpretation) uses one set of motion parameters (Δx0, y0, refIdx0) to generate motion-compensated predictions from a reference picture. For example, using single prediction, each prediction block uses at most one motion-compensated prediction signal to generate P prediction units.

[0094] A PU may contain data relating to the prediction process (e.g., motion parameters or other preferred data). For example, when a PU is encoded using intra-prediction, the PU may contain data describing the intra-prediction mode of the PU. As another example, when a PU is encoded using inter-prediction, the PU may contain data defining the motion vector of the PU. The data defining the motion vector of the PU may describe, for example, the horizontal component (Δx) of the motion vector, the vertical component (Δy) of the motion vector, the resolution of the motion vector (e.g., integer precision, 1 / 4 pixel precision, or 1 / 8 pixel precision), the reference picture pointed to by the motion vector, the reference index, the reference picture list of the motion vector (e.g., List 0, List 1, or List C), or any combination thereof.

[0095] AV1 includes two common techniques for encoding and decoding coding blocks of video data. The two common techniques are intra-prediction (e.g., intra-frame prediction or spatial prediction) and inter-prediction (e.g., inter-frame prediction or temporal prediction). In the context of AV1, when predicting a block of the current frame of video data using the intra-prediction mode, the encoding device 104 and decoding device 112 do not use video data from other frames of the video data. For most intra-prediction modes, the video encoding device 104 encodes the block of the current frame based on the difference between the sample value in the current block and the predicted value generated from a reference sample in the same frame. The video encoding device 104 determines the predicted value generated from the reference sample based on the intra-prediction mode.

[0096] After performing predictions using intra-prediction and / or inter-prediction, the encoding device 104 can perform transformations and quantization. For example, following predictions, the encoder engine 106 may compute residual values ​​corresponding to the PU. The residual values ​​may include pixel difference values ​​between the current block (PU) of the pixels being coded and the prediction block used to predict the current block (e.g., the predicted version of the current block). For example, after generating a prediction block (e.g., emitting an inter-prediction or intra-prediction), the encoder engine 106 may generate a residual block by subtracting the prediction block generated by the prediction unit from the current block. The residual block contains a set of pixel difference values ​​that quantify the difference between the pixel values ​​of the current block and the pixel values ​​of the prediction block. In some examples, the residual block may be represented in a two-dimensional block format (e.g., a two-dimensional matrix or two-dimensional array of pixel values). In such examples, the residual block is a two-dimensional representation of the pixel values.

[0097] Any residual data that may remain after the prediction has been performed is transformed using a block transform, which may be based on a discrete cosine transform, discrete sine transform, integer transform, wavelet transform, other suitable transform functions, or any combination thereof. In some cases, one or more block transforms (e.g., of sizes 32×32, 16×16, 8×8, 4×4, or other suitable sizes) may be applied to the residual data in each CU. In some examples, a TU may be used for the transformation and quantization processes implemented by the encoder engine 106. A given CU having one or more PUs may also include one or more TUs. As will be described in more detail below, the residual values ​​may be transformed into transformation coefficients using a block transform, and may be quantized and scanned using a TU to generate serial transformation coefficients for entropy coding.

[0098] In some examples, following intra-predictive coding or inter-predictive coding using the PU of the CU, the encoder engine 106 may compute residual data for the TU of the CU. The PU may contain pixel data in the spatial domain (or pixel domain). The TU may consist of coefficients in the transformation domain after applying a block transformation. As mentioned above, the residual data may correspond to the pixel difference between the pixels of the unencoded picture and the predicted values ​​corresponding to the PU. The encoder engine 106 may form a TU containing the residual data for the CU, and then transform the TU to generate transformation coefficients for the CU.

[0099] The encoder engine 106 may perform quantization of the conversion coefficients. Quantization achieves further compression by quantizing the conversion coefficients to reduce the amount of data used to represent the coefficients. For example, quantization may reduce the bit depth associated with some or all of the coefficients. In one example, a coefficient with an n-bit value may be truncated to an m-bit value during quantization, where n is greater than m.

[0100] Once quantization is performed, the coded video bitstream contains quantized transformation coefficients, prediction information (e.g., prediction mode, motion vector, block vector, etc.), piecewise information, and any other appropriate data, such as other syntax data. Different elements of the coded video bitstream can be entropy coded by the encoder engine 106. In some examples, the encoder engine 106 may use a predefined scan order to scan the quantization transformation coefficients to generate a serialized vector that can be entropy coded. In some examples, the encoder engine 106 may perform adaptive scanning. After scanning the quantization transformation coefficients to form a vector (e.g., a one-dimensional vector), the encoder engine 106 may entropy coded the vector. For example, the encoder engine 106 may use context-adaptive variable-length coding, context-adaptive binary arithmetic coding, syntax-based context-adaptive binary arithmetic coding, probability interval piecewise entropy coding, or another preferred entropy coding technique.

[0101] The output unit 110 of the encoding device 104 may send NAL units constituting encoded video bitstream data to the decoding device 112 of the receiving device via the communication link 120. The input unit 114 of the decoding device 112 may receive the NAL units. The communication link 120 may include a channel provided by a wireless network, a wired network, or a combination of a wired network and a wireless network. The wireless network may include any wireless interface or combination of wireless interfaces, and may include any suitable wireless network (e.g., the Internet or other wide area networks, packet-based networks, WiFi®, radio frequency (RF), UWB, WiFi-Direct, cellular, Long-Term Evolution (LTE), WiMAX®, etc.). The wired network may include any wired interface (e.g., fiber, Ethernet, power line Ethernet, Ethernet over coaxial cable, digital signal line (DSL), etc.). The wired network and / or wireless network may be implemented using various devices such as base stations, routers, access points, bridges, gateways, and switches. The encoded video bitstream data may be modulated according to a communication standard such as a wireless communication protocol and transmitted to a receiving device.

[0102] In some examples, the encoding device 104 may store the encoded video bitstream data in storage 108. The output unit 110 may retrieve the encoded video bitstream data from the encoder engine 106 or from storage 108. Storage 108 may include any of a variety of distributed or locally accessed data storage media. For example, storage 108 may include a hard drive, a storage disk, flash memory, volatile or non-volatile memory, or any other suitable digital storage medium for storing encoded video data. Storage 108 may also include a decoded picture buffer (DPB) for storing a reference picture for use in interpretation. In further examples, storage 108 may correspond to a file server or another intermediate storage device capable of storing encoded video generated by the source device. In such cases, a receiving device including a decoding device 112 can access the stored video data from the storage device via streaming or download. The file server may be any type of server capable of storing encoded video data and transmitting that encoded video data to the receiving device. An exemplary file server may include a web server (for a website, for example), an FTP server, a network-attached storage (NAS) device, or a local disk drive. The receiving device may access encoded video data through any standard data connection, including an internet connection, and may include wireless channels (e.g., Wi-Fi connection), wired connections (e.g., DSL, cable modem, etc.), or a combination of both, which are preferred for accessing encoded video data stored on the file server. Transmission of encoded video data from storage 108 may be streaming transmission, download transmission, or a combination thereof.

[0103] The input 114 of the decoding device 112 may receive encoded video bitstream data and provide the video bitstream data to the decoder engine 116 or to storage 118 for later use by the decoder engine 116. For example, storage 118 may include a DPB for storing reference pictures for use in interpretation. A receiving device including the decoding device 112 may receive encoded video data to be decoded via storage 108. The encoded video data may be modulated according to a communication standard such as a wireless communication protocol and transmitted to a receiving device. The communication medium for the transmitted encoded video data may comprise any wireless or wired communication medium, such as a radio frequency (RF) spectrum or one or more physical transmission lines. The communication medium may form part of a packet-based network, such as a local area network, a wide area network, or a global network such as the Internet. The communication medium may include routers, switches, base stations, or any other equipment that may be useful to facilitate communication from the source device to the receiving device.

[0104] The decoder engine 116 can decode the encoded video bitstream data by entropy decoding and extracting elements of one or more coded video sequences that make up the encoded video data (for example, using an entropy decoder). The decoder engine 116 can rescale and perform an inverse transform on the encoded video bitstream data. The residual data is passed to the prediction stage of the decoder engine 116. The decoder engine 116 predicts blocks of pixels (for example, PUs). In some examples, the predictions are added to the output of the inverse transform (residual data).

[0105] The decoding device 112 may output the decoded video to the video destination device 122, which may include a display or other output device for displaying the decoded video data to the content consumer. In some embodiments, the video destination device 122 may be part of a receiving device that includes the decoding device 112. In some embodiments, the video destination device 122 may be part of a separate device other than the receiving device.

[0106] In some examples, the video encoding device 104 and / or the video decoding device 112 may be integrated with an audio encoding device and an audio decoding device, respectively. The video encoding device 104 and / or the video decoding device 112 may also include other hardware or software necessary to implement the coding techniques described above, such as one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic, software, hardware, firmware, or any combination thereof. The video encoding device 104 and the video decoding device 112 may be integrated as part of a composite encoder / decoder (codec) in each device. An example of specific details of the encoding device 104 is described below with reference to Figure 8. An example of specific details of the decoding device 112 is described below with reference to Figure 9.

[0107] The exemplary system shown in Figure 1 is one exemplary example that may be used herein. The techniques for processing video data using the techniques described herein can be performed by any digital video coding and / or decoding device. Generally, the techniques of this disclosure are performed by a video coding device or a video decoding device, but the techniques can also be performed by a composite video encoder / decoder, usually called a “codec”. Furthermore, the techniques of this disclosure can also be performed by a video preprocessor. Source devices and receiving devices are merely examples of coding devices, where the source device generates coded video data for transmission to the receiving device. In some examples, the source and receiving devices may operate substantially symmetrically, such that each device contains video coding and decoding components. Thus, the exemplary system may support one-way or two-way video transmission between video devices for, for example, video streaming, video playback, video broadcasting, or video telephony.

[0108] Extensions to the HEVC standard include a multi-view video coding extension called MV-HEVC and a scalable video coding extension called SHVC. Both the MV-HEVC and SHVC extensions share the concept of layered coding, where different layers are included within the coded video bitstream. Each layer in the coded video sequence is addressed by a unique layer identifier (ID). The layer ID may be present in the header of a NAL unit to identify the associated layer. In MV-HEVC, different layers can represent different views of the same scene within the video bitstream. SHVC provides different scalable layers that represent the video bitstream at different spatial resolutions (i.e., picture resolutions) or different reconstruction fidelity. The scalable layers may include a base layer (with layer ID=0) and one or more enhancement layers (with layer IDs=1, 2, ..., n). The base layer may conform to the profile of the first version of HEVC and represent the lowest available layer in the bitstream. An enhancement layer offers increased spatial resolution, temporal resolution or frame rate, and / or reconstruction fidelity (i.e., quality) compared to the base layer. Enhancement layers are organized hierarchically and may (or may not) depend on lower layers. In some examples, different layers may be coded using a single standard codec (e.g., all layers are encoded using HEVC, SHVC, or another coding standard). In some examples, different layers may be coded using multiple standard codecs. For example, the base layer may be coded using AVC, and one or more enhancement layers may be coded using the SHVC extension and / or MV-HEVC extension to the HEVC standard.

[0109] Generally, a layer contains a set of VCL NAL units and a corresponding set of non-VCL NAL units. NAL units are assigned a specific layer ID value. Layers can be hierarchical in the sense that they may depend on lower layers. A layer set refers to a set of layers represented within a self-contained bitstream, meaning that a layer within a layer set can depend on other layers within the layer set in the decoding process, but does not depend on any other layer for decoding. Therefore, layers within a layer set can form independent bitstreams capable of representing video content. A set of layers within a layer set can be obtained from another bitstream through the operation of a sub-bitstream extraction process. A layer set can correspond to a set of layers to be decoded when a decoder wants it to operate according to certain parameters.

[0110] As previously described, an HEVC bitstream includes a group of NAL units, including VCL NAL units and non-VCL NAL units. A VCL NAL unit contains coded picture data that forms the coded video bitstream. For example, a sequence of bits that forms the coded video bitstream resides in a VCL NAL unit. Non-VCL NAL units may include parameter sets that have high-level information about the coded video bitstream, in addition to other information. For example, a parameter set may include a video parameter set (VPS), a sequence parameter set (SPS), and a picture parameter set (PPS). Examples of the purpose of a parameter set include providing bitrate efficiency, error resilience, and a system layer interface. Each slice refers to a single active PPS, SPS, and VPS to access information that the decoding device 112 can use to decode the slice. Identifiers (IDs) may be coded for each parameter set and include a VPS ID, SPS ID, and PPS ID. The SPS includes the SPS ID and VPS ID. The PPS includes the PPS ID and SPS ID. Each slice header contains a PPS ID. This ID allows the active parameter set to be identified for a given slice.

[0111] The PPS contains information that applies to all slices within a given picture. In some examples, all slices within a picture refer to the same PPS. Slices within different pictures may also refer to the same PPS. The SPS contains information that applies to all pictures within the same coded video sequence (CVS) or bitstream. As previously described, a coded video sequence is a series of AUs, starting with a random access point picture with certain characteristics (described above) in the base layer (e.g., an instantaneous decoded reference (IDR) picture or a broken link access (BLA) picture, or another suitable random access point picture) and ending immediately before the next access unit (AU) with a random access point picture with certain characteristics in the base layer (or the end of the bitstream). The information in the SPS may not change for each picture within the coded video sequence. Pictures within a coded video sequence may use the same SPS. The VPS contains information that applies to all layers within a coded video sequence or bitstream. The VPS contains a syntax structure with syntax elements that apply to the entire coded video sequence. In some embodiments, the VPS, SPS, or PPS may be transmitted in-band along with the encoded bitstream. In some embodiments, the VPS, SPS, or PPS may be transmitted out-of-band in a separate transmission from the NAL unit containing the encoded video data.

[0112] This disclosure may generally refer to “signaling” certain information, such as syntax elements. The term “signaling” may generally refer to the communication of values ​​for syntax elements and / or other data used to decode the encoded video data. For example, video encoding device 104 may signal values ​​for syntax elements in the bitstream. Generally, signaling refers to generating values ​​in the bitstream. As described above, video source 102 may transport the bitstream to video destination device 122, either substantially in real time or not in real time, such as when storing syntax elements in storage 108 for later retrieval by video destination device 122.

[0113] A video bitstream may also contain Supplemental Enhancement Information (SEI) messages. For example, an SEI NAL unit may be part of a video bitstream. In some cases, SEI messages may contain information that is not required by the decoding process. For example, information in an SEI message may not be essential for the decoder to decode the video picture in the bitstream, but the decoder may use that information to improve the display or processing of the picture (e.g., the decoded output). The information in an SEI message may be embedded metadata. In one exemplary example, information in an SEI message may be used by a decoder-side entity to improve the readability of the content. In some cases, certain application standards may require the presence of such SEI messages in the bitstream so that quality improvements can be brought to all devices conforming to the application standard (e.g., carrying frame packing SEI messages for frame-compatible planar stereoscopic 3DTV video formats where SEI messages are carried for every frame of video, processing recovery point SEI messages, and the use of pan scan scan rectangle SEI messages in DVB, in addition to many other examples).

[0114] As described above, the encoding device 104 may encode one or more blocks or rectangular regions of pictures in the original video sequence using intra-prediction and / or intra-frame prediction to remove spatial redundancy. The decoding device 112 may decode the encoded blocks by using the same intra-prediction mode used by the encoding device 104. An intra-prediction mode describes a different variant or technique for calculating pixel values ​​in the area being coded based on reference pixel values. In the VVC standard, one or more smoothing and interpolation filters may be selected based on the intra-prediction mode and then applied to the reference pixels and / or intra-prediction of the current block. In this technique, the same selection of smoothing or interpolation filters used for intra-prediction is provided for all block sizes, for example, a fixed degree of smoothing is applied to all possible block sizes. Different directional intra-prediction modes are provided in the VVC standard.

[0115] Figure 2B shows an illustrative Figure 200b of directional intra-prediction modes (also called “angle intra-prediction modes”) in VVC. In some examples, the planar and DC modes maintain the same state in VVC as they do in HEVC. As shown, intra-prediction modes with even indices between 2 and 66 may be equal to 33 HEVC intra-prediction modes, and the remaining intra-prediction modes in Figure 2B represent newly added intra-predictions in VVC. As an illustrative example, to better capture any edge direction presented in natural video, the number of directional intra-prediction modes in VTM5 (VVC Test Model 5) was increased from 33 HEVC directions to a total of 93 directions. The intra-prediction mode is described in more detail in B. Bross, J. Chen, and S. Liu, "Versatile Video Coding (Draft 10)," 19th JVET Meeting, Teleconference, July 2020, JVET-S2001, which is incorporated herein by reference in its entirety for all purposes. In some cases, the higher-density directional intra-prediction modes introduced in the VVC standard can be applied to all block sizes and to both lumane and chromane intra-prediction. In some cases, these directional intra-prediction modes can be used in combination with either multiple reference lines (MLR) and / or intra-subpartition mode (ISP). Further details are provided in J. Chen, Y. Ye, and S. Kim, "Algorithm description for Versatile Video Coding and Test Model 10 (VTM10)," 19th JVET Meeting, Teleconference, July 2020, JVET-S2002, which is incorporated herein by reference in its entirety for all purposes.

[0116] In some examples, mode-dependent intra-smoothing (MDIS) may be used to smooth the intra-prediction signal by applying a smoothing filter and / or type of smoothing based on the intra-prediction mode of the block currently being coded. Figure 3 is a flowchart showing an example of an MDIS process 300 that may be used for intra-prediction. In the exemplary example, the exemplary MDIS process in Figure 3 may be the same as the MDIS process in the VVC standard. The exemplary MDIS process 300 may be used to select a specific interpolation filter and / or a specific smoothing filter that will be used in the intra-prediction for the block currently being coded. As will be discussed in more detail below, in some examples, the selection of the interpolation and / or smoothing filter may be based at least in part on the intra-prediction mode of the block currently being coded.

[0117] An exemplary MDIS process 300 may begin in operation 302 by determining whether the intra-prediction mode of the block currently being coded is horizontal intra-prediction mode or vertical intra-prediction mode. Referring to the directional intra-prediction modes shown in Figure 2B, the horizontal intra-prediction mode is shown as mode 18 and the vertical intra-prediction mode is shown as mode 50. In response to the determination in operation 302 that the intra-prediction mode is either horizontal or vertical (e.g., a "Yes" output in 302), the exemplary MDIS process may proceed to operation 304. As shown, operation 304 terminates the MDIS process without performing reference pixel smoothing or applying an interpolation filter. In some examples, no smoothing or no interpolation may be performed for both the horizontal and vertical intra-prediction modes because the reference pixel values ​​for these two modes can be directly copied when determining the predicted pixel values ​​for the block currently being coded.

[0118] If the intra-prediction mode is neither horizontal nor vertical (e.g., a “No” output in operation 302), the exemplary MDIS process can proceed to determine whether smoothing is required for the current block. As shown, the decision of whether smoothing should be performed on the current block may be made in operation 306, at least in part, based on the intra-prediction mode of the current block. For example, the intra-prediction mode may be used to calculate the minimum distance minDistVerHor, where, for example, minDistVerHor is the minimum of {|intra-prediction mode number - vertical intra-prediction mode number|, |intra-prediction mode number - horizontal intra-prediction mode number|}. The minimum distance minDistVerHor may also be called the minimum angular offset and / or minimum angular distance. In the exemplary example, the vertical intra-prediction mode number may be 50 and the horizontal intra-prediction mode number may be 18. Therefore, if the current number of intra-prediction modes for a block is 30, the minimum angular offset can be calculated as min{|30-50|,|30-18|}=min{20,12}=12.

[0119] In operation 306, the minimum angular offset minDistVerHor may then be compared to a threshold intraHorVerDistThres[nTbS], where the threshold intraHorVerDistThres[nTbS] may be a predetermined threshold given by the VVC standard, determined in some examples, for example, by providing the current transformed block size nTbS as an index to a lookup function or lookup table intraHorVerDistThres. As shown in Figure 3, if the minimum angular offset minDistVerHor is less than or equal to the threshold intraHorVerDistThres[nTbS], operation 306 may determine that no smoothing is required for the current block, for example, outputting "No" in 306.

[0120] If smoothing is not required, the exemplary MDIS process can proceed from operation 306 to operation 307, which is shown here as applying an interpolation filter without any reference pixel smoothing. In some examples, the interpolation filter applied by operation 307 may be a cubic interpolation filter, such as the 4-tap (6-bit) cubic interpolation filter shown in Figure 3. Since operation 306 determined that direct reference pixel smoothing is not required, operation 307 may apply only the 4-tap cubic interpolation filter; for example, no reference pixel smoothing is performed because the minimum angular offset in operation 306 indicated that the intra-prediction mode was within a threshold distance from either the horizontal or vertical mode.

[0121] If operation 306 determines that the minimum angular offset minDistVerHor is greater than the threshold intraHorVerDistThres[nTbS], operation 306 may determine that smoothing is required for the current block, for example, by outputting "Yes". In response to the determination that smoothing is required, the intra predictive mode for the current block may be further analyzed in the subsequent operation 308.

[0122] In some examples, operation 308 may analyze the intra-prediction mode for the current block to determine whether it is an integer-angled intra-prediction mode or a fractional-angled intra-prediction mode (also known as "integer-angled mode" and "fractional-angled mode," respectively). As previously mentioned, the integer-angled mode is associated with a specific integer-value reference pixel location in the current block, while the fractional-angled mode is not associated with a specific integer-value reference pixel location. Instead, the fractional-angled mode is associated with some intermediate (e.g., fractional) location between adjacent integer-value reference pixel locations.

[0123] Based on the determination that operation 308 determines that the current intra-prediction mode for the block is integer angle mode (e.g., the "Yes" output of 308), operation 308 can proceed to operation 309. As shown, in some cases it is determined that interpolation is not necessary for integer angle mode, so operation 309 may perform reference pixel smoothing but not interpolation. For example, since the integer angle intra-prediction mode can use the reference pixel value directly, only reference pixel smoothing is performed. In some examples, the reference pixel smoothing in operation 309 may be performed by applying a low-pass filter, such as the

[0121] filter, which calculates the average of a sum of numbers consisting of twice the reference pixel value plus the values ​​of the directly adjacent left and right (or above and below) reference pixel positions.

[0124] Based on operation 308 determining that the intra-prediction mode for the current block is fractional angle mode (e.g., non-integer angle mode, output "No" for 308), the subsequent operation 310 may, in some cases, calculate interpolated values ​​for the fractional reference pixel locations associated with the intra-prediction mode. For example, operation 310 may calculate interpolated fractional reference pixel location values, which may be calculated based on one or more reference pixel values ​​obtained from one or more adjacent integer reference pixel locations. Recalling that in operation 306 it was previously determined that smoothing should be performed on the intra-prediction of the current block (for example, because operation 306 determined that the minimum angle offset minDistVerHor > threshold intraHorVerDistThres[nTbS]), the "No" output of operation 308 may correspond to a scenario where both smoothing and interpolation are applied to the current block.

[0125] In some examples, as shown in Figure 3, the smoothing and interpolation operations may be performed in a single combined step, for example, by applying a smoothing interpolation filter. In an exemplary example, the smoothing interpolation filter may be provided as a Gaussian interpolation filter that smooths the generated intra-predicted signal and simultaneously interpolates the fractional reference pixel position values. The smoothing interpolation filter, such as the Gaussian interpolation filter described above, may apply smoothing without performing direct reference pixel smoothing. In some examples, the smoothing interpolation filter may include a 4-tap (6-bit) Gaussian interpolation filter, as shown in operation 310.

[0126] In the context of the exemplary MDIS process 300 in Figure 3, it should be noted that the MDIS process (and the VVC standard) does not use variable degree smoothing based on block size or other characteristics. In some examples, the systems and techniques described herein may provide variable degree smoothing and / or interpolation based at least partially on factors including the intra-prediction mode of the current block, the size of the current block, the width of the current block, the height of the current block, etc.

[0127] In some examples, video coding techniques may include using a directional intra-prediction mode, along with one or more of the main reference line extension (MRL) and / or intra-subpartition mode (ISP), to perform intra-prediction. In an exemplary example, intra-prediction may include using one or more side reference pixels to extend the main reference pixel line for intra-prediction.

[0128] Figure 4 shows an illustrative figure 400 of reference line extension using one or more side reference pixels. Shown for block 405 currently being coded is an overline 410 of the reference pixels containing a set of calculated reference line extension pixels 420. A set of left reference pixels 430 is also shown. For intra-prediction in vertical mode (e.g., intra-prediction mode >= 34, not to be mixed with a particular vertical intra-prediction mode 50), one or more pixels from the left reference pixels 430 of block 405 currently being coded may be used to extend the overline 410 of the reference pixels, for example, by generating or possibly calculating values ​​for the reference line extension pixels 420. The calculation of the reference line extension pixels 420 may be used to extend the length of the overline 410 of the reference pixels so that it extends beyond the left edge of block 405, as shown in Figure 4.

[0129] In the current VVC standard, the top line 410 of a reference pixel can be extended by identifying its nearest neighbor in the left reference pixel 430, and the value of the identified nearest neighbor is set to equal to the value of at least one of the reference line extension pixels 420. In an exemplary example, Figure 4 shows a point P (indicated by, for example, 423) located in the reference line extension pixels 420 of the top line 410 of a reference pixel. The top reference pixel line 410 is extended based on the left reference pixel 430. In the current VVC standard, the reference line extension process proceeds by determining which of the left reference pixels 430 is the nearest neighbor to the extended reference line pixel P / 423, and then setting the value of the extended reference line pixel P to equal to the value of the identified nearest neighbor in the left reference pixel 430. As shown in Figure 4, the nearest neighbor in the column of left reference pixels 430 is indicated as X1, and the pixel value at position X1 is therefore used to create the extended reference line pixel P (indicated by, 423). The top line 410 of the reference pixel may be extended to a desired length using this methodology, and then intra-prediction is performed using the extended reference line formed by the original top line 410 of the reference pixel and the extended reference line pixel 420. In some examples, a similar process may be applied to intra-prediction in horizontal mode (e.g., intra-prediction mode < 34, so as not to be mixed with a particular horizontal intra-prediction mode 18), where the values ​​of identified nearest neighbor pixels in the top reference line are projected to extend the left line of the reference pixel.

[0130] Various improvements to the VVC intra-prediction process are proposed in JVET-D0119, described in X. Zhao, V. Seregin, and M. Karczewicz, "Six tap intra-interpolation filter," 4th JVET Meeting, Chengdu, China, October 2016, JVET-D0119, which is incorporated herein by reference in its entirety for all purposes. For example, JVET-D0119 proposes improving the intra-prediction process by introducing two methods: (1) using 6-tap (8-bit) cubic interpolation instead of 4-tap (6-bit) cubic interpolation as described above for performing the exemplary MDIS process in Figure 3; and (2) using the same 4-tap (6-bit) cubic interpolation (again, as described above for the exemplary MDIS process in Figure 3) to perform the exemplary reference line extension described with respect to Figure 4, instead of projecting the most recent neighbor pixel values.

[0131] As mentioned earlier, in some examples, larger block sizes can benefit from applying a higher degree of smoothing during intra-prediction. However, VVC uses a fixed degree of smoothing (e.g., 4-tap Gaussian interpolation or

[0121] filtering) for all block sizes, which, in light of the observations above, can result in inefficient or inefficient intra-prediction. Regarding JVET-D0119 discussed above, the use of 4-tap cubic interpolation to extend one or more lines of the reference pixels (e.g., the top and / or left reference pixel lines) may be problematic because, when intra-prediction is performed using the extended portion of the extended reference lines, its use can result in over-smoothing, thereby introducing inaccuracy and / or inefficiency into the entire intra-prediction process.

[0132] For example, oversmoothing may occur in such situations because the extended pixels of the extended reference line are subject to at least two distinct interpolation operations, each of which results in some degree of smoothing and edge degradation. The first interpolation operation is a 4-tap cubic interpolation to determine the extended upper / left reference pixel line value based on the most recent neighbor values ​​from the left / upper reference pixel. The interpolated reference pixel values ​​of the extended reference pixel line may then be involved in a second interpolation operation during intra-prediction for the current block, such as the interpolation operation described with respect to the MDIS process illustrated in Figure 3. For example, the interpolated reference pixel values ​​of the extended reference pixel line may be used in one or more of the 4-tap cubic interpolation, 4-tap Gaussian smoothing interpolation, and / or low-pass

[0121] reference pixel smoothing, each of which may result in oversmoothing throughout the intra-prediction process.

[0133] As previously mentioned, this specification describes systems and techniques for performing intra-prediction using one or more extended interpolation filters. The systems and techniques may be performed by an encoding device 104, a decoding device 112, both encoding device 104 and decoding device 112, and / or by other devices. The embodiments described herein may be applied independently and / or in combination. In some examples, the systems and techniques described herein may be used to perform one or more intra-prediction modes (for example, for filtering used during or in conjunction with the application of an intra-prediction mode).

[0134] In some examples, the systems and techniques described herein may provide a variable degree of reference pixel smoothing using block-level switching. For example, multiple smoothing filters and / or Gaussian interpolation filters (also called “Gaussian smoothing interpolation filters”), each having a different degree of smoothing, may be used to smooth the reference pixels during intra-prediction. In some cases, the selection of the determined smoothing filters and / or the determined interpolation filters may be explicitly signaled at different coding levels, for example, at the predictive block level, the coding block level, the CTU level, the slice level, and / or the sequence level (e.g., within an SPS). In some examples, the selection of the determined smoothing and / or interpolation filters may be implicitly determined using decoding information, which may include, but is not limited to, block size, predictive mode, QP, and / or CU level mode flags (MRL, ISP, etc.), in which case explicit signaling of filter selection is not required. For example, in some cases, the encoding device 104 and / or decoding device 112 may implicitly determine or select smoothing filters and / or interpolation filters to be used in intra-prediction based on determinations such as the block currently being coded having a certain size, a width and / or height greater than a threshold, or a width and / or height less than a threshold.

[0135] In one exemplary example, the processing of fractional angle (e.g., non-integer angle) intra-prediction modes could be extended from the method described in the VVC standard to include selecting from either a first Gaussian smoothing interpolation filter that applies a higher degree of smoothing or at least a second Gaussian smoothing interpolation filter that applies a lower degree of smoothing. As previously discussed with respect to Figure 3, the method used by the VVC standard utilizes the same 4-tap Gaussian smoothing interpolation filter for all fractional angle intra-prediction modes, regardless of the size of the block currently being coded.

[0136] Figure 5 is an exemplary diagram showing an example of a process 500 for performing switchable smoothing and / or interpolation to apply a variable degree of intra-predictive smoothing based on at least the intra-predictive mode of the current block and the size of the current block. In the context of the example discussed immediately above, the systems and techniques of the present disclosure for intra-prediction using extended interpolation filters may include selecting either a first filter with a 6-tap Gaussian smoothing filter or a second filter with a 4-tap Gaussian smoothing interpolation filter for a fractional-angle intra-predictive mode. The 6-tap Gaussian smoothing interpolation filter may apply a higher degree of smoothing than the 4-tap Gaussian smoothing interpolation filter. In some examples, the 4-tap Gaussian smoothing interpolation filter in Figure 5 may be the same as or similar to the 4-tap Gaussian smoothing interpolation filter described with respect to the exemplary VVC MDIS process 300 in Figure 3. In some examples, the filtering, interpolation, and / or smoothing degree selection processes may be implicit depending on the block size of the block being coded, as seen in Figure 5.

[0137] In some examples, the variable smoothing filtering and interpolation process for a reference pixel using block-level switching as shown in Figure 5 may be the same as or similar to the exemplary VVC MDIS process in Figure 3, except for operation 510 (e.g., comparing one or more of the width and height of the block currently being coded to at least a first threshold T) and subsequent operations 512 (e.g., selecting and applying a 6-tap Gaussian smoothing interpolation filter with a relatively high degree of smoothing in response to an exceedance of the first threshold T) and 514 (e.g., selecting and applying a 4-tap Gaussian smoothing interpolation filter with a relatively low degree of smoothing in response to not exceeding the first threshold T).

[0138] In operation 502, the process may determine whether the intra-prediction mode for the block currently being decoded is a horizontal intra-prediction mode (e.g., mode 18) or a vertical intra-prediction mode (e.g., mode 50). If the intra-prediction mode is either horizontal or vertical, the process decides not to perform reference pixel smoothing (referred to as “reference pixel smoothing” in Figure 5) and interpolation filtering in block 504, as previously described with respect to the exemplary MDIS process in Figure 3. The process then may proceed to processing the block currently being coded and perform intra-prediction without applying reference pixel smoothing or interpolation filtering.

[0139] In operation 506, the process may determine whether the minimum angular offset minDistVerHor is greater than the threshold intraHorVerDistThres[nTbS]. In some cases, one or more of minDistVerHor and / or intraHorVerDistThres[nTbS] may be the same as or similar to the corresponding variable values ​​discussed above for the exemplary MDIS process in Figure 3. In the exemplary example, the angular offset variable minDistVerHor may be set to equal Min(Abs(predModeIntra-50), Abs(predModeIntra-18)), where predModeIntra represents the intra-prediction mode number, 50 represents the vertical intra-prediction mode number, and 18 represents the horizontal intra-prediction mode number. In some cases, predModeIntra may be set to equal IntraPredModeY[xCb][yCb] or IntraPredModeC[xCb][yCb]. In some examples, the threshold variable intraHorVerDistThres[nTbS] may be given as specified in Table 2 below for different values ​​of the currently coded transformation block size nTbS.

[0140] [Table 2]

[0141] In some examples, if operation 506 determines that the angular offset minDistVerHor is less than or equal to the value of the threshold variable intraHorVerDistThres[nTbS] (for example, minDistVerHor ≤ intraHorVerDistThres[nTbs]), the process may, in operation 507, decide not to perform reference pixel smoothing and may further decide to apply a 4-tap cubic interpolation filter for intra-prediction of the block currently being coded. For example, the process may apply a 4-tap cubic filter to predict or interpolate one or more reference pixels without performing any reference pixel smoothing.

[0142] If operation 506 determines that the angular offset minDistVerHor is greater than the threshold intraHorVerDistThres[nTbS] (for example, minDistVerHor > intraHorVerDistThres[nTbS]), the process may then determine in operation 508 whether an integer angular mode exists in the intra-prediction mode of the block currently being coded, as previously described with respect to the exemplary MDIS process in Figure 3.

[0143] For example, when operation 508 determines that an integer angle mode exists in the intra-prediction mode of the block currently being coded, the process may decide in operation 509 to perform reference pixel smoothing using the

[0121] low-pass filter and not perform interpolation filtering. The process may then terminate in operation 509 after performing reference pixel smoothing to smooth the reference pixels using the

[0121] filter. No interpolation is performed, and the smoothed reference pixels are directly copied for intra-prediction of the block currently being coded.

[0144] In one example, when operation 508 determines that a fractional (e.g., non-integer) angle mode exists in the intra-prediction mode of the block currently being coded, the process may proceed to operation 510, which may determine whether the width of the block is greater than or equal to a threshold T and / or whether the height of the block is greater than or equal to a threshold T. In some examples, operation 510 may include determining which of the block width and / or block height is greater than or equal to a threshold T. In some examples, the value of the threshold T may be a predetermined value, for example, 16, 32, 64, or one or more other predetermined values.

[0145] If in operation 510 it is determined that the width and height of the block are greater than or equal to a threshold T (for example, height ≥ T && width ≥ T), the process may then decide in operation 512 not to perform reference pixel smoothing and terminate by applying a 6-tap Gaussian smoothing interpolation filter to the intra prediction of the block currently being coded. For example, the process may apply a 6-tap Gaussian smoothing interpolation filter to predict one or more pixels of the current block without any reference pixel smoothing.

[0146] If the width or height of the block is not greater than a threshold value T (e.g., height < T and / or width < T), the process may determine not to perform reference pixel smoothing in operation 514 and end by applying a 4-tap Gaussian smoothing interpolation filter. For example, the process may apply a 4-tap (6-bit) Gaussian smoothing interpolation filter to predict one or more pixels of the currently coded block without any reference pixel smoothing. As previously described, for example, operation 514 may be triggered in response to operation 510 determining that the currently coded block has a relatively small size. Thus, the 4-tap Gaussian smoothing interpolation filter of operation 514 may apply a lower degree of smoothing than the 6-tap Gaussian smoothing interpolation filter of operation 512. Similarly, recalling that the 6-tap Gaussian smoothing interpolation filter applies a higher degree of smoothing and that a larger block size can benefit more from a greater smoothing compared to a smaller block size, the 6-tap Gaussian smoothing interpolation filter of operation 512 may be triggered at least in part in response to operation 510 determining that the currently coded block has a relatively large block size.

[0147] In some cases, an exemplary 6-tap Gaussian smoothing interpolation filter applied in operation 514 may be derived using a convolution of one or more different phases of a [1 4 6 4 1] low-pass filter and a bilinear filter.

[0148] In one exemplary example, such as with respect to a scenario in which operation 508 determines that the intra-prediction mode of the block currently being coded is integer angle mode, operation 509 shown in Figure 5 may be extended to include a choice between a larger tap smoothing filter (e.g., a [1 4 6 4 1] low-pass filter, not shown) or a smaller

[0121] low-pass filter, which is currently shown as being applied in relation to operation 509. In some examples, the selection criteria for choosing between a larger tap [1 4 6 4 1] filter and a smaller tap

[0121] filter may be performed in the same or similar manner as the selection criteria implemented in operation 510. For example, one or more of the width and height of the block currently being coded may be compared to at least one threshold, and larger blocks (determined to be greater than or equal to the threshold) may have a larger tap [1 4 6 4 1] filter applied to the intra-prediction, and smaller blocks (determined to be less than the threshold) may have a smaller tap

[0121] filter applied to the intra-prediction. In some cases, one or more of the same or similar explicit and / or explicit selection processes described with respect to operation 510, which are based on factors such as block size, may be used in such examples where integer angle reference pixel smoothing of operation 509 is extended to select from different tap filters and / or degrees based on the currently coded block size.

[0149] In some examples, the systems and techniques described herein may perform weak filtering interpolation for reference line extension, avoiding or minimizing the oversmoothing problem discussed above, which may potentially occur when reference line extension is based on 4-tap cubic interpolation, and then undergoing further interpolation during intra-prediction. For example, instead of using 4-tap cubic filtering to interpolate values ​​for reference line extension pixels (e.g., interpolated based on the values ​​of the most recent neighboring pixels of a vertical pixel reference), weaker filter-based interpolation may be used to reduce or mitigate the possible oversmoothing problem that may occur in the context of extending reference lines. By utilizing weaker interpolation to determine the values ​​of reference line extension pixels, the rest of the intra-prediction process, and its associated interpolation and smoothing operations described herein, may remain in the same state without incurring the aforementioned oversmoothing problem.

[0150] In one exemplary example, 4-tap sync-based interpolation (e.g., with appropriate windowing) may be used to provide weak interpolation for calculating interpolation values ​​for reference line extension pixels. In some examples, 4-tap sync-based interpolation may be weaker than cubic interpolation, such as 4-tap cubic interpolation (e.g., with a higher cutoff frequency). In one exemplary example, weak interpolation for reference line extension pixels may be provided as a 6-bit 4-tap weak filter, an example of which is provided below (note that the coefficients at position (32-i) / 32 are mirror versions of i / 32): {0,64,0,0}, / / 0 / 32 position {-1,64,1,0}, / / 1 / 32 position {-3,65,3,-1}, / / 2 / 32 position {-3,63,5,-1}, / / 3 / 32 position {-4,63,6,-1}, / / 4 / 32 position {-5,62,9,-2}, / / 5 / 32 position {-5,60,11,-2}, / / 6 / 32 position {-5,58,13,-2}, / / 7 / 32 position {-6,57,16,-3}, / / position 8 / 32 {-6,55,18,-3}, / / position 9 / 32 {-7,54,21,-4}, / / position 10 / 32 {-7,52,23,-4}, / / position 11 / 32 {-6,48,26,-4}, / / position 12 / 32 {-7,47,29,-5}, / / position 13 / 32 {-6,43,32,-5}, / / position 14 / 32 {-6,41,34,-5}, / / position 15 / 32 {-5,37,37,-5}, / / position 16 / 32

[0151] These systems and techniques enable predictions (e.g., intra-predictions) to be performed using extended interpolation filters. The systems and techniques described herein may, in some examples, offer advantages over other techniques that utilize multiple interpolation filters. For example, in some cases, multiple interpolation filters with different interpolation filter taps may be applied within a single block, slice, tile, and / or picture. In one example, the interpolation filter type and interpolation filter tap (length) may depend on neighbor decoding information, including the block height and / or width, block shape (width-to-height ratio), block area size, intra-prediction mode, and / or, but not limited to, reconstructed sample values ​​and intra-prediction mode. In such cases, when the intra-prediction is a vertical angular intra-prediction mode and the width is 8 or less or of other sizes, a 6-tap 6th-order interpolation filter is used, and in some cases, a 4-tap Gaussian interpolation filter is used. When the intra-prediction is a horizontal intra-prediction mode and the width is 8 or less or of other sizes, a 6-tap 6th-order interpolation filter is used, and in some cases, a 4-tap Gaussian interpolation filter is used. In one example using the systems and techniques described herein, a 6-tap Gaussian filter is used (and pixel smoothing is not applied) when the width and height of the coding block are greater than or equal to a threshold T, and in some cases, a 4-tap Gaussian filter is used (and pixel smoothing is not applied).

[0152] Figure 6 is a flowchart showing an example of a process 600 for processing image and / or video data. In block 602, the process 600 may include the step of determining an intra-prediction mode for predicting blocks of video data.

[0153] In block 604, process 600 may include the step of determining the type of smoothing filter to use for blocks of video data. For example, process 600 may determine the type of smoothing filter at least in part on comparing at least one of the width and height of the video data blocks to a first threshold. In some embodiments, the type of smoothing filter is signaled in the video bitstream. In some cases, the type of smoothing filter is signaled for individual sets of prediction blocks, coding blocks, coding tree units (CTUs), slices, or sets of sequences. In block 606, process 600 may include the step of performing intra-prediction on blocks of video data using the determined type of smoothing filter and an intra-prediction mode.

[0154] In some examples, process 600 may include the step of using a first smoothing interpolation filter as the determined type of smoothing filter, at least in part on the block width, the block height, or the determination that the block width and height are greater than a first threshold. In one exemplary example, the first smoothing interpolation filter includes a 6-tap Gaussian filter. In such an example, process 600 may further include the step of using the first smoothing interpolation filter to determine a reference pixel for intra-prediction of a block of video data.

[0155] In some examples, process 600 may include a step of using a second smoothing interpolation filter as the determined type of smoothing filter, at least in part on the determination that the width of a block, the height of a block, or the width and height of a block are less than or equal to a first threshold (e.g., less than). In one exemplary example, the second smoothing interpolation filter includes a 4-tap Gaussian filter. In such an example, process 600 may further include using the second smoothing interpolation filter to determine a reference pixel for intra-prediction of a block of video data.

[0156] In some cases, process 600 may include the step of determining the minimum offset between the angular direction of the intra-prediction mode and one of the vertical intra-prediction mode and the horizontal intra-prediction mode. Process 600 may further include the step of determining the type of smoothing filter to use for a block of video data based on comparing the determined minimum offset with a second threshold. In one example, process 600 may include the step of determining a low-pass filter as the type of smoothing filter based at least in part on the determination that the determined minimum offset is greater than the second threshold and the determination that the intra-prediction mode is an integer angular mode associated with an integer reference pixel position. In one exemplary example, the low-pass filter includes a

[0121] filter that performs reference pixel smoothing without interpolation.

[0157] In another example, process 600 may include the step of determining a Gaussian filter as the type of smoothing filter, at least in part on the determination that the determined minimum offset is greater than a second threshold and the determination that the intra-prediction mode is a fractional angle mode associated with a fractional reference pixel position. In some cases, the Gaussian filter performs smoothing interpolation without reference pixel smoothing. In one exemplary example, the Gaussian filter includes a 6-tap Gaussian filter based on the determination that at least one of the block width and block height is greater than a first threshold. In another exemplary example, the Gaussian filter includes a 4-tap Gaussian filter based on the determination that at least one of the block width and block height is less than or equal to a first threshold.

[0158] In some embodiments, process 600 may include a step of using an interpolation filter as the determined type of smoothing filter, at least in part on the determination that the determined minimum offset is less than or equal to a second threshold (e.g., less than that). In one exemplary example, the interpolation filter includes a 4-tap cubic filter. Process 600 may further include a step of performing an intra-prediction for blocks of video data using the interpolation filter without applying reference pixel smoothing.

[0159] In some examples, process 600 may include the step of determining a low-pass filter as the type of smoothing filter, at least partially based on the determination that the intra-prediction mode is integer angle mode and the determination that the determined minimum offset is greater than a second threshold. In some cases, process 600 may include the step of performing reference pixel smoothing using a large-tap low-pass filter, at least partially based on the determination that the block width, block height, or block width and height are greater than a first threshold. The large-tap low-pass filter applies a higher degree of reference pixel smoothing than the small-tap low-pass filter. In some cases, process 600 may include the step of performing reference pixel smoothing using a small-tap low-pass filter, at least partially based on the determination that the block width, block height, or block width and height are less than or equal to a first threshold (e.g., less than). The small-tap low-pass filter applies a lower degree of reference pixel smoothing than the large-tap low-pass filter.

[0160] In some cases, the process 600 may include the step of determining that the intra-prediction mode is an integer angle mode, at least in part on comparing the tilt of the intra-prediction mode with one or more pixel positions determined from the width and height of the block.

[0161] In some embodiments, process 600 may include the step of determining that the offset between the angular direction of the intra-prediction mode and the vertical intra-prediction mode or the horizontal intra-prediction mode is less than a second threshold. Based on the determination that the offset between the angular direction of the intra-prediction mode and the vertical intra-prediction mode or the horizontal intra-prediction mode is less than the second threshold, process 600 may further include the step of performing intra-prediction on blocks of video data using a cubic interpolation filter.

[0162] In some examples, process 600 may include a step of performing reference line extension using a weak interpolation filter. In some cases, the weak interpolation filter is used to perform reference line extension prior to the use of a cubic interpolation filter for performing intra prediction. In some cases, the cubic interpolation filter has a higher cutoff frequency than the weak interpolation filter and applies a higher degree of smoothing than the weak interpolation filter. In some embodiments, the weak interpolation filter includes a 4-tap sync-based interpolation filter and a 6-bit 4-tap interpolation filter.

[0163] In some embodiments, the process 600 may include the step of determining the type of smoothing filter based on the width of a block, the height of a block, or the width and height of a block, without using information explicitly signaled in the video bitstream.

[0164] In some cases, process 600 may be performed by a decoding device (for example, decoding device 112 in Figures 1 and 8). For example, process 600 may further include the step of determining blocks of residual data with respect to blocks of video data. Process 600 may also include the step of decoding blocks of video data using blocks of residual data and predicted blocks determined based on performing intra-predictions on blocks of video data.

[0165] In some cases, process 600 may be performed by an encoding device (for example, encoding device 104 in Figures 1 and 7). For example, process 600 may include the step of generating an encoded video bitstream containing information related to blocks of video data. In some examples, process 600 may include the step of storing the encoded video bitstream (for example, in at least one memory of the device). In some examples, process 600 may include the step of transmitting the encoded video bitstream (for example, using a transmitter of the device).

[0166] In some implementations, the process (or method) described herein may be performed by a computing device or apparatus, such as system 100 shown in Figure 1. For example, the process may be performed by an encoding device 104 shown in Figures 1 and 8, by another video source-side device or video transmission device, by a decoding device 112 shown in Figures 1 and 9, and / or by another client-side device, such as a player device, display, or any other client-side device. In some cases, the computing device or apparatus may include a processor, microprocessor, microcomputer, or other components of a device configured to perform the steps of the process described herein. In some examples, the computing device or apparatus may include a camera configured to capture video data (e.g., a video sequence) containing video frames. In some examples, the camera or other capture device that captures the video data is separate from the computing device, in which case the computing device receives or acquires the captured video data. The computing device may further include a network interface configured to communicate the video data. The network interface may be configured to communicate Internet Protocol (IP) based data or other types of data. In some examples, a computing device or apparatus may include a display for showing output video content, such as a sample picture of a video bitstream.

[0167] A process is described in relation to a logical flow diagram, and its operation represents a sequence of actions that can be implemented by hardware, computer instructions, or a combination thereof. In the context of computer instructions, an action represents a computer-executable instruction stored in one or more computer-readable storage media that, when executed by one or more processors, performs the described action. Generally, computer-executable instructions include routines, programs, objects, components, data structures, etc., that perform a particular function or implement a particular data type. The order in which actions are described is not intended to be interpreted as restrictive, and any number of actions described may be combined in any order and / or in parallel to implement a process.

[0168] In addition, a process can be executed under the control of one or more computer systems consisting of executable instructions, and can be implemented as code (e.g., executable instructions, one or more computer programs, or one or more applications) that is executed together by hardware or a combination thereof on one or more processors. As described above, the code can be stored in a computer-readable or machine-readable storage medium, for example, in the form of a computer program comprising multiple instructions executable by one or more processors. The computer-readable or machine-readable storage medium may be non-temporary.

[0169] The coding techniques described herein may be implemented in exemplary video coding and decoding systems (e.g., System 100). In some examples, the system includes a source device that provides coded video data to be later decoded by a destination device. Specifically, the source device provides the video data to the destination device via a computer-readable medium. The source and destination devices may comprise any of a wide range of devices, including desktop computers, notebook (i.e., laptop) computers, tablet computers, set-top boxes, telephone handsets such as so-called "smart" phones, so-called "smart" pads, televisions, cameras, display devices, digital media players, video gaming consoles, video streaming devices, and the like. In some cases, the source and destination devices may be equipped for wireless communication.

[0170] The destination device may receive encoded video data to be decoded via a computer-readable medium. The computer-readable medium may comprise any type of medium or device capable of moving encoded video data from the source device to the destination device. For example, the computer-readable medium may comprise a communication medium that enables the source device to transmit encoded video data directly to the destination device in real time. The encoded video data may be modulated according to a communication standard, such as a wireless communication protocol, and transmitted to the destination device. The communication medium may comprise any wireless or wired communication medium, such as a radio frequency (RF) spectrum or one or more physical transmission lines. The communication medium may form part of a packet-based network, such as a local area network, a wide area network, or a global network such as the Internet. The communication medium may include routers, switches, base stations, or any other equipment that may be useful in facilitating communication from the source device to the destination device.

[0171] In some examples, encoded data may be output to a storage device via an output interface. Similarly, encoded data may be accessed from the storage device via an input interface. The storage device may include any of a variety of distributed or locally accessed data storage media, such as a hard drive, Blu-ray disc, DVD, CD-ROM, flash memory, volatile or non-volatile memory, or any other suitable digital storage medium for storing encoded video data. In further examples, the storage device may correspond to a file server or another intermediate storage device capable of storing encoded video generated by the source device. The destination device may access the stored video data from the storage device via streaming or download. The file server may be any type of server capable of storing encoded video data and sending that encoded video data to the destination device. Exemplary file servers include web servers (for example, for websites), FTP servers, network-attached storage (NAS) devices, or local disk drives. The destination device may access the encoded video data through any standard data connection, including an internet connection. This may include a wireless channel (e.g., Wi-Fi connection), a wired connection (e.g., DSL, cable modem, etc.), or a combination of both, suitable for accessing encoded video data stored on a file server. Transmission of encoded video data from the storage device may be via streaming, download, or a combination thereof.

[0172] The techniques of this disclosure are not necessarily limited to wireless applications or configurations. The techniques may be applied to video coding supporting any of a variety of multimedia applications, such as over-the-air television broadcasting, cable television transmission, satellite television transmission, internet streaming video transmission such as Dynamic Adaptive Streaming over HTTP (DASH), digital video encoded on a data storage medium, decoding of digital video stored on a data storage medium, or other applications. In some examples, the system may be configured to support one-way or two-way video transmission to support applications such as video streaming, video playback, video broadcasting, and / or video telephony.

[0173] In one example, the source device includes a video source, a video encoder, and an output interface. The destination device may include an input interface, a video decoder, and a display device. The video encoder of the source device may be configured to apply the techniques disclosed herein. In other examples, the source and destination devices may include other components or arrangements. For example, the source device may receive video data from an external video source, such as an external camera. Similarly, the destination device may interface with an external display device rather than including an integrated display device.

[0174] The exemplary system described above is merely an example. Techniques for processing video data in parallel can be performed by any digital video encoding and / or decoding device. Generally, the techniques of this disclosure are performed by video encoding devices, but the techniques can also be performed by video encoders / decoders, commonly referred to as “codecs.” Furthermore, the techniques of this disclosure can also be performed by video preprocessors. Source and destination devices are merely examples of such encoding devices, where the source device generates encoded video data for transmission to the destination device. In some examples, the source and destination devices may operate substantially symmetrically, such that each device contains video encoding and decoding components. Thus, the exemplary system may support one-way or two-way video transmission between video devices for, for example, video streaming, video playback, video broadcasting, or video telephony.

[0175] A video source may include a video capture device such as a video camera, a video archive containing previously captured video, and / or a video feed interface for receiving video from a video content provider. Alternatively, the video source may generate computer graphics-based data as source video, or a combination of live video, archived video, and computer-generated video. In some cases, if the video source is a video camera, the source and destination devices may form a so-called cameraphone or videophone. However, as stated above, the techniques described in this disclosure may generally be applicable to video coding and may be applicable to wireless and / or wired applications. In each case, captured, pre-captured, or computer-generated video may be encoded by a video encoder. The encoded video information may then be output onto a computer-readable medium by an output interface.

[0176] As stated above, computer-readable media may include temporary media such as wireless broadcasting or wired network transmission, or storage media (i.e., non-temporary storage media) such as hard disks, flash drives, compact discs, digital video discs, and Blu-ray discs, or other computer-readable media. In some examples, a network server (not shown) may, for example, receive encoded video data from a source device via network transmission and provide the encoded video data to a destination device. Similarly, a computing device in a media manufacturing facility, such as a disc stamping facility, may receive encoded video data from a source device and manufacture a disc containing the encoded video data. Thus, computer-readable media may be understood to include one or more computer-readable media in various forms in various examples.

[0177] The input interface of the destination device receives information from a computer-readable medium. The information on the computer-readable medium may include syntax information defined by a video encoder, which includes syntax elements describing the characteristics and / or processing of blocks and other coded units, such as picture groups (GOPs), and this syntax information is also used by a video decoder. A display device displays the decoded video data to the user and may comprise any of various display devices, such as a cathode ray tube (CRT), liquid crystal display (LCD), plasma display, organic light-emitting diode (OLED) display, or another type of display device. Various embodiments of this application have been described.

[0178] Specific details of the encoding device 104 and the decoding device 112 are shown in Figures 8 and 9, respectively. Figure 8 is a block diagram showing an exemplary encoding device 104 that may implement one or more of the techniques described herein. The encoding device 104 may, for example, generate the syntax structures described herein (e.g., VPS, SPS, PPS, or syntax structures of other syntax elements). The encoding device 104 may perform intra-predictive and inter-predictive coding of video blocks within a video slice. As previously described, intra-coding reduces or eliminates spatial redundancy within a given video frame or picture by relying at least partially on spatial prediction. Intercoding relies at least partially on temporal prediction to reduce or eliminate temporal redundancy within adjacent or surrounding frames of a video sequence. Intra-mode (I-mode) may refer to any of several spatial-based compression modes. Inter-mode, such as unidirectional prediction (P-mode) or bidirectional prediction (B-mode), may refer to any of several temporal-based compression modes.

[0179] The encoding device 104 includes a partitioning unit 35, a prediction processing unit 41, a filter unit 63, a picture memory 64, an adder 50, a transformation processing unit 52, a quantization unit 54, and an entropy encoding unit 56. The prediction processing unit 41 includes a motion estimation unit 42, a motion compensation unit 44, and an intra-prediction processing unit 46. For video block reconstruction, the encoding device 104 also includes an inverse quantization unit 58, an inverse transformation processing unit 60, and an adder 62. The filter unit 63 is intended to represent one or more loop filters, such as a deblocking filter, an adaptive loop filter (ALF), and a sample-adaptive offset (SAO) filter. Although the filter unit 63 is shown as an in-loop filter in Figure 8, in other configurations, the filter unit 63 may be implemented as a post-loop filter. The post-processing device 57 may perform additional processing on the encoded video data generated by the encoding device 104. The techniques of this disclosure may be implemented by the encoding device 104 in some cases. However, in other cases, one or more of the techniques of this disclosure may be implemented by the post-processing device 57.

[0180] As shown in Figure 8, the encoding device 104 receives video data, and the partitioning unit 35 partitions the data into video blocks. Partitioning can also include partitioning into slices, slice segments, tiles, or other larger units, as well as video block partitioning by a quadtree structure of LCUs and CUs, for example. The encoding device 104 generally represents the components that encode the video blocks within the video slice to be encoded. A slice can be divided into multiple video blocks (and possibly into sets of video blocks called tiles). The prediction processing unit 41 may select one of several possible coding modes for the current video block, such as one of several intra-predictive coding modes or one of several inter-predictive coding modes, based on the error results (e.g., coding rate and distortion level). The prediction processing unit 41 may provide the resulting intra- or inter-coded blocks to the adder 50 to generate residual block data and to the adder 62 to reconstruct the encoded blocks for use as reference pictures.

[0181] An intra-prediction processing unit 46 within the prediction processing unit 41 may perform intra-prediction coding of the current video block for one or more adjacent blocks in the same frame or slice as the current block to be coded, in order to perform spatial compression. A motion estimation unit 42 and a motion compensation unit 44 within the prediction processing unit 41 perform inter-prediction coding of the current video block for one or more predicted blocks in one or more reference pictures, in order to perform temporal compression.

[0182] The motion estimation unit 42 may be configured to determine the inter-prediction mode for video slices according to a predetermined pattern for a video sequence. The predetermined pattern may designate video slices in the sequence as P slices, B slices, or GPB slices. The motion estimation unit 42 and the motion compensation unit 44 may be highly integrated, but are shown separately for conceptual purposes. The motion estimation performed by the motion estimation unit 42 is the process of generating motion vectors that estimate the motion of video blocks. The motion vectors may, for example, represent the displacement of the prediction unit (PU) of the current video frame or video block in the picture relative to the prediction block in the reference picture.

[0183] A prediction block is a block that has been found to closely match the PU of the video block to be coded with respect to pixel differences that can be determined by absolute difference sum (SAD), squared difference sum (SSD), or other difference metrics. In some examples, the encoding device 104 may compute values ​​for sub-integer pixel positions of a reference picture stored in picture memory 64. For example, the encoding device 104 may interpolate values ​​for quarter-pixel, eighth-pixel, or other fractional pixel positions of the reference picture. Thus, the motion estimation unit 42 can perform motion searches for full pixel positions and fractional pixel positions and output motion vectors with fractional pixel positions.

[0184] The motion estimation unit 42 calculates a motion vector for the PU of a video block in an intercoded slice by comparing the PU's position with the predicted block's position in a reference picture. The reference picture can be selected from a first reference picture list (list 0) or a second reference picture list (list 1), each of which identifies one or more reference pictures stored in the picture memory 64. The motion estimation unit 42 sends the calculated motion vector to the entropy coding unit 56 and the motion compensation unit 44.

[0185] Motion compensation performed by the motion compensation unit 44 may, in some cases, involve fetching or generating predicted blocks based on motion vectors determined by motion estimation, which perform interpolation to sub-pixel precision. Upon receiving the motion vector of the current video block from the PU, the motion compensation unit 44 may locate the position of the predicted block pointed to by the motion vector in the reference picture list. The encoding device 104 forms the residual video block by subtracting the pixel values ​​of the predicted block from the pixel values ​​of the current video block being coded to form a pixel difference value. The pixel difference value forms the residual data for the block and may include both lumern difference and chromern difference components. The adder 50 represents one or more components that perform this subtraction operation. The motion compensation unit 44 may also generate syntax elements associated with the video block and video slice for use by the decoding device 112 when decoding the video block of the video slice.

[0186] The intra-prediction processing unit 46 may intra-predict the current block as an alternative to the inter-prediction performed by the motion estimation unit 42 and the motion compensation unit 44, as described above. Specifically, the intra-prediction processing unit 46 may determine which intra-prediction mode to use to encode the current block. In some examples, the intra-prediction processing unit 46 may encode the current block using various intra-prediction modes, for example, between separate encoding passes, and the intra-prediction processing unit 46 may select the appropriate intra-prediction mode to use from the tested modes. For example, the intra-prediction processing unit 46 may use rate-distortion analysis to calculate rate-distortion values ​​for various tested intra-prediction modes and select the intra-prediction mode with the best rate-distortion characteristics from the tested modes. Rate-distortion analysis generally determines the amount of distortion (or error) between the encoded block and the original unencoded block encoded to generate the encoded block, as well as the bit rate (i.e., number of bits) used to generate the encoded block. The intra-prediction processing unit 46 can calculate a ratio from the strain and rate for various coded blocks in order to determine which intra-prediction mode shows the best rate strain value for the block.

[0187] In either case, after selecting an intra-prediction mode for a block, the intra-prediction processing unit 46 may provide the entropy coding unit 56 with information indicating the selected intra-prediction mode for the block. The entropy coding unit 56 may encode the information indicating the selected intra-prediction mode. The coding device 104 may include in the transmitted bitstream configuration data definitions for various blocks, as well as instructions for the most probable intra-prediction mode to be used for each context, an intra-prediction mode index table, and a modified intra-prediction mode index table. The bitstream configuration data may include multiple intra-prediction mode index tables and multiple modified intra-prediction mode index tables (also called codeword mapping tables).

[0188] After the prediction processing unit 41 generates a prediction block for the current video block via either interpretation or intraprediction, the encoding device 104 forms a residual video block by subtracting the prediction block from the current video block. The residual video data in the residual block is included in one or more TUs and can be applied to the transformation processing unit 52. The transformation processing unit 52 converts the residual video data into residual transformation coefficients using a transformation such as a discrete cosine transform (DCT) or a conceptually similar transformation. The transformation processing unit 52 can convert the residual video data from the pixel domain to a transformation domain such as the frequency domain.

[0189] The conversion processing unit 52 may send the obtained conversion coefficients to the quantization unit 54. The quantization unit 54 quantizes the conversion coefficients to further reduce the bit rate. The quantization process may reduce the bit depth associated with some or all of the coefficients. The degree of quantization can be modified by adjusting the quantization parameters. In some examples, the quantization unit 54 may then perform a scan of the matrix containing the quantized conversion coefficients. Alternatively, the entropy coding unit 56 may perform the scan.

[0190] Following quantization, the entropy coding unit 56 entropy-codes the quantization transformation coefficients. For example, the entropy coding unit 56 can perform context-adaptive variable-length coding (CAVLC), context-adaptive binary arithmetic coding (CABAC), syntax-based context-adaptive binary arithmetic coding (SBAC), probability interval piece entropy (PIPE) coding, or another entropy coding technique. Following entropy coding by the entropy coding unit 56, the coded bitstream may be transmitted to the decoding device 112 or archived for later transmission or retrieval by the decoding device 112. The entropy coding unit 56 may also entropy-code the motion vector and other syntax elements of the currently coded video slice.

[0191] The inverse quantization unit 58 and the inverse transformation processing unit 60 apply inverse quantization and inverse transformation, respectively, to reconstruct residual blocks in the pixel region for later use as reference blocks of the reference picture. The motion compensation unit 44 may compute a reference block by adding the residual block to a predicted block of one of the reference pictures in the reference picture list. The motion compensation unit 44 may also apply one or more interpolation filters to the reconstructed residual block to compute sub-integer pixel values ​​for use in motion estimation. The adder 62 adds the reconstructed residual block to the motion-compensated predicted block generated by the motion compensation unit 44 to generate a reference block for storage in the picture memory 64. The reference block may be used by the motion estimation unit 42 and the motion compensation unit 44 as a reference block for interpreting blocks in subsequent video frames or pictures.

[0192] Thus, the encoding device 104 in Figure 8 represents an example of a video encoder configured to perform the techniques described herein. For example, the encoding device 104 can perform any of the techniques described herein, including the processes described herein. In some cases, some of the techniques of this disclosure may also be implemented by a post-processing device 57.

[0193] Figure 9 is a block diagram showing an exemplary decoding device 112. The decoding device 112 includes an entropy decoding unit 80, a prediction processing unit 81, an inverse quantization unit 86, an inverse transform processing unit 88, an adder 90, a filter unit 91, and a picture memory 92. The prediction processing unit 81 includes a motion compensation unit 82 and an intra-prediction processing unit 84. In some examples, the decoding device 112 may perform a decoding path that is generally the reverse of the encoding path described with respect to the encoding device 104 from Figure 8.

[0194] During the decoding process, the decoding device 112 receives an encoded video bitstream representing video blocks and associated syntax elements of an encoded video slice sent by the encoding device 104. In some embodiments, the decoding device 112 may receive the encoded video bitstream from the encoding device 104. In some embodiments, the decoding device 112 may receive the encoded video bitstream from a network entity 79, such as a server, a media-aware network element (MANE), a video editor / splitter, or another such device configured to implement one or more of the techniques described above. The network entity 79 may or may not include the encoding device 104. Some of the techniques described in this disclosure may be implemented by the network entity 79 before it sends the encoded video bitstream to the decoding device 112. In some video decoding systems, the network entity 79 and the decoding device 112 may be part of a separate device, but in other cases, the functions described with respect to the network entity 79 may be performed by the same device comprising the decoding device 112.

[0195] The entropy decoding unit 80 of the decoding device 112 entropy-decodes the bitstream to generate quantized coefficients, motion vectors, and other syntax elements. The entropy decoding unit 80 transfers the motion vectors and other syntax elements to the prediction processing unit 81. The decoding device 112 may receive syntax elements at the video slice level and / or video block level. The entropy decoding unit 80 may process and parse both fixed-length and variable-length syntax elements in one or more parameter sets such as VPS, SPS, and PPS.

[0196] When a video slice is coded as an intra-coded (I) slice, the intra-prediction processing unit 84 of the prediction processing unit 81 may generate prediction data for the video block of the current video slice based on the signaled intra-prediction mode and data from previously decoded blocks of the current frame or picture. When a video frame is coded as an intercoded (i.e., B, P, or GPB) slice, the motion compensation unit 82 of the prediction processing unit 81 generates prediction blocks for the video block of the current video slice based on motion vectors and other syntax elements received from the entropy decoding unit 80. Prediction blocks may be generated from one of the reference pictures in the reference picture list. The decoding device 112 may construct reference frame lists, i.e., list 0 and list 1, using default construction techniques based on the reference pictures stored in picture memory 92.

[0197] The motion compensation unit 82 determines prediction information about the video blocks of the current video slice by parsing motion vectors and other syntax elements, and uses the prediction information to generate prediction blocks for the currently decoded video blocks. For example, the motion compensation unit 82 may use one or more syntax elements in the parameter set to determine the prediction mode used to code the video blocks of the video slice (e.g., intra-prediction or inter-prediction), the inter-prediction slice type (e.g., B-slice, P-slice, or GPB-slice), construction information about one or more reference picture lists for the slice, motion vectors for each intercoded video block of the slice, the inter-prediction status for each intercoded video block of the slice, and other information for decoding the video blocks in the current video slice.

[0198] The motion compensation unit 82 can also perform interpolation based on an interpolation filter. The motion compensation unit 82 may calculate interpolated values ​​for sub-integer pixels of a reference block using an interpolation filter, such as the one used by the encoding device 104 during the encoding of the video block. In this case, the motion compensation unit 82 may determine the interpolation filter used by the encoding device 104 from the received syntax elements and use that interpolation filter to generate the predicted block.

[0199] The inverse quantization unit 86 inversely quantizes or dequantizes the quantization transformation coefficients provided in the bitstream and decoded by the entropy decoding unit 80. The inverse quantization process may include determining the degree of quantization using quantization parameters calculated by the encoding device 104 for each video block in the video slice, and similarly determining the degree of inverse quantization to be applied. The inverse transformation processing unit 88 applies an inverse transformation (e.g., inverse DCT or other preferred inverse transformation), an inverse integer transformation, or a conceptually similar inverse transformation process to the transformation coefficients to generate residual blocks in the pixel region.

[0200] After the motion compensation unit 82 generates predicted blocks for the current video block based on motion vectors and other syntax elements, the decoding device 112 forms the decoded video block by adding the residual block from the inverse processing unit 88 with the corresponding predicted block generated by the motion compensation unit 82. The adder 90 represents one or more components that perform this addition operation. Loop filters (either in or after the coding loop), if desired, may also be used to smooth pixel transitions or to improve video quality in other ways. The filter unit 91 is intended to represent one or more loop filters, such as a deblocking filter, an adaptive loop filter (ALF), and a sample-adaptive offset (SAO) filter. The filter unit 91 is shown in Figure 9 as an in-loop filter, but in other configurations, the filter unit 91 may be implemented as a post-loop filter. The decoded video block in a given frame or picture is then stored in the picture memory 92, which stores a reference picture used for subsequent motion compensation. The picture memory 92 also stores the decoded video so that it can be later presented on a display device such as the video destination device 122 shown in Figure 1.

[0201] Thus, the decoding device 112 in Figure 9 represents an example of a video decoder configured to perform the techniques described herein. For example, the decoding device 112 can perform any of the techniques described herein, including the processes described herein.

[0202] As used herein, the term “computer-readable medium” includes, but is not limited to, portable or non-portable storage devices, optical storage devices, and various other media capable of storing, containing, or transporting instructions and / or data. Computer-readable medium may include non-transient media capable of storing data and not containing carrier waves and / or transient electronic signals that propagate wirelessly or via wired connections. Examples of non-transient media include, but are not limited to, magnetic disks or tapes, optical storage media such as compact discs (CDs) or digital multipurpose discs (DVDs), flash memory, memory, or memory devices. Computer-readable medium may store code and / or machine-executable instructions that may represent procedures, functions, subprograms, programs, routines, subroutines, modules, software packages, classes, or any combination of instructions, data structures, or program statements. Code segments may be coupled to other code segments or hardware circuits by passing and / or receiving information, data, arguments, parameters, or memory contents. Information, arguments, parameters, data, etc., may be passed, transferred, or transmitted via any appropriate means, including memory sharing, message passing, token passing, network transmission, etc.

[0203] In some embodiments, computer-readable storage devices, media, and memory may include cables or wireless signals, such as bitstreams. However, non-temporary computer-readable storage media, as referred to, explicitly exclude media such as energy, carrier signals, electromagnetic waves, and signals themselves.

[0204] Specific details are provided in the above description to give a complete understanding of the embodiments and examples provided herein. However, it will be understood by those skilled in the art that embodiments can be practiced without these specific details. For clarity of description, in some cases the art may be presented as including individual functional blocks comprising a device, device components, and software, or steps or routines in a manner embodied in a combination of hardware and software. Additional components other than those shown in the figures and / or described herein may be used. For example, circuits, systems, networks, processes, or other components may be shown as components in the form of block diagrams so as not to obscure the embodiments with unnecessary details. In other cases, well-known circuits, processes, algorithms, structures, and techniques may be shown without unnecessary details so as not to obscure the embodiments.

[0205] Individual embodiments may be described above as processes or methods represented as flowcharts, flow diagrams, data flow diagrams, structural diagrams, or block diagrams. While flowcharts may describe operations as sequential processes, many operations can be performed in parallel or simultaneously. In addition, the order of operations may be rearranged. A process terminates when its operation is complete, but it may have additional steps not shown in the diagram. A process may correspond to a method, function, procedure, subroutine, subprogram, etc. When a process corresponds to a function, its termination may correspond to the function returning to a calling function or main function.

[0206] The processes and methods described above may be implemented using computer-executable instructions stored or otherwise available from a computer-readable medium. Such instructions may include instructions and data that cause a general-purpose computer, a dedicated computer, or a processing device to perform a particular function or group of functions, or to otherwise configure a general-purpose computer, a dedicated computer, or a processing device to perform a particular function or group of functions. The portion of computer resources used may be accessible over a network. Computer-executable instructions may be binary or intermediate format instructions, such as assembly language, firmware, or source code. Examples of computer-readable media that may be used to store instructions, information used, and / or information created during the methods described above include magnetic or optical disks, flash memory, USB devices with non-volatile memory, and network-connected storage devices.

[0207] Devices implementing the processes and methods described herein may include hardware, software, firmware, middleware, microcode, hardware description languages, or any combination thereof, and may take any of a variety of form factors. When implemented in software, firmware, middleware, or microcode, program code or code segments (e.g., computer program products) for performing the required tasks may be stored in computer-readable or machine-readable media. A processor may perform the required tasks. Typical examples of form factors include laptops, smartphones, mobile phones, tablet devices or other small form factor personal computers, personal digital assistants, rack-mount devices, and standalone devices. The functions described herein may also be embodied in peripheral devices or add-in cards. Such functions may, as a further example, be implemented on different processes running on circuit boards in different chips or in a single device.

[0208] Instructions, a medium for transmitting such instructions, computing resources for executing the instructions, and other structures for supporting such computing resources are exemplary means for providing the functionality described herein.

[0209] While the above description illustrates aspects of this application with respect to specific embodiments, those skilled in the art will recognize that this application is not limited thereto. Although exemplary embodiments of this application have been described in detail herein, it should be understood that the concepts of the present invention can be embodied and employed in various other ways, and that, unless limited by the prior art, the appended claims are intended to be interpreted as including such variations. The various features and aspects of the applications described above can be used individually or in combination. Furthermore, embodiments can be used in any number of environments and applications other than those described herein without departing from the broader spirit and scope of this specification. Thus, this specification and the drawings should be considered illustrative rather than restrictive. For illustrative purposes, the methods have been described in a particular order. It should be understood that in alternative embodiments, the methods may be performed in a different order than described.

[0210] Those skilled in the art will understand that the symbols or terms less than ("<") and greater than (">") used herein may be replaced by the symbols less than or equal to ("≦") and greater than or equal to ("≧"), respectively, without departing from the scope of this description.

[0211] When components are described as “configured to” perform certain operations, such configurations can be achieved, for example, by designing electronic circuits or other hardware to perform the operations, by programming programmable electronic circuits (e.g., a microprocessor or other suitable electronic circuit) to perform the operations, or by any combination thereof.

[0212] The phrase "combined" refers to any component that is physically connected to another component, either directly or indirectly, and / or any component that communicates with another component, either directly or indirectly (for example, connected to another component via a wired or wireless connection and / or other preferred communication interface).

[0213] The wording of a claim that includes “at least one of” a set and / or “one or more” of a set indicates that one element of a set or multiple elements of a set (in any combination) satisfy the claim. For example, the wording of a claim that includes “at least one of A and B” means A, B, or A and B. In another example, the wording of a claim that includes “at least one of A, B, and C” means A, B, C, or A and B, or A and C, or B and C, or A and B and C. The wording “at least one of” a set and / or “one or more” of a set does not limit the set to items enumerated in the set. For example, the wording of a claim that includes “at least one of A and B” could mean A, B, or A and B, and could also include items not enumerated in the set A and B.

[0214] The various exemplary logic blocks, modules, circuits, and algorithmic steps described in relation to the embodiments disclosed herein may be implemented as electronic hardware, computer software, firmware, or a combination thereof. To clearly demonstrate this hardware- and software compatibility, various exemplary components, blocks, modules, circuits, and steps have been described above in general terms with respect to their function. Whether such functions are implemented as hardware or as software depends on the specific application and the design constraints imposed on the overall system. Those skilled in the art may implement the described functions in various ways for each specific application, but such implementation decisions should not be construed as causing a departure from the scope of this application.

[0215] The techniques described herein may also be implemented in electronic hardware, computer software, firmware, or any combination thereof. Such techniques may be implemented in any of a variety of devices, such as general-purpose computers, wireless communication device handsets, or integrated circuit devices with multiple applications, including applications in wireless communication device handsets and other devices. Any feature described as a module or component may be implemented together in an integrated logic device, or separately as individual but interoperable logic devices. When implemented in software, the technique may be at least partially realized by a computer-readable data storage medium comprising program code that, when executed, performs one or more of the methods described above. The computer-readable data storage medium may form part of a computer program product, which may include packaging materials. The computer-readable medium may comprise memory or data storage media such as random access memory (RAM), read-only memory (ROM), non-volatile random access memory (NVRAM), electrically erasable programmable read-only memory (EEPROM), FLASH memory, magnetic or optical data storage media, etc. The technique may, as an addition or alternative, be at least partially implemented by a computer-readable communication medium that carries or communicates program code in the form of instructions or data structures, such as propagating signals or waves, and which can be accessed, read, and / or executed by a computer.

[0216] The program code may be executed by a processor which may include one or more processors, such as one or more digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), field-programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuit configurations. Such a processor may be configured to perform any of the techniques described herein. A general-purpose processor may be a microprocessor, but alternatively, the processor may be any conventional processor, controller, microcontroller, or state machine. The processor may also be implemented as a combination of computing devices, for example, a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors working with a DSP core, or any other such configuration. Accordingly, the term “processor” as used herein may refer to any of the above structures, any combination thereof, or any other structure or device suitable for implementing the techniques described herein. In addition, in some embodiments, the functions described herein may be provided in a dedicated software or hardware module configured for encoding and decoding, or incorporated into a composite video encoder / decoder (codec).

[0217] Examples used to illustrate this disclosure include the following:

[0218] Embodiment 1: A method for processing video data, comprising the steps of: acquiring a block of video data; processing the block using an intra-predictive mode; and determining the type of interpolation filter to use for the block based on at least one of the width and height of the block.

[0219] Embodiment 2: The method of Embodiment 1, further comprising the steps of determining that at least one of the width and height of a base block is greater than a threshold, determining a first type of interpolation filter to use for the block, and determining a reference pixel for the block using the first type of interpolation filter.

[0220] Embodiment 3: The method of Embodiment 1, wherein the first type of interpolation filter includes a 6-tap Gaussian filter.

[0221] Embodiment 4: The method of Embodiment 1, further comprising the steps of determining that at least one of the width and height of a base block is less than or equal to a threshold, determining a second type of interpolation filter to use on the block, and determining a reference pixel for the block using the second type of interpolation filter.

[0222] Embodiment 5: The method of Embodiment 4, wherein the second type of interpolation filter includes a 4-tap Gaussian filter.

[0223] Embodiment 6: Any one of embodiments 1 to 5, wherein the type of interpolation filter is explicitly signaled in the video bitstream.

[0224] Embodiment 7: The method of Embodiment 6, wherein the type of interpolation filter is explicitly signaled on a prediction block, coding block, coding tree unit (CTU), slice, or sequence basis.

[0225] Embodiment 8: Any one of embodiments 1 to 5, further comprising the step of determining the type of interpolation filter based on at least one of the width and height of a block, without using information explicitly signaled in the video bitstream.

[0226] Embodiment 9: An apparatus comprising a memory configured to store video data and a processor, wherein the processor is configured to acquire blocks of video data, process the blocks using an intra-predictive mode, and determine the type of interpolation filter to use for the blocks based on at least one of the width and height of the blocks.

[0227] Embodiment 10: The apparatus of Embodiment 9, wherein the processor is configured to determine that at least one of the width and height of a base block is greater than a threshold, to determine a first type of interpolation filter to use for the block, and to determine a reference pixel for the block using the first type of interpolation filter.

[0228] Embodiment 11: The apparatus of Embodiment 9, wherein the first type of interpolation filter includes a 6-tap Gaussian filter.

[0229] Embodiment 12: The apparatus of Embodiment 9, wherein the processor is configured to determine that at least one of the width and height of a base block is less than or equal to a threshold, to determine a second type of interpolation filter to use for the block, and to determine a reference pixel for the block using the second type of interpolation filter.

[0230] Embodiment 13: The apparatus of Embodiment 12, wherein the second type of interpolation filter is a 4-tap Gaussian filter.

[0231] Embodiment 14: Any one of embodiments 9 to 13, wherein the type of interpolation filter is explicitly signaled in the video bitstream.

[0232] Embodiment 15: The apparatus of Embodiment 14, wherein the type of interpolation filter is explicitly signaled on a prediction block, coding block, coding tree unit (CTU), slice, or sequence basis.

[0233] Embodiment 16: Any one of embodiments 9 to 13, wherein the processor is configured to determine the type of interpolation filter based on at least one of the width and height of a block, without using information explicitly signaled in the video bitstream.

[0234] Embodiment 17: Any one of embodiments 9 to 16, wherein the device includes an encoder.

[0235] Embodiment 18: Any one of embodiments 9 to 17, wherein the device includes a decoder.

[0236] Embodiment 19: Any one of embodiments 9 to 18, wherein the device is a mobile device.

[0237] Embodiment 20: An apparatus according to any of Embodiments 9 to 19, wherein the apparatus is an augmented reality device.

[0238] Embodiment 21: Any one of embodiments 9 to 20, further comprising a display configured to display video data.

[0239] Embodiment 22: Any one of embodiments 9 to 21, further comprising a camera configured to capture one or more pictures.

[0240] Embodiment 23: A computer-readable medium storing instructions that, when executed by a processor, perform any of the methods described in Embodiments 1 to 22.

[0241] Embodiment 24: An apparatus including means for performing any of the operations of Embodiments 1 to 22.

[0242] Embodiment 25: A method for processing video data, comprising the steps of: acquiring a block of video data; processing the block using an intra-predictive mode; and determining the type of smoothing filter to use on the block based on at least one of the width and height of the block.

[0243] Embodiment 26: The method of Embodiment 25, comprising the step of determining whether the angle of the intra-prediction mode is an integer angle, further comprising the step of determining the type of smoothing filter, which is further based on the determination that the angle of the intra-prediction mode is an integer angle.

[0244] Embodiment 27: Any one of Embodiments 25 or 26, further comprising the steps of determining that at least one of the width and height of a base block is greater than a threshold, determining a first type of smoothing filter to use on the block, and processing at least one predicted pixel for the block using the first type of smoothing filter.

[0245] Embodiment 28: The method of Embodiment 27, wherein the first type of smoothing filter includes a [1 4 6 4 1] filter.

[0246] Embodiment 29: Any one of Embodiments 25 or 26, further comprising the steps of determining that at least one of the width and height of a base block is less than or equal to a threshold, determining a second type of smoothing filter to use on the block, and processing at least one predicted pixel for the block using the second type of smoothing filter.

[0247] Embodiment 30: The method of Embodiment 29, wherein the second type of smoothing filter includes a

[0121] filter.

[0248] Embodiment 31: Any one of embodiments 25 to 30, wherein the type of smoothing filter is explicitly signaled in the video bitstream.

[0249] Embodiment 32: The method of Embodiment 31, wherein the type of interpolation filter is explicitly signaled on a prediction block, coding block, coding tree unit (CTU), slice, or sequence basis.

[0250] Embodiment 33: Any one of embodiments 25 to 30, further comprising the step of determining the type of smoothing filter based on at least one of the width and height of a block, without using information explicitly signaled in the video bitstream.

[0251] Embodiment 34: An apparatus comprising a memory configured to store video data and a processor, wherein the processor is configured to acquire blocks of video data, process the blocks using an intra-predictive mode, and determine the type of smoothing filter to use on the blocks based on at least one of the width and height of the blocks.

[0252] Embodiment 35: The apparatus of Embodiment 34, wherein the processor is configured to determine whether the angle of the intra-prediction mode is an integer angle, and to determine the type of smoothing filter, which is further based on the determination that the angle of the intra-prediction mode is an integer angle.

[0253] Embodiment 36: An apparatus of any one embodiment 34 or 35, wherein the processor is configured to determine that at least one of the width and height of a base block is greater than a threshold, to determine a first type of smoothing filter to use on the block, and to process at least one predicted pixel for the block using the first type of smoothing filter.

[0254] Embodiment 37: The apparatus of Embodiment 36, wherein the first type of smoothing filter includes a [1 4 6 4 1] filter.

[0255] Embodiment 38: An apparatus in any one of Embodiments 34 or 35, wherein the processor is configured to determine that at least one of the width and height of a base block is less than or equal to a threshold, to determine a second type of smoothing filter to use on the block, and to process at least one predicted pixel for the block using the second type of smoothing filter.

[0256] Embodiment 39: The apparatus of Embodiment 38, wherein the second type of smoothing filter includes a

[0121] filter.

[0257] Embodiment 40: Any one of embodiments 34 to 39, wherein the type of smoothing filter is explicitly signaled in the video bitstream.

[0258] Embodiment 41: The apparatus of Embodiment 40, wherein the type of interpolation filter is explicitly signaled on a prediction block, coding block, coding tree unit (CTU), slice, or sequence basis.

[0259] Embodiment 42: Any one of embodiments 34 to 39, wherein the processor is configured to determine the type of smoothing filter based on at least one of the width and height of a block, without using information explicitly signaled in the video bitstream.

[0260] Embodiment 43: Any one of embodiments 34 to 42, wherein the device includes an encoder.

[0261] Embodiment 44: Any one of embodiments 34 to 43, wherein the device includes a decoder.

[0262] Embodiment 45: Any one of Embodiments 34 to 44, wherein the device is a mobile device.

[0263] Embodiment 46: Any one of embodiments 34 to 45, wherein the device is an augmented reality device.

[0264] Embodiment 47: Any one of embodiments 34 to 46, further comprising a display configured to display video data.

[0265] Embodiment 48: Any one of embodiments 34 to 47, further comprising a camera configured to capture one or more pictures.

[0266] Embodiment 49: A computer-readable medium storing instructions that, when executed by a processor, perform any of the methods described in Embodiments 25 to 48.

[0267] Embodiment 50: An apparatus comprising means for performing any of the operations described in Embodiments 25 to 48.

[0268] Embodiment 51: A computer-readable medium storing instructions that, when executed by a processor, perform any of the methods of Embodiments 1 to 22 and Embodiments 25 to 48.

[0269] Embodiment 52: An apparatus comprising means for performing any of the operations of Embodiments 1 to 22 and Embodiments 25 to 48.

[0270] Embodiment 53: An apparatus for processing video data, comprising at least one memory and at least one processor coupled to the at least one memory, wherein the at least one processor is configured to determine an intra-prediction mode for predicting blocks of video data, and to determine a type of smoothing filter to use for blocks of video data, wherein the type of smoothing filter is determined at least in part on comparing at least one of the width and height of the blocks of video data to a first threshold, and to perform an intra-prediction for blocks of video data using the determined type of smoothing filter and the intra-prediction mode.

[0271] Embodiment 54: The apparatus of Embodiment 53, wherein at least one processor is configured to use a first smoothing interpolation filter as a determined type of smoothing filter, at least in part on the determination that at least one of the block width and block height is greater than a first threshold, and to use the first smoothing interpolation filter to determine a reference pixel for intra-prediction of a block of video data.

[0272] Aspect 55: An apparatus according to any of aspects 53 to 54, wherein the first smoothing interpolation filter includes a 6-tap Gaussian filter.

[0273] Aspect 56: An apparatus according to aspect 55, wherein at least one processor is configured to use a second smoothing interpolation filter as the determined type of smoothing filter, at least partially based on a determination that at least one of the width and height of a block is below a first threshold, and to determine reference pixels for intra prediction of a block of video data using the second smoothing interpolation filter.

[0274] Aspect 57: An apparatus according to aspect 56, wherein the second smoothing interpolation filter includes a 4-tap Gaussian filter.

[0275] Aspect 58: An apparatus according to any of aspects 53 to 57, wherein at least one processor is configured to determine a minimum offset between an angular direction of an intra prediction mode and one of a vertical intra prediction mode and a horizontal intra prediction mode, and to determine a type of smoothing filter for use with a block of video data based on comparing the determined minimum offset with a second threshold.

[0276] Aspect 59: An apparatus according to aspect 58, wherein at least one processor is configured to determine a low-pass filter as the type of smoothing filter, at least partially based on a determination that the determined minimum offset is greater than a second threshold and a determination that the intra prediction mode is an integer angle mode associated with integer-valued reference pixel positions.

[0277] Aspect 60: An apparatus according to aspect 59, wherein the low-pass filter performs reference pixel smoothing without interpolation, and the low-pass filter is a

[0121] filter.

[0278] Embodiment 61: The apparatus of Embodiment 58, wherein at least one processor is configured to determine a Gaussian filter as the type of smoothing filter, at least in part on the determination that the determined minimum offset is greater than a second threshold and the determination that the intra-prediction mode is a fractional angle mode associated with a fractional reference pixel position.

[0279] Embodiment 62: The apparatus of Embodiment 61, wherein the Gaussian filter performs smoothing interpolation without smoothing of the reference pixels.

[0280] Embodiment 63: The apparatus of Embodiment 61, wherein the Gaussian filter includes a 6-tap Gaussian filter based on the determination that at least one of the block width and block height is greater than a first threshold.

[0281] Embodiment 64: The apparatus of Embodiment 61, wherein the Gaussian filter includes a four-tap Gaussian filter based on the determination that at least one of the block width and block height is not greater than a first threshold.

[0282] Embodiment 65: The apparatus of Embodiment 58, wherein at least one processor uses an interpolation filter as the determined type of smoothing filter, at least on the basis of a determination that the determined minimum offset is less than or equal to a second threshold, wherein the interpolation filter includes a 4-tap cubic filter, and is configured to perform intra-prediction of blocks of video data using the interpolation filter without applying reference pixel smoothing.

[0283] Embodiment 66: The apparatus of Embodiment 58, wherein at least one processor is configured to determine a low-pass filter as the type of smoothing filter, at least in part on the determination that the intra-prediction mode is an integer-angle mode and the determination that the determined minimum offset is greater than a second threshold.

[0284] Embodiment 67: An apparatus of Embodiment 67, wherein at least one processor performs reference pixel smoothing using a large-tap low-pass filter, at least on the basis of determining that at least one of the block width and block height is greater than a first threshold, the large-tap low-pass filter is configured to perform the smoothing of reference pixels to a greater degree than a small-tap low-pass filter.

[0285] Embodiment 68: The apparatus of Embodiment 67, wherein at least one processor performs reference pixel smoothing using a small-tap low-pass filter, at least partially based on a determination that at least one of the block width and block height is less than or equal to a first threshold, wherein the small-tap low-pass filter is configured to perform, perform, apply a lower degree of reference pixel smoothing than a large-tap low-pass filter.

[0286] Embodiment 69: An apparatus of any embodiment 53 to 68, wherein at least one processor is configured to determine that the intra-predictive mode is an integer angle mode, at least in part on comparing the tilt of the intra-predictive mode with one or more pixel positions determined from the width and height of the block.

[0287] Embodiment 70: An apparatus of any one of embodiments 53 to 69, wherein at least one processor is configured to perform intra-prediction on blocks of video data using a cubic interpolation filter based on the determination that the offset between the angular direction of an intra-prediction mode and a vertical intra-prediction mode or a horizontal intra-prediction mode is less than a second threshold.

[0288] Embodiment 71: The apparatus of Embodiment 70, wherein at least one processor is configured to perform reference line extension using a weak interpolation filter, the weak interpolation filter is used to perform reference line extension prior to the use of a cubic interpolation filter for performing intra prediction, the cubic interpolation filter has a higher cutoff frequency than the weak interpolation filter and applies a higher degree of smoothing than the weak interpolation filter.

[0289] Embodiment 72: The apparatus of Embodiment 71, wherein the weak interpolation filter includes a 4-tap sync-based interpolation filter and a 6-bit 4-tap interpolation filter.

[0290] Embodiment 73: An apparatus from any of embodiments 53 to 72, wherein the type of smoothing filter is signaled in the video bitstream.

[0291] Embodiment 74: An apparatus according to any of Embodiments 53 to 73, wherein the type of smoothing filter is signaled to individual sets of prediction blocks, coding blocks, coding tree units (CTUs), slices, or sequences.

[0292] Embodiment 75: An apparatus of any embodiment 53 to 74, wherein at least one processor is configured to determine the type of smoothing filter based on at least one of the width and height of a block, without using information explicitly signaled in the video bitstream.

[0293] Embodiment 76: An apparatus of any embodiment 53 to 75, wherein at least one processor is configured to determine blocks of residual data with respect to blocks of video data, and to decode blocks of video data using blocks of residual data and predicted blocks determined based on performing intra-predictions on blocks of video data.

[0294] Aspect 77: An apparatus according to any of aspects 53 to 75, wherein at least one processor is configured to generate an encoded video bitstream including information related to a block of video data.

[0295] Aspect 78: The apparatus of aspect 77, further comprising storing the encoded video bitstream in at least one memory.

[0296] Aspect 79: The apparatus according to any of aspects 77 to 78, further comprising a transmitter configured to transmit the encoded video bitstream.

[0297] Aspect 80: A method of processing video data, comprising determining an intra prediction mode for predicting a block of video data; determining a type of smoothing filter for use with the block of video data, the type of smoothing filter being determined based at least in part on comparing at least one of the width of the block of video data and the height of the block of video data to a first threshold; and performing an intra prediction for the block of video data using the determined type of smoothing filter and the intra prediction mode.

[0298] Aspect 81: The method of aspect 80, further comprising using a first smoothing interpolation filter as the determined type of smoothing filter based at least in part on a determination that at least one of the width of the block and the height of the block is greater than the first threshold; and determining reference pixels for intra prediction of the block of video data using the first smoothing interpolation filter. [[ID=十七]]

[0299] Aspect [[ID=二十一]]

[0300] Embodiment 83: Any method of Embodiments 80 to 82, further comprising the steps of using a second smoothing interpolation filter as a determined type of smoothing filter, at least in part on the determination that at least one of the width and height of a block is less than or equal to a first threshold, and using the second smoothing interpolation filter to determine a reference pixel for intra-prediction of a block of video data.

[0301] Embodiment 84: The method of Embodiment 83, wherein the second smoothing interpolation filter includes a 4-tap Gaussian filter.

[0302] Embodiment 85: Any method of Embodiments 80 to 84, further comprising the steps of determining a minimum offset between the angular direction of an intra-prediction mode and one of a vertical intra-prediction mode and a horizontal intra-prediction mode, and determining a type of smoothing filter to use for a block of video data based on comparing the determined minimum offset with a second threshold.

[0303] Embodiment 86: The method of Embodiment 85, further comprising the step of determining a low-pass filter as the type of smoothing filter, at least in part on the determination that the determined minimum offset is greater than a second threshold and the determination that the intra-prediction mode is an integer angle mode associated with an integer value reference pixel position.

[0304] Embodiment 87: The method of Embodiment 86, wherein the low-pass filter performs reference pixel smoothing without interpolation, and the low-pass filter includes a

[0121] filter.

[0305] Embodiment 88: The method of Embodiment 85, further comprising the step of determining a Gaussian filter as the type of smoothing filter, at least in part on the determination that the determined minimum offset is greater than a second threshold and the determination that the intra-prediction mode is a fractional angle mode associated with a fractional reference pixel position.

[0306] Apparatus 89: The method of Apparatus 88, wherein the Gaussian filter performs smoothing interpolation without smoothing of the reference pixels.

[0307] Embodiment 90: The method of Embodiment 88, wherein the Gaussian filter includes a 6-tap Gaussian filter based on the determination that at least one of the block width and block height is greater than a first threshold.

[0308] Embodiment 91: The method of Embodiment 88, wherein the Gaussian filter includes a four-tap Gaussian filter based on the determination that at least one of the block width and block height is less than or equal to a first threshold.

[0309] Embodiment 92: The method of Embodiment 85, further comprising the steps of using an interpolation filter as a determined type of smoothing filter, at least in part on the determination that the determined minimum offset is less than or equal to a second threshold, wherein the interpolation filter includes a 4-tap cubic filter, and performing an intra-prediction for blocks of video data using the interpolation filter without applying reference pixel smoothing.

[0310] Embodiment 93: The method of Embodiment 85, further comprising the step of determining a low-pass filter as the type of smoothing filter, at least in part on the determination that the intra-prediction mode is an integer angle mode and the determination that the determined minimum offset between the intra-prediction mode and the horizontal or vertical mode is greater than a second threshold.

[0311] Embodiment 94: The method of Embodiment 93, further comprising the step of applying reference pixel smoothing using a large-tap low-pass filter, at least in part on the determination that at least one of the width and height of a block is greater than a first threshold, wherein the large-tap low-pass filter applies a greater degree of reference pixel smoothing than the small-tap low-pass filter.

[0312] Embodiment 95: The method of Embodiment 93, further comprising the step of applying a reference pixel smoothing using a small-tap low-pass filter, at least in part on the determination that at least one of the width and height of a block is less than or equal to a first threshold, wherein the small-tap low-pass filter applies a lower degree of reference pixel smoothing than the large-tap low-pass filter.

[0313] Embodiment 96: Any method of Embodiments 80 to 95, further comprising the step of determining that the intra-prediction mode is an integer angle mode, at least in part on comparing the inclination of the intra-prediction mode with one or more pixel positions determined from the width and height of the block.

[0314] Embodiment 97: Any method of Embodiments 80 to 96, further comprising the steps of determining that the offset between the angular direction of an intra-prediction mode and a vertical intra-prediction mode or a horizontal intra-prediction mode is less than a second threshold, and performing intra-prediction on a block of video data using a cubic interpolation filter based on the determination that the determined offset is less than a second threshold.

[0315] Embodiment 98: The method of Embodiment 97, further comprising the step of performing a reference line extension using a weak interpolation filter, wherein the weak interpolation filter is used to perform a reference line extension prior to the use of a cubic interpolation filter for performing an intra prediction, and the cubic interpolation filter has a higher cutoff frequency than the weak interpolation filter and applies a higher degree of smoothing than the weak interpolation filter.

[0316] Embodiment 99: The method of Embodiment 98, wherein the weak interpolation filter includes a 4-tap sync-based interpolation filter and a 6-bit 4-tap interpolation filter.

[0317] Embodiment 100: Any method of Embodiments 80 to 99, wherein the type of smoothing filter is signaled in the video bitstream.

[0318] Embodiment 101: Any method of Embodiments 80 to 100, wherein the type of smoothing filter is signaled for individual sets of prediction blocks, coding blocks, coding tree units (CTUs), slices, or sequences.

[0319] Embodiment 102: Any method of Embodiments 80 to 101, further comprising the step of determining the type of smoothing filter based on at least one of the width and height of a block, without using information explicitly signaled in the video bitstream.

[0320] Embodiment 103: Any method of Embodiments 80 to 102, further comprising the steps of determining a block of residual data with respect to a block of video data, and decoding a block of video data using a predicted block determined based on the block of residual data and the block of video data.

[0321] Embodiment 104: Any method of Embodiments 80 to 102, further comprising the step of generating an encoded video bitstream that includes information relating to blocks of video data.

[0322] Embodiment 105: The method of Embodiment 104, further comprising the step of storing an encoded video bitstream.

[0323] Embodiment 106: Any method of Embodiment 104 or 105, further comprising the step of transmitting an encoded video bitstream.

[0324] Embodiment 107: Embodiment 43: A computer-readable medium storing instructions that, when executed by a processor, perform any of the methods described in Embodiments 53 to 106.

[0325] Embodiment 108: An apparatus comprising means for performing any of the operations described in Embodiments 53 to 106. [Explanation of symbols]

[0326] 2 modes 18 Intra mode, Mode 26 Intra Mode 34 modes 35 division units 41 Prediction Processing Unit 42 Motion Estimation Unit 44 Motion compensation unit 46 Intra Prediction Processing Unit 50 modes, vertical intra prediction mode, adder 52 Conversion Processing Unit 54 Quantization Units 56 Entropy coding unit 57 Post-processing devices 58 Inverse Quantization Unit 60 Inverse Transform Processing Unit 62 Adder 63 filter units 64 Picture Memory 66 Mode 79 Network Entities 80 Entropy Decoding Unit 81 Prediction Processing Unit 82 Motion Compensation Unit 84 Intra Prediction Processing Unit 86 Inverse Quantization Unit 88 Inverse Transformation Processing Unit 90 Adder 91 Filter Unit 92 Picture Memory 100 Systems 102 Video Sources 104 Encoding devices, video encoding devices 106 Encoder Engine 108 storage 110 Output section 112 Decoding devices, video decoding devices 114 Input section 116 Decoder Engine 118 storage 120 Communication Links 122 Video Destination Devices Figure 200a 200b Figure 300 MDIS processes Figure 400 405 Currently coding block, current block 410 reference pixel top line, top reference pixel line 420 Reference Line Extended Pixels 423 Reference line pixel P 430 left reference pixels 500 processes 600 processes

Claims

1. A device for processing video data, At least one memory, The at least one memory is coupled to at least one processor, and the at least one processor is Determining an intra-prediction mode for predicting blocks of video data, For non-integer angle intra-prediction mode, the selection of a first Gaussian smoothing interpolation filter or a second Gaussian smoothing interpolation filter for use on the block of video data, wherein the first Gaussian smoothing interpolation filter applies a higher degree of smoothing than the second Gaussian smoothing interpolation filter when interpolating fractional reference pixel positions, and the selection is at least in part based on comparing at least one of the width of the block of video data and the height of the block of video data with a first threshold. A device configured to perform intra-prediction on the block of video data using the selected smoothing interpolation filter and the intra-prediction mode for the non-integer angle intra-prediction mode.

2. Regarding the non-integer angle intra prediction mode, at least one processor, Based at least in part on the determination that at least one of the width and height of the block is greater than the first threshold, the first Gaussian smoothing interpolation filter is used as the selected smoothing interpolation filter. The apparatus according to claim 1, configured to use the first Gaussian smoothing interpolation filter to determine a reference pixel for intra-prediction of the block of video data.

3. The apparatus according to claim 2, wherein the first Gaussian smoothing interpolation filter includes a 6-tap Gaussian filter.

4. Regarding the non-integer angle intra prediction mode, at least one processor, A second Gaussian smoothing interpolation filter is used as the selected smoothing interpolation filter, at least in part on the determination that at least one of the width and height of the block is less than or equal to the first threshold. The apparatus according to claim 1, configured to use the second Gaussian smoothing interpolation filter to determine a reference pixel for intra-prediction of the block of video data.

5. The apparatus according to claim 4, wherein the second Gaussian smoothing interpolation filter includes a 4-tap Gaussian filter.

6. The aforementioned at least one processor, Determine the block of residual data with respect to the aforementioned block of video data. The apparatus according to claim 1, configured to decode the blocks of video data using the blocks of residual data and predicted blocks determined based on performing the intra-prediction on the blocks of video data.

7. The aforementioned at least one processor, The apparatus according to claim 1, configured to generate an encoded video bitstream that includes information relating to the block of video data.

8. The apparatus according to claim 7, wherein the at least one processor is further configured to cause the encoded video bitstream to be stored in the at least one memory.

9. A transmitter configured to transmit the encoded video bitstream. The apparatus according to claim 7, further comprising:

10. A method for processing video data, The steps include determining an intra-prediction mode for predicting blocks of video data, For a non-integer angle intra-prediction mode, a step of selecting a first Gaussian smoothing interpolation filter or a second Gaussian smoothing interpolation filter to use on the block of video data, wherein the first Gaussian smoothing interpolation filter applies a greater degree of smoothing than the second Gaussian smoothing interpolation filter when interpolating fractional reference pixel positions, and the selection is at least in part based on comparing at least one of the width of the block of video data and the height of the block of video data with a first threshold; A method comprising the step of performing an intra-prediction on the block of video data using the selected smoothing interpolation filter and the intra-prediction mode for the non-integer angle intra-prediction mode.

11. Regarding the aforementioned non-integer angle intra prediction mode, The steps of using the first Gaussian smoothing interpolation filter as the selected smoothing interpolation filter, at least in part on the determination that at least one of the width and height of the block is greater than the first threshold, The steps include: determining a reference pixel for intra-prediction of the block of video data using the first Gaussian smoothing interpolation filter described above; It further includes, The method according to claim 10, wherein the first Gaussian smoothing interpolation filter includes a 6-tap Gaussian filter.

12. Regarding the aforementioned non-integer angle intra prediction mode, The steps of using the second Gaussian smoothing interpolation filter as the selected smoothing interpolation filter, at least in part on the determination that at least one of the width and height of the block is less than or equal to the first threshold, The steps include determining a reference pixel for intra-prediction of the block of video data using the second Gaussian smoothing interpolation filter described above, and It further includes, The method according to claim 10, wherein the second Gaussian smoothing interpolation filter includes a 4-tap Gaussian filter.

13. A step of determining the block of residual data relating to the block of video data, The steps of decoding the blocks of video data using the blocks of residual data and the predicted blocks determined based on performing the intra-prediction on the blocks of video data, The method according to claim 10, further comprising:

14. The step of generating an encoded video bitstream that includes information related to the block of video data. The method according to claim 10, further comprising:

Images