Intra-mode JVET coding

Abstract

This provides a suitable method for dividing video coding blocks for JVET. [Solution] The MPM set includes a set of intra predictive coding modes other than the six, and is encoded using truncated unary binarization. The 16 selected intra predictive coding modes are encoded using 4 bits of fixed-length code, and the remaining unselected coding modes are encoded using truncated binary coding. The JVET coding tree unit is coded as the root node of a QTBT structure. The QTBT structure has a quadtree branching from the root node and binary trees branching from each of the multiple leaf nodes of the quadtree, and using asymmetric binary partitioning, the coding unit represented by the leaf nodes of the quadtree is divided into multiple child nodes, and the multiple child nodes are represented as multiple leaf nodes of binary trees branching from the leaf nodes of the quadtree.

Description

Technical Field

[0001] The present disclosure relates to video coding, more specifically to the field of efficient intra-mode coding.

Background Art

[0002] Technical improvements in evolving video coding standards tend to increase coding efficiency and achieve higher bitrates, higher resolutions, and better video quality. The Joint Video Exploration Team is developing a new video coding method called JVET. Similar to other video coding methods such as HEVC (High Efficiency Video Coding), JVET is a block-based hybrid spatial and temporal prediction coding method. However, compared to HEVC, JVET includes many changes to the bitstream structure, syntax, constraints, and mapping for generating a plurality of decoded images. JVET is implemented in the JEM (Joint Exploration Model) encoder and decoder.

[0003] The current JVET standard describes a total of 67 intra prediction modes, including the planar mode, the DC mode, and 65 directional angular intra modes. To efficiently code these 67 modes, all intra modes are subdivided into three sets: a set of 6 most probable modes (MPM), a set of 16 selected modes, and a set of 45 non-selected modes.

[0004] The six MPMs are derived from the modes of the available neighboring blocks, the derived intra-modes, and the default intra-mode. The intra-modes of the five neighboring blocks of the current block are shown in Figure 1a. These are left (L), top (A), bottom left (BL), top right (AR), and top left (AL), and are used to form the MPM list of the current block. The initial MPM list is created by inserting the five neighboring intra-modes, the planar mode, and the DC mode into the MPM list. A pruning process is used to remove duplicate modes so that only unique modes are included in the MPM list. The order in which multiple initial modes are included is left, top, planar, DC, bottom left, top right, and top left.

[0005] If the MPM list is incomplete, derived modes are added. These intra modes are derived by adding "-1" or "+1" to the angular modes already included in the MPM list. If the MPM list is still incomplete, multiple default modes are added in the order of vertical, horizontal, mode 2, and diagonal modes. This process results in a unique list of six MPM modes.

[0006] In the entropy coding of the six MPMs, the truncated unary binarization shown in Figure 1b is currently used. The first three bins of the MPM modes are coded in multiple contexts that depend on the MPM mode associated with the bin currently being signaled. The MPM modes are classified into one of three categories: (a) primarily horizontal modes (i.e., MPM mode numbers are smaller than diagonal mode numbers), (b) primarily vertical modes (i.e., MPM modes are larger than diagonal mode numbers), and (c) non-angular (DC and planar) classes. Thus, three contexts are used to signal the MPM index based on this classification.

[0007] The coding for selecting the remaining 61 non-MPM modes is performed as follows: The 61 non-MPM modes are first divided into two sets: a selected mode set and a deselected mode set. The selected mode set contains 16 modes, and the remaining 45 modes are assigned to the deselected mode set. The mode set to which the current mode belongs is indicated by a flag in the bitstream. If the indicated mode is in the selected mode set, the selected mode is signaled with a 4-bit fixed-length code; if the indicated mode is from the deselected mode set, the selected mode is signaled with a truncated binary code. As an example, the selected mode set is generated by subsampling the 61 non-MPM modes as follows:

[0008] Select mode set = {0, 4, 8, 12, 16, 20…60} Non-selected mode set = {1, 2, 3, 5, 6, 7, 9, 10…59} The current JVET intra-mode coding is summarized in Figure 1b below.

[0009] As shown in Figure 1b, the last two entries in the MPM list require six bins, the same number as the bins assigned to the 16 selection modes. Such a configuration does not offer any advantage in terms of coding performance for the last two modes in the MPM list. Furthermore, since the first three bins of the MPM modes are coded with context-based entropy coding, the complexity of coding the six bins of the MPM modes is higher than coding the six bins of the selection modes.

[0010] There is a need for systems and methods to reduce the coding burden and bandwidth associated with intra-mode coding. [Overview of the Initiative]

[0011] This disclosure provides a video coding method for JVET intra prediction, which includes defining a set of unique intra prediction coding modes, which in some embodiments may be 67 modes; identifying and instantiating in memory a subset of unique MPM intra prediction coding modes from the set of unique intra prediction coding modes, which in some embodiments may be 5 or fewer out of 7 or more modes; also providing a method for identifying and instantiating in memory a selected subset of unique intra prediction coding modes, which in some embodiments may include 16 coding modes, from a set of unique intra prediction coding modes other than the subset of unique MPM intra prediction coding modes; and identifying and instantiating in memory a subset of unselected unique intra prediction coding modes that constitute a balance of intra prediction modes, from a set of unique intra prediction coding modes other than the subset of unique MPM intra prediction coding modes and other than the selected subset of unique intra prediction coding modes. Next, the subset of unique MPM intra prediction coding modes is coded using truncated unary binarization.

[0012] The Disclosure also provides a video coding system for JVET intra prediction, in some embodiments the video coding system comprises: instantiating in memory a set of 67 unique intra prediction coding modes; instantiating in memory a subset of unique MPM intra prediction coding modes from the set of unique intra prediction coding modes; instantiating in memory a subset of 16 unique selected intra prediction coding modes from a set of unique intra prediction coding modes other than the subset of unique MPM intra prediction coding modes; instantiating in memory a subset of unselected unique intra prediction coding modes from a set of unique intra prediction coding modes other than the subset of unique MPM intra prediction coding modes and other than the subset of unique selected intra prediction coding modes; encoding the subset of unique MPM intra prediction coding modes using truncated unary binarization; and encoding the subset of 16 selected unique intra prediction coding modes using a 4-bit fixed-length code. [Brief explanation of the drawing]

[0013] Further details of the present invention will be explained with reference to the accompanying drawings. [Figure 1a] This shows the adjacent blocks related to the current coding block. [Figure 1b] A table of current JVET coding for intra-mode prediction is shown. [Figure 1c] This shows the division of a frame into multiple coding tree units (CTUs). [Figure 2] This example demonstrates the partitioning of a CTU into multiple coding units (CUs) using quadtree partitioning and symmetric 2-part partitioning. [Figure 3]Shows the QTBT (quadtree plus binary tree) representation of the split in FIG. 2. [Figure 4] Shows four possible types of asymmetric binary splitting of a CU into two smaller CUs. [Figure 5] Shows an exemplary split of a CTU into multiple CUs using quadtree splitting, symmetric binary splitting, and asymmetric binary splitting. [Figure 6] Shows the QTBT representation of the split in FIG. 5. [Figure 7] Shows a simplified block diagram of CU coding in the JVET encoder. [Figure 8] Shows 67 possible intra prediction modes for the luminance component of JVET. [Figure 9] Shows a simplified block diagram of CU coding in the JVET encoder. [Figure 10] Shows an embodiment of a method of CU coding in the JVET encoder. [Figure 11] Shows a simplified block diagram of CU coding in the JVET encoder. [Figure 12] Shows a simplified block diagram of CU decoding in the JVET decoder. [[ID=二十八]] [[ID=二十九]] [Figure 13] Shows an alternative simplified block diagram of JVET coding for intra mode prediction. [Figure 14] Shows an alternative table of JVET coding for intra mode prediction. [Figure 15] Shows an embodiment of a computer system adapted and / or configured to process a method of CU coding. [Figure 16] Shows an embodiment of an encoding / decoding system for CU encoding / decoding in a JVET encoder / decoder.

Embodiments for Carrying Out the Invention

[0014] FIG. 1 shows the division of a frame into a plurality of Coding Tree Units (CTUs) 100. The frame can be an image of a video sequence. The frame can include a matrix or a set of matrices having a plurality of pixel values representing intensity measurements within the image. Thus, a video sequence can be generated by this set of matrices. The plurality of pixel values can be defined to represent color and brightness in full-color video coding where a plurality of pixels are divided into three channels. For example, in the YCbCr color space, a plurality of pixels have a luminance value Y representing the intensity of the gray level of the image, and two chrominance values Cb, Cr representing the color differences from gray to blue and red. In other embodiments, the plurality of pixel values can be represented by values in different color spaces or models. The number of pixels in a frame is determined by the resolution of the video. The higher the resolution, the more pixels there are, and the sharper the image, but the requirements for bandwidth, storage, and transmission also increase.

[0015] A plurality of frames of a video sequence can be encoded and decoded using JVET. JVET is a video coding method developed by the Joint Video Exploration Team. A plurality of versions of JVET are implemented in JEM (Joint Exploration Model) decoders and multiplexers. Similar to other video coding methods such as HEVC (High Efficiency Video Coding), JVET is a block-based hybrid spatial and temporal prediction coding method. In coding with JVET, a frame is first divided into a plurality of square blocks called CTUs 100 as shown in FIG. 1. For example, the plurality of CTUs 100 can be a plurality of blocks of 128x128 pixels.

[0016] Figure 2 shows an exemplary division of a CTU100 into multiple CU102s. Each CTU100 within a frame can be divided into one or more CU102s. One or more CU102s can be used for prediction and transformation as described below. Unlike HEVC, in JVET, multiple CU102s may be rectangular or square and can be coded without further division into multiple prediction units or multiple transformation units. Multiple CU102s may be the same size as their root CTU100, or they may be smaller subdivisions of a root CTU100, as small as a 4x4 block.

[0017] In JVET, CTU100 can be partitioned into multiple CU102 according to the QTBT (quadtree plus binary tree) scheme. In this scheme, CTU100 is recursively partitioned into multiple square blocks according to a quadtree, and these square blocks can be recursively partitioned horizontally or vertically according to a binary tree. Several parameters can be set to control the partitioning according to the QTBT, such as the CTU size, the minimum size of leaf nodes in the quadtree and binary tree, the maximum size of leaf nodes in the binary tree, and the maximum depth of the binary tree.

[0018] In some embodiments, JVET can restrict the binary partitioning of the QTBT's binary tree portion to symmetric partitioning, where multiple blocks are divided in half either vertically or horizontally along the midline.

[0019] As a non-restrictive example, Figure 2 shows a CTU100 divided into multiple CU102, where solid lines represent quadtree partitioning and dashed lines represent symmetric binary tree partitioning. As illustrated, the binary partitioning allows for symmetric horizontal and vertical partitioning, defining the structure of the CTU and its subdivision into multiple CUs.

[0020] Figure 3 shows the QTBT representation of the partition in Figure 2. The quadtree root node represents CTU100, which has each child node of the quadtree portion representing one of the four square blocks partitioned from the parent square block. Multiple square blocks represented by multiple quadtree leaf nodes are partitioned symmetrically zero or more times using a binary tree, and multiple quadtree leaf nodes are multiple root nodes of the binary tree. At each level of the binary tree portion, the block can be partitioned symmetrically vertically or horizontally. A flag set to "0" indicates that the block is partitioned symmetrically horizontally, and a flag set to "1" indicates that the block is partitioned symmetrically vertically.

[0021] In other embodiments, JVET can enable either symmetric or asymmetric partitioning in the binary tree portion of the QTBT. Asymmetrical motion partitioning (AMP) is possible in different contexts of HEVC when partitioning multiple prediction units (PUs). However, when partitioning multiple CU102 within JVET according to the QTBT structure, asymmetrical partitioning can result in an improved partition compared to symmetrical partitioning when the multiple correlated areas of the CU102 are not located on either side of a midline passing through the center of the CU102. As a non-restrictive example, if a CU102 represents one object close to the center of the CU and another object on the side of the CU102, the CU102 can be asymmetrically partitioned to place each object into separate, smaller CU102s of different sizes.

[0022] Figure 4 shows four possible types of asymmetrical bipartite divisions in which CU102 is divided into two smaller CU102s along a line that crosses the length or height of CU102, with one of the two smaller CU102s being 25% of the size of the parent CU102 and the other being 75% of the size of the parent CU102. The four types of asymmetrical bipartite divisions shown in Figure 4 allow CU102 to be divided along lines 25% from the left side of CU102, 25% from the right side of CU102, 25% from the top of CU102, or 25% from the bottom of CU102. In another embodiment, the asymmetrical dividing line through which CU102 is divided may be located at any other arbitrary position such that CU102 is not divided symmetrically in half.

[0023] Figure 5 shows a non-restrictive example of CTU100 partitioned into multiple CU102 using a scheme that allows both symmetric and asymmetric binary partitioning in the binary tree portion of the QTBT. In Figure 5, the dashed lines indicate asymmetric binary partitioning lines, and the parent CU102 is partitioned using one of the multiple partitioning types shown in Figure 4.

[0024] Figure 6 shows the QTBT representation of the partition in Figure 5. In Figure 6, the two solid lines extending from the nodes represent symmetric partitions in the binary tree portion of the QTBT, and the two dashed lines extending from the nodes represent asymmetric partitions in the binary tree portion.

[0025] The syntax showing how CTU100 was partitioned into multiple CU102s can be encoded in a bitstream. As a non-restrictive example, the syntax can be encoded in a bitstream to indicate which nodes were partitioned by quadtree partitioning, symmetric div partitioning, and asymmetric div partitioning. Similarly, the syntax can be encoded in a bitstream for multiple nodes partitioned using asymmetric div partitioning to indicate which type of asymmetric div partitioning was used, such as one of the four types shown in Figure 4.

[0026] In some embodiments, the use of asymmetric partitioning can be limited to partitioning multiple CU102s at multiple leaf nodes in a quadtree portion of the QTBT. In these embodiments, the CU102s of multiple child nodes partitioned from a parent node using quadtree partitioning in a quadtree portion may be the final CU102, or they may be further partitioned using quadtree partitioning, symmetric 2-partitioning, or asymmetric 2-partitioning. The multiple child nodes of a binary tree portion partitioned using symmetric 2-partitioning may be the final CU102, or they may be recursively partitioned one or more times using only symmetric 2-partitioning. The multiple child nodes of a binary tree portion partitioned from a QT leaf node using asymmetric 2-partitioning may be the final CU102 that is not further partitioned.

[0027] In these embodiments, the complexity of the search can be reduced and / or the overhead bit can be limited by restricting the use of asymmetric partitioning to the partitioning of quadtree leaf nodes. Since only quadtree leaf nodes are partitioned by asymmetric partitioning, using asymmetric partitioning allows the termination of a branch in the QT portion to be indicated directly without other syntax or further signaling.

[0028] Similarly, since multiple nodes that have been asymmetrically divided cannot be divided further, the use of asymmetric division in a node can also directly indicate that its asymmetrically divided multiple child nodes are the final CU102 without any other syntax or further signaling.

[0029] In alternative embodiments, such as when limiting the complexity of the search and / or limiting the number of append bits is not a major concern, asymmetric partitioning may be used to partition multiple nodes generated by quadtree partitioning, symmetric binary partitioning, and / or asymmetric binary partitioning.

[0030] Following quadtree and binary tree partitioning using any of the above-described QTBT structures, the multiple blocks represented by the leaf nodes of the QTBT indicate the final coded CU102, such as coding using inter-prediction or intra-prediction. In the case of multiple slices or multiple full frames coded with inter-prediction, different partitioning structures can be used for the luminance and chroma components. For example, in the case of an inter-slice, the CU102 may have coding blocks (CBs) for different color components, such as one luminance CB and two chroma CBs. In the case of multiple slices or multiple full frames coded with intra-prediction, the partitioning structures for the luminance and chroma components are the same.

[0031] In an alternative embodiment, JVET can use a two-level coding block structure as an alternative to or extension of the QTBT partitioning described above. In a two-level coding block structure, CTU100 may first be divided into multiple base units (BUs) at a higher level. The multiple BUs may then be divided into multiple operating units (OUs) at a lower level.

[0032] In embodiments employing a two-level coding block structure, at the higher level, CTU100 can be divided into multiple BUs according to one of the multiple QTBT structures described above, or according to a quadtree (QT) structure such as that used in HEVC. A block can be divided into only four subblocks of the same size. As a non-limiting example, CTU102 can be divided into multiple BUs according to the QTBT structure described above with respect to Figures 5-6. Multiple leaf nodes in a quadtree portion can be divided using quadtree partitioning, symmetric binary partitioning, or asymmetric binary partitioning.

[0033] In this example, the final leaf node of the QTBT can be multiple BUs instead of multiple CUs. At the lower level of the two-level coding block structure, each BU derived from CTU100 can be further divided into one or more OUs. In some embodiments, if a BU is square, it can be divided into multiple OUs using quadtree division or bipartite division, such as symmetric or asymmetric bipartite division. However, if a BU is not square, it can only be divided into multiple OUs using bipartite division. Restricting the types of divisions that can be used for non-square BUs can limit the number of bits used to indicate the type of division used to generate multiple BUs.

[0034] The following description concerns the coding of CU102, but in embodiments using a two-level coding block structure, BUs and OUs can be coded instead of CU102. As a non-limiting example, multiple BUs may be used for higher-level coding operations such as intra-prediction or inter-prediction, while smaller multiple OUs may be used for lower-level coding operations such as transformations or the generation of transformation coefficients. Thus, the syntax coded for multiple BUs indicates whether they are coded in intra-prediction or inter-prediction, or identifies information that identifies a particular intra-prediction mode or motion vector used to code the multiple BUs. Similarly, the syntax for multiple OUs may identify a particular transformation operation or quantization transformation coefficient used to code the multiple OUs.

[0035] Figure 7 shows a simplified block diagram of CU coding in the JVET encoder. The main steps of video coding are, as described above, to split and identify multiple CU102s, then to encode the multiple CU102s using predictions in 704 or 706, to generate residual CU710s in 708, to transform in 712, to quantize in 716, and to entropy code in 720.

[0036] The encoder and coding process shown in Figure 7 also includes the decoding process, which will be described in more detail below. Given the current CU102, the encoder can obtain the predicted CU702 by spatially using intra-prediction at 704 or temporally using inter-prediction at 706. The basic idea of ​​predictive coding is to transmit a difference signal or residual signal between the original signal and its prediction. On the receiver side, the original signal can be reconstructed by adding the residual and prediction, as described below. Since the difference signal is less correlated than the original signal, fewer bits are required for transmission.

[0037] A slice that is entirely coded by an intra-predictive CU102, such as the entire image or a portion of an image, can be an "I" slice, which is decoded without referencing other slices, and can be a point from which decoding can begin. Slices coded by at least several inter-predictive CUs can be predictive (P) slices or bi-predictive (B) slices, which can be decoded based on one or more reference images. A P slice can use intra-predictive and inter-predictive with a previously coded slice. For example, a P slice can be further compressed than an "I" slice using inter-predictive, but coding requires coding of a previously coded slice. A B slice can use data from slices before and / or after coding, using intra-predictive or inter-predictive with interpolated prediction from two different frames, thus improving the accuracy of the motion estimation process. In some cases, P and B slices can be coded using an intra-block copy, which uses data from other parts of the same slice, or alternatively, they can be coded.

[0038] As described below, intra-prediction or inter-prediction may be performed based on multiple CU734s reconstructed from multiple previously coded CU102s, such as multiple adjacent CU102s or multiple CU102s in a reference image.

[0039] By spatially encoding CU102 using intra-prediction in 704, it is possible to identify an intra-prediction mode that optimally predicts multiple pixels of CU102 based on multiple samples from multiple adjacent CU102s in the image.

[0040] When coding the luminance component of a CU, the encoder can generate a list of candidate intra-prediction modes. While HEVC had 35 possible intra-prediction modes for the luminance component, JVET has 67 possible intra-prediction modes for the luminance component. These include planar modes that use a three-dimensional plane of multiple values ​​generated from multiple adjacent pixels, DC modes that use multiple values ​​averaged from multiple adjacent pixels, and 65 directional modes shown in Figure 8 that use multiple values ​​copied from multiple adjacent pixels along a specified direction.

[0041] When generating a list of candidate intra-prediction modes for the luminance component of a CU, the number of candidate modes on the list depends on the size of the CU. The candidate list may include a portion of the 35 HEVC modes with the lowest SATD (Sum of Absolute Transform Difference) cost, multiple new directional modes added to the adjacent JVET to multiple candidates identified from multiple HEVC modes, and multiple modes from a set of six most probable modes (MPMs) of CU102, identified based on a list of multiple intra-prediction modes and multiple default modes used for multiple previously coded adjacent blocks.

[0042] When coding the chroma component of a CU, a list of candidate intra-prediction modes can be generated. This list includes modes generated by a cross-component linear model projection from luminance samples, multiple intra-prediction modes identified by luminance CBs at specific array positions within a chroma block, and multiple chroma prediction modes previously identified in adjacent blocks. The encoder identifies the candidate modes on the list with the lowest rate distortion cost and uses those intra-prediction modes when coding the luminance and chroma components of the CU. The syntax can be encoded into a bitstream indicating the multiple intra-prediction modes used to encode each CU102.

[0043] After the optimal intra-prediction mode for CU102 is selected, the encoder can use those modes to generate the prediction CU402. If the selected mode is a directional mode, a 4-tap filter can be used to improve directional accuracy. The top or left column or row of the prediction block can be adjusted with boundary prediction filters such as 2-tap or 3-tap filters.

[0044] The predicted CU702 can be further smoothed by a PDPC (position dependent intraprediction combination) process, which adjusts the predicted CU702 generated based on filtered samples from multiple adjacent blocks using multiple unfiltered samples from multiple adjacent blocks, or by adaptive reference sample smoothing using a 3-tap or 5-tap low-pass filter to process multiple reference samples.

[0045] When CU102 is temporally encoded using interpretation in 706, a set of motion vectors (MVs) can be identified that point to multiple samples in multiple reference images that best predict multiple pixels of CU102. Interpretation utilizes temporal redundancy between multiple slices by representing the displacement of blocks of multiple pixels within a slice. The displacement is determined according to the values ​​of multiple pixels in the preceding or succeeding slice through a process called motion compensation. Motion vectors and associated reference indices indicating pixel displacement relative to a particular reference image, along with the residuals between the original pixels and the motion-compensated pixels, can be provided in a bitstream to the decoder. The decoder can use the signaled motion vectors and reference indices of the residuals to reconstruct blocks of multiple pixels within the reconstructed slice.

[0046] In JVET, the accuracy of the motion vector is stored in 1 / 16 pel, and the difference between the motion vector and the predicted motion vector of the CU can be coded with a resolution of 1 / 4 pel or an integer pel resolution.

[0047] In JVET, multiple motion vectors can be identified for multiple subCUs within CU102 using techniques such as advanced temporal motion vector prediction (ATMVP), spatial-temporal motion vector prediction (STMVP), affine motion compensation prediction, pattern-matched motion vector derivation (PMMVD), and / or bi-directional optical flow (BIO).

[0048] The encoder may use ATMVP to identify the time vector of CU102 pointing to the corresponding block in the reference image. The time vector may be identified based on multiple motion vectors and multiple reference images previously identified for multiple adjacent CU102s that have been coded. Using the reference block indicated by the time vector of the entire CU102, the motion vector of each subCU within CU102 may be identified.

[0049] STMVP can identify the motion vectors of sub-CUs by scaling and averaging multiple motion vectors identified in multiple adjacent blocks previously coded in interpretation, along with the time vector.

[0050] Affine motion compensation prediction can be used to predict the field of multiple motion vectors for each subCU within a block, based on two control motion vectors identified at the top corner of the block. For example, multiple motion vectors for multiple subCUs can be derived based on multiple motion vectors at the top corners identified in each 4x4 block within CU102.

[0051] PMMVD can identify the initial motion vector of the current CU102 using either bilateral matching or template matching. Bilateral matching identifies the current CU102 and reference blocks in two different reference images along the motion trajectory, while template matching can retrieve reference images identified by corresponding blocks and templates within the current CU102.

[0052] Next, the initial motion vector identified for CU102 can be individually refined for each subCU. BIO is used when performing interpretation with biprediction based on preceding and succeeding reference images, and the motion vector of a subCU can be identified based on the gradient of the difference between two reference images.

[0053] Depending on the situation, local illumination compensation (LIC) can be used at the CU level to determine the values ​​of the scaling factor parameter and offset parameter based on multiple samples adjacent to the current CU102 and the corresponding multiple samples adjacent to the reference block identified by the candidate motion vector. In JVET, multiple LIC parameters can be modified and signaled at the CU level. In some of the methods described above, multiple motion vectors identified for each sub-CU of the CU can be signaled to the decoder at the CU level. In other methods such as PMMVD and BIO, motion information is not signaled in a bitstream to save overhead, and the decoder can derive motion vectors in the same process.

[0054] After the motion vectors of CU102 are identified, the encoder can use those motion vectors to generate a predicted CU702. In some cases, if multiple motion vectors are identified in an individual subCU, Overlapped Block Motion Compensation (OBMC) may be used when generating the predicted CU702 by combining those motion vectors with motion vectors previously identified in one or more adjacent subCUs.

[0055] Using bilateral prediction, JVET can identify multiple motion vectors using decoder-side motion vector refinement (DMVR). DMVR uses a bilateral template matching process to identify motion vectors based on two motion vectors identified by bilateral prediction. DMVR identifies weighted combinations of multiple prediction CU702s generated using each of the two motion vectors, and the two motion vectors can be refined by replacing them with new motion vectors that best represent the combined prediction CU702.

[0056] Using the two improved motion vectors, the final prediction CU702 can be generated. As described above, if the predicted CU702 is identified in the intra prediction at 704 or the inter prediction at 706, then at 708, the encoder may subtract the predicted CU702 from the current CU102 to identify the residual CU710.

[0057] The encoder may use one or more transformation operations in 712 to transform the residual CU710 into transformation coefficients 714 that represent the residual CU710 within the transformation domain, for example, by using a discrete cosine block transform (DCT) to transform the data into the transformation domain. JVET allows for more types of transformation operations than HEVC, including DCT-II, DST-VII, DCT-VIII, DST-I, and DCT-V operations. These multiple possible transformation operations are grouped into multiple subsets, and the encoder may signal which subset and which specific operation within those subsets was used. In some cases, a large block size transformation is used to zero out high-frequency transformation coefficients in CU102 larger than a certain size, so that only low-frequency transformation coefficients are retained for these CU102.

[0058] In some cases, the mode-dependent non-separable secondary transform (MDNSST) can be applied to the low-frequency transformation coefficient 714 after the forward core transform. The Hypercube-Givens Transform (HyGT) can be used in MDNSST operations based on rotational data. If used, an index value identifying a particular MDNSST operation may be signaled from the encoder.

[0059] In 716, the encoder can quantize the conversion coefficients 714 to quantization conversion coefficients 716. The quantization of each coefficient can be calculated by dividing the value of the coefficient by a quantization step derived from the quantization parameter (QP). In some embodiments, Qstep is 2 (QP-4) / 6 It is defined as follows. Since the high-precision conversion coefficient 714 can be converted to a quantization conversion coefficient 716 which has a finite number of possible values, quantization can assist in data compression.

[0060] Therefore, quantization of the conversion coefficients may limit the amount of bits generated and transmitted by the conversion process. However, quantization is a lossy operation, and the losses caused by quantization are irrecoverable. The quantization process presents a trade-off between the quality of the reconstructed sequence and the amount of information required to represent the sequence. For example, a lower QP value improves the quality of the decoded video, but may require a large amount of data for representation and transmission. In contrast, a higher QP value reduces the quality of the reconstructed video sequence, but requires less data and bandwidth.

[0061] JVET can utilize a variance-based adaptive quantization technique, which allows each CU102 to use different quantization parameters for its coding process (instead of using the same frame QP in coding each CU102 of a frame). The variance-based adaptive quantization technique adaptively decreases the quantization parameters of certain blocks and increases them of others. The variance of the CU is calculated to select a particular QP for a CU102. That is, if the variance of the CU is higher than the mean variance of the frame, a QP higher than the frame's QP may be assigned to the CU102. If the CU102 exhibits a variance lower than the mean variance of the frame, a lower QP may be assigned.

[0062] In 720, the encoder can identify multiple final compressed bits 722 by entropy coding multiple quantization conversion coefficients 718. Entropy coding aims to remove statistical redundancy from the transmitted information. In JVET, quantization conversion coefficients 718 can be coded using CABAC (Context Adaptive Binary Arithmetic Coding), which uses a probability measure to remove statistical redundancy. For multiple CU102 with non-zero quantization conversion coefficients 718, the quantization conversion coefficients 718 can be converted to binary. Each bit ("bin") of the binary representation can then be coded using a context model. The CU102 is divided into three regions, each region having its own set of context models to use for multiple pixels within that region.

[0063] Multiple scans may be performed to encode multiple bins. During the pass to encode the first three bins (bin0, bin1, bin2), an index value indicating which context model should be used for a bin can be determined by calculating the sum of its bin position in up to five adjacent quantization transformation coefficients 718 that were coded before being identified by the template.

[0064] The context model can be based on the probability that a bin value is "0" or "1". Once the values ​​are coded, the context model probabilities can be updated based on the actual number of "0" and "1" values. While HEVC used a fixed table to reinitialize the context model for each new image, JVET allows the context model probabilities for multiple new interpretation images to be initialized based on the context model generated for previously coded interpretation images.

[0065] The encoder may generate a bitstream containing entropy coded bits 722 of multiple residual CU710s, prediction information such as a selected intra-prediction mode or motion vector, an indicator of how multiple CU102s were partitioned from CTU100 according to the QTBT structure, and / or other information about the encoded video. The bitstream can be decoded by a decoder, as described below.

[0066] In addition to using the quantization conversion coefficients 718 to identify the final compressed bit 722, the encoder can also generate multiple reconstructed CU734s by using the quantization conversion coefficients 718 and following the same decoding process that the decoder uses to generate multiple reconstructed CU734s. Thus, once the conversion coefficients are computed and quantized by the encoder, the quantization conversion coefficients 718 can be sent to the decoding loop within the encoder. After quantizing the conversion coefficients of multiple CUs, the decoding loop can cause the encoder to generate the same reconstructed CU734s that the decoder generates in the decoding process. Thus, when performing intra- or inter-prediction of new CU102s, the encoder can use the same reconstructed multiple CU734s that the decoder uses for multiple adjacent CU102s or multiple reference images. The reconstructed multiple CU102s, reconstructed multiple slices, or fully reconstructed frames may serve as references for further prediction stages.

[0067] To obtain multiple pixel values ​​of the reconstructed image, an inverse quantization process may be performed in the decoder's decoding loop (see below for the same operation in the decoder). To inverse quantize a frame, for example, the quantized value of each pixel in the frame is multiplied by a quantization step such as the Qstep described above to obtain the reconstructed inverse quantization conversion coefficient 726. For example, in the decoding process shown in Figure 7 in the encoder, the quantization conversion coefficient 718 of the residual CU710 can be inversely quantized at 724 to obtain the inverse quantization conversion coefficient 726. If the MDNSST operation is performed in encoding, that operation may be reversed after inverse quantization.

[0068] In 728, the inverse quantization transformation coefficient 726 can be used to identify the reconstructed residual CU730, which is obtained by inverse transformation, such as by applying DCT to multiple values ​​to obtain the reconstructed image. In 732, the reconstructed residual CU730 can be added to the corresponding prediction CU702, which is identified in the intra-prediction in 704 or the inter-prediction in 706, in order to identify the reconstructed CU734.

[0069] In 736, one or more filters may be applied to the reconstructed data during the decoding process (in the encoder or decoder, as described below) at either the image level or the CU level. For example, the encoder may apply a deblocking filter, a sample adaptive offset (SAO) filter, and / or an adaptive loop filter (ALF). The encoder's decoding process may implement filters that estimate the optimal filter parameters to address potential artifacts in the reconstructed image and send them to the decoder. Such improvements enhance the objective and subjective quality of the reconstructed video.

[0070] In deblocking filtering, multiple pixels near the sub-CU boundary are corrected, whereas in SAO, multiple pixels within CTU100 can be corrected using either edge offset or band offset classification. JVET's ALF can use circularly symmetric filters for each 2x2 block. The size and identity of the filter used for each 2x2 block can be signaled.

[0071] If the reconstructed images are reference images, in 706 they may be stored in the reference buffer 738 for future interpretation of CU102. In the steps described above, JVET can use content adaptive clipping operations to adjust color values ​​to fit between the upper and lower limits of the clipping boundary. Multiple clipping boundaries can be modified for each slice, and multiple parameters that identify the boundaries can be transmitted via the bitstream.

[0072] Figure 9 shows a simplified block diagram of CU coding in the JVET decoder. The JVET decoder may receive a bitstream containing information about the encoded CU102. The bitstream may show how multiple CU102s of an image were partitioned from a CTU100 according to a QTBT structure. As a non-restrictive example, the bitstream may use quadtree partitioning, symmetric binary partitioning, and / or asymmetric binary partitioning to identify how multiple CU102s were partitioned from each CTU100 in the QTBT. The bitstream may also show prediction information for multiple CU102s, such as intra-predictive modes or motion vectors, and multiple bits 902 representing entropy-encoded residual CUs.

[0073] In 904, the decoder may decode the entropy-encoded bits 902 using the CABAC context model signaled in the bitstream by the encoder. The decoder may update the probabilities of the context model in the same way that they were updated during encoding, using several parameters signaled by the encoder.

[0074] After reversing the entropy coding at 904 to identify the quantization transformation coefficients 906, the decoder may dequantize them at 908 to identify the dequantization transformation coefficients 910. If an MDNSST operation is performed in the coding, that operation may be reversed by the decoder after dequantization.

[0075] In 912, the inversely quantized transformation coefficients 910 can be inversely transformed to identify the reconstructed residual CU914. In 916, the reconstructed residual CU914 can be added to the corresponding prediction CU926 identified in the intra-prediction in 922 or the inter-prediction in 924 to identify the reconstructed CU918.

[0076] In 920, one or more filters can be applied to the reconstructed data, either at the image level or the CU level. For example, the decoder can apply a deblocking filter, a sample adaptive offset (SAO) filter, and / or an adaptive loop filter (ALF). As mentioned above, in-loop filters in the decoder's decoding loop can be used to estimate optimal filter parameters and improve the objective and subjective quality of the frames. In 920, these parameters are sent to the decoder to filter the reconstructed frames to match the filtered reconstructed frames within the decoder.

[0077] After a reconstructed image is generated by identifying multiple reconstructed CU918s and applying multiple signal-transmitted filters, the decoder may output the reconstructed image as an output video 928. If the reconstructed image is used as a reference image, in 924, these may be stored in a reference buffer 930 for future interpretation of CU102.

[0078] Figure 10 shows an embodiment of the method for CU coding 1000 in the JVET decoder. In the embodiment shown in Figure 10, in step 1002, the encoded bitstream 902 is received; in step 1004, the CABAC context model associated with the encoded bitstream 902 is determined; and then, in step 1006, the encoded bitstream 902 is decoded using the determined CABAC context model.

[0079] In step 1008, a plurality of quantization conversion coefficients 906 associated with the encoded bitstream 902 are determined, and in step 1010, an inverse quantization conversion coefficient 910 may be determined from the plurality of quantization conversion coefficients 906.

[0080] In step 1012, it may be determined whether an MDNSST operation was performed during encoding and / or whether bitstream 902 contains an instruction that an MDNST operation was applied to bitstream 902. If it is determined that an MDNSST operation was performed during the encoding process or that bitstream 902 contains an instruction that an MDNSST operation was applied to bitstream 902, then the inverse MDNSST operation 1014 may be performed on bitstream 902 in step 1016 before the inverse transform operation 912 is performed. Alternatively, if no inverse MDNSST operation is applied in step 1014, then the inverse transform operation 912 may be performed on bitstream 902 in step 1016. The inverse transform operation 912 in step 1016 may determine and / or construct the reconstructed residual CU 914.

[0081] In step 1018, the reconstructed residual CU914 from step 1016 may be combined with the prediction CU918. The prediction CU918 may be one of the intra-prediction CU922 determined in step 1020 and the inter-prediction unit 924 determined in step 1022.

[0082] In step 1024, any one or more filters 920 may be applied to the reconfigured CU914 and output in step 1026. In some embodiments, multiple filters 920 may not be applied in step 1024.

[0083] In some embodiments, in step 1028, the reconfigured CU918 may be stored in the reference buffer 930. Figure 11 shows a simplified block diagram 1100 of CU coding in the JVET encoder. In step 1102, the JVET coding tree unit may be represented as the root node of a QTBT (quadtree plus binary tree) structure. In some embodiments, the QTBT may have a quadtree branching from the root node and / or binary trees branching from one or more leaf nodes of the quadtree. The representation from step 1102 can proceed to steps 1104, 1106, or 1108.

[0084] In step 1104, an asymmetric bipartition may be used to divide the represented quadtree node into two blocks of unequal sizes. In some embodiments, the divided blocks may be represented by a binary tree branching from the quadtree node as multiple leaf nodes representing multiple final coding units. In some embodiments, the binary tree branching from the quadtree node as leaf nodes represents final coding units from which further division is not permitted. In some embodiments, the asymmetric bipartition divides the coding unit into multiple blocks of unequal sizes, where the first block represents 25% of the quadtree node and the second block represents 75% of the quadtree node.

[0085] In step 1106, a quadtree partition may be used to divide the represented quadtree note into four square blocks of equal size. In some embodiments, the divided blocks may be represented as quadtree nodes representing multiple final coding units, or as multiple child nodes that are further divided by a quadtree partition, symmetric binary partition, or asymmetric binary partition.

[0086] In step 1108, a quadtree partition may be used to divide the represented quadtree note into two blocks of equal size. In some embodiments, the divided blocks may be represented as quadtree nodes representing multiple final coding units, or as multiple child nodes that are further divided by a quadtree partition, symmetric binary partition, or asymmetric binary partition.

[0087] In step 1110, multiple child nodes from step 1106 or step 1108 may be represented as multiple child nodes configured to be encoded. In some embodiments, multiple child nodes may be represented in JVET by multiple leaf notes of a binary tree.

[0088] In step 1112, multiple coding units from step 1104 or 1110 may be coded using JVET. Figure 12 shows a simplified block diagram 1200 of CU decoding in the JVET decoder. In the embodiment shown in Figure 12, step 1202 may receive a bitstream showing how the coding tree unit has been divided into multiple coding units according to the QTBT structure. The bitstream may show how the quadtree node is divided in at least one of quadtree partitioning, symmetric duplicating, or asymmetric duplicating.

[0089] In step 1204, multiple coding units represented by multiple leaf nodes of the QTBT structure can be identified. In some embodiments, the multiple coding units can indicate whether a node was split from a quadtree leaf node using an asymmetric 2-partition. In some embodiments, the coding units can indicate that a node represents the final coding unit from which it is decoded.

[0090] In step 1206, one or more identified coding units can be decrypted using JVET. Figure 13 shows an alternative simplified block diagram 1300 of JVET coding for intra-mode prediction. In the embodiment shown in Figure 13, in step 1302, a set of MPMs is identified and instantiated in memory; in step 1304, a set of 16 selected modes is identified and instantiated in memory; and in step 1304, a balance of 67 modes may be defined and instantiated in memory. In some embodiments, the set of MPMs may be reduced from a standard set of 6 MPMs. In some embodiments, the set of MPMs may include 5 unique modes, the selected modes may include 16 unique modes, and the set of unselected modes may include the remaining 46 unselected unique modes. However, in alternative embodiments, the set of MPMs may include fewer unique modes, the selected modes may remain fixed at 16 unique modes, and the size of the set of unselected unique modes may be appropriately adjusted to accommodate a total of 67 modes. As a non-restrictive example, in some embodiments where the set of MPMs includes five unique modes instead of six MPMs, truncated unary binarization is used, and if a new binarization for five MPMs is utilized, the number of bins assigned to the MPM modes may be equal to or less than five bins.

[0091] Therefore, in some embodiments, the 16 modes selected from the remaining 62 intra-modes are generated by equally subsampling these 62 intra-modes, each encoded with a fixed-length 4-bit code. As a non-restrictive example, assuming the remaining 62 modes are indexed as {0, 1, 2, ..., 61}, the 16 selected modes are {0, 4, 8, 12, 16, 20, 24, 28, 32, 36, 40, 44, 48, 52, 56, 60}. The remaining 46 unselected modes are {1, 2, 3, 5, 6, 7, 9, 10...59, 61}, and these 46 unselected modes can be encoded with a truncated binary code.

[0092] Figure 14 shows Table 1400 of alternative JVET coding for intra-mode prediction according to Figure 13. In the embodiment shown in Figure 14, the multiple intra-prediction modes 1402 are shown to include 5 MPMs, 16 select modes, and 46 deselect modes, where multiple bin strings 1404 for the MPMs are coded using truncated unary binarization, the 16 select modes are coded using 4 bits of fixed-length code, and the 46 deselect modes may be coded using truncated binary coding.

[0093] In the alternative embodiment shown in Figure 13, six MPMs are available, but only the first five MPMs in the MPM list are binarized and coded in a way that is based on the current context as described in the current JVET, as shown in Figure 14. The sixth MPM in the MPM list is considered one of the 16 selection modes and is coded with a fixed-length 4-bit code along with the other 15 selection modes.

[0094] As a non-restrictive example, if the remaining 61 modes are indexed as {0, 1, 2, ..., 60}, then 15 selected modes can be obtained by equally subsampling the remaining 61 intra modes as follows: The set of 15 selected modes can be {0, 5, 10, 14, 18, 22, 26, 30, 34, 38, 42, 46, 50, 55, 60}, where, in addition to the 15 selected modes, a sixth The MPM is coded in 4 bits of fixed-length code, such as the set {6th MPM, 0, 5, 10, 14, 18, 22, 26, 30, 34, 38, 42, 46, 50, 55, 60}, and the balance of the 46 unselected modes is represented as a set of unselected modes = {1, 2, 3, 4, 6, 7, 8, 9, 11, 12…49, 51, 52, 53, 54, 56, 57, 58, 59}, and is coded in truncated binary code.

[0095] In a further alternative embodiment of Figure 13, only the first five MPMs in the MPM list are binarized and coded in a current context-based manner as described in the current JVET standard, as shown in Figure 14. In such an embodiment, the sixth MPM in the MPM list is considered one of the 16 selected modes and is coded with a fixed-length 4-bit code along with the other 15 selected modes. Thus, the selection of the other 15 selected modes can be established using any known convenient and / or desired selection process. In non-limiting examples, they may be selected with respect to MPM modes, or with respect to statistically well-known (content-based) modes, or with respect to trained or historically well-known modes, or using other methods or processes.

[0096] In this case, as before, the selection of five MPMs is merely an unrestrictive example, and in alternative embodiments, the set of MPMs may be further reduced to four or three MPMs, or expanded to more than six. There are still 16 selected modes, and the balance of 67 (or any other known, convenient, and / or desired total) intracoding modes is included in the set of unselected intracoding modes. That is, embodiments are conceivable in which the total number of intracoding modes is greater than or less than 67, in which the set of MPMs includes any known convenient or desired number of MPMs, and the number of selected modes may be any known convenient and / or desired number.

[0097] The execution of the sequence of instructions necessary to implement multiple embodiments can be performed by a computer system 1500, as shown in Figure 15. In one embodiment, the execution of the sequence of instructions is performed by a single computer system 1500. According to other embodiments, multiple computer systems 1500 connected by a communication link 1515 can cooperate to execute the sequence of instructions. Although only one computer system 1500 is described below, it should be understood that any number of computer systems 1500 can be used to implement multiple embodiments.

[0098] Next, a computer system 1500 according to one embodiment will be described with reference to Figure 15, which is a block diagram of several functional components of the computer system 1500. The term computer system 1500 as used herein is broadly used to describe any computing device that can store and independently execute one or more programs.

[0099] Each computer system 1500 may include a communication interface 1514 connected to a bus 1506. The communication interface 1514 provides bidirectional communication between multiple computer systems 1500. The communication interface 1514 of each computer system 1500 sends and receives electrical, electromagnetic, or optical signals, which include data streams representing various types of signal information, such as commands, messages, and data. A communication link 1515 links one computer system 1500 to another computer system 1500. For example, if the communication link 1515 is a LAN, then the communication interface 1514 is a LAN card; or if the communication link 1515 is a PSTN, then the communication interface 1514 is an Integrated Services Digital Network (ISDN) card or modem; or if the communication link 1515 is the Internet, then the communication interface 1514 may be a dial-up, cable, or wireless modem.

[0100] The computer system 1500 can send and receive messages, data, and instructions, including programs, i.e., applications, and code, via its corresponding communication links 1515 and communication interfaces 1514. The received program code is executed by each receiving processor 1507 and / or stored in a storage device 1510 or other relevant non-volatile medium for later execution.

[0101] In one embodiment, the computer system 1500 operates in conjunction with a data storage system 1531, for example, a data storage system 1532 that is easily accessible by the computer system 1500. The computer system 1500 communicates with the data storage system 1531 via a data interface 1533. The data interface 1533, connected to a bus 1506, transmits and receives electrical, electromagnetic, or optical signals, including data streams representing various types of signal information, such as commands, messages, and data. In several embodiments, the functions of the data interface 1533 may be performed by a communication interface 1514.

[0102] The computer system 1500 includes a bus 1506 or other communication mechanism for communicating instructions, messages, and data, and information collectively, and one or more processors 1507 connected to the bus 1506 for processing the information. The computer system 1500 also includes main memory 1508, such as random access memory (RAM) or other dynamic storage device connected to the bus 1506, for storing dynamic data and instructions executed by one or more processors 1507. The main memory 1508 may also be used to store temporary data, i.e., variables, or other intermediate information during the execution of instructions by one or more processors 1507.

[0103] The computer system 1500 may further include read-only memory (ROM) 1509 or other static storage device connected to a bus 1506 for storing static data and instructions for one or more processors 1507. A storage device 1510, such as a magnetic disk or optical disk, may also be provided and connected to the bus 1506 for storing data and instructions for one or more processors 1507.

[0104] The computer system 1500 may, but is not limited to, be connected to a display device 1511, such as a cathode ray tube (CRT) or liquid crystal display (LCD) monitor, via bus 1506 to display information to the user. Input devices 1512, such as alphanumeric and other keys, are connected to bus 1506 to communicate information and command selections to the processor 1507.

[0105] According to one embodiment, individual computer systems 1500 perform specific operations by corresponding one or more processors 1507 that execute one or more sequences of one or more instructions contained in main memory 1508. Such instructions may be read into main memory 1508 from another computer-available medium, such as ROM 1509 or storage device 1510. By executing the sequence of instructions contained in main memory 1508, one or more processors 1507 perform the processes described herein. In alternative embodiments, hardwired circuitry may be used instead of or in combination with software instructions. Thus, the multiple embodiments are not limited to any particular combination of hardware circuitry and / or software.

[0106] As used herein, the term “computer-usable medium” refers to any medium that provides information or is usable by one or more processors 1507. Such mediums can take many forms, including but not limited to non-volatile mediums, volatile mediums, and transmission mediums. Non-volatile mediums, i.e., mediums that can retain information without power, include ROM 1509, CD-ROMs, magnetic tapes, and magnetic disks. Volatile mediums, i.e., mediums that cannot retain information without power, include main memory 1508. Transmission mediums include coaxial cables, copper wires, and optical fibers, including the wires that make up the bus 1506. Transmission mediums can also take the form of carrier waves, i.e., electromagnetic waves that are modulated in frequency, amplitude, or phase to transmit information signals. Furthermore, transmission mediums can take the form of sound waves or light waves, such as those generated during radio waves or infrared data communications.

[0107] In the aforementioned specification, several embodiments were described with reference to their specific components. However, it is clear that various modifications and changes can be made without departing from the broader spirit and scope of the embodiments. For example, the specific ordering and combination of several process actions shown in the process flow diagrams described herein are merely illustrative, and it should be understood that embodiments can be carried out using different or additional process actions, or using different combinations or orderings of several process actions. Accordingly, this specification and the drawings should be considered illustrative rather than restrictive.

[0108] Furthermore, it should be noted that the present invention can be implemented in various computer systems. The various techniques described herein may be embodied in hardware, software, or a combination of both. Preferably, these techniques are embodied in programmable computer programs that run on multiple computers, each including a processor, a processor-readable storage medium (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device. The program code is applied to data input using the input device, performs the functions described above, and generates output information. The output information is applied to one or more output devices. Each program is preferably implemented in a schematic procedural or object-oriented programming language for communication with the computer system. However, the program may be embodied in assembly language or machine language as needed. In any case, the language may be a compiled language or an interpreted language. Each such computer program is preferably stored in a general-purpose or special-purpose programmable computer-readable storage medium or device (e.g., ROM or magnetic disk) for configuring and operating the computer when the storage medium or device is read by the computer to perform the procedures described above. Furthermore, the system is considered to embody a computer-readable storage medium configured to hold computer programs, and such a storage medium causes the computer to operate in a specific predetermined manner. In addition, the storage element of the exemplary computing application can be a relational or sequential (flat file) type computing database capable of storing data in various combinations and configurations.

[0109] Figure 16 is a schematic diagram of a source device 1612 and a destination device 1610 incorporating the features of the system and devices described herein. As shown in Figure 16, the exemplary video coding system 1610 includes a source device 1612 and a destination device 1616, in this example the source device 1612 generates encoded video data. Therefore, the source device 1612 may be referred to as a video coding device. The destination device 1616 can decode the encoded video data generated by the source device 1612. Therefore, the destination device 1616 may be referred to as a video decoding device. The source device 1612 and the destination device 1616 may be examples of video coding devices.

[0110] The destination device 1616 may receive encoded video data from the source device 1612 via channel 1616. Channel 1616 may comprise a medium or device of a type that can transfer encoded video data from the source device 1612 to the destination device 1616. In one example, channel 1616 may comprise a communication medium that enables the source device 1612 to directly transmit encoded video data to the destination device 1616 in real time.

[0111] In this example, the source device 1612 may modulate video data encoded according to a communication standard such as a wireless communication protocol and transmit the modulated video data to the destination device 1616. The communication medium may include a wireless or wired communication medium such as a radio frequency (RF) spectrum or one or more physical transmission lines. The communication medium may form part of a packet-based network such as a local area network, a wide area network, or a global network such as the Internet. The communication medium may include routers, switches, base stations, or other devices that enable communication from the source device 1612 to the destination device 1616. In another example, channel 1616 may correspond to a storage medium that stores encoded video data generated by the source device 1612.

[0112] In the example in Figure 16, the source device 1612 includes a video source 1618, a video encoder 1620, and an output interface 1622. In some cases, the output interface 1628 may include a modulator / demodulator (modem) and / or transmitter. In the source device 1612, the video source 1618 may include sources such as a video capture device like a video camera, a video archive containing previously captured video data, a video feed interface for receiving video data from a video content provider, and / or a computer graphics system for generating video data, or a combination of such sources.

[0113] The video encoder 1620 may encode captured, pre-captured, or computer-generated video data. An input image may be received by the video encoder 1620 and stored in the input frame memory 1621. The general-purpose processor 1623 may load the information from there and perform encoding. A program for driving the general-purpose processor may be loaded from a storage device, such as the exemplary memory module shown in Figure 16. The general-purpose processor performs encoding using the processing memory 1622, and the output of the encoded information by the general-purpose processor may be stored in a buffer, such as the output buffer 1626.

[0114] The video encoder 1620 may include a resampling module 1625 configured to encode (e.g., encode) video data using a scalable video coding scheme that defines at least one base layer and at least one enhancement layer. The resampling module 1625 may resample at least some video data as part of the encoding process, and the resampling may be performed adaptively using a resampling filter.

[0115] Encoded video data, for example, an encoded bitstream, may be transmitted directly to the destination device 1616 via the output interface 1628 of the source device 1612. In the example in Figure 16, the destination device 1616 includes an input interface 1638, a video decoder 1630, and a display device 1632. In some cases, the input interface 1628 may include a receiver and / or modem. The input interface 1638 of the destination device 1616 receives the encoded video data via channel 1616. The encoded video data may include various syntax elements generated by the video encoder 1620 representing the video data. Such syntax elements may be included in the encoded video data transmitted on a communication medium, stored on a storage medium, or stored on a file server.

[0116] The encoded video data may also be stored in a storage medium or file server for later access by the destination device 1616 for decoding and / or playback. For example, the encoded bitstream may be temporarily stored in the input buffer 1631 and then loaded into the general-purpose processor 1633. A program for driving the general-purpose processor may be loaded from storage or memory. The general-purpose processor may perform decoding using process memory 1632. The video decoder 1630 may also include a resampling module 1635 similar to the resampling module 1625 used in the video encoder 1620.

[0117] Although Figure 16 shows the resampling module 1635 separately from the general-purpose processor 1633, it will be understood by those skilled in the art that the resampling function is performed by a program executed by the general-purpose processor, and the processing in the video encoder is achieved using one or more processors. One or more decoded images may be stored in the output frame buffer 1636 and then transmitted to the input interface 1638.

[0118] The display device 1638 may be integrated with the destination device 1616 or be external. In some examples, the destination device 1616 may include an integrated display device or be configured to interface with an external display device. In other examples, the destination device 1616 may be a display device. Generally, the display device 1638 displays the decoded video data to the user.

[0119] The video encoder 1620 and video decoder 1630 can operate according to video compression standards. ITU-T VCEG (Q6 / 16) and ISO / IEC MPEG (JTC 1 / SC 29 / WG 11) are currently studying the potential need for standardization of future video coding techniques that have compression capabilities far exceeding those of the current high-efficiency video coding HEVC standard (including its current and near-future extensions for screen content coding and high dynamic range coding). These groups are collaborating in this research activity, a joint collaboration known as JVET (Joint Video Exploration Team), to evaluate compression technique designs proposed by their experts in this field. A recent record of JVET development is found in "Algorithm Description of Joint Exploration Test Model 5 (JEM 5)" by J. Chen, E. Alshina, G. Sullivan, J. Ohm, and J. Boyce, JVET-E1001-V2.

[0120] Additionally or alternatively, video encoder 1620 and video decoder 1630 may operate in accordance with other proprietary or industry standards that work in conjunction with the disclosed JVET features. That is, other standards include the ITU-T H.264 standard, alternatively MPEG-4, Part 10, Advanced Video Coding (AVC), or extensions of such standards. Therefore, the techniques of this disclosure, while newly developed for JVET, are not limited to any specific coding standard or technique. Other examples of video compression standards and techniques include MPEG-2, ITU-T H.263, and proprietary or open-source compression formats and related formats.

[0121] The video encoder 1620 and video decoder 1630 can be embodied in hardware, software, firmware, or any combination thereof. For example, the video encoder 1620 and decoder 1630 can use one or more processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic, or any combination thereof.

[0122] If the video encoder 1620 and decoder 1630 are partially embodied in software, the device may store a number of software instructions in a suitable, non-temporary, computer-readable storage medium and execute the number of instructions in hardware using one or more processors to perform the techniques of the present disclosure. Each of the video encoder 1620 and video decoder 1630 may be included in one or more encoders or decoders, any of which may be integrated as part of a composite encoder / decoder (codec) within the respective device.

[0123] The aspects of the subject matter described herein may be described in the overall context of multiple computer-executable instructions, such as program modules, which are executed by computers like the general-purpose processors 1623 and 1633 described above. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a particular task or implement a particular abstract data type. The aspects of the subject matter described herein may also be implemented in a distributed computing environment in which tasks are executed by remote processing units linked over a communication network. In a distributed computing environment, multiple program modules may reside on both local and remote computer storage media, including memory storage devices.

[0124] Multiple examples of memory include random access memory (RAM), read-only memory (ROM), or both. Memory may store multiple instructions, such as source code or binary code, for performing the techniques described above. Memory may also be used to store variables or other intermediate information during the execution of multiple instructions by a processor such as processors 1623 and 1633.

[0125] The storage device may store multiple instructions, such as source code or binary code, for performing the techniques described above. The storage device may further store data used and manipulated by the computer processor. For example, the storage device in the video encoder 1620 or video decoder 1630 may be a database accessed by the computer system 1623 or 1633.

[0126] Other examples of storage devices include random access memory (RAM), read-only memory (ROM), hard drives, magnetic disks, optical disks, CD-ROMs, DVDs, flash memory, USB memory cards, or any other media that a computer can read.

[0127] Memory or storage devices may be examples of non-temporary computer-readable storage media for use by or in connection with video encoders and / or decoders. A non-temporary computer-readable storage medium includes a plurality of instructions for controlling a computer system configured to perform a plurality of functions described in a particular embodiment. The plurality of instructions may be configured, when executed by one or more computer processors, to perform what is described in a particular embodiment.

[0128] Furthermore, it should be noted that some embodiments are described as processes shown as flowcharts or block diagrams. Although each is described as a sequential process of multiple operations, many of these operations can be performed in parallel or simultaneously. In addition, the order of the operations can be rearranged. The process may have additional steps not included in the diagrams.

[0129] Certain embodiments may be implemented on a non-temporary computer-readable storage medium for use by or in connection with an instruction execution system, apparatus, system, or machine. The computer-readable storage medium contains instructions for controlling a computer system to perform the method, as described by a particular embodiment. The computer system may include one or more computing devices. When executed by one or more computer processors, the instructions may be configured to perform those described in a particular embodiment.

[0130] In this description and throughout the following claims, unless the context explicitly indicates otherwise, “a, an” and “the” include plural references. Also, in this description and throughout the following claims, unless the context explicitly indicates otherwise, the meaning of “in” includes “in” and “on.”

[0131] While exemplary embodiments of the present invention have been described in detail and language specific to the structural features and / or methodological operations described above, those skilled in the art will readily understand that many additional modifications are possible in exemplary embodiments without substantially departing from the novel teachings and merits of the present invention. Furthermore, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or operations described above. Accordingly, these and all such modifications are intended to fall within the scope of the present invention, as interpreted in accordance with the breadth and scope of the appended claims.

Claims

1. A method for decoding video data from a bitstream, (a) receiving the bitstream indicating how the coding tree unit has been divided into a plurality of coding units, wherein the plurality of coding units are rectangular, (b) A step of determining a first set of most likely modes (MPMs) for the current block of the video data, wherein the first set of MPMs is selectable based on an MPM index, one of the first set of MPMs selectable based on the MPM index includes a true horizontal mode, another of the first set of MPMs selectable based on the MPM index includes a true vertical mode, another of the first set of MPMs selectable based on the MPM index includes an angular mode, and the first set of MPMs includes only five different modes. (c) From the bitstream, (i) the MPM flag containing a total of 1 bit and (ii) another bit The steps include deriving an index, where at least one of the MPM flags and another index indicates whether the intra-mode for predicting the current block is one of the first set of MPMs, (d) If at least one of the MPM flag and the other index is used to indicate that the intra-mode for predicting the current block is one of the first set of MPMs that can be selected based on the MPM index, the steps of selecting the intra-mode for the current block based on the MPM index decoded from the bitstream of one of the first set of MPMs, (e) If at least one of the MPM flag and the other index indicates that the intra mode for predicting the current block is not one of the first set of MPMs, then the MPM flag and the other index are used to (i) determine at least one of a second set of modes, and (ii) determine at least one of a third set of modes, (f) Here, the first set, the second set, and the third set include different modes, and the combination of the first set, the second set, and the third set includes 67 different modes, (g) determining the intra-mode of the current block for at least one of the second set of modes based on a first combination of the MPM flag and the other index that does not include any of the first set of MPMs that are selectable based on the MPM index included in the first set of MPMs, (h) A method comprising the step of determining the intra-mode of the current block for at least one of the third set of modes based on a second combination of the MPM flag and the other index, which does not include any of the first set of MPMs that are selectable based on the MPM index included in the first set of MPMs.

2. A program that, when executed by a decoder, causes the decoder to perform each step of the method for decoding video data from a bitstream as described in claim 1.

3. A method for encoding video data using an encoder, (a) supplying a bitstream indicating how a coding tree unit is divided into a plurality of coding units, wherein the plurality of coding units are rectangular, (b) The bitstream includes data suitable for determining a first set of most likely modes (MPMs) for the current block of the video data, wherein the first set of MPMs is selectable based on an MPM index, one of the first set of MPMs selectable based on the MPM index includes a true horizontal mode, another of the first set of MPMs selectable based on the MPM index includes a true vertical mode, another of the first set of MPMs selectable based on the MPM index includes an angular mode, and the first set of MPMs includes only five different modes. (c) The bitstream contains data suitable for deriving from the bitstream an MPM flag and another index, which together comprise one bit, where at least one of the MPM flag and the other index indicates whether the intra-mode for predicting the current block is one of the first set of MPMs. (d) The bitstream includes data suitable for selecting the intra-mode of the current block based on the MPM index decoded from the bitstream of one of the first set of MPMs, where at least one of the MPM flags and the other index is used to indicate that the intra-mode for predicting the current block is one of the first set of MPMs which can be selected based on the MPM index, (e) The bitstream includes, if at least one of the MPM flag and the other index indicates that the intra-mode for predicting the current block is not one of the first set of MPMs, the MPM flag and the other index, the steps of (i) determining at least one of a second set of modes, and (ii) data suitable for determining at least one of a third set of modes, (f) Here, the first set, the second set, and the third set include different modes, and the combination of the first set, the second set, and the third set includes 67 different modes, (g) The bitstream includes data suitable for determining the intra-mode of the current block for at least one of a second set of modes based on a first combination of the MPM flags and the other index, which does not include any of the first set of MPMs that are selectable based on the MPM index included in the first set of MPMs, (h) A method wherein the bitstream includes data suitable for determining the intra-mode of the current block for at least one of a third set of modes based on a second combination of the MPM flags and the other index, which does not include any of the first set of MPMs that are selectable based on the MPM index included in the first set of MPMs.

Images