Intra prediction mode extension

Abstract

A method and apparatus for performing intra prediction for encoding or decoding adds multiple prediction modes to the prediction modes of an existing intra prediction scheme. A horizontal mode, a vertical mode, and a diagonal mode are added. The reference pixels for each of the added modes can come from a number of pixel locations away from the block being encoded or decoded. The reference pixels to the left and above the block being reconstructed from already encoded pixels. The reference pixels from the right and below the block are estimated or extrapolated and can be based on other pixels around the block. An index indicating which prediction was used can be sent in the bitstream.

Description

Technical Field

[0001] At least one embodiment of the present invention relates to a method or apparatus for video encoding or decoding. Background Technology

[0002] To achieve high compression efficiency, image and video decoding schemes typically employ prediction (including spatial and / or motion vector prediction) and transform to utilize spatial and temporal redundancy in the video content. Intra-frame or inter-frame prediction is usually used to leverage intra-frame or inter-frame correlations, followed by transform, quantization, and entropy decoding of the difference between the original and predicted images, typically represented as prediction error or prediction residuals. To reconstruct the video, the compressed data is decoded through inverse processing corresponding to entropy decoding, quantization, transform, and prediction. Summary of the Invention

[0003] The shortcomings and disadvantages of the prior art can be addressed by the main aspects described in this paper, which involve block shape adaptive intra-prediction direction in encoding and decoding.

[0004] According to a first aspect, a method is provided. The method includes the steps of: determining an extended intra-prediction mode to be used for encoding a video data block; using the determined extended intra-prediction mode to predict a target pixel of the video data block based on one or more pixels adjacent to the block, wherein the prediction includes: horizontally predicting the target pixel using reference pixels of the left adjacent row and a reference pixel of the upper right of the block, or vertically predicting the target pixel using reference pixels of the upper adjacent row and a reference pixel of the upper left of the block, or diagonally predicting the target pixel using reference pixels above and to the upper right of the block or reference pixels to the left and to the lower left of the block; and using the prediction to encode the video block of video data.

[0005] According to a second aspect, a method is provided. The method includes the steps of: determining an extended intra-frame prediction mode to be used for decoding a block of video data; using the determined extended intra-frame prediction mode to predict a target pixel of the video data block based on one or more pixels adjacent to the block, wherein the prediction includes: horizontally predicting the target pixel using reference pixels of the left adjacent row and a reference pixel of the upper right of the block, or vertically predicting the target pixel using reference pixels of the upper adjacent row and a reference pixel of the upper left of the block, or diagonally predicting the target pixel using reference pixels above and to the upper right of the block or reference pixels to the left and to the lower left of the block; and using the prediction to decode the block of video data.

[0006] According to another aspect, an apparatus is provided. The apparatus includes a processor. The processor can be configured to encode blocks of video or decode bitstreams by performing any of the methods described above.

[0007] According to another key aspect of at least one embodiment, an apparatus is provided comprising means according to any of the decoding embodiments; and at least one of: (i) an antenna configured to receive a signal including the video block, (ii) a band limiter configured to limit the received signal to a band including the video block, or (iii) a display configured to display an output representing the video block.

[0008] According to another key aspect of at least one embodiment, a non-transitory computer-readable medium is provided, which contains data content formed according to any of the described encoding embodiments or variations.

[0009] According to another key aspect of at least one embodiment, a signal comprising video data generated according to any of the described encoding embodiments or variations is provided.

[0010] According to another key aspect of at least one embodiment, the bitstream is formatted to include data content generated according to any of the described encoding embodiments or variations.

[0011] According to another key aspect of at least one embodiment, a computer program product including instructions is provided that, when a computer executes the program, cause the computer to perform any of the described decoding embodiments or variations.

[0012] These and other aspects, the features and advantages of the main aspects, will become apparent from the following detailed description of exemplary embodiments, which are read in conjunction with the accompanying drawings. Attached Figure Description

[0013] Figure 1 A reference sample for intra-frame prediction in VTM is shown.

[0014] Figure 2 Multiple reference lines are shown for intra-frame prediction in VTM.

[0015] Figure 3 The intra-prediction direction in the VTM is shown for a square target block.

[0016] Figure 4 The planar_horizontal (plane_horizontal) prediction is shown under the main aspects described.

[0017] Figure 5The planar_vertical (plane_vertical) prediction is shown under the main aspects described.

[0018] Figure 6 The planar_diagonal prediction mode under the main aspects described is shown.

[0019] Figure 7 The top and left decoding unit (CU) positions are shown for deriving the list of most probable modes (MPMs) from previous proposals for different target block shapes.

[0020] Figure 8 This is a flowchart illustrating encoding using the prediction pattern of Example 1.

[0021] Figure 9 This is a flowchart illustrating the decoding using the prediction pattern of Example 1.

[0022] Figure 10 An embodiment of the encoding method based on the described main aspects is shown.

[0023] Figure 11 An embodiment of the decoding method based on the main aspects described is shown.

[0024] Figure 12 An embodiment of an apparatus for encoding or decoding using intra-frame prediction mode extension is shown.

[0025] Figure 13 This illustrates a general standard encoding scheme.

[0026] Figure 14 This illustrates a typical standard decoding scheme.

[0027] Figure 15 A typical processor arrangement in which the described embodiments can be implemented is shown. Detailed Implementation

[0028] The embodiments described herein belong to the field of video compression and relate to video compression, as well as video encoding and decoding. In the Universal Video Decoding (VVC) Test Model (VTM), any target block in intra-frame prediction can have one of 67 prediction modes. Similar to HEVC, one is a planar mode, one is a DC mode, and the remaining 65 are directional modes. These 65 directional modes are selected from 95 directions, which includes 65 regular angles spanning from 45 degrees to -135 degrees if the target block is square, and may include 28 wide-angle directions when the target block is rectangular. VTM encodes the prediction modes of the block using a set of most probable modes (MPMs) consisting of 6 prediction modes. If a prediction mode does not belong to the MPM set, it is truncated into binary encoding with 5 or 6 bits.

[0029] The purpose of this invention is to improve prediction efficiency in intra-frame prediction. It proposes three additional prediction modes for any target block, in addition to the existing 67 modes. By including these modes, the MPM set can be modified based on whether the prediction modes of the left and top adjacent blocks are directional or non-directional, or whether they have newly introduced prediction modes. If the prediction mode of the current block belongs to the MPM set, it is encoded using a variable-length decoding scheme in the VTM 3.0 software; otherwise, it is decoded using a fixed-length six-bit scheme instead of the truncated binary encoding scheme in VTM 3.0. The proposed modes are inherently very general and are not limited to use only in VTM, but can be included in any future video decoding standard.

[0030] In this invention, three additional modes are proposed to model slowly varying intensity regions along the horizontal, vertical, and diagonal directions. These modes are referred to as Planar_horizontal, Planar_vertical, and Planar_diagonal modes, respectively. Including these three modes increases the total number of intra-prediction modes used by the target block to 70. With the introduction of these three new prediction modes, the remaining set of prediction modes will contain 64 prediction modes. Therefore, any prediction mode not belonging to the MPM set will simply be encoded with a fixed length of 6 bits.

[0031] Intra-frame prediction in video compression refers to spatial prediction of pixel blocks using information from causally adjacent blocks (i.e., neighboring blocks that have already been decoded within the same frame). This is a powerful decoding tool because it allows for high compression efficiency in INTRA frames, as well as high compression efficiency in INTER frames, provided there is no better temporal prediction. Therefore, intra-frame prediction has been included as a core decoding tool in all video compression standards, including H.264 / AVC, HEVC, etc. In the following text, for illustrative purposes, we will refer to intra-frame prediction in the Universal Video Decoding (VVC) Software Test Model (VTM).

[0032] In VTM, the encoding of frames in a video sequence is based on a quadtree (QT) / multi-type tree (MTT) block structure. A frame is divided into non-overlapping square decoder tree units (CTUs), each of which undergoes QT / MTT-based segmentation to become multiple decoder units (CUs) based on a rate distortion criterion. In intra-frame prediction, CUs are spatially predicted from causally adjacent CUs (i.e., the top and left CUs). For this purpose, VTM uses a simple spatial model called prediction modes. Based on the decoded pixel values ​​called reference pixels in the top and left CUs, the encoder constructs different predictions for the target block and selects the one that results in the best RD performance. Of the 95 defined modes, one is a planar mode (indexed as mode 0), one is a DC mode (indexed as mode 1), and the remaining 93 (indexed as mode -14…-1,2…80) are angular modes. Of the 93 angular modes, for any target CU, only 65 adjacent modes are selected based on its shape. Angular modes are designed to model the directional structure of objects in a frame. Therefore, the decoded pixel values ​​in the top and left CUs are repeated only along the defined directions to fill the target CU. Some prediction modes can cause discontinuities along the top and left reference boundaries, so those prediction modes include subsequent post-processing called Position-Related Intra-Prediction Combination (PDPC), which aims to smooth the pixel values ​​along those boundaries.

[0033] The defined prediction modes model slowly varying intensity regions and object orientation well; however, when the object intensity varies slowly along certain directions, the corresponding directional mode or one of two non-directional modes will be selected. To better model such regions, we propose three directional plane modes to be used in conjunction with the existing 67 modes. Before describing the proposed modes, intra-frame prediction in VTM 3.0 is briefly introduced below. For ease of reference, we will use the terms "CU" and "block" interchangeably in this paper.

[0034] The intra-frame prediction process in VTM consists of three steps: (1) reference sample generation, (2) intra-frame sample prediction, and (3) post-processing of predicted samples. Figure 1The diagram illustrates the reference sample generation process, showing reference samples used for intra-frame prediction in VTM. The reference pixel value at coordinates (x, y) is represented by R(x, y) in the diagram. H and W represent the height and width of the current block, respectively. For a CU of size H x W, a row of 2W decoded samples is formed at the top from the top and top-right pixels of the previously reconstructed current CU. Similarly, a column of 2H samples is formed on the left from the reconstructed left and bottom-left pixels. The corner pixels at the top-left position are also used to fill the gap between the top row and the left column reference. If some of the top or left samples are unavailable because the corresponding CU is not in the same slice, or the current CU is at a frame boundary, etc., then a method called reference sample replacement is performed, in which the missing samples are copied from the available samples in a clockwise direction. The reference samples are then filtered using a low-pass filter with coefficients [1 / 4, 1 / 2, 1 / 4], depending on the current CU size and prediction mode, which is applied vertically to the left reference column and horizontally to the top reference row.

[0035] VTM 3.0 also supports intra-frame prediction with multiple reference lines (MRLs). Its idea is based on... Figure 2 Several sets of reference lines are shown for prediction, and then the reference line that gives the best rate distortion performance is selected. The reference line used is signaled to the decoder in a variable-length code. Figure 2 Multiple reference lines are shown for intra-frame prediction in VTM. These reference lines are indexed in order of their distance from the target block. The reference line closest to the target block is indexed as 0. For example, the other two reference lines used for prediction are indexed as 1 and 3. For example, ... Figure 2 The three reference lines, used in VTM 3.0, are signaled with bits "0", "10", and "11" respectively for indices 0, 1, and 3. Different reference lines can be used. Furthermore, to limit the increased complexity of searching for the optimal prediction pattern, reference lines 1 and 3 are tested using only six-angle prediction patterns derived in the same way as the patterns in the MPM list, but excluding the PLANA and DC patterns. In the following text, for easier understanding, we will limit our description to only the first reference line, as... Figure 1 As shown, the use of multiple reference lines is obvious and can be readily inferred from the following description. As in VTM 3.0, the use of multiple reference lines still applies, and the use of a first reference line to apply a new pattern is never prohibited. This is obvious and can be readily inferred from the following description.

[0036] The next step (i.e., in-sample prediction) involves predicting the pixels of the target CU based on a reference sample. As mentioned earlier, VTM supports a range of prediction models to effectively predict different types of content. Planar and DC prediction modes are used to predict smooth and gradually changing regions, while angular prediction modes are used to capture different oriented structures. VTM supports 95 oriented prediction modes, indexed from -14 to -1 and from 2 to 80. For square CUs, only prediction modes 2-66 are used. These prediction modes correspond to different prediction directions from 45 degrees to -135 degrees in a clockwise direction, such as... Figure 3 The diagram illustrates the intra-prediction direction in the VTM for a square target block. Typically, non-square blocks can also be used with extended prediction directions, but this diagram shows a square block. The numbers represent the prediction mode indices associated with the corresponding directions. Modes 2 through 33 indicate horizontal prediction, while modes 34 through 66 indicate vertical prediction.

[0037] Patterns with indices -14 to -1 and indices 67 to 80 are wide-angle patterns used for rectangular blocks of different shapes. Patterns -14 to -1 are defined as exceeding pattern 2 (exceeding an angle of 45 degrees) and are used for tall rectangular blocks (blocks whose height is greater than their width). Similarly, patterns 67 to 80 are defined as exceeding pattern 66 (exceeding an angle of -135 degrees) and are used for flat rectangular blocks (blocks whose width is greater than their height). The number of wide-angle patterns used for rectangular blocks depends on the aspect ratio of the block. In any case, the total number of angle patterns used for any block is 65, and these patterns are always continuous in direction. The angle prediction patterns used for different block shapes are summarized in Table 1.

[0038]

[0039] Table 1: Range of intra-prediction modes for different target block shapes in VTM. W / H refers to the ratio of block width to height.

[0040] To define the angle corresponding to any prediction mode, VTM specifies the angle parameter A, or intraPredAngle, which is given as the offset of the predictor reference sample position relative to the target pixel at (0, 0). This offset is expressed as a multiple of (1 / 32) of the sampling interval. In other words, the prediction direction has a resolution of (1 / 32) of the sampling interval. In VTM 3.0, any predictor reference sample within this sampling interval is computed as a linear interpolation of the two nearest reference samples. Except for the wide angle, the angle parameter values ​​have been slightly modified from those specified in HEVC to accommodate different rectangular block shapes, such that the prediction mode for any target block is aligned with its vertical diagonal in a clockwise direction. Therefore, the intra-frame prediction direction adapts to the target block shape. Table 2 shows the intraPredAngle values ​​for the angle prediction mode of a square CU.

[0041]

[0042]

[0043] Table 2: Angle parameters A or intraPredAngle for the angle prediction mode of the square CU in VTM 3.0

[0044] The angular prediction modes can describe image regions containing object structures with different orientations. PLANAR and DC modes describe constant and gradually changing regions without any specific orientation. If any region has slowly changing areas along certain directions, it cannot be adequately described by the corresponding angular mode, or either the PLANAR or DC mode. We propose three simple modes to describe such regions. Using these three modes, if the prediction mode is not in the MPM list, encoding the prediction mode will simply require 6 bits instead of the existing truncated binary encoding.

[0045] For generality, we will assume in the following text that the rectangular block has a width W and a height H. The square target block is a special case where W = H. The following three prediction modes are proposed for use with the other sixty-seven modes in the normal way of the top reference row and left reference column in HEVC. In the case of multi-reference line intra-frame prediction, such as in VTM 3.0, it is proposed to use them only with the first reference line.

[0046] The PLANAR mode (indexed as mode 0) consists of two interpolated averages: a horizontal interpolation between the left reference sample and the upper-right reference sample (i.e., the reference sample at (W, -1)), and a vertical interpolation between the top reference sample and the bottom-left reference sample (i.e., the reference sample at (-1, H)). This prediction is followed by PDPC to smooth the boundary samples. In the planar_horizontal mode, we only use the horizontal interpolation and then simply filter the first row of samples. Similarly, in the planar_vertical mode, we only use the vertical interpolation and then simply filter the first column of samples.

[0047] In the planar_horizontal mode, using the upper-right reference sample R(W, -1) and the reference samples R(-1, y) on the left side of the target block, 0 <= y < H, the linear interpolation is performed as follows:

[0048] P h (x, y) = (W - 1 - x) * R(-1, y) + (1 + x) * R(W, -1), 0 ≤ x < W, 0 ≤ y < H

[0049] P(x, y) = (P h (x, y) + W / 2) >> log2(W), 0 ≤ x < W, 0 ≤ y < H

[0050] The planar_horizontal mode is shown in Figure 4 below.

[0051] Then, the samples on the first row are smoothed using the filter [1, 2, 1] as:

[0052] P(x, 0) = (2 * P(x, 0) + P(x, 1) + R(x, -1)) >> 2, 0 ≤ x < W

[0053] In the planar_vertical mode, using the bottom_left reference sample R(-1, H) and the reference samples R(x, -1) on the top of the target block, 0 <= x < W, the linear interpolation is performed as:

[0054] P v (x, y) = (H - 1 - y) * R(x, -1) + (1 + y) * R(-1, H), 0 ≤ x < W, 0 ≤ y < H;

[0055] P(x, y) = (P v (x, y) + H / 2) >> log2(H), 0 ≤ x < W, 0 ≤ y < H

[0056] planar_vertical mode in Figure 5 As shown in the image.

[0057] Then, the samples in the first column are smoothed using the filter [1, 2, 1] as follows:

[0058] P(0,y)=(2*P(0,y)+P(1,y)+R(-1,y))>>2,0≤y <H

[0059] In both prediction methods, we can optionally replace the simple smoothing filter with a more advanced filter, such as the filter used in PDPC for pure horizontal and pure vertical prediction modes (modes 18 and 50).

[0060] The planar diagonal mode is a variation of the diagonal mode (mode 34). Instead of repeating reference samples along the diagonal direction at the top and left, we linearly interpolate the samples using reference samples at the top and left of the block, as well as "estimated" reference samples at the right and bottom, as in the PLANA mode. This latter reference sample is first calculated using linear interpolation between the top_right and bottom_left reference samples and the estimated bottom_right sample. The bottom_right sample is calculated as the average of the reference samples at (2W-1, -1) and (-1, 2H-1):

[0061] First, we estimate the bottom_right sample as:

[0062] P(W,H)=(R(-1,2H-1)+R(2W-1,-1)+1>>1

[0063] Alternatively, we can use top_right and bottom_left reference samples to estimate the bottom_right sample, as described below, but this has higher complexity:

[0064] P(W,H)=(H*R(-1,H)+W*R(W,-1)+(W+H) / 2 / (W+H)

[0065] Then, using the bottom_left reference sample R(-1, H) and the bottom_right sample, we linearly interpolate the sample at the bottom of the target block:

[0066] P b (x,H)=(W-1-x)*R(-1,H)+(1+x)*P(W,H),0≤x <W

[0067] P b (x,H)=(P b(x,H)+W / 2)>>log2(W),0≤x <W

[0068] Similarly, using the top_right reference sample R(W, -1) and the bottom_right sample, we linearly interpolate the samples to the right of the target block:

[0069] P r (W,y)=(H-1-y)*R(W,-1)+(1+y)*P(W,H),0≤y <H;

[0070] P r (W,y)=(P r (W,y)+H / 2)>>log2(H),0≤y <H;

[0071] Using these samples, we constructed an array of size 1+W+H, called RightColumn, as follows:

[0072] rightColumn[k] = P r (W,k),0≤k <H;

[0073] rightColumn[H] = P(W,H);

[0074] rightColumn[H+k]=P b (Wk,H),1≤k≤W

[0075] Similarly, using the reference samples from the top and left, we construct another array of size 1 + W + H, called LeftColumn, as follows:

[0076] leftColumn[k]=R(W-1-k,-1),0≤k <W;

[0077] leftColumn[W] = R(-1, -1);

[0078] leftColumn[W+k]=R(-1,k-1),1≤k≤H

[0079] Then, we calculate the predicted sample values ​​by linear interpolation between LeftColumn and RightColumn, as follows:

[0080] P(x,y)=(L-1-p)*leftColumn[k]+(1+p)*rightColumn[k],0≤x <W,0≤y<H;

[0081] P(x,y)=(P(x,y)+L / 2)>>log2(L),0≤x <W,0≤y<H;

[0082] Where k = W - x + y, L = min(W, H), and p = min(x, y).

[0083] Figure 6 The diagram shows interpolation in the planar_diagonal mode.

[0084] VTM uses the concept of MPM sets to encode prediction patterns for the luminance component. Each MPM consists of six different prediction patterns, which are derived using the prediction patterns of the left and top adjacent blocks. Figure 7 The image shows the top-left and top-left blocks used to derive the MPM list in existing recommendations for different target block shapes.

[0085] In one existing proposal, the MPM list is constructed as follows:

[0086] L≡Predicted pattern of the left-hand CU (values ​​in the range [0–66])

[0087] A≡Predicted pattern of CU above (values ​​in the range [0-66])

[0088] offset=NUM_LUMA_MODES–6(=61)

[0089] mod = offset + 3 (= 64)

[0090] initialization:

[0091] MPM[0] = L;

[0092] MPM[1]=! L(0if L≠0,else 1)

[0093] MPM[2] = VER_IDX

[0094] MPM[3] = HOR_IDX

[0095] MPM[4] = VER_IDX-4

[0096] MPM[5] = VER_IDX + 4

[0097] If L = A

[0098] If L>DC_IDX

[0099] MPM[0] = L;

[0100] MPM[1] = PLANA_IDX

[0101] MPM[2] = DC_IDX

[0102] MPM[3]=((L+offset)%mod)+2

[0103] MPM[4]=((L–1)%mod)+2

[0104] MPM[5]=((L+offset-1)%mod)+2

[0105] Else,

[0106] Use the initial values.

[0107] Else (i.e., if L≠A)

[0108] If L>DC_IDX and A>DC_IDX

[0109] MPM[0] = L;

[0110] MPM[1]=A

[0111] MPM[2] = PLANA_IDX

[0112] MPM[3] = DC_IDX

[0113] MPM[4]=((max(L,A)+offset)%mod)+2if L and A are not adjacent

[0114] =((max(L,A)+offset-1)%mod)+2,otherwise

[0115] MPM[5]=((max(L,A)-1)%mod)+2if L and A are not adjacent

[0116] =((max(L,A)-0)%mod)+2otherwise

[0117] Else if L + A >= 2 (i.e., only one of L and A is DC or PLANA)

[0118] MPM[0] = L;

[0119] MPM[1]=A

[0120] MPM[2]=! (min(L,A))(0if min(L,A)=1,else 0)

[0121] MPM[3]=((max(L,A)+offset)%mod)+2

[0122] MPM[4]=((max(L,A)-1)%mod)+2

[0123] MPM[5]=((max(L,A)+offset-1)%mod)+2

[0124] Else (i.e., L and A are either PLANA or DC)

[0125] Use the initial values.

[0126] Using a circular adjacency over the range [2-66] (i.e., 2 and 65 are adjacent), we can equivalently write it as:

[0127] ((L+offset)%mod)+2≡L–1

[0128] ((L+offset-1)%mod)+2≡L–2

[0129] ((L–1)%mod)+2≡L+1

[0130] ((L–0)%mod)+2≡L+2

[0131] Using these equivalence relations, we can show the derivation of the MPM list for different cases as shown in Table 3.

[0132]

[0133] Table 3: MPM derivation in the existing scheme. A and L represent the prediction modes of the upper CU and the left CU, respectively.

[0134] If the prediction mode of the current block is equal to one of the six MPM modes, this is indicated by setting the mpmFlag (mpm flag) to 1 and then encoding the candidate modes from the MPM list using the variable-length decoding scheme shown in Table 4.

[0135]

[0136]

[0137] Table 4: MPM Coding in Existing Proposals

[0138] Otherwise, mpmFlag is set to 0, and the indices of the patterns in the remaining list of 61 patterns are truncated into binary encoding. The first three patterns in the list (indices 0-2) are encoded with 5 bits, while the remaining 58 (indices 3-60) are each encoded with 6 bits.

[0139] Using the newly introduced pattern, we propose the following method for constructing the MPM list:

[0140] Let's index these three new modes as PLANA_HOR_IDX (index = 67), PLANA_DIA_IDX (index = 68), and PLANA_VER_IDX (index = 69). Note that these indices are for encoding purposes only and should not be confused with the actual indexes of the prediction modes. As we mentioned earlier, the prediction modes with indices 67-69 are already associated with the wide-angle direction. However, since VTM only supports 67 prediction modes, it's appropriate to index the newly introduced modes as described above, since we've expanded the number of modes, as it indexes them from 0 to 66 for encoding purposes. L ≡ Prediction mode of the left CU (values ​​in the range [0–69])

[0141] A≡Predicted pattern of CU above (values ​​in the range [0–69])

[0142]

[0143]

[0144]

[0145]

[0146] If the prediction mode of the current block is equal to one of the six MPM modes, this is indicated by setting mpmFlag to 1 and then encoding the candidate modes from the MPM list using the variable-length decoding scheme in Table 4. Otherwise, mpmFlag is set to 0 and the indices of the modes in the remaining list of 64 modes are binary encoded using 6 bits.

[0147] It should be noted that the proposed patterns are non-angular and are intended to describe intensity regions that change slowly along the associated direction. In the MPM list construction method described above, we give these patterns a higher priority than angular patterns in the same way that we give higher priority to PLANA and DC patterns in the existing MPM list construction in VTM 3.0. The reason for this is that we expect these new patterns to be more likely than other angular patterns, but possibly less likely than PLANA or DC patterns. Therefore, other MPM list constructions without using this implicit concept are also possible. For example, the original MPM list construction method can be used with a mapping of new patterns to the nearest directional pattern or to a PLANA pattern. If the predicted pattern of the left neighboring block or the top neighboring block is equal to the planar_vertical pattern, it can be replaced with the vertical pattern VER_IDX or the PLANA pattern. If the predicted pattern of the left neighboring block or the top neighboring block is equal to the planar_horizontal pattern, it can be replaced with the horizontal pattern HOR_IDX or the PLANA pattern. Similarly, if the predicted pattern of the left-side neighboring block or the top-side neighboring block is equal to the planar_diagonal pattern, then it can be replaced by the diagonal pattern DIA_IDX or the PLANAR pattern. In these cases, the newly introduced pattern will belong to the list of remaining patterns, and each will require 6 bits for encoding. Another approach to MPM construction is to include the new pattern in the MPM list if it happens to be the predicted pattern of the left-side or top-side neighboring block, but also include the associated directional pattern (i.e., VER_IDX, HOR_IDX, or DIA_IDX) as well as the PLANAR and DC patterns. In this case, if the number of candidates is still less than six, the neighboring patterns of the corresponding directional pattern can also be included in the MPM list. We describe several embodiments below that combine these methods of MPM list construction.

[0148] Example 1: In this example, we include the proposed three prediction modes in intra-frame prediction. In addition to the existing 67 prediction modes, the RD performance of these three modes is checked when considering the first reference line. When considering other reference lines, only the RD performance of the six-angle modes is checked, as in VTM 3.0. If the first reference line produces the best prediction, this prediction is signaled with a bit "0", and then the corresponding prediction mode is encoded using the MPM list (MPM list 0) as derived in the previous section. If the prediction mode belongs to the MPM list, the MPMflag is encoded as "1", and then the index of the prediction mode in the MPM list is encoded. Otherwise, the MPMflag is encoded as "0", and then the index of the prediction mode in the remaining mode set is encoded with a fixed length of 6 bits. Otherwise, if the second or fourth reference line produces the best prediction, this is signaled with bits "10" (for the second reference line) or "11" (for the fourth reference line), and the prediction mode is encoded using the MPM list (MPM list 1) as derived in VTM 3.0 for reference lines with indices 1 and 3. In this case, since the prediction mode always belongs to the MPM list (MPM list 1), the MPMflag is not encoded. On the decoder side, the decoder first decodes the reference line index. If the reference line index is equal to "0", it decodes the MPMflag, and then, based on the MPMflag value, it decodes the index of the prediction mode in MPM list 0 or the remaining mode list (which contains all modes except those in MPM list 0). Otherwise, if the reference line index is greater than 0, the decoder simply decodes the index of the prediction mode in MPM list 1. For any reference line, the encoder and decoder derive the same MPM list (MPM list 0 or list 1); therefore, the decoder correctly decodes the predicted mode from the decoding index value in the MPM list (list 0 with reference line #0 or list 1 with reference line #1 or #3) or the remaining mode list (reference line #0). Figure 8 and Figure 9 Flowcharts for encoding and decoding the prediction pattern for the current block are shown respectively.

[0149] Example 2: In this example, the inclusion of the three proposed modes is optional, utilizing a mode extension flag. If the flag is valid, then, as in Example 1, 70 intra-prediction modes are considered for intra-prediction using the first reference line, and encoding and decoding of prediction modes for arbitrary target blocks are performed. Otherwise, as in VTM 3.0, the existing 67 intra-prediction modes are considered, and encoding and decoding of these prediction modes are performed. The mode extension flag can be set at the CU level or slice level in the PPS or SPS.

[0150] Example 3: In this example, we include three intra-frame prediction modes utilizing the first reference line as proposed in Example 1 or 2, but the construction of the MPM list to be used with the first reference line is any other method different from the method described in the previous sections. For example, if the new mode happens to be the prediction mode of the left or upper adjacent block, we can map them to the associated directional mode. In this case, we can use the existing MPM list construction method in VTM 3.0 using these directional modes. Therefore, the new prediction mode will be forced to belong to the list of the remaining 64 modes, thus requiring 6 bits for encoding. Another example is to include neighboring modes in the MPM list (even if it is one of the new modes), and also include the corresponding directional modes as well as the PLANA and DC modes.

[0151] The proposed prediction mode is implemented by incorporating it into the VTM 3.0 software. This implementation can be performed as in Example 1. On the encoder side, a fast RD pass test is first performed on the three prediction modes along with the PLANA and DC modes using a first reference line. For reference lines 1 and 3, the MPM list structure remains unchanged (List 1), and the RD performance of the modes in this list is tested using reference lines 1 and 3. The BD-rate performance of the test method is shown in Table 5. It can be observed that the proposed method results in a luminance BD-rate gain of -0.07%. This gain is most significant for A1 class sequences (where the proposed mode is more likely due to its very high frame resolution).

[0152]

[0153] Table 5: BD-rate performance of Example 1 compared to VTM 3.0 anchor.

[0154] Figure 10An embodiment of a method 1000 for encoding a video data block using the main aspects described herein is illustrated. The method begins at a start box 1001 and control proceeds to a function box 1005 for determining an extended intra-prediction mode to be used to encode the video data block. Control then proceeds from box 1005 to box 1010 to predict target pixels of the video data block using the determined extended intra-prediction mode. Depending on the determined extended intra-prediction mode, control proceeds to one of several predictions. If the determined extended prediction mode is a horizontal planar mode, control proceeds to function box 1012 for horizontally predicting the target pixel using reference pixels from the left adjacent row and a reference pixel from the upper right of the block. Alternatively, if the determined extended prediction mode is a vertical planar mode, control proceeds to function box 1015 for vertically predicting the target pixel using reference pixels from the upper adjacent row and a reference pixel from the lower left of the block. Alternatively, if the determined extended prediction mode is a planar diagonal mode, control proceeds to function block 1017 to diagonally predict the target pixel using reference pixels located above and to the upper right of the block, or to the left and to the lower left of the block. Control then proceeds from block 1010 to block 1020 to encode the video block using the prediction in an extended intra-frame decoding mode.

[0155] Figure 11 An embodiment of a method 1100 for decoding a video data block using the main aspects described herein is illustrated. The method begins at a start box 1101 and control proceeds to a function box 1105 for determining an extended intra-prediction mode to be used to decode the video data block. Control then proceeds from box 1105 to box 1110 for predicting a target pixel of the video data block using the determined extended intra-prediction mode. Depending on the determined extended intra-prediction mode, control proceeds to one of several predictions. If the determined extended prediction mode is a horizontal planar mode, control proceeds to function box 1112 for horizontally predicting the target pixel using reference pixels from the left adjacent row and a reference pixel from the upper right of the block. Alternatively, if the determined extended prediction mode is a vertical planar mode, control proceeds to function box 1115 for vertically predicting the target pixel using reference pixels from the upper adjacent row and a reference pixel from the lower left of the block. Alternatively, if the determined extended prediction mode is a planar diagonal mode, control proceeds to function block 1117 to diagonally predict the target pixel using reference pixels located above and to the upper right of the block or to the left and to the lower left of the block. Control then proceeds from block 1110 to block 1120 to decode the video block using the prediction in an extended intra-frame decoding mode.

[0156] Figure 12An embodiment of an apparatus 1200 for encoding or decoding video data blocks is shown. The apparatus includes a processor 1210 and can be interconnected with a memory 1220 via at least one port. Both the processor 1210 and the memory 1220 may also have one or more additional interconnects to external connections.

[0157] The processor 1210 is configured to encode or decode video data using an extended prediction mode, and to encode or decode the video data blocks using predictions in an extended intra-frame decoding mode.

[0158] The main aspects described aim to improve intra-frame prediction efficiency by introducing three new modes and making the necessary changes required for intra-frame mode coding using an MPM list. The advantage is higher compression efficiency without much additional complexity.

[0159] This application describes various aspects, including tools, features, embodiments, models, methods, etc. Many of these aspects are described as specific and are generally described in a manner that may sound restrictive, at least to illustrate individual characteristics. However, this is for the purpose of clarity and does not limit the application or scope of those aspects. In fact, all the different aspects can be combined and interchanged to provide other aspects. Furthermore, these aspects can also be combined and interchanged with those described in earlier documents.

[0160] The aspects described and anticipated in this application can be implemented in many different forms. The following... Figure 13 , 14 Some embodiments are provided in sections 1 and 15, but other embodiments are conceivable, and... Figure 13 , 14 The discussion in section 15 does not limit the breadth of implementation. At least one of the aspects relates primarily to video encoding and decoding, and at least one other aspect relates primarily to transmitting the generated or encoded bitstream. These and other aspects can be implemented as methods, apparatus, computer-readable storage media having instructions thereon stored thereon for encoding or decoding video data according to any of the described methods, and / or computer-readable storage media having bitstreams generated according to any of the described methods stored thereon.

[0161] In this application, the terms "reconstruction" and "decoding" are used interchangeably, as are the terms "pixel" and "sample," and the terms "image," "picture," and "frame." Typically, but not necessarily, the term "reconstruction" is used on the encoder side, while "decoding" is used on the decoder side.

[0162] This document describes various methods, and each method includes one or more steps or actions for implementing the described method. Unless the correct operation of the method requires a specific order of steps or actions, the order and / or use of specific steps and / or actions may be modified or combined.

[0163] The various methods and other aspects described in this application can be used to modify the module, for example, Figure 13 and Figure 14 The intra-frame prediction, entropy decoding, and / or decoding modules (160, 360, 145, 330) of the video encoder 100 and decoder 200 are shown. Furthermore, the invention is not limited to VVC or HEVC and can be applied to, for example, other standards and proposals (whether pre-existing or developed in the future) and any extensions to such standards and proposals (including VVC and HEVC). Unless otherwise indicated or technically excluded, the aspects described in this application may be used individually or in combination.

[0164] Various numerical values ​​are used in this application. Specific values ​​are for illustrative purposes, and the aspects described are not limited to these specific values.

[0165] Figure 13 Encoder 100 is shown. Variations of encoder 100 can be envisioned, but for clarity, encoder 100 is described below without describing all anticipated variations.

[0166] Before being encoded, the video sequence may undergo pre-coding processing (101), such as applying a color transformation to the input color image (e.g., a conversion from RGB 4:4:4 to YCbCr 4:2:0), or performing a remapping of the input image components to obtain a signal distribution that is more resilient to compression (e.g., histogram equalization using one of the color components). Metadata may be associated with this pre-processing and appended to the bitstream.

[0167] In encoder 100, the image is encoded by encoder elements as described below. The image to be encoded is partitioned (102) in units such as CUs. Each unit is encoded using, for example, an intra-frame or inter-frame mode. When a unit is encoded in intra-frame mode, intra-frame prediction (160) is performed. In inter-frame mode, motion estimation (175) and compensation (170) are performed. The encoder determines (105) which of the intra-frame or inter-frame modes to use to encode the unit, and indicates the intra-frame / inter-frame decision by, for example, a prediction mode flag. For example, the prediction residual is calculated by subtracting (110) the prediction block from the original image block.

[0168] Then, the predicted residual is transformed (125) and quantized (130). The quantized transform coefficients, motion vectors, and other syntax elements are entropy decoded (145) to output a bitstream. The encoder may skip the transform and directly apply quantization to the untransformed residual signal. The encoder may bypass both the transform and quantization, i.e., directly decode the residual without applying the transform or quantization.

[0169] The encoder decodes the coded blocks to provide a reference for further prediction. The quantized transform coefficients are dequantized (140) and inversely transformed (150) to decode the prediction residuals. The decoded prediction residuals and prediction blocks are combined (155) to reconstruct an image block. An in-loop filter (165) is applied to the reconstructed image to perform, for example, unblocking / SAO (sample adaptive offset) filtering, thereby reducing coding artifacts. The filtered image is stored in a reference image buffer (180).

[0170] Figure 14 A block diagram of a video decoder 200 is shown. In decoder 200, the bitstream is decoded by decoder elements as described below. Video decoder 200 typically performs operations similar to... Figure 13 The encoding process described herein is the inverse of the decoding process. The encoder 100 typically also performs video decoding as part of the encoded video data.

[0171] Specifically, the input to the decoder includes a video bitstream, which can be generated by the video encoder 100. The bitstream is first entropy decoded (230) to obtain transform coefficients, motion vectors, and other decoding information. Image partitioning information indicates how the image is partitioned. The decoder can therefore partition (235) the image based on the decoded image partitioning information. The transform coefficients are dequantized (240) and inverse transformed (250) to decode the prediction residuals. The decoded prediction residuals are combined with prediction blocks (255) to reconstruct image blocks. The prediction blocks can be obtained from intra-frame prediction (260) or motion-compensated prediction (i.e., inter-frame prediction) (275) (270). An in-loop filter (265) is applied to the reconstructed image. The filtered image is stored in a reference image buffer (280).

[0172] The decoded image can undergo further post-decoding processing (285), such as inverse color transformation (e.g., conversion from YCbCr 4:2:0 to RGB 4:4:4) or inverse remapping of the remapping process performed in the pre-encoding process (101). The post-decoding processing may use metadata derived in the pre-encoding process and signaled in the bitstream.

[0173] Figure 15A block diagram illustrating an example system in which various aspects and embodiments are implemented is shown. System 1000 can be implemented as a device including the various components described below and configured to perform one or more aspects described herein. Examples of such devices include, but are not limited to, various electronic devices such as personal computers, laptop computers, smartphones, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected home appliances, and servers. Elements of system 1000 can be implemented individually or in combination in a single integrated circuit (IC), multiple ICs, and / or discrete components. For example, in at least one embodiment, the processing and encoder / decoder elements of system 1000 are distributed across multiple ICs and / or discrete components. In various embodiments, system 1000 is communicatively coupled to one or more other systems or other electronic devices via, for example, a communication bus or through dedicated input and / or output ports. In various embodiments, system 1000 is configured to implement one or more aspects described herein.

[0174] The system 1000 includes at least one processor 1010 configured to execute instructions loaded therein for implementing various aspects, such as those described herein. The processor 1010 may include embedded memory, input / output interfaces, and various other circuitry known in the art. The system 1000 includes at least one memory 1020 (e.g., a volatile memory device and / or a non-volatile memory device). The system 1000 includes a storage device 1040, which may include non-volatile memory and / or volatile memory, including but not limited to electrically erasable programmable read-only memory (EEPROM), read-only memory (ROM), programmable read-only memory (PROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, disk drives, and / or optical disk drives. As a non-limiting example, the storage device 1040 may include internal storage devices, attached storage devices (including removable and non-removable storage devices), and / or network-accessible storage devices.

[0175] System 1000 includes an encoder / decoder module 1030 configured to, for example, process data to provide encoded or decoded video, and the encoder / decoder module 1030 may include its own processor and memory. The encoder / decoder module 1030 represents one or more modules that may be included in a device to perform encoding and / or decoding functions. As is known, a device may include one or both of the encoding and decoding modules. Alternatively, the encoder / decoder module 1030 may be implemented as a separate element of system 1000 or may be incorporated into processor 1010 as a combination of hardware and software as known to those skilled in the art.

[0176] Program code to be loaded onto processor 1010 or encoder / decoder 1030 to execute the various aspects described herein may be stored in storage device 1040 and subsequently loaded onto memory 1020 for execution by processor 1010. According to various embodiments, one or more of processor 1010, memory 1020, storage device 1040, and encoder / decoder module 1030 may store one or more of various items during the execution of the processes described herein. These stored items may include, but are not limited to, input video, decoded video or portions of the decoded video, bitstreams, matrices, variables, and intermediate or final results from the processing of equations, formulas, operations, and operational logic.

[0177] In some embodiments, the memory within the processor 1010 and / or the encoder / decoder module 1030 is used to store instructions and provide working memory for processing required during encoding or decoding. However, in other embodiments, external memory (e.g., the processing device may be the processor 1010 or the encoder / decoder module 1030) is used for one or more of these functions. External memory may be memory 1020 and / or storage device 1040, such as volatile memory and / or non-volatile flash memory. In several embodiments, external non-volatile flash memory is used to store, for example, the operating system of a television. In at least one embodiment, a fast external dynamic volatile memory, such as RAM, is used as working memory for video decoding and decoding operations, such as working memory for MPEG-2 (MPEG stands for Moving Picture Experts Group; MPEG-2 is also known as ISO / IEC 13818, and 13818-1 is also known as H.222, and 13818-2 is also known as H.262), HEVC (HEVC stands for High Efficiency Video Decoding, also known as H.265 and MPEG-H Part 2), or VVC (Universal Video Decoding, a new standard developed by the Joint Video Team experts JVET).

[0178] As shown in box 1130, input to the components of system 1000 can be provided through various input devices. Such input devices include, but are not limited to: (i) an RF section that receives radio frequency (RF) signals transmitted over the air, for example by a broadcaster; (ii) component (COMP) input terminals (or a set of component input terminals); (iii) universal serial bus (USB) input terminals; and / or (iv) high-definition multimedia interface (HDMI) input terminals. Figure 15 Other examples not shown include composite videos.

[0179] In various embodiments, the input device of block 1130 has associated corresponding input processing elements known in the art. For example, the RF section may be associated with elements suitable for: (i) selecting a desired frequency (also known as selecting a signal, or limiting a signal band to a band), (ii) downconverting the selected signal, (iii) again limiting the band to a narrower band to select (e.g.,) a signal band that may be referred to as a channel in some embodiments), (iv) demodulating the downconverted and band-limited signal, (v) performing error correction, and (vi) demultiplexing to select a desired data packet stream. The RF section in various embodiments includes one or more elements to perform these functions, such as frequency selectors, signal selectors, band limiters, channel selectors, filters, downconverters, demodulators, error correctors, and demultiplexers. The RF section may include tuners that perform various of these functions, including, for example, downconverting received signals to a lower frequency (e.g., intermediate frequency or near-baseband frequency) or baseband. In one set-top box embodiment, the RF section and its associated input processing elements receive RF signals transmitted via a wired (e.g., cable) medium and perform frequency selection to a desired frequency band through filtering, down-conversion, and re-filtering. Various embodiments rearrange the order of the above (and other) elements, remove some of these elements, and / or add other elements that perform similar or different functions. Adding elements may include inserting elements between existing elements, such as inserting amplifiers and analog-to-digital converters. In various embodiments, the RF section includes an antenna.

[0180] Additionally, the USB and / or HDMI terminals may include corresponding interface processors for connecting the system 1000 to other electronic devices via USB and / or HDMI connections. It should be understood that various aspects of input processing (e.g., Solomon error correction) can be implemented as needed, for example, within a separate input processing IC or processor 1010. Similarly, various aspects of USB or HDMI interface processing can be implemented as needed within a separate interface IC or within processor 1010. The demodulated, error-corrected, and demultiplexed streams are provided to various processing elements, including, for example, processor 1010 and encoder / decoder 1030, which operate in conjunction with memory and storage elements to process the data streams as needed for presentation on the output device.

[0181] Various components of the system 1000 can be housed within an integrated housing. Within this integrated housing, the various components can be interconnected and transmit data therebetween using suitable connection arrangements (e.g., internal buses known in the art, including inter-IC (I2C) buses, wiring, and printed circuit boards).

[0182] The system 1000 includes a communication interface 1050 that enables communication with other devices via a communication channel 1060. The communication interface 1050 may include, but is not limited to, a transceiver configured to send and receive data via the communication channel 1060. The communication interface 1050 may include, but is not limited to, a modem or network interface card (NIC), and the communication channel 1060 may be implemented, for example, in a wired and / or wireless medium.

[0183] In various embodiments, a wireless network (e.g., a Wi-Fi network, such as IEEE 802.11 (IEEE refers to the Institute of Electrical and Electronics Engineers)) is used to stream or otherwise provide data to the system 1000. In these embodiments, the Wi-Fi signal is received via a communication channel 1060 and a communication interface 1050 suitable for Wi-Fi communication. The communication channel 1060 in these embodiments is typically connected to an access point or router that provides access to external networks, including the Internet, to allow streaming applications and other cloud-based communications. Other embodiments use a set-top box that transmits data via an HDMI connection to input box 1130 to provide streaming data to the system 1000. Still other embodiments use an RF connection to input box 1130 to provide streaming data to the system 1000. As described above, various embodiments provide data in a non-streaming manner. Additionally, various embodiments use wireless networks other than Wi-Fi, such as cellular networks or Bluetooth networks.

[0184] The system 1000 can provide output signals to various output devices, including a display 1100, a speaker 1110, and other peripheral devices 1120. The display 1100 in various embodiments includes one or more of the following: for example, a touchscreen display, an organic light-emitting diode (OLED) display, a flexible display, and / or a foldable display. The display 1100 can be used in televisions, tablets, laptops, cellular phones (mobile phones), or other devices. The display 1100 can also be integrated with other components (e.g., in a smartphone) or stand alone (e.g., an external monitor for a laptop computer). In various examples of embodiments, the other peripheral devices 1120 include one or more of the following: a standalone digital video disc (or digital multifunction disc) (DVR, for both), a disc player, a stereo system, and / or a lighting system. Various embodiments use one or more peripheral devices 1120 that provide functionality based on the output of the system 1000. For example, a disc player performs the function of playing the output of the system 1000.

[0185] In various embodiments, signaling (such as AV.Link, Consumer Electronics Control (CEC), or other communication protocols that enable device-to-device control with or without user intervention) is used to transmit control signals between system 1000 and display 1100, speaker 1110, or other peripheral devices 1120. The output devices may be communicatively coupled to system 1000 via dedicated connections through corresponding interfaces 1070, 1080, and 1090. Alternatively, the output devices may be connected to system 1000 via communication interface 1050 using communication channel 1060. The display 1100 and speaker 1110 may be integrated into a single unit within an electronic device (e.g., a television set) along with other components of system 1000. In various embodiments, display interface 1070 includes a display driver, such as a timing controller ((TCon) chip).

[0186] For example, if the RF portion of input 1130 is part of a separate set-top box, the display 1100 and speaker 1110 may alternatively be separated from one or more other components. In various embodiments where the display 1100 and speaker 1110 are external components, the output signal may be provided via a dedicated output connection, such as an HDMI port, a USB port, or a COMP output.

[0187] These embodiments can be implemented by processor 1010 or by computer software implemented in hardware or by a combination of hardware and software. As a non-limiting example, the embodiments can be implemented by one or more integrated circuits. The memory 1020 can be of any type suitable for the technical environment and can be implemented using any suitable data storage technology, such as optical memory devices, magnetic memory devices, semiconductor-based memory devices, fixed memory, and removable memory, as non-limiting examples. The processor 1010 can be of any type suitable for the technical environment and, as a non-limiting example, can include one or more of the following: microprocessors, general-purpose computers, special-purpose computers, and processors based on multi-core architectures.

[0188] Various implementations involve decoding. As used herein, "decoding" can include, for example, all or part of a process performed on a received encoded sequence to produce a final output suitable for display. In various embodiments, such processes include one or more of the processes typically performed by a decoder, such as entropy decoding, inverse quantization, inverse transform, and differential decoding. In various embodiments, such processes also or alternatively include processes performed by decoders of the various implementations described herein.

[0189] As a further example, in one embodiment, "decoding" refers only to entropy decoding; in another embodiment, "decoding" refers only to differential decoding; and in yet another embodiment, "decoding" refers to a combination of entropy decoding and differential decoding. Whether the phrase "decoding process" is intended to specifically refer to a subset of operations or generally to a broader decoding process will be clear based on the specific context of the description and is believed to be fully understood by those skilled in the art.

[0190] Various implementations involve encoding. In a manner similar to the above discussion of “decoding,” “encoding,” as used herein, can include all or part of a process performed, for example, on an input video sequence to produce an encoded bitstream. In various embodiments, such processes include one or more processes typically performed by an encoder, such as partitioning, differential coding, transform, quantization, and entropy decoding. In various embodiments, such processes also, or alternatively, include processes performed by encoders of the various implementations described herein.

[0191] As a further example, in one embodiment, "encoding" refers only to entropy decoding; in another embodiment, "encoding" refers only to differential encoding; and in yet another embodiment, "encoding" refers to a combination of differential encoding and entropy decoding. Whether the phrase "encoding process" is intended to specifically refer to a subset of operations or generally to a broader encoding process will become clear based on the specific context of the description and is believed to be fully understood by those skilled in the art.

[0192] Note that the grammatical elements used in this article are descriptive terms. Therefore, the use of other grammatical element names is not excluded.

[0193] When the accompanying drawings are presented as flowcharts, it should be understood that they also provide block diagrams of the corresponding devices. Similarly, when the accompanying drawings are presented as block diagrams, it should be understood that they also provide flowcharts of the corresponding methods / processes.

[0194] Various implementations may involve parametric models or rate distortion optimization. Specifically, during the encoding process, a balance or trade-off between rate and distortion is typically considered, usually with constraints on computational complexity. This can be measured by rate distortion optimization (RDO), or by least mean square (LMS), mean absolute error (MAE), or other such measures. RDO is typically formulated as minimizing a rate distortion function that is a weighted sum of rate and distortion. Different approaches exist to address the rate distortion optimization problem. For example, these approaches can be based on extensive testing of all encoding options, including all considered mode or decoding parameter values, and a complete evaluation of their decoding costs and the associated distortion of the reconstructed signal after decoding and decoding. Faster methods can also be used to save encoding complexity, particularly by calculating approximate distortion based on predicting or predicting the residual signal rather than the reconstructed signal. A hybrid of these two approaches can also be used, for example, by using approximate distortion only for some possible encoding options and full distortion for others. Other methods evaluate only a subset of possible encoding options. More generally, many methods employ any of a variety of techniques to perform the optimization, but the optimization is not necessarily a complete evaluation of both decoding cost and associated distortion.

[0195] The implementations and aspects described herein can be implemented, for example, in a method or process, apparatus, software program, data stream, or signal. Even if discussed only in the context of a single form of implementation (e.g., discussed only as a method), implementations of the discussed features can also be implemented in other forms (e.g., apparatus or program). For example, an apparatus can be implemented with appropriate hardware, software, and firmware. The method can be implemented, for example, in a processor, which generally refers to a processing device, including, for example, a computer, microprocessor, integrated circuit, or programmable logic device. Processors also include communication devices, such as computers, cellular phones, portable / personal digital assistants (“PDAs”), and other devices that facilitate information communication between end users.

[0196] The reference to "an embodiment," "an embodiment," "an implementation," or "an implementation," and other variations, means that a particular feature, structure, characteristic, etc., described in connection with that embodiment is included in at least one embodiment. Therefore, the phrases "in an embodiment," "in one embodiment," "in one implementation," or "in one implementation," and any other variations appearing in various places throughout this application, do not necessarily refer to the same embodiment.

[0197] Additionally, this application may involve "determining" various types of information. Determining the information may include, for example, one or more of the following: estimating the information, calculating the information, predicting the information, or retrieving the information from memory.

[0198] Furthermore, this application may relate to "accessing" various types of information. Accessing such information may include, for example, one or more of the following: receiving the information, retrieving the information (e.g., retrieving the information from a memory), storing the information, moving the information, copying the information, calculating the information, determining the information, predicting the information, or estimating the information.

[0199] Additionally, this application can refer to "receiving" various types of information. Like "accessing," receiving is intended to be a broad term. Receiving said information may include, for example, one or more of the following: accessing the information or (e.g., retrieving the information from memory). Furthermore, "receiving" is generally referred to in one or more ways during operations such as storing information, processing information, sending information, moving information, copying information, erasing information, calculating information, determining information, predicting information, or estimating information.

[0200] It should be understood that, for example, in the cases of “A / B,” “A and / or B,” and “at least one of A and B,” the use of any of the following “ / ,” “and / or,” and “at least one of” is intended to cover the selection of only the first listed option (A), or only the second listed option (B), or the selection of both options (A and B). As a further example, in the cases of “A, B, and / or C” and “at least one of A, B, and C,” such wording is intended to include selecting only the first listed option (A), or only the second listed option (B), or only the third listed option (C), or only the first and second listed options (A and B), or only the first and third listed options (A and C), or only the second and third listed options (B and C), or selecting all three options (A, B, and C). As will be apparent to those skilled in the art and related fields, this can be extended to multiple listed items.

[0201] Furthermore, as used herein, the term "signal" specifically refers to indicating something to the corresponding decoder. For example, in some embodiments, the encoder signals a particular one of multiple transforms, decoding modes, or flags. Thus, in one embodiment, the same transform, parameter, or mode is used on both the encoder and decoder sides. Therefore, for example, the encoder can send (explicitly signal) a specific parameter to the decoder so that the decoder can use that same specific parameter. Conversely, if the decoder already has the specific parameter as well as other parameters, signaling can be used without sending (implicitly signal) to simply allow the decoder to know and select the specific parameter. Bit savings are achieved in various embodiments by avoiding the transmission of any actual functionality. It should be understood that signaling can be implemented in various ways. For example, in various embodiments, one or more syntax elements, flags, etc., are used to signal information to the corresponding decoder. While the foregoing refers to the verb form of the term "signal," the term "signal" can also be used as a noun herein.

[0202] As will be apparent to those skilled in the art, implementations can generate various signals formatted to carry, for example, information that can be stored or transmitted. This information may include, for example, instructions for performing a method, or data generated by one of the described implementations. For example, the signal may be formatted to carry a bitstream of the described embodiments. Such a signal may be formatted as, for example, electromagnetic waves (e.g., using the radio frequency portion of the spectrum) or baseband signals. The formatting may include, for example, encoding the data stream and modulating a carrier wave using the encoded data stream. The information carried by the signal may be, for example, analog or digital information. As is known, the signal can be transmitted via various wired or wireless links. The signal may be stored on a processor-readable medium.

[0203] We have described several embodiments that can be provided across various claim classes and types. Features of these embodiments may be provided individually or in any combination. Furthermore, across various claim classes and types, embodiments may include one or more of the following features, devices, or aspects, individually or in any combination:

[0204] • A process or apparatus for performing intra-frame encoding and decoding using mode extension.

[0205] • A process or apparatus that uses multiple reference lines to perform intra-frame encoding and decoding using mode extension.

[0206] • A process or apparatus for performing intra-frame encoding and decoding with mode extension using a list of MPMs (most probable modes) and variable-length codes indicating which mode to use.

[0207] • A process or apparatus for performing intra-frame encoding and decoding, which utilizes mode expansion and filtering of predicted pixels across block boundaries.

[0208] • A process or apparatus for performing intra-frame encoding and decoding with mode extension on rectangular blocks.

[0209] • Includes a bitstream or signal of one or more of the described syntax elements or their variations.

[0210] • A bitstream or signal including a syntax that conveys information generated according to any of the described embodiments.

[0211] • Create and / or send and / or receive and / or decode according to any of the embodiments described.

[0212] • Methods, processes, apparatus, media for storing instructions, media for storing data, or signals according to any of the described embodiments.

[0213] • Inserting syntax elements into the signaling enables the decoder to determine the decoding mode in a manner corresponding to that used by the encoder.

[0214] • Creating and / or sending and / or receiving and / or decoding bitstreams or signals that include one or more described syntax elements or variations thereof.

[0215] TVs, set-top boxes, cellular phones, tablets, or other electronic devices that perform one or more transformation methods according to any of the described embodiments.

[0216] A TV, set-top box, cellular phone, tablet computer, or other electronic device that performs one or more transformation methods according to any of the described embodiments to determine and display (e.g., using a monitor, screen, or other type of display) the resulting image.

[0217] TVs, set-top boxes, cellular phones, tablets, or other electronic devices that select, band-limit, or tune (e.g., using a tuner) a channel to receive signals including encoded images, and perform transformation methods (one or more) according to any of the described embodiments.

[0218] • TVs, set-top boxes, cellular phones, tablets, or other electronic devices that receive signals including encoded images over the air (e.g., using an antenna) and perform transformation methods (one or more).

Claims

1. An encoding method, comprising: Determine the extended intra-prediction mode to be used to encode video data blocks, wherein the extended intra-prediction mode is one of the following: a horizontal plane mode, a vertical plane mode, or a diagonal plane mode; Using the determined extended intra-frame prediction mode, target pixels of the video data block are predicted based on one or more pixels adjacent to the block, wherein the prediction includes: If the determined extended prediction mode is a planar horizontal mode, the target pixel is predicted horizontally using the upper right reference pixel of the block and the reference pixels of the adjacent left row, and then the first row of samples is filtered, or If the determined extended prediction pattern is a planar vertical pattern, the target pixel is predicted vertically using the lower left reference pixel of the block and the reference pixels of the adjacent row above it, and then the first column of samples is filtered. If the determined extended prediction mode is a planar diagonal mode, the target pixel is predicted diagonally using reference pixels above and to the upper right of the block, or reference pixels to the left of the block and to the lower left of the block. Modify the most likely pattern list based on whether the prediction modes of the left and top adjacent blocks are directional or non-directional, or whether the left and top adjacent blocks use extended prediction modes; and The video block that uses the prediction to encode video data, wherein the prediction is made based on reference pixels including one or more pixels from adjacent columns or rows of the block, and the syntax indicates the extended prediction mode that uses the reference pixels to make the prediction.

2. The method of claim 1, wherein the reference pixel is a reconstructed pixel or an estimated pixel.

3. The method according to claim 1, wherein, At least one of the horizontal and vertical predictions is assigned an index corresponding to its pattern.

4. The method of claim 3, wherein a flag is used to indicate the prediction corresponding to a pattern included in the list of most likely patterns.

5. The method according to claim 1, wherein, The prediction is made based on reference pixels from one or more pixels of the block.

6. The method of claim 5, wherein the syntax indicates which reference pixels are used for the prediction.

7. The method according to claim 1, wherein, The prediction is selected based on rate distortion optimization using a first reference pixel line around the block.

8. The method according to claim 1, wherein, Based on the prediction used, filtering of the predicted pixels is performed on the boundaries of the block.

9. An encoding device, comprising: The processor, which is configured to execute: Determine the extended intra-prediction mode to be used to encode video data blocks, wherein the extended intra-prediction mode is one of the following: a horizontal plane mode, a vertical plane mode, or a diagonal plane mode; Using the determined extended intra-frame prediction mode, target pixels of the video data block are predicted based on one or more pixels adjacent to the block, wherein the prediction includes: If the determined extended prediction mode is a planar horizontal mode, the target pixel is predicted horizontally using the upper right reference pixel of the block and the reference pixels of the adjacent left row, and then the first row of samples is filtered, or If the determined extended prediction pattern is a planar vertical pattern, the target pixel is predicted vertically using the lower left reference pixel of the block and the reference pixels of the adjacent row above it, and then the first column of samples is filtered. If the determined extended prediction mode is a planar diagonal mode, the target pixel is predicted diagonally using reference pixels above and to the upper right of the block, or reference pixels to the left of the block and to the lower left of the block. Modify the most likely pattern list based on whether the prediction modes of the left and top adjacent blocks are directional or non-directional, or whether the left and top adjacent blocks use extended prediction modes; and The video block that uses the prediction to encode video data, wherein the prediction is made based on reference pixels including one or more pixels from adjacent columns or rows of the block, and the syntax indicates the extended prediction mode that uses the reference pixels to make the prediction.

10. The apparatus of claim 9, wherein the reference pixel is a reconstructed pixel or an estimated pixel.

11. The apparatus according to claim 9, wherein, At least one of the horizontal and vertical predictions is assigned an index corresponding to its pattern.

12. The apparatus of claim 11, wherein a flag is used to indicate the prediction corresponding to a pattern included in a list of most likely patterns.

13. The apparatus according to claim 9, wherein, The prediction is made based on reference pixels from one or more pixels of the block.

14. The apparatus of claim 13, wherein the syntax indicates which reference pixels are used for the prediction.

15. The apparatus according to claim 9, wherein, The prediction is selected based on rate distortion optimization using a first reference pixel line around the block.

16. The apparatus according to claim 9, wherein, Based on the prediction used, filtering of the predicted pixels is performed on the boundaries of the block.

17. A decoding method, comprising: Determine the extended intra-prediction mode to be used for decoding video data blocks, wherein the extended intra-prediction mode is one of the following: a horizontal plane mode, a vertical plane mode, or a diagonal plane mode; Using the determined extended intra-frame prediction mode, target pixels of the video data block are predicted based on one or more pixels adjacent to the block, wherein the prediction includes: If the determined extended prediction mode is a planar horizontal mode, the target pixel is predicted horizontally using the upper right reference pixel of the block and the reference pixels of the adjacent left row, and then the first row of samples is filtered, or If the determined extended prediction pattern is a planar vertical pattern, the target pixel is predicted vertically using the lower left reference pixel of the block and the reference pixels of the adjacent row above it, and then the first column of samples is filtered. If the determined extended prediction mode is a planar diagonal mode, the target pixel is predicted diagonally using reference pixels above and to the upper right of the block, or reference pixels to the left of the block and to the lower left of the block. Modify the most likely pattern list based on whether the prediction modes of the left and top adjacent blocks are directional or non-directional, or whether the left and top adjacent blocks use extended prediction modes; and The video block is used to decode the video data using the prediction, wherein the prediction is made based on reference pixels including one or more pixels from adjacent columns or rows of the block, and the syntax indicates the extended prediction mode for making the prediction using the reference pixels.

18. The method of claim 17, wherein the reference pixel is a reconstructed pixel or an estimated pixel.

19. The method of claim 17, wherein, At least one of the horizontal and vertical predictions is assigned an index corresponding to its pattern.

20. The method of claim 19, wherein a flag is used to indicate the prediction corresponding to a pattern included in the list of most likely patterns.

21. The method according to claim 17, wherein, The prediction is made based on reference pixels from one or more pixels of the block.

22. The method of claim 21, wherein the syntax indicates which reference pixels are used for the prediction.

23. The method according to claim 17, wherein, The prediction is selected based on rate distortion optimization using a first reference pixel line around the block.

24. The method according to claim 17, wherein, Based on the prediction used, filtering of the predicted pixels is performed on the boundaries of the block.

25. A decoding device, comprising: The processor, which is configured to execute: Determine the extended intra-prediction mode to be used for decoding video data blocks, wherein the extended intra-prediction mode is one of the following: a horizontal plane mode, a vertical plane mode, or a diagonal plane mode; Using the determined extended intra-frame prediction mode, target pixels of the video data block are predicted based on one or more pixels adjacent to the block, wherein the prediction includes: If the determined extended prediction mode is a planar horizontal mode, the target pixel is predicted horizontally using the upper right reference pixel of the block and the reference pixels of the adjacent left row, and then the first row of samples is filtered, or If the determined extended prediction pattern is a planar vertical pattern, the target pixel is predicted vertically using the lower left reference pixel of the block and the reference pixels of the adjacent row above it, and then the first column of samples is filtered. If the determined extended prediction mode is a planar diagonal mode, the target pixel is predicted diagonally using reference pixels above and to the upper right of the block, or reference pixels to the left of the block and to the lower left of the block. Based on whether the prediction modes of the left and top adjacent blocks are oriented or non-oriented, or whether the left and top adjacent blocks use extended prediction modes, modify the most likely mode list, and The video block is used to decode the video data using the prediction, wherein the prediction is made based on reference pixels including one or more pixels from adjacent columns or rows of the block, and the syntax indicates the extended prediction mode for making the prediction using the reference pixels.

26. The apparatus of claim 25, wherein the reference pixel is a reconstructed pixel or an estimated pixel.

27. The apparatus according to claim 25, wherein, At least one of the horizontal and vertical predictions is assigned an index corresponding to its pattern.

28. The apparatus of claim 27, wherein a flag is used to indicate the prediction corresponding to a pattern included in the list of most likely patterns.

29. The apparatus according to claim 25, wherein, The prediction is made based on reference pixels from one or more pixels of the block.

30. The apparatus of claim 29, wherein the syntax indicates which reference pixels are used for the prediction.

31. The apparatus according to claim 25, wherein, The prediction is selected based on rate distortion optimization using a first reference pixel line around the block.

32. The apparatus according to claim 25, wherein, Based on the prediction used, filtering of the predicted pixels is performed on the boundaries of the block.

33. An apparatus comprising: The apparatus according to any one of claims 25 to 32; as well as At least one of the following: (1) an antenna configured to receive a signal including the video data block, (ii) a band limiter configured to limit the received signal to the band including the video data block, and (iii) a display configured to display an output representing the video data block.

34. A non-transitory computer-readable medium comprising data content generated by the method of any one of claims 1-8 or by the apparatus of any one of claims 9-16, for playback using a processor.

35. A computer program product comprising instructions which, when executed by a computer, cause the computer to perform the method according to any one of claims 1-8 and 17-24.

Images