Time domain intra mode prediction

By extracting the motion information and motion compensation reference samples of video blocks, and using time-domain intra prediction technology, the problems of low encoding efficiency and decoding delay in the intra prediction mode in the prior art are solved, and more efficient video encoding and decoding are achieved.

CN119096549APending Publication Date: 2024-12-06INTERDIGITAL CE PATENT HOLDINGS SAS
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202380032978.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2022-04-07
Filing Date
2023-03-22
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

The existing video encoding techniques have problems with low encoding efficiency and decoding delay in intra prediction mode, especially in inter-frame slices, resulting in poor trade-offs on rate distortion and increased decoder complexity.

Method used

A method is proposed to obtain motion compensation reference samples by extracting the motion information of video blocks, and determine intra prediction based on these reference samples. Time domain intra prediction technology is used to replace traditional intra prediction, and use motion compensation reference samples for Encoding and decoding.

Benefits of technology

The encoding efficiency of intra prediction mode is improved, decoding delay is reduced, the accuracy and rate distortion trade-offs of inter prediction are improved, and the encoding and decoding complexity is reduced.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119096549A_ABST
    Figure CN119096549A_ABST
Patent Text Reader

Abstract

One of several embodiments is used to determine intra prediction samples for encoding in a video encoder and decoding in a video decoder. In at least one embodiment, motion information is derived from a reference frame to determine a reference sample to be used. A prediction sample is determined using a reference sample in an intra prediction mode. Reference samples are used in encoding or decoding. In at least one embodiment, motion information may be extracted using one of several motion models. Motion models, intra modes, or reference frames may be signaled from an encoder to a decoder. The signaling may use an index. The global motion model is computed at an encoder and sent to a corresponding decoder in a slice header or picture header. In at least one embodiment, different references are used for reference frames.
Need to check novelty before this filing date? Find Prior Art

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of EPO application serial number 22305482.6 filed on April 7, 2022, which is incorporated herein by reference in its entirety. Technical Field

[0003] At least one of the present embodiments generally relates to a method or apparatus for video encoding or decoding, compression or decompression. Background Art

[0004] To achieve high compression efficiency, image and video coding schemes typically use prediction (including motion vector prediction) and transforms to exploit spatial and temporal redundancy in video content. Typically, intra-frame or inter-frame prediction is used to exploit intra-frame or inter-frame correlations, and then the difference between the original image and the predicted image (usually expressed as a prediction error or prediction residual) is transformed, quantized, and entropy encoded. To reconstruct the video, the compressed data is decoded through the inverse process corresponding to entropy coding, quantization, transform, and prediction. Summary of the invention

[0005] At least one of the present embodiments generally relates to a method or apparatus for video encoding or decoding, and more specifically, to a method or apparatus for improving the encoding efficiency of an intra-frame prediction mode.

[0006] According to a first aspect, a method is provided. The method comprises the steps of: extracting motion information for a video block; obtaining motion compensation reference samples based on the motion information for the video block; determining intra prediction based on the motion compensation reference samples for the video block; and encoding at least a portion of the video block using intra prediction.

[0007] According to a second aspect, a method is provided. The method comprises the steps of: extracting motion information for a video block; obtaining motion compensation reference samples based on the motion information for the video block; determining intra prediction based on the motion compensation reference samples for the video block; and decoding at least a portion of the video block using the intra prediction.

[0008] According to another aspect, a device is provided. The device includes a processor. The processor can be configured to encode a video block or decode a bitstream by executing any of the aforementioned methods.

[0009] According to another general aspect of at least one embodiment, an apparatus is provided, comprising: an apparatus according to any decoding embodiment; and at least one of: (i) an antenna configured to receive a signal, the signal comprising a video block; (ii) a band limiter configured to limit the received signal to a frequency band comprising the video block, or (iii) a display configured to display an output representing the video block.

[0010] According to another general aspect of at least one embodiment, a non-transitory computer-readable medium is provided, comprising data content generated according to any of the described encoding embodiments or variations.

[0011] According to another general aspect of at least one embodiment, there is provided a signal comprising video data generated according to any of the encoding embodiments or variants described.

[0012] According to another general aspect of at least one embodiment, a bitstream is formatted to contain data content generated according to any of the encoding embodiments or variations described.

[0013] According to another general aspect of at least one embodiment, there is provided a computer program product comprising instructions that, when a computer executes the program, cause the computer to perform any one of the decoding embodiments or variants.

[0014] These and other aspects, features and advantages of the general aspects will become apparent from the following detailed description of illustrative embodiments, which is to be read in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 67 intra prediction modes in VVC and ECM are shown.

[0016] Figure 2 The matrix-weighted intra prediction process is shown.

[0017] Figure 3 The definition of samples used by PDPC applied to diagonal and adjacent-corner intra modes is shown.

[0018] Figure 4 An example of four reference lines adjacent to a prediction block is shown.

[0019] Figure 5 Intra prediction direction is shown.

[0020] Figure 6 An example workflow for temporal intra prediction is shown.

[0021] Figure 7 An example of a shifted reference sample is shown.

[0022] Figure 8An example of a simplified one pass motion compensation and filtering process is shown.

[0023] Fig. 9 An example showing the availability of new reference samples.

[0024] Fig.10 An example of a new intra prediction direction is shown.

[0025] Fig.11 An example of internal sample selection is shown.

[0026] Fig.12 An embodiment of a method for performing the described aspects is shown.

[0027] Fig.13 Another embodiment of a method for performing the described aspect is shown.

[0028] Fig.14 An embodiment of an apparatus for implementing the aspects is shown.

[0029] Fig.15 A general video encoding or compression system is shown.

[0030] Fig.16 A general video decoding or decompression system is shown.

[0031] Fig.17 A processor-based system for implementing the described aspects is shown. DETAILED DESCRIPTION

[0032] Intra prediction is a basic coding tool in video compression. The encoder selects the best prediction mode and signals its index to the decoder so that the same prediction is performed. Intra prediction is performed using reference samples, which are decoded samples around the current block to be decoded. Currently, there are several modes available in the VVC standard:

[0033] -DC: Use reference samples to create unified predictions

[0034] - Planar mode: Uses reference samples to create a smooth prediction of the block

[0035] - Directional mode: uses reference samples to "pad" these samples along a specific direction to create predictions

[0036] - MIP (Matrix-based Intra Prediction): uses linear combinations of reference samples to create predictions All of these modes use spatial information to create predictions.

[0037] In inter-coding, samples from one or more reference frames are used to create a prediction.

[0038] While inter prediction can provide better prediction when applicable, prediction from motion compensation of blocks is not always suitable. In particular, for inter slices far away from the reference frame (e.g. the last picture of a GOP in a random access configuration), the temporal gap may be so large that motion compensated blocks do not necessarily provide a good RD (rate-distortion) trade-off, even for areas common between 2 frames. One reason is that even if the global motion is correct, the local variation within the block is too large to obtain a good inter prediction. For example, the proportion of intra coded blocks in the first inter frame of a GOP of size 32 is typically above 50%.

[0039] Another aspect is that intra mode prediction increases the complexity of the decoder, since it introduces a delay during inter slice decoding: all inter and intra blocks around the current block should be reconstructed first in order to start reconstruction of the current intra block. As for low-latency decoders, it may introduce further delays in the pipeline. On the other hand, inter-based prediction allows full parallelism of inter-based reconstruction.

[0040] Intra-mode coding with 67 intra-prediction modes

[0041] To capture arbitrary edge directions present in natural videos, the number of directional intra modes in VVC is expanded from 33 used in HEVC to 65. New directional modes not present in HEVC are introduced in Figure 1 Planar and DC modes remain unchanged, as indicated by the red dashed arrows in . These denser directional intra prediction modes apply to all block sizes and both luma and chroma intra prediction.

[0042] In VVC, for non-square blocks, several traditional angular intra prediction modes are adaptively replaced with wide-angle intra prediction modes. Wide-angle intra prediction is described elsewhere in this specification.

[0043] In HEVC, each intra-coded block has a square shape, with each side length being a power of 2. Therefore, no division operation is required to generate the intra predictor using the DC mode. In VVC, blocks can have a rectangular shape, which necessitates the use of a division operation for each block in the general case. To avoid division operations for DC prediction, only the longer side is used to calculate the average for non-square blocks.

[0044] Intra-mode coding

[0045] In order to keep the complexity of the most probable mode (MPM) list generation low, an intra mode coding method with 6 MPMs is used by considering two available adjacent intra modes. The following three aspects are considered when constructing the MPM list:

[0046] -Default intra mode

[0047] - Adjacent Intra Mode

[0048] -Derivation of intra mode

[0049] Regardless of whether the MRL and ISP coding tools are applied, a unified 6-MPM list is used for intra blocks. The MPM list is constructed based on the intra modes of the left and upper neighboring blocks. Assuming that the mode on the left is denoted as Left and the mode of the upper block is denoted as Above, the unified MPM list is constructed as follows:

[0050] - When a neighboring block is not available, its intra mode is set to Planar by default.

[0051] -If both Left and Above modes are non-angle modes:

[0052] -MPM List → {Planar, DC, V, H, V-4, V+4}

[0053] -If one of the Left and Above modes is an angle mode and the other is a non-angle mode:

[0054] - Set the mode Max to the larger of Left and Above

[0055] -MPM List → {Planar, Max, Max-1, Max+1, Max-2, Max+2}

[0056] -If Left and Above are both angle modes and are different:

[0057] - Set the mode Max to the larger of Left and Above

[0058] - Set Mode Min to the smaller of Left and Above

[0059] -If Max–Min is equal to 1:

[0060] -MPM list → {Planar, Left, Above, Min–1, Max+1, Min–2}

[0061] - Otherwise, if Max–Min is greater than or equal to 62:

[0062] -MPM list → {Planar, Left, Above, Min+1, Max–1, Min+2}

[0063] - Otherwise, if Max–Min is equal to 2:

[0064] -MPM list → {Planar, Left, Above, Min+1, Min–1, Max+1}

[0065] -otherwise:

[0066] -MPM list → {Planar, Left, Above, Min–1, –Min+1, Max–1}

[0067] -If Left and Above are both angled and they are the same:

[0068] -MPM list → {Planar, Left, Left-1, Left+1, Left–2, Left+2}

[0069] In addition, the first bit (bin) of the mpm index codeword is CABAC context coding. A total of three contexts are used, corresponding to whether the current intra-frame block is MRL-enabled, ISP-enabled, or a normal intra-frame block.

[0070] During the generation of the 6 MPM lists, pruning is used to remove repeated patterns so that only unique patterns can be included in the MPM lists. For the entropy coding of the 61 non-MPM patterns, truncated binary code (TBC) is used.

[0071] Matrix Weighted Intra Prediction (MIP)

[0072] The matrix-weighted intra prediction (MIP) method is a newly added intra prediction technique in VVC. To predict the samples of a rectangular block of width W and height H, the matrix-weighted intra prediction (MIP) takes a row of H reconstructed adjacent boundary samples on the left side of the block and a row of W reconstructed adjacent boundary samples above the block as input. If the reconstructed samples are not available, they are generated according to the traditional intra prediction method. The generation of the prediction signal is based on the following three steps, namely averaging, matrix-vector multiplication, and linear interpolation, as shown in Figure 19.

[0073] Position-dependent intra prediction combination

[0074] In VVC, the intra prediction results for DC, planar, and several angle modes are further modified by the Position Dependent Intra Prediction Combination (PDPC) method. PDPC is an intra prediction method that calls for a combination of boundary reference samples and HEVC-style intra prediction using filtered boundary reference samples. PDPC is applied to the following intra modes without signaling interaction: planar, DC, intra angles less than or equal to horizontal, intra angles greater than or equal to vertical and less than or equal to 80. PDPC is not applied if the current block is in Bdpcm mode or the MRL index is greater than 0.

[0075] The prediction sample pred(x',y') is predicted using a linear combination of the intra prediction mode (DC, planar, angular) and the reference sample according to the following equation 3-8:

[0076] pred(x',y')=Clip(0,(1< <BitDepth)–1,(wL×R -1,y’ +wT×R x’,-1 +(64-wL-wT)×

[0077] pred(x',y') + 32)>>6) (3-8)

[0078] Where R x,-1 , R -1,y Respectively represent the reference samples located at the top and left boundaries of the current sample (x, y).

[0079] When PDPC is applied to DC, planar, horizontal and vertical intra modes, no additional boundary filtering is required, which is different from the requirement in the case of HEVC DC mode boundary filtering or horizontal / vertical mode edge filtering. The PDPC process is the same for DC and planar modes. For angular modes, if the current angular mode is HOR_IDX or VER_IDX, the left or top reference samples are not used, respectively. The PDPC weights and scaling factors depend on the prediction mode and block size. PDPC is applied to blocks with width and height both greater than or equal to 4.

[0080] Figure 3 The reference samples (R x,-1 and R -1,y ). The prediction sample pred(x',y') is located at (x',y') in the prediction block. For example, for diagonal mode, the reference sample R x,-1 The coordinate x of is given by x=x'+y'+1, and the reference sample R -1,y The coordinate y of is similarly given by y=x'+y'+1. For other angle modes, the reference sample R x,-1 and R -1,y Can be at a fractional sample position. In this case, the sample value at the nearest integer sample position is used.

[0081] Multiple Reference Line (MRL) Intra Prediction

[0082] Multiple reference line (MRL) intra prediction uses more reference lines for intra prediction. Figure 4An example of 4 reference lines is depicted in Figure 1, where the samples of segments A and F are not taken from the reconstructed neighboring samples, but are filled with the closest samples from segments B and E, respectively. HEVC intra prediction uses the nearest reference line (i.e., reference line 0). In MRL, 2 additional lines are used (reference line 1 and reference line 3).

[0083] The index of the selected reference line (mrl_idx) is signaled and used to generate the intra predictor. For reference line idx greater than 0, only additional reference line modes are included in the MPM list and only the MPM index is signaled instead of the remaining modes. The reference line index is signaled before the intra prediction mode, and if a non-zero reference line index is signaled, the planar mode is excluded from the intra prediction mode.

[0084] MRL is disabled for blocks in the first row within a CTU to prevent the use of extended reference samples outside the current CTU row. In addition, PDPC is disabled when additional rows are used. For MRL mode, the DC value derivation in DC intra prediction mode for non-zero reference row index is consistent with the DC value derivation for reference row index 0. MRL requires 3 adjacent luma reference rows to be stored along with the CTU to generate the prediction. Note that the Cross Component Linear Model (CCLM) tool also requires 3 adjacent luma reference rows for its downsampling filtering. MRL is designed to use the same 3 rows as CCLM to reduce the storage requirements of the decoder.

[0085] specification

[0086] 8.4.5.2.10 Reference sample filtering process

[0087] The input to this process is:

[0088] - variable refIdx which specifies the intra prediction reference line index;

[0089] - the variable nTbW which specifies the transform block width;

[0090] - a variable nTbH which specifies the transform block height;

[0091] - a variable refW which specifies the reference sample width;

[0092] - a variable refH which specifies the reference sample height;

[0093] - variable refFilterFlag which specifies the value of the reference filter flag;

[0094] - (unfiltered) neighboring samples refUnfilt[x][y], where x=-1-refIdx, y=-1-refIdx..refH-1 and x=-refIdx..refW-1, y=-1-refIdx,

[0095] - A variable cIdx which specifies the color component of the current block.

[0096] The output of this process is the reference samples p[x][y], where x=-1-refldx, y=-1-refldx..refH-1 and x=-refldx..refW-1, y=-1-refldx.

[0097] The variable filterFlag is derived as follows:

[0098] - Set filterFlag to 1 if all of the following are true:

[0099] -refIdx is equal to 0;

[0100] -nTbW*nTbH is greater than 32;

[0101] -cIdx is equal to 0;

[0102] -IntraSubPartitionsSplitType is equal to ISP_NO_SPLIT;

[0103] -refFilterFlag is equal to 1;

[0104] - Otherwise, set filterFlag to 0.

[0105] For the derivation of the reference sample p[x][y], the following processing is applied:

[0106] - If filterFlag is equal to 1, the filtered sample values ​​p[x][y] for x=-1, y=-1..refH-1 and x=0..refW-1, y=-1 are derived as follows:

[0107] p[-1][-1]=(refUnfilt[-1][0]+2*refUnfilt[-1][-1]+refUnfilt[0][-1]+2)>>2 (315)

[0109] p[-1][y]=(refUnfilt[-1][y+1]+2*refUnfilt[-1][y]+refUnfilt[-1][y-1]+2)>>2

[0110] for y=0..refH-2 (316)

[0111] p[-1][refH-1]=refUnfilt[-1][refH-1] (317)

[0112] p[x][-1]=(refUnfilt[x-1][-1]+2*refUnfilt[x][-1]+refUnfilt[x+1][-1]+2)>>2

[0113] for x=0..refW-2 (318)

[0114] p[refW-1][-1]=refUnfilt[refW-1][-1] (319)

[0115] Otherwise, the reference sample value p[x][y] is set equal to the unfiltered sample value refUnfilt[x][y], where x=-1-refIdx, y=-1-refIdx..refH-1 and x=-refIdx..refW-1, y=-1-refIdx.

[0116] 8.4.5.2.11 INTRA_PLANAR Intra prediction mode specification

[0117] The input to this process is:

[0118] - the variable nTbW which specifies the transform block width;

[0119] - a variable nTbH which specifies the transform block height;

[0120] - Neighboring samples p[x][y], where x=-1, y=-1..nTbH and x=0..nTbW, y=-1.

[0121] The output of this process is the prediction samples predSamples[x][y], where x=0..nTbW-1, y=0..nTbH-1.

[0122] The values ​​of the predicted samples predSamples[x][y] for x=0..nTbW-1 and y=0..nTbH-1 are derived as follows:

[0123] predV[x][y]=((nTbH-1-y)*p[x][-1]+(y+1)*p[-1][nTbH])< <Log2(nTbW) (320)

[0125] predH[x][y]=((nTbW-1-x)*p[-1][y]+(x+1)*p[nTbW][-1])< <Log2(nTbH) (321)

[0127] predSamples[x][y]=(predV[x][y]+predH[x][y]+nTbW*nTbH)>>(Log2(nTb W)+Log2(nTbH)+1)(322)

[0128] 8.4.5.2.12 Specification of INTRA_DC intra prediction mode

[0129] The input to this process is:

[0130] - the variable nTbW which specifies the transform block width;

[0131] - a variable nTbH which specifies the transform block height;

[0132] - variable refIdx which specifies the intra prediction reference line index;

[0133] - Neighboring samples p[x][y], where x=-1-refIdx, y=0..nTbH-1 and x=0..nTbW-1, y=-1-refIdx.

[0134] The output of this process is the prediction samples predSamples[x][y], where x=0..nTbW-1, y=0..nTbH-1.

[0135] The value of the predicted sample predSamples[x][y] for x=0..nTbW-1, y=0..nTbH-1 is derived by the following ordered steps:

[0136] 1. Derived variable dcVal as follows:

[0137] -When nTbW is equal to nTbH:

[0138]

[0139] -When nTbW is greater than nTbH:

[0140]

[0141] -When nTbW is less than nTbH:

[0142]

[0143] 2. Derived prediction samples predSamples[x][y] are as follows:

[0144] predSamples[x][y]=dcVal, where x=0..nTbW-1, y=0..nTbH-1 (326)

[0145] 8.4.5.2.13 Specification of INTRA_ANGULAR2..INTRA_ANGULAR66 intra prediction modes The input to this process is:

[0146] - Intra prediction mode predModeIntra,

[0147] - variable refIdx which specifies the intra prediction reference line index;

[0148] - the variable nTbW which specifies the transform block width;

[0149] - a variable nTbH which specifies the transform block height;

[0150] - a variable refW which specifies the reference sample width;

[0151] - a variable refH which specifies the reference sample height;

[0152] - variable nCbW which specifies the coding block width;

[0153] - variable nCbH which specifies the coding block height;

[0154] - variable refFilterFlag which specifies the value of the reference filter flag;

[0155] - a variable cIdx which specifies the current block color component;

[0156] - Neighboring samples p[x][y], where x=-1-refIdx, y=-1-refIdx..refH-1 and x=-refIdx..refW-1, y=-1-refIdx.

[0157] The output of this process is the prediction samples predSamples[x][y], where x=0..nTbW-1, y=0..nTbH-1.

[0158] Set the variable nTbS to (Log2(nTbW)+Log2(nTbH))>>1.

[0159] The variable filterFlag is derived as follows:

[0160] - Set filterFlag equal to 0 if one or more of the following conditions are true:

[0161] -refFilterFlag is equal to 1;

[0162] -refIdx is not equal to 0;

[0163] -IntraSubPartitionsSplitType is not equal to ISP_NO_SPLIT.

[0164] - Otherwise, the following applies:

[0165] - Set the variable minDistVerHor equal to Min(Abs(predModeIntra-50),Abs(predModeIntra-18)).

[0166] -Specify the variable intraHorVerDistThres[nTbS] in Table 23.

[0167] -Derivation of the variable filterFlag is as follows:

[0168] - If minDistVerHor is greater than intraHorVerDistThres[nTbS], set filterFlag to 1.

[0169] - Otherwise, set filterFlag to 0.

[0170] Table 23 - Specification of intraHorVerDistThres[nTbS] for various transform block sizes nTbS

[0171]

[0172] Fig. 9 93 prediction directions are shown, with the dashed directions associated with the wide-angle mode applicable only to non-square blocks.

[0173] Temporal Intra Prediction

[0174] To improve intra prediction, we introduce temporal intra prediction. Instead of relying on reference samples around the current block, it uses reference samples in the reference frame. In order to obtain reliable reference samples, motion compensation is introduced.

[0175] Note: In the following, we always refer to prediction as “intra” prediction to point out that after defining the reference samples, the prediction process is very similar to regular intra prediction ( Figure 6 However, it is understood that since the prediction uses a different reference frame than the current frame, the prediction is technically an "inter" prediction.

[0176] The new temporal intra prediction follows Figure 6 The process in:

[0177] -First, calculate the motion information for the current block

[0178] - Use a motion compensated version of the reference sample template to construct the reference samples used in intra prediction.

[0179] Motion vector export

[0180] Global Motion Model

[0181] Since the first B-frame (after the initial I-frame) contains more intra-coded blocks and less motion information, global motion information is sent at the slice level. Typically, at least one global motion model is calculated at the encoder and sent in the slice header (or picture header).

[0182] A typical way to calculate global motion is:

[0183] performing block matching between a reference frame and a current frame and extracting dominant motion; and

[0184] Salient points (e.g. Harris corners or SIFT points) are extracted and robustly matched according to a general motion model (e.g. using RANSAC).

[0185] The motion model can be an affine 4, affine 6 or homography model. Alternatively, multiple models can be transmitted. Advantageously, the model parameters can be sent in the form of differentially encoded frame angular motion:

[0186] Top left: Use the codec's MVD encoding to send motion vectors

[0187] Top right: Sends the mvd of the difference between the top right motion and the decoded top left motion

[0188] Lower left corner: same as upper right

[0189] Bottom right: MVD of the difference between the bottom right motion and the average of the top left and bottom left motion vectors

[0190] From the global motion model, the motion of the current block is calculated, for example by using the motion of the center of the block. Alternatively, the motion of the block can be calculated using the top left or other position.

[0191] If multiple models are available, the index of the model to be used is signaled for the block using the pattern. Advantageously, context-based coding (using neighboring blocks (usually top and left blocks)) can be used to reduce the cost of signaling the index.

[0192] Variants of motion derivation

[0193] Another way to obtain motion information is to use the same process used to infer motion in inter-coded blocks. Typically, a list of possible motion vector candidates is calculated, for example by using a process similar to merging the inter list. The first candidate in the list is then used to infer motion for the current block.

[0194] Alternatively, the indices of candidates for inferring motion are sent.

[0195] Alternatively, use template-based re-ranking similar to ARMC-TM to select the most likely candidate in the list.

[0196] Co-located frame variants

[0197] In a variant, the co-located frame used as reference frame for motion vectors is signaled in the slice header. Alternatively, the index is signaled at block level, in the same way as reference frame index is signaled in inter-coded blocks.

[0198] Reference sample construction

[0199] Once the motion vectors are available, reference samples can be constructed.

[0200] In contrast to conventional intra prediction where some reference samples may not be available due to decoding causality, in case of temporal intra prediction (TIP) all reference samples are available (except when the sample is not in a frame or slice).

[0201] To create the reference sample array, the motion vectors decoded at the previous stage are applied to the co-located block positions.

[0202] Pixel-level motion compensation and smooth adaptation

[0203] In a variant, to reduce complexity, motion compensation is performed pixel by pixel.

[0204] However, since the reference samples are filtered (eg, using Gaussian filtering as shown in 8.4.5.2.10 Reference Sample Filtering Process), the sub-pixel motion compensation filtering and Gaussian filtering stages can be completed in 1 stage.

[0205] In a variant, horizontal reference samples (corresponding to the top neighbors in the block) are filtered only in the horizontal direction, i.e. sub-pixel motion compensation filtering and Gaussian filtering are applied only in the horizontal direction. The same logic applies to vertical samples.

[0206] In a variant, the resulting filter of sub-pixel motion compensation and Gaussian filtering is reduced to 3 taps, making it the same complexity as intra smoothing of the current reference sample.

[0207] We are Figure 8An example of sub-pixel motion of 1 / 4 pixel using a bilinear filter and a 3-tap Gaussian filter is given in.

[0208] In a variant, filtering is performed in both directions and more reference samples are needed to perform a complete reference sample construction.

[0209] Negative Angle Prediction

[0210] Since all reference samples (surrounding the block) are available, some directions can use reference samples closer to the block than regular intra prediction. Typically, negative directions and directions above 66 (see Figure 5 )Use a new reference sample, such as Fig. 9 shown.

[0211] In another variant, new intra-frame directions are available, such as Fig.10 Add 16 new negative directions (i.e. the last direction is -30), and add 15 new directions after 80 (up to 95).

[0212] In another variation, the MIP mode is also extended to take into account the newly available samples.

[0213] Full plane mode

[0214] Since all reference samples (surrounding blocks) are available, planar mode can take advantage of this.

[0215] Typically, the original process:

[0216]

[0217] Replace with:

[0218]

[0219] Now we can get the reference sample values ​​p[x][y] for the bottom (p[x][nTbH]) and right part (p[nTbW][y]).

[0220] Full DC mode

[0221] Using the same reference sample, the DC value can also be adjusted:

[0222]

[0223] Internal reference sample time domain intra mode

[0224] Since reference samples are also available within the block, they can be used to provide better predictions (note: this is technically still inter prediction, but it does not try to accurately predict every pixel of the block).

[0225] Internal sample template prediction

[0226] In a variant, all intra reference samples are taken as in conventional prediction, but on the inside rather than outside of the block boundaries, e.g. Fig.11 shown.

[0227] As an example, the flat mode becomes:

[0228]

[0229] Full in-sample forecasting

[0230] A more general variant is to use all samples of the block. However, the goal of the prediction is not to make an accurate prediction of the interior of the block (otherwise, an inter-based mode would be chosen).

[0231] To create this pattern, one needs to create a prediction similar to MIP: the reference samples are put into a vector, and a linear function (matrix-vector multiplication) is computed to create the prediction samples (or a subset of the prediction samples).

[0232] - Learn a matrix to predict blocks based on internal samples

[0233] - Creating a dataset using intra blocks from the dataset encoding, where the motion vectors are inferred using the method described above for the intra blocks. Compensated blocks that differ too much from the original blocks (according to a certain threshold) are not used in the dataset.

[0234] - As in MIP, create subsets of predictions and use linear interpolation to fill the entire prediction to limit complexity.

[0235] - For the same reason, not all samples are used during matrix-vector multiplication, but only a subset of samples, or a subsampled version of the block.

[0236] Signal Notification

[0237] Block Level

[0238] The following table shows an excerpt of decoding a coding unit, in particular intra mode decoding. A flag is added on top of the intra syntax to signal the use of temporal intra prediction. In the following examples, we assume that the temporal predictor used is inferred (e.g., using a global motion model). The same syntax as before is retained to describe intra prediction, but the mode will be applied to the shifted blocks.

[0239] Code unit syntax

[0240] CU decoding contains new syntax elements related to temporal intra prediction. When true, the remainder of the intra directions is slightly changed (using intra_luma_mpm_remainder_ext) to account for the added negative directions. The final syntax looks like this:

[0241]

[0242] Slice level

[0243] At slice / picture / sequence level, if temporal prediction is used, a flag is signaled, such as sps_intra_temporal_prediction_enabled_flag.

[0244] When at least one global motion model is used, the global motion model parameters are also transmitted, for example by using the following syntax:

[0245]

[0246] The syntax element model_order is an integer between 0 and 3 that controls the order of the motion model (translation, affine4, affine6, or homography).

[0247] In a variant, the cpmv is differentially encoded using already available angle and correlation models to predict each angle.

[0248] Fig.12 An embodiment of a method 1200 according to the general aspects described herein is shown. The method begins at a start block 1201 and control proceeds to block 1210 to extract motion information for a video block. Control proceeds from block 1210 to block 1220 to obtain motion compensated reference samples based on the motion information for the video block. Control proceeds from block 1220 to block 1230 to determine an intra prediction based on the motion compensated reference samples for the video block. Control proceeds from block 1230 to block 1240 to encode at least a portion of the video block using intra prediction.

[0249] Fig.13 An embodiment of a method 1300 according to the general aspects described herein is shown. The method begins at a start block 1301 and control proceeds to block 1310 to extract motion information for a video block. Control proceeds from block 1310 to block 1320 to obtain motion compensated reference samples based on the motion information for the video block. Control proceeds from block 1320 to block 1330 to determine an intra prediction based on the motion compensated reference samples for the video block. Control proceeds from block 1330 to block 1340 to decode at least a portion of the video block using intra prediction.

[0250] Fig.14An embodiment of an apparatus 1400 for encoding, decoding, compressing or decompressing video data using any of the above methods or variations is shown. The apparatus includes a processor 1910 and may be interconnected to a memory 1920 via at least one port. The processor 1910 and the memory 1920 may also have one or more additional interconnects to external connections.

[0251] Processor 1410 is also configured to insert or receive information in a bitstream and perform compression, encoding or decoding using any of the described aspects.

[0252] The embodiments described herein include multiple aspects, including tools, features, embodiments, models, schemes, etc. Many of these aspects are described as being specific and, at least to show the individual features, are often described in a manner that may sound restrictive. However, this is for the purpose of describing clarity and does not limit the application or scope of these aspects. In fact, all of the different aspects can be combined and interchanged to provide further aspects. In addition, these aspects can also be combined and interchanged with aspects described in previous submissions.

[0253] The aspects described and contemplated in this application may be embodied in many different forms. Fig.15 , 16 and 17 provide some embodiments, but other embodiments are contemplated, and Fig.15 , 16 The discussion of 17 does not limit the breadth of the embodiments. At least one aspect generally relates to video encoding and decoding, and at least one other aspect generally relates to transmitting a generated or encoded bitstream. These and other aspects can be implemented as a method, an apparatus, a computer-readable storage medium storing instructions for encoding or decoding video data according to any of the described methods, and / or a computer-readable storage medium storing a bitstream generated according to any of the described methods.

[0254] In this application, the terms "reconstruction" and "decoding" can be used interchangeably, the terms "pixel" and "sample" can be used interchangeably, and the terms "image", "picture" and "frame" can be used interchangeably. Usually, but not necessarily, the term "reconstruction" is used on the encoder side, while "decoding" is used on the decoder side.

[0255] This article describes a variety of methods, and each method includes one or more steps or actions for implementing the method. Unless the correct operation of the method requires a specific order of steps or actions, the order and / or use of specific steps and / or actions can be modified or combined.

[0256] Various methods and other aspects described in this application can be used to modify modules, such as Fig.15 and Fig.16Intra-frame prediction, entropy coding and / or decoding modules 160, 360, 145, 330 of the video encoder 100 and decoder 200 are shown. In addition, the present aspects are not limited to VVC or HEVC, and can be applied to, for example, other standards and recommendations (whether existing or developed in the future) and extensions of any such standards and recommendations (including VVC and HEVC). Unless otherwise stated or technically excluded, the aspects described in this application can be used alone or in combination.

[0257] Various numerical values ​​are used in this application. The specific values ​​are only for example purposes, and the aspects are not limited to these specific values.

[0258] Fig.15 An encoder 100 is shown. Variations of the encoder 100 are contemplated, but for clarity, only the encoder 100 is described below, rather than describing all contemplated variations.

[0259] Prior to encoding, the video sequence may undergo pre-encoding processing (101), such as applying a color transform to an input color picture (e.g., converting from RGB 4:4:4 to YCbCr 4:2:0), or performing a remapping of input picture components to obtain a signal distribution more amenable to compression (e.g., using histogram equalization of one of the color components). Metadata may be associated with the pre-processing and appended to the bitstream.

[0260] In encoder 100, an image is encoded by encoder elements as described below. The image to be encoded is segmented (102) and processed in units such as CUs. Each unit is encoded using, for example, intra or inter mode. When the unit is encoded in intra mode, it performs intra prediction (160). In inter mode, motion estimation (175) and compensation (170) are performed. The encoder decides (105) which of intra mode or inter mode to use to encode the unit and indicates the intra / inter decision by, for example, a prediction mode flag. For example, a prediction residual is calculated by subtracting (110) the predicted block from the original image block.

[0261] The prediction residual is then transformed (125) and quantized (130). The quantized transform coefficients, along with motion vectors and other syntax elements, are entropy encoded (145) to output a bitstream. The encoder may skip the transform and apply quantization directly to the untransformed residual signal. The encoder may bypass the transform and quantization, i.e., encode the residual directly without applying the transform or quantization process.

[0262] The encoder decodes the coded block to provide a reference for further prediction. The quantized transform coefficients are inverse quantized (140) and inverse transformed (150) to decode the prediction residual. The decoded prediction residual and the prediction block are combined (155) to reconstruct the image block. Loop filtering (165) is applied to the reconstructed image to perform, for example, deblocking / SAO (sample adaptive offset) / ALF (adaptive loop filter) filtering to reduce coding artifacts. The filtered image is stored in a reference picture buffer (180).

[0263] Fig.16 2 shows a block diagram of a video decoder 200. In the decoder 200, the bitstream is decoded by the decoder elements as described below. The video decoder 200 generally performs the same Fig.15 The encoding process shown is the reverse of the decoding process. Encoder 100 also typically performs video decoding as part of encoding the video data.

[0264] Specifically, the input to the decoder includes a video bitstream, which may be generated by the video encoder 100. The bitstream is first entropy decoded (230) to obtain transform coefficients, motion vectors, and other encoding information. Picture partition information indicates how the picture is partitioned. Therefore, the decoder can divide (235) the picture according to the decoded picture partition information. The transform coefficients are inverse quantized (240) and inverse transformed (250) to decode the prediction residual. The decoded prediction residual and the prediction block are combined (255) to reconstruct the image block. The prediction block can be obtained (270) by intra-frame prediction (260) or motion compensated prediction (i.e., inter-frame prediction) (275). Loop filtering (265) is applied to the reconstructed image. The filtered image is stored in a reference picture buffer (280).

[0265] The decoded picture may be further subjected to post-decoding processing (285), such as an inverse color transform (e.g., from YCbCr 4:2:0 to RGB 4:4:4) or inverse remapping of the inverse of the remapping process performed in the pre-encoding process (101). The post-decoding processing may use metadata derived in the pre-encoding process and signaled in the bitstream.

[0266] Fig.17A block diagram of an example of a system implementing various aspects and embodiments is shown. System 1000 may be embodied as a device including a variety of components described below, and is configured to perform one or more aspects described in this document. Examples of such devices include, but are not limited to, a variety of electronic devices, such as personal computers, notebook computers, smart phones, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, networked home appliances, and servers. The elements of system 1000 may be embodied in a single integrated circuit (IC), multiple ICs, and / or discrete components, either individually or in combination. For example, in at least one embodiment, the processing and encoder / decoder elements of system 1000 are distributed over multiple ICs and / or discrete components. In various embodiments, system 1000 is coupled to one or more other systems or other electronic devices in a communication manner, such as by a communication bus or by a dedicated input and / or output port. In various embodiments, system 1000 is configured to implement one or more aspects described in this document.

[0267] The system 1000 includes at least one processor 1010 configured to execute instructions loaded therein to implement, for example, various aspects described in this document. The processor 1010 may include embedded memory, input-output interfaces, and various other circuits known in the art. The system 1000 includes at least one memory 1020 (e.g., a volatile memory device and / or a non-volatile memory device). The system 1000 includes a storage device 1040, which may include a non-volatile memory and / or a volatile memory, including but not limited to an electrically erasable programmable read-only memory (EEPROM), a read-only memory (ROM), a programmable read-only memory (PROM), a random access memory (RAM), a dynamic random access memory (DRAM), a static random access memory (SRAM), a flash memory, a magnetic disk drive, and / or an optical disk drive. As non-limiting examples, the storage device 1040 may include an internal storage device, an additional storage device (including removable and non-removable storage devices), and / or a network accessible storage device.

[0268] The system 1000 includes an encoder / decoder module 1030, which is configured to, for example, process data to provide encoded video or decoded video, and the encoder / decoder module 1030 may include its own processor and memory. The encoder / decoder module 1030 represents a module that can be included in a device to perform encoding and / or decoding functions. As is well known, a device may include one or both of the encoding and decoding modules. In addition, the encoder / decoder module 1030 may be implemented as a separate element of the system 1000, or may be incorporated into the processor 1010 as a combination of hardware and software known to those skilled in the art.

[0269] Program code loaded onto the processor 1010 or the encoder / decoder 1030 to perform various aspects described in this document may be stored in the storage device 1040 and subsequently loaded onto the memory 1020 for execution by the processor 1010. According to various embodiments, one or more of the processor 1010, the memory 1020, the storage device 1040, and the encoder / decoder module 1030 may store one or more of a variety of items during the execution of the processes described in this document. These stored items may include, but are not limited to, input video, decoded video or portions of decoded video, bitstreams, matrices, variables, and intermediate or final results from equations, formulas, operations, and arithmetic logic processing.

[0270] In some embodiments, memory internal to the processor 1010 and / or the encoder / decoder module 1030 is used to store instructions and provide processing working memory required during encoding or decoding. However, in other embodiments, memory external to the processing device (e.g., the processing device may be the processor 1010 or the encoder / decoder module 1030) is used for one or more of these functions. The external memory may be memory 1020 and / or a storage device 1040, such as a dynamic volatile memory and / or a non-volatile flash memory. In various embodiments, the external non-volatile flash memory is used to store, for example, an operating system for a television. In at least one embodiment, a fast external dynamic volatile memory (e.g., RAM) is used as working memory for video encoding and decoding operations (e.g., MPEG-2 (MPEG refers to Moving Picture Experts Group, MPEG-2 is also known as ISO / IEC 13818, 13818-1 is also known as H.222, 13818-2 is also known as H.262), HEVC (HEVC refers to High Efficiency Video Coding, also known as H.265 and MPEG-H Part 2), or VVC (Versatile Video Coding, a new standard being developed by the Joint Video Experts Group JVET)).

[0271] Inputs to the elements of system 1000 may be provided through various input devices as shown in block 1130. Such input devices include, but are not limited to: (i) a radio frequency (RF) section that receives RF signals transmitted over the air by, for example, a broadcaster; (ii) a component (COMP) input terminal (or a set of COMP input terminals); (iii) a universal serial bus (USB) input terminal; and / or (iv) a high-definition multimedia interface (HDMI) input terminal. Fig.17 Other examples not shown include composite video.

[0272] In various embodiments, the input device of block 1130 has associated input processing elements known in the art. For example, the RF section may be associated with elements adapted to (i) select a desired frequency (also referred to as selecting a signal or bandwidth limiting a signal to a frequency band), (ii) downconvert the selected signal, (iii) again bandwidth limit a narrower frequency band to select (for example) a signal frequency band (which may be referred to as a channel in some embodiments), (iv) demodulate the downconverted and bandwidth limited signal, (v) perform error correction, and (vi) demultiplex to select a desired packet stream. The RF section of various embodiments includes one or more elements to perform these functions, such as a frequency selector, a signal selector, a bandwidth limiter, a channel selector, a filter, a downconverter, a demodulator, an error corrector, and a demultiplexer. The RF section may include a tuner that performs a variety of these functions, including, for example, downconverting a received signal to a lower frequency (e.g., an intermediate frequency or near baseband frequency) or baseband. In a set-top box embodiment, the RF part and its associated input processing element receive the RF signal transmitted by wired (for example, cable) medium, and filter to the desired frequency band again by filtering, down-conversion and perform frequency selection. Various embodiments rearrange the order of above-mentioned (and other) elements, remove some of them, and / or add other elements of similar or different functions. Adding element can include inserting element between existing element, for example inserting amplifier and analog / digital converter. In various embodiments, the RF part comprises antenna.

[0273] In addition, the USB and / or HDMI terminals may include respective interface processors for connecting the system 1000 to other electronic devices via USB and / or HDMI connections. It should be appreciated that aspects of input processing (e.g., Reed-Solomon error correction) may be implemented, for example, within a separate input processing IC or within the processor 1010 as desired. Similarly, aspects of USB or HDMI interface processing may be implemented, for example, within a separate interface IC or within the processor 1010 as desired. The demodulated, error corrected, and demultiplexed streams are provided to a variety of processing elements, including, for example, the processor 1010 and the encoder / decoder 1030, which operate in conjunction with memory and storage elements to process the data streams as desired for presentation on an output device.

[0274] The various components of system 1000 may be disposed within an integrated housing. Within the integrated housing, the various components may be interconnected and data may be transmitted between them using a suitable connection arrangement (e.g., an internal bus known in the art, including an Inter-IC (I2C) bus, wiring, and a printed circuit board).

[0275] The system 1000 includes a communication interface 1050 that enables communication with other devices via a communication channel 1060. The communication interface 1050 may include, but is not limited to, a transceiver configured to send and receive data via the communication channel 1060. The communication interface 1050 may include, but is not limited to, a modem or a network card, and the communication channel 1060 may be implemented, for example, within a wired and / or wireless medium.

[0276] In various embodiments, data is provided to the system 1000 in a streaming or otherwise manner using a wireless network (e.g., a Wi-Fi network, such as IEEE 802.11 (IEEE refers to the Institute of Electrical and Electronics Engineers)). The Wi-Fi signals of these embodiments are received through a communication channel 1060 and a communication interface 1050 suitable for Wi-Fi communication. The communication channel 1060 of these embodiments is typically connected to an access point or router that provides access to an external network (including the Internet) to allow for streaming applications and other over-the-top communications. Other embodiments provide streaming data to the system 1000 using a set-top box that transmits data through an HDMI connection of an input block 1130. Other embodiments provide streaming data to the system 1000 using an RF connection of an input block 1130. As described above, various embodiments provide data in a non-streaming manner. In addition, various embodiments use wireless networks other than Wi-Fi, such as cellular networks or Bluetooth networks.

[0277] The system 1000 can provide output signals to a variety of output devices, including a display 1100, a speaker 1110, and other peripherals 1120. The display 1100 of various embodiments includes one or more of a touch screen display, an organic light emitting diode (OLED) display, a curved display, and / or a foldable display. The display 1100 can be used for a television, a tablet computer, a laptop computer, a mobile phone, or other devices. The display 1100 can also be integrated with other components (e.g., in a smartphone) or be separate (e.g., an external display of a laptop computer). In examples of various embodiments, other peripherals 1120 include one or more of a stand-alone digital video disc (or digital versatile disc) (DVR, for both terms), a disc player, a stereo system, and / or a lighting system. Various embodiments use one or more peripherals 1120 that provide functions based on the output of the system 1000. For example, a disk player performs the function of playing the output of the system 1000.

[0278] In various embodiments, control signals are communicated between the system 1000 and the display 1100, speakers 1110, or other peripheral devices 1120 using signaling such as AV.Link, Consumer Electronics Control (CEC), or other communication protocols that enable device-to-device control with or without user intervention. Output devices may be communicatively coupled to the system 1000 through dedicated connections via respective interfaces 1070, 1080, and 1090. Alternatively, output devices may be connected to the system 1000 through a communication interface 1050 using a communication channel 1060. The display 1100 and speakers 1110 may be integrated with other components of the system 1000 in a single unit in an electronic device, such as a television. In various embodiments, the display interface 1070 includes a display driver, such as a timing controller (T Con) chip.

[0279] The display 1100 and the speaker 1110 may also be separate from one or more other components, for example, if the RF portion of the input 1130 is part of a separate set-top box. In various embodiments where the display 1100 and the speaker 1110 are external components, the output signal may be provided via a dedicated output connection including, for example, an HDMI port, a USB port, or a COMP output.

[0280] The embodiments may be implemented by computer software or hardware or a combination of hardware and software implemented by the processor 1010. As a non-limiting example, the embodiments may be implemented by one or more integrated circuits. The memory 1020 may be of any type suitable for the technical environment and may be implemented using any appropriate data storage technology, such as optical memory devices, magnetic memory devices, semiconductor-based memory devices, fixed memory, and removable memory as non-limiting examples. The processor 1010 may be of any type suitable for the technical environment and may include one or more microprocessors, general-purpose computers, special-purpose computers, and processors based on multi-core architectures as non-limiting examples.

[0281] Various implementations involve decoding. "Decoding" as used in this application may encompass, for example, all or part of a process performed on a received coded sequence to produce a final output suitable for display. In various embodiments, such a process includes one or more processes typically performed by a decoder, such as entropy decoding, inverse quantization, inverse transform, and differential decoding. In various embodiments, such a process also includes or alternatively includes a process performed by a decoder of the various embodiments described in this application.

[0282] As a further example, in one embodiment, "decoding" refers only to entropy decoding, in another embodiment, "decoding" refers only to differential decoding, and in another embodiment, "decoding" refers to a combination of entropy decoding and differential decoding. Whether the phrase "decoding process" is intended to specifically refer to a subset of operations or to generally refer to a broader decoding process will become clear based on the context of the specific description and is believed to be well understood by those skilled in the art.

[0283] Various implementations involve encoding. Similar to the above discussion about "decoding", "encoding" as used in this application can cover, for example, all or part of the processes performed on an input video sequence to produce an encoded bitstream. In various embodiments, such processes include one or more processes typically performed by an encoder, such as partitioning, differential encoding, transforms, quantization, and entropy encoding. In various embodiments, such processes also include or alternatively include processes performed by the encoder of the various embodiments described in this application.

[0284] As a further example, in one embodiment, "encoding" refers only to entropy encoding, in another embodiment, "encoding" refers only to differential encoding, and in another embodiment, "encoding" refers to a combination of differential encoding and entropy encoding. Whether the phrase "encoding process" is intended to refer specifically to a subset of operations or generally to a broader encoding process will become clear based on the context of the specific description and is believed to be well understood by those skilled in the art.

[0285] It should be noted that the syntax elements used in this article are descriptive terms. Therefore, they do not exclude the use of other syntax element names.

[0286] When a diagram is presented as a flow chart, it should be understood that it also provides a block diagram of the corresponding device. Similarly, when a diagram is presented as a block diagram, it should be understood that it also provides a flow chart of the corresponding method / process.

[0287] Various embodiments may involve parameter models or rate-distortion optimization. Specifically, during the encoding process, a balance or trade-off between rate and distortion is usually considered, usually taking into account constraints on computational complexity. It can be measured by a rate-distortion optimization (RDO) metric, or by a least mean square (LMS), mean absolute error (MAE) or other such measurements. Rate-distortion optimization is usually expressed as minimizing a rate-distortion function, which is a weighted sum of rate and distortion. There are different approaches to solving the rate-distortion optimization problem. For example, these approaches can be based on extensive testing of all coding options, including all considered modes or coding parameter values, and a complete evaluation of their coding costs and the associated distortion of the reconstructed signal after encoding and decoding. Faster methods can also be used to save coding complexity, in particular, to calculate approximate distortion based on a prediction or prediction residual signal rather than a reconstructed signal. The two approaches can also be mixed, for example, using approximate distortion only for some possible coding options and using complete distortion for other coding options. Other schemes only evaluate a subset of possible coding options. More generally, many schemes employ any of a variety of techniques to perform optimization, but the optimization is not necessarily a complete assessment of the coding cost and associated distortion.

[0288] The embodiments and aspects described herein may be implemented as, for example, methods or processes, devices, software programs, data streams or signals. Even if only discussed in the context of a single implementation form (e.g., discussed only as a method), the implementation of the features discussed may also be implemented in other forms (e.g., devices or programs). The device may, for example, be implemented as appropriate hardware, software, and firmware. The method may, for example, be implemented as a processor, which generally refers to a processing device, including, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device. The processor also includes a communication device, such as a computer, a mobile phone, a portable / personal digital assistant ("PDA"), and other devices that facilitate information communication between end users.

[0289] References to "an embodiment" or "an embodiment" or "an implementation" or "an implementation" and other variations thereof mean that the particular features, structures, characteristics, etc. described in connection with the embodiment are included in at least one embodiment. Therefore, the phrases "in an embodiment" or "in an embodiment" or "in an implementation" or "in an implementation" and any other variations appearing in multiple places in the present application do not necessarily all refer to the same embodiment.

[0290] Furthermore, the present application may involve “determining” a variety of information. Determining information may include one or more of the following, such as estimating information, calculating information, predicting information, or retrieving information from memory.

[0291] Furthermore, the present application may involve "accessing" various information. Accessing information may include one or more of the following, such as receiving information, retrieving information (e.g., from a memory), storing information, moving information, copying information, calculating information, determining information, predicting information, or estimating information.

[0292] Furthermore, the present application may involve "receiving" a variety of information. Like "accessing," receiving is intended to be a broad term. Receiving information may include one or more of the following, such as accessing information or retrieving information (e.g., from a memory). Furthermore, "receiving" generally involves in some way during an operation such as storing information, processing information, transmitting information, moving information, copying information, erasing information, calculating information, determining information, predicting information, or estimating information.

[0293] It should be understood that the use of any of the following " / ", "and / or", and "at least one", such as in the case of "A / B", "A and / or B", and "at least one of A and B", is intended to cover selecting only the first listed option (A), or only the second listed option (B), or selecting both options (A and B). As a further example, in the case of "A, B, and / or C" and "at least one of A, B, and C", such wording is intended to cover selecting only the first listed option (A), or only the second listed option (B), or only the third listed option (C), or only the first and second listed options (A and B), or only the first and third listed options (A and C), or only the second and third listed options (B and C), or all three options (A and B and C). This can be extended to as many items as listed as will be apparent to one of ordinary skill in this and related arts.

[0294] In addition, as used herein, the term "signal" refers, among other things, to indicating something to a corresponding decoder. For example, in some embodiments, an encoder signals a specific one of a plurality of transforms, coding modes, or flags. Thus, in one embodiment, the same transform, parameter, or mode is used on the encoder side and the decoder side. Thus, for example, an encoder may transmit (explicitly signal) a specific parameter to a decoder so that the decoder may use the same specific parameter. Conversely, if the decoder already has the specific parameter as well as other parameters, it may be signaled (implicitly signaled) without transmission to simply allow the decoder to know and select the specific parameter. Bit savings are achieved in various embodiments by avoiding transmission of any actual functionality. It should be understood that signaling may be accomplished in a variety of ways. For example, in various embodiments, one or more grammatical elements, flags, etc. are used to signal information to a corresponding decoder. Although the verb form of the word "signal" is mentioned above, the word "signal" may also be used as a noun here.

[0295] As will be appreciated by those of ordinary skill in the art, embodiments may generate a variety of formatted signals to carry information that may be, for example, stored or transmitted. The information may include, for example, instructions for executing a method or data generated by one of the embodiments. For example, a signal may be formatted to carry a bitstream of the embodiment. Such a signal may be formatted as, for example, an electromagnetic wave (e.g., using a radio frequency portion of a spectrum) or a baseband signal. Formatting may include, for example, encoding a data stream and modulating a carrier with the encoded data stream. The information carried by the signal may be, for example, analog or digital information. As is well known, a signal may be transmitted over a variety of different wired or wireless links. The signal may be stored on a processor readable medium.

[0296] In the previous section, multiple embodiments are described, involving multiple claim categories and types. The features of these embodiments may be provided individually or in any combination. In addition, embodiments may include one or more of the following features, devices, or aspects, provided individually or in any combination, involving multiple claim categories and types:

[0297] An embodiment includes predicting a video block using an intra coding mode and reference samples from a temporally co-located reference frame.

[0298] An embodiment includes using an intra prediction motion model to predict samples from a reference frame.

[0299] An embodiment includes the method above, wherein the intra-coding mode is determined at an encoder and the mode and / or reference frames are signaled to a corresponding decoder.

[0300] An embodiment includes the above method wherein a different motion model is used to determine the reference frame or the motion vector.

[0301] An embodiment includes any of the methods above, wherein an index is signaled to indicate a motion model, reference frame, or motion vector to be used for encoding / decoding.

[0302] An embodiment includes any of the above methods using reference samples surrounding the motion compensated block from a reference frame.

[0303] An embodiment includes a bitstream or signal including one or more syntax elements for performing the above functions or variations thereof.

[0304] An embodiment comprises a bitstream or signal comprising syntax conveying information generated according to any of the described embodiments.

[0305] An embodiment comprises creating and / or sending and / or receiving and / or decoding according to any of the described embodiments.

[0306] An embodiment includes a method, process, apparatus, medium storing instructions, medium storing data, or signal according to any of the described embodiments.

[0307] An embodiment includes inserting syntax elements in the signaling syntax elements so that the decoder can determine the decoding information in a manner corresponding to that used by the encoder.

[0308] An embodiment comprises creating and / or sending and / or receiving and / or decoding a bitstream or signal comprising one or more of the described syntax elements or variants thereof.

[0309] An embodiment includes a television, a set-top box, a mobile phone, a tablet computer or other electronic device, which executes the conversion method according to any of the embodiments.

[0310] One embodiment includes a television, set-top box, mobile phone, tablet computer, or other electronic device that performs a transformation method according to any of the described embodiments and displays (eg, using a monitor, screen, or other type of display) the resulting image.

[0311] An embodiment includes a television, set-top box, mobile phone, tablet or other electronic device that selects channels, bandwidth limits or tunes (e.g., using a tuner) to receive a signal including an encoded image and performs a conversion method according to any of the described embodiments.

[0312] One embodiment includes a television, set-top box, mobile phone, tablet computer or other electronic device that wirelessly receives (eg, using an antenna) a signal including an encoded image and performs the transformation method.

Claims

1. A method, include: extracting motion information for a video block; obtaining motion compensated reference samples based on the motion information for the video block; determining an intra prediction based on the motion compensated reference samples for the video block; as well as At least a portion of the video block is encoded using the intra prediction.

2. A device, include: A processor configured to perform: extracting motion information for a video block; obtaining motion compensated reference samples based on the motion information for the video block; determining an intra prediction based on the motion compensated reference samples for the video block; as well as At least a portion of the video block is encoded using the intra prediction.

3. A method, include: extracting motion information for a video block; obtaining motion compensated reference samples based on the motion information for the video block; determining an intra prediction based on the motion compensated reference samples for the video block; as well as At least a portion of the video block is decoded using the intra prediction.

4. A device, include: A processor configured to perform: extracting motion information for a video block; obtaining motion compensated reference samples based on the motion information for the video block; determining an intra prediction based on the motion compensated reference samples for the video block; as well as At least a portion of the video block is decoded using the intra prediction.

5. The method according to claim 1 or 3, or the apparatus according to claim 2 or 4, wherein the extracted motion information comprises a motion vector.

6. A method according to claim 1, 3 or 5, or an apparatus according to claim 2, 4 or 5, wherein the motion compensated reference samples are obtained from a reference frame.

7. The method or device of claim 6, wherein the reference frame is co-located with a frame comprising a video block.

8. The method according to claim 1, 3 or 5 to 7, or the apparatus according to claim 2, 4 or 5 to 7, wherein the motion information is determined by a motion model.

9. The method or apparatus according to claim 8, wherein the motion model is calculated in an encoder and sent to a corresponding decoder as header information.

10. A method according to claim 1, 3 or 5 to 9, or an apparatus according to claim 2, 4 or 5 to 9, wherein the reference samples from the reference frame can comprise samples surrounding the block in the reference frame.

11. A method according to claim 1, 3 or 5 to 10, or an apparatus according to claim 2, 4 or 5 to 10, wherein the reference co-located frame to be used or the index of the reference frame to be used is signaled.

12. A device, include: The device according to claim 4; and At least one of: (i) an antenna configured to receive a signal, the signal comprising a video block; (ii) a frequency band limiter configured to limit the received signal to a frequency band comprising the video block; and (iii) a display configured to display an output representing the video block.

13. A non-transitory computer-readable medium comprising data content generated based on the method according to any one of claims 1 and 5 to 11 or by the apparatus according to any one of claims 2 and 5 to 11 for playback using a processor.

14. A signal comprising video data generated based on the method according to any one of claims 1 and 5 to 11 or by the apparatus according to any one of claims 2 and 5 to 11 for playback using a processor.

15. A computer program product comprising instructions, wherein the instructions cause a computer to perform the method according to any one of claims 1, 3 and 5 to 11 when the computer executes the program.