Extended reference intra-picture prediction

Abstract

The video encoder is configured to encode an image of a video into coded data by block-based predictive coding including intra-image prediction. For intra-image prediction, the video encoder is configured to use a plurality of nearest reference samples and a plurality of extended reference samples of images immediately adjacent to the predictive block to encode a predictive block of the image, where each extended reference sample of the plurality of extended reference samples is separated from the predictive block by at least one nearest reference sample of the plurality of reference samples. The video encoder is further configured to sequentially determine availability or unavailability of each of the plurality of nearest reference samples and replace a nearest reference sample determined to be unavailable with a replacement sample. The video encoder is configured to use the replacement samples for intra-image prediction.

Description

[Technical field]

[0001] The present invention relates to video coding, in particular hybrid video coding including intra-picture prediction. The present invention further relates to a video encoder, a video decoder and methods for video encoding and decoding, respectively. [Background technology]

[0002] H.265 / HEVC is a video codec that already provides tools to improve or enable parallel processing in the encoder and / or decoder. For example, HEVC supports subdivision of an image into an array of tiles that are coded independently of each other. Another concept supported in HEVC is related to WPP, according to which CTU rows or CTU lines of an image can be processed in parallel (i.e., in stripes) from left to right, if a minimum CTU offset is respected in the processing of successive CTU lines. However, it would be preferable to have a video codec at hand that supports parallel processing capabilities of a video encoder and / or video decoder even more efficiently. Summary of the Invention [Problem to be solved by the invention]

[0003] It is therefore an object of the present invention to provide a video codec that allows more efficient processing in an encoder and / or decoder with regard to reference samples used to predict a predictive block. [Means for solving the problem]

[0004] This object is achieved by the subject matter of the independent claims of the present application.

[0005] According to an embodiment, a video encoder is configured to encode an image of a video into encoded data by block-based predictive coding, the block-based predictive coding including intra-image prediction. The video encoder is configured to use a plurality of extended reference samples of the image for encoding a predictive block of the image for intra-image prediction, each extended reference sample of the plurality of extended reference samples being separated from the predictive block by at least one nearest reference sample of the plurality of reference samples that is directly adjacent to the predictive block. The video encoder is further configured to sequentially determine the availability or unavailability of each of the plurality of nearest reference samples and to replace the nearest reference sample determined to be unavailable by a replacement sample. The video encoder is configured to use the replacement sample for intra-image prediction. This allows the concept of prediction using the nearest reference sample to be used even when such a sample is unavailable, which may occur, for example, when having a line or row of samples in a buffer / memory and not actually having a column of samples in the memory and therefore unavailable.

[0006] According to an embodiment, a video encoder is configured to encode an image of a video into encoded data by block-based predictive coding, the block-based predictive coding including intra-image prediction. The video encoder is configured to use a plurality of extended reference samples of an image in the intra-image prediction to encode a predictive block of the image, each extended reference sample of the plurality of extended reference samples being separated from the predictive block by at least one closest reference sample of the plurality of reference samples that is directly adjacent to the predictive block. The video encoder is further configured to filter at least a subset of the plurality of extended reference samples using a bilateral filter to obtain a plurality of filtered extended reference samples, and to use the plurality of filtered extended reference samples for the intra-image prediction.

[0007] According to an embodiment, a video encoder is configured to encode an image of a video into encoded data by block-based predictive coding, the block-based predictive coding including intra-image prediction. The video encoder is configured to use a plurality of extended reference samples of an image in the intra-image prediction to encode a predictive block of the image, each extended reference sample of the plurality of extended reference samples being separated from the predictive block by at least one nearest reference sample of the plurality of reference samples that is directly adjacent to the predictive block, the plurality of nearest reference samples being arranged along a first image direction of the predictive block and along a second image direction of the predictive block, and to map at least a portion of the nearest reference samples arranged along the second direction to the extended reference samples arranged along the first direction such that the mapped reference samples exceed an extension of the predictive block along the first image direction. The video encoder is configured to use the mapped extended reference samples for prediction.

[0008] According to an embodiment, a video encoder is configured to encode an image of a video into encoded data by block-based predictive coding, the block-based predictive coding including intra-image prediction. The video encoder is configured to use, for intra-image prediction, a plurality of nearest reference samples of an image directly adjacent to the predictive block and a plurality of extended reference samples to encode a predictive block of the image, each extended reference sample of the plurality of reference samples being separated from the predictive block by at least one nearest reference sample of the plurality of reference samples, the video encoder is configured to boundary filter in a mode in which the extended samples are not used and not use boundary filtering when the extended samples are used, or the video encoder is configured to boundary filter at least a subset of the plurality of nearest reference samples and not use boundary filtering for the extended samples.

[0009] According to an embodiment, a video encoder is configured to encode an image of a video into encoded data by block-based predictive coding, the block-based predictive coding including intra-image prediction. The video encoder is configured to determine, in the intra-image prediction, a plurality of closest reference samples of an image directly adjacent to the predictive block and a plurality of extended reference samples for encoding a predictive block of the image, where each extended reference sample of the plurality of reference samples is separated from the predictive block by at least one closest reference sample of the plurality of extended reference samples. The video encoder is further configured to determine a prediction of the predictive block using the extended reference samples, filter the extended reference samples to obtain a plurality of filtered extended reference samples, and combine the prediction and the filtered extended reference samples to obtain a combined prediction of the predictive block.

[0010] According to an embodiment, a video encoder is configured to encode an image of a video into encoded data by block-based predictive coding, the block-based predictive coding including intra-image prediction. The video encoder is configured to use a plurality of closest reference samples and / or a plurality of extended reference samples of images directly adjacent to the predictive block to encode a predictive block of the image for intra-image prediction, each extended reference sample of the plurality of extended reference samples being separated from the predictive block by at least one closest reference sample of the plurality of reference samples, determine a first prediction of the predictive block using a first prediction mode of a set of prediction modes, the first prediction mode including a prediction mode using the plurality of closest reference samples in the absence of an extended reference sample, and determine a second prediction of the predictive block using a second prediction mode of a second set of prediction modes, the second set including a subset of the prediction modes of the first set, the subset being associated with the plurality of extended reference samples. The video encoder is configured to determine a first prediction of the predictive block using a first prediction mode of a set of prediction modes, the second prediction mode including a subset of the prediction modes of the first set, the subset being associated with the plurality of extended reference samples. 0 (x, y)) and the second prediction (p i (x, y)) and weighted (w 0 ;w i) to obtain a combined prediction (p(x,y)) as a prediction for a predictive block of encoded data.

[0011] According to an embodiment, a video encoder encodes an image of a video into encoded data by block-based predictive coding including intra-image prediction, and for the intra-image prediction, for encoding a predictive block of the image, using a plurality of nearest reference samples and / or a plurality of extended reference samples of images directly neighboring the predictive block, each extended reference sample of the plurality of extended reference samples being separated from the predictive block by at least one nearest reference sample of the plurality of reference samples, e.g., in the absence of an extended reference sample, using a prediction mode that is one of a first set of prediction modes for predicting the predictive block using the nearest reference samples, or using a prediction mode for predicting the predictive block using the extended reference samples. The decoder is configured to enable a conclusion in the decoder that a particular value of the parameter is selected or determined by using a prediction mode that is one of a second set of modes, the second set of prediction modes being a subset of the first set of prediction modes, signaling mode information (m) indicating a prediction mode used to predict the prediction block, and then signaling parameter information (i) indicating a subset of extended reference samples used for the prediction mode if the prediction mode is included in the second set of prediction modes, and skipping the signaling of the parameter information if the prediction mode used is not included in the second set of prediction modes, where a predetermined property allows the skipping of the signaling, i.e., the absence of a signal is given a beneficial meaning. For example, the absence can indicate that the nearest reference sample should be used.

[0012] According to an embodiment, a video encoder is configured to encode an image of a video into encoded data by block-based predictive coding including intra-image prediction, and for intra-image prediction, to encode a predictive block of the image, using a plurality of reference samples including a nearest reference sample of an image directly adjacent to the predictive block and a plurality of extended reference samples, each extended reference sample of the plurality of extended reference samples being separated from the predictive block by at least one nearest reference sample of the plurality of reference samples, and using a prediction mode that is one of a first set of prediction modes for predicting the predictive block using the nearest reference sample or a prediction mode that is one of a second set of prediction modes for predicting the predictive block using the extended reference samples, the second set of prediction modes being a subset of the first set of prediction modes. The video encoder can generate the first set and / or the second set using available reference data and / or can determine the set using information derived from the image. The second set being a subset of the first set includes the case where both sets are equal. The video encoder is configured to signal parameter information indicating a subset of a plurality of reference samples used for a prediction mode, the subset of the plurality of reference samples including only the nearest reference sample or the extended reference sample, and then signal mode information (m) indicating a prediction mode to be used to predict a prediction block, the mode information indicating a prediction mode from the subset of modes, the subset being restricted to a set of allowed prediction modes according to the parameter information (i). Based on the association of the used reference sample, i.e., nearest or extended, only the prediction mode associated with the indicated reference sample is applied, thus enabling identification of the restricted set.

[0013] According to an embodiment, a video encoder is configured to encode an image of a video into encoded data by block-based predictive coding including intra-image prediction, and for the intra-image prediction, to encode a predictive block of the image, using a plurality of nearest reference samples and / or a plurality of extended reference samples of images directly neighboring the predictive block, each extended reference sample of the plurality of extended reference samples being separated from the predictive block by at least one nearest reference sample of the plurality of reference samples, to determine a first prediction of the predictive block using a first prediction mode of a set of prediction modes, the first set of prediction modes including a prediction mode using the plurality of nearest reference samples in the absence of an extended reference sample, to determine a second prediction of the predictive block using a second prediction mode of a second set of prediction modes, the second set of prediction modes including a subset of the prediction modes of the first set being associated with the plurality of extended reference samples. The video encoder is configured to combine the first prediction and the second prediction to obtain a combined prediction as a prediction of the predictive block in the encoded data.

[0014] According to an embodiment, a video encoder is configured to encode an image of a video into encoded data by block-based predictive coding including intra-image prediction, and in the intra-image prediction, to use a plurality of extended reference samples of the image for encoding a predictive block of the image, each extended reference sample of the plurality of extended reference samples being separated from the predictive block by at least one nearest reference sample of the plurality of reference samples directly neighboring the predictive block, and to use the plurality of extended reference samples according to a predetermined set of the plurality of extended reference samples. The plurality of extended reference samples can, for example, be included in a list of area indexes identified by an identifier.

[0015] According to an embodiment, a video encoder is configured to encode a plurality of predictive blocks into encoded data by block-based predictive encoding including intra-image prediction, and for the intra-image prediction, to use a plurality of extended reference samples of an image for encoding a predictive block of the plurality of predictive blocks, where each extended reference sample of the plurality of extended reference samples is separated from the predictive block by at least one nearest reference sample of the plurality of reference samples directly neighboring the predictive block. The video encoder is configured to determine the extended reference sample to be at least a partial part of a neighboring predictive block of the plurality of predictive blocks, determine that the neighboring predictive block has not yet been predicted, and signal information indicative of the extended predictive sample associated with the predictive block and located in the neighboring predictive block as an unavailable sample.

[0016] According to an embodiment, a video decoder is configured to decode an image encoded by encoding data into video by block-based predictive decoding including intra-image prediction, in which a plurality of extended reference samples of the image are used to encode a predictive block of the image, each extended reference sample of the plurality of extended reference samples being separated from the predictive block by at least one nearest reference sample of the plurality of reference samples that is directly adjacent to the predictive block, sequentially determine the availability or unavailability of each of the plurality of nearest reference samples, replace the nearest reference sample determined to be unavailable by the replacement sample, and use the replacement sample for the intra-image prediction.

[0017] According to an embodiment, a video decoder is configured to decode an image encoded by encoding data into video by block-based predictive decoding including intra-image prediction, in which the video decoder uses a plurality of extended reference samples of the image to decode a predictive block of the image, each extended reference sample of the plurality of extended reference samples being separated from the predictive block by at least one nearest reference sample of the plurality of reference samples directly neighboring the predictive block, filter at least a subset of the plurality of extended reference samples using a bilateral filter to obtain a plurality of filtered extended reference samples, and use the plurality of filtered extended reference samples for the intra-image prediction.

[0018] According to an embodiment, a video decoder is configured to decode an image encoded by encoding data into video by block-based predictive decoding including intra-image prediction, and in the intra-image prediction, to use a plurality of extended reference samples of the image to decode a predictive block of the image, each extended reference sample of the plurality of extended reference samples being separated from the predictive block by at least one nearest reference sample of the plurality of reference samples directly adjacent to the predictive block, the plurality of nearest reference samples being arranged along a first image direction of the predictive block and along a second image direction of the predictive block, to map at least a portion of the nearest reference samples arranged along the second direction to the extended reference samples arranged along the first direction such that the mapped reference samples exceed an extension of the predictive block along the first image direction, and to use the mapped extended reference samples for prediction.

[0019] According to an embodiment, a video decoder is configured to decode an image encoded by encoding data into video by block-based predictive decoding including intra-image prediction, and for the intra-image prediction, to use a plurality of nearest reference samples of an image directly neighboring the predictive block and a plurality of extended reference samples for decoding a predictive block of the image, each extended reference sample of the plurality of extended reference samples being separated from the predictive block by at least one nearest reference sample of the plurality of reference samples. The video decoder is configured to perform boundary filtering in a mode in which the extended samples are not used, and to not use boundary filtering when the extended samples are used, or to border filter at least a subset of the plurality of nearest reference samples and not use boundary filtering for the extended samples.

[0020] According to an embodiment, a video decoder is configured to decode an image encoded by encoding data into video by block-based predictive decoding including intra-image prediction, and in the intra-image prediction, to decode a predictive block of the image, determine a plurality of nearest reference samples and a plurality of extended reference samples of the image directly adjacent to the predictive block, each extended reference sample of the plurality of extended reference samples being separated from the predictive block by at least one nearest reference sample of the plurality of reference samples, determine a prediction of the predictive block using the extended reference samples, filter the extended reference samples to obtain a plurality of filtered extended reference samples, and combine the prediction and the filtered extended reference samples to obtain a combined prediction of the predictive block.

[0021] According to an embodiment, a video decoder is configured to decode an image encoded by encoding data into a video by block-based predictive decoding including intra-image prediction, and for the intra-image prediction, to decode a predictive block of the image, using a plurality of nearest reference samples and / or a plurality of extended reference samples of images directly adjacent to the predictive block, each extended reference sample of the plurality of extended reference samples being separated from the predictive block by at least one nearest reference sample of the plurality of reference samples, to determine a first prediction of the predictive block using a first prediction mode of a set of prediction modes, the first set of prediction modes including a prediction mode using the plurality of nearest reference samples in the absence of an extended reference sample, and to determine a second prediction of the predictive block using a second prediction mode of a second set of prediction modes, the second set of prediction modes including a subset of the prediction modes of the first set, the subset being associated with the plurality of extended reference samples. The video decoder is configured to weightedly combine the first prediction and the second prediction to obtain a combined prediction as a prediction of the predictive block in the encoded data.

[0022] According to an embodiment, a video decoder is configured to decode an image encoded by encoding data into video by block-based predictive decoding including intra-image prediction, and for the intra-image prediction, to use a plurality of nearest reference samples and / or a plurality of extended reference samples of an image directly adjacent to the predictive block to decode a predictive block of the image, wherein each extended reference sample of the plurality of extended reference samples is separated from the predictive block by at least one nearest reference sample of the plurality of reference samples, to use a prediction mode that is one of a first set of prediction modes for predicting the predictive block using the nearest reference sample or to use a prediction mode that is one of a second set of prediction modes for predicting the predictive block using the extended reference samples, wherein the second set of prediction modes is a subset of the first set of prediction modes, to receive mode information (m) indicating a prediction mode used to predict the predictive block, and then to receive parameter information (i) indicating a subset of the extended reference samples used for the prediction mode, thereby indicating that the prediction mode is included in the second set of prediction modes, and if no parameter information is received, to determine that the used prediction mode is not included in the second set of prediction modes and to determine the use of the nearest reference sample for prediction.

[0023] According to an embodiment, a video decoder is configured to decode an image encoded by encoding data into video by block-based predictive decoding including intra-image prediction, and for the intra-image prediction, to decode a predictive block of the image, use a plurality of reference samples and a plurality of extended reference samples including a nearest reference sample of an image directly adjacent to the predictive block, each extended reference sample of the plurality of extended reference samples being separated from the predictive block by at least one nearest reference sample of the plurality of reference samples, use a prediction mode that is one of a first set of prediction modes for predicting the predictive block using the nearest reference sample or use a prediction mode that is one of a second set of prediction modes for predicting the predictive block using the extended reference samples, the second set of prediction modes being a subset of the first set of prediction modes, receive parameter information (i) indicating a subset of the plurality of reference samples used for the prediction mode, the subset of the plurality of reference samples including only the nearest reference sample or at least one extended reference sample, then receive mode information (m) indicating a prediction mode to be used to predict the predictive block, wherein the mode information indicates a prediction mode from the subset of modes, the subset being restricted to a set of allowed prediction modes according to the parameter information (i).

[0024] According to an embodiment, a video decoder is configured to decode an image encoded by encoding data into a video by block-based predictive decoding including intra-image prediction, and for the intra-image prediction, to decode a predictive block of the image, using a plurality of nearest reference samples and / or a plurality of extended reference samples of images directly neighboring the predictive block, each extended reference sample of the plurality of extended reference samples being separated from the predictive block by at least one nearest reference sample of the plurality of reference samples, to determine a first prediction of the predictive block using a first prediction mode of a set of prediction modes, the first set of prediction modes including a prediction mode using the plurality of nearest reference samples in the absence of an extended reference sample, to determine a second prediction of the predictive block using a second prediction mode of a second set of prediction modes, the second set of prediction modes including a subset of the prediction modes of the first set being associated with the plurality of extended reference samples. The video decoder is configured to combine the first prediction and the second prediction to obtain a combined prediction as a prediction of a predictive block in the encoded data.

[0025] According to an embodiment, a video decoder is configured to decode an image encoded by encoding data into video by block-based predictive decoding including intra-image prediction, and in the intra-image prediction, to use a plurality of extended reference samples of the image to decode a predictive block of the image, each extended reference sample of the plurality of extended reference samples being separated from the predictive block by at least one nearest reference sample of the plurality of reference samples directly adjacent to the predictive block, and to use the plurality of extended reference samples according to a predetermined set of the plurality of extended reference samples.

[0026] According to an embodiment, a video decoder is configured to decode images encoded by encoding data into video by block-based predictive decoding including intra-picture prediction, where for each image a plurality of predictive blocks are decoded, and for the intra-picture prediction, a plurality of extended reference samples of the image are used to decode a predictive block of the plurality of predictive blocks, where each extended reference sample of the plurality of extended reference samples is separated from the predictive block by at least one closest reference sample of the plurality of reference samples directly neighboring the predictive block. The video decoder is configured to determine the extended reference sample to be at least a partial part of a neighboring predictive block of the plurality of predictive blocks, determine that the neighboring predictive block has not yet been predicted, and receive information indicating the extended predictive sample associated with the predictive block and located as an unavailable sample in the neighboring predictive block.

[0027] Further embodiments relate to methods for encoding and decoding video, and to computer program products.

[0028] With regard to the aforementioned embodiments of the present patent application, it should be noted that same may be combined such that two or more of the aforementioned embodiments, such as all the embodiments, are implemented in a video codec simultaneously.

[0029] Further advantageous embodiments of the present patent application are the subject of the dependent claims, preferred embodiments of which are described below with reference to the figures therein. [Brief description of the drawings]

[0030] [Figure 1] 1 shows a schematic block diagram of a video encoder according to an embodiment, comprising a decoder according to an embodiment; [Diagram 2] 1 shows a schematic flow chart of a method for encoding a video stream according to an embodiment; [Diagram 3] 2 shows examples of immediately neighboring (nearest) reference samples and extended reference samples used in an embodiment. [Figure 4a] 1 illustrates an example of intra-picture prediction angles for 5 angles for a 4x2 block of prediction samples according to an embodiment. [Figure 4b] 1 illustrates an example of intra-picture prediction angles for 5 angles for a 4x2 block of prediction samples according to an embodiment. [Figure 4c] 1 illustrates an example of intra-picture prediction angles for 5 angles for a 4x2 block of prediction samples according to an embodiment. [Figure 4d] 1 illustrates an example of intra-picture prediction angles for 5 angles for a 4x2 block of prediction samples according to an embodiment. [Figure 4e] 1 illustrates an example of intra-picture prediction angles for 5 angles for a 4x2 block of prediction samples according to an embodiment. [Figure 4f] 1 shows a schematic diagram to illustrate the direction of angular prediction used in an embodiment; [Figure 4g] 1 shows a table to illustrate an example of the dependency of the number of taps used in the filter, which number depends on the block size and prediction mode of the prediction block. [Figure 4h] 1 shows a table to illustrate an example of the dependency of the number of taps used in the filter, which number depends on the block size and prediction mode of the prediction block. [Figure 5a] 1 illustrates an embodiment relating to angle prediction using angular parameter definitions. [Figure 5b] 1 illustrates an embodiment relating to angle prediction using angular parameter definitions. [Figure 5c] 1 illustrates an embodiment relating to angle prediction using angular parameter definitions. [Figure 6a] 13 illustrates the derivation of a vertical offset associated with a mapping of reference samples according to an embodiment. [Figure 6b] 13 illustrates the derivation of a vertical offset associated with a mapping of reference samples according to an embodiment. [Figure 6c]13 illustrates the derivation of a vertical offset associated with a mapping of reference samples according to an embodiment. [Figure 7a] 1 illustrates the derivation of a horizontal offset according to an embodiment. [Figure 7b] 1 illustrates the derivation of a horizontal offset according to an embodiment. [Figure 7c] 1 illustrates the derivation of a horizontal offset according to an embodiment. [Figure 8] The upper left corner of the diagonal shows an embodiment and the use of the nearest reference sample according to the embodiment. [Figure 9] 13 illustrates an example of projection of an extended left reference sample as a side reference adjacent to an extended top reference sample as a primary reference in the case of upper left diagonal prediction according to an embodiment. [Figure 10] 13 illustrates an example of a projection of the nearest left reference sample as a side reference next to an extended top reference sample according to an embodiment. [Figure 11] 1 illustrates an exemplary truncated single code for a particular set of reference regions according to an embodiment. [Figure 12a] 1 shows a schematic diagram of available block sizes according to an embodiment; [Figure 12b] 1 shows a schematic diagram of available block sizes according to an embodiment; [Figure 13] 13 shows a schematic diagram of an embodiment of vertical angle prediction with a prediction block angle of 45 degrees. [Figure 14a] 1 illustrates an example of nearest reference samples and extended reference samples required for diagonal vertical intra prediction according to an embodiment. [Figure 14b] 1 illustrates an example of nearest reference samples and extended reference samples required for diagonal vertical intra prediction according to an embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0031] Equal or equivalent elements or elements having equal or equivalent functionality are indicated in the following description by equal or equivalent reference signs even if they occur in different figures.

[0032] In the following description, a number of details are set forth to provide a more complete description of the embodiments of the present invention. However, it will be apparent to those skilled in the art that the embodiments of the present invention can be practiced without these specific details. In other instances, well-known structures and devices are shown in block diagram form rather than in detail to avoid obscuring the embodiments of the present invention. Furthermore, the features of different embodiments described below can be combined with each other unless otherwise specified.

[0033] In hybrid video coding, intra-picture prediction is used to code regions of image samples by generating a prediction signal from available neighboring samples, i.e. reference samples. The prediction signal is subtracted from the original signal to obtain a residual signal. This residual signal or prediction error is further transformed, scaled, quantized and entropy coded, for example as shown in FIG. 1, which shows a schematic block diagram of a video encoder 1000 according to an embodiment, a hybrid video encoder with an intra-picture prediction block 1001. The video encoder 1000 is configured to receive an input video signal 1002 comprising a plurality of images, the sequence of images forming a video. The video encoder 1000 comprises a block 1004 configured to divide the signal 1002 into regions of samples, i.e. to form blocks from the input video signal 1002. A controller 1006 of the video encoder 1000 is configured to control the block 1004 and to control a decoder 1008, which may be part of the encoder 1000. A decoder for receiving and decoding the output bitstream 1012 and the generated output video signal 1014, i.e. the encoded data, can be implemented accordingly. In particular, the transform, scaling and quantization block 1016, together with a block 1018 for motion estimation of a signal 1022, which is the input video signal 1002 divided into blocks by the block 1004, can provide information on both the quantized transform coefficients and the motion information to enable entropy coding of the output bitstream 1012.

[0034] The quantized transformed coefficients are then scaled and inverse transformed to generate a reconstructed residual signal, prior to a potential in-loop filtering operation. This signal can be added back to the prediction signal to obtain a reconstruction that is also available to the decoder. The reconstructed signal can be used to predict subsequent samples in coding order within the same image.

[0035] For details of intra-picture prediction, please refer to FIG. 2. First, the reference samples used for prediction are generated in block 1042 based on the reconstructed samples. This stage also includes, for example, the replacement of neighboring samples that are unavailable at the boundaries of pictures, slices or tiles. Second, in block 1044, the reference samples can be filtered to eliminate discontinuities in the reference signal. Third, in block 1046, prediction samples are calculated using the reference samples according to an intra-prediction mode. The prediction mode describes how the prediction signal is generated from the reference samples, for example, by averaging them in DC mode or copying them along one prediction angle in angle prediction mode. The encoder needs to determine which intra-prediction mode to select, and the selected intra-prediction mode is signaled in the bitstream by entropy coding to the decoder. At the decoder side, the intra-prediction mode is extracted from the bitstream by entropy decoding. Fourth, and perhaps finally, in block 1048, the prediction samples can also be filtered to smooth the signal. In other words, FIG. 2 shows a flowchart of an intra-picture prediction process or method. Generally, the correlation between samples in an image decreases as the distance increases. Therefore, directly adjacent samples are generally suitable as reference samples for predicting the area of ​​a sample. However, there are cases where directly adjacent reference samples represent edges or objects in homogeneous regions (occlusions). In these cases, the correlation between the sample to be predicted (homogeneous or textured regions) and the directly adjacent reference samples (edges) is low. Extended reference intra-picture prediction solves this problem by incorporating more distant reference samples that are not directly adjacent. Although the concept of extending the nearest reference sample is known, some novel improvements to all parts of the intra-picture prediction process and notification are defined in the embodiments of the present invention and are described below.

[0036] Extended reference intra-picture prediction allows generating a prediction signal for a sample region using extended reference samples. Extended reference samples are available reference samples that are not directly adjacent. In the following, improved reference sample generation, filtering, prediction, and prediction filtering using extended reference samples according to embodiments are described in more detail. Special cases that combine prediction using extended reference samples with prediction using directly adjacent samples or unfiltered reference samples are described later. Then, various methods according to embodiments are described to improve prediction modes and extended reference region signaling of extended reference samples. Furthermore, embodiments for facilitating parallel coding using extended reference samples are described.

[0037] For the generation of reference samples, current video coding standards predict the current block using directly adjacent samples. In addition to the nearest directly adjacent samples, it has been proposed in the literature to use multiple reference lines. The additional reference lines used in intra-picture prediction are further referred to as extended reference samples in detail below. Then, an improved method for replacing unavailable extended reference samples according to an embodiment is described.

[0038] An example showing the nearest reference sample line and three extended reference sample lines for a predicted 16×8 block is shown with the directly adjacent (nearest) reference sample 1062 and extended reference sample 1064. 1 , 1064 2 and 1064 3 An example of this is shown in Figure 3.

[0039] Closest reference sample 1062 and extended reference sample 1064 1 , 1064 2 and 1064 3are located adjacent to the prediction block 1066 to be predicted and are arranged along two directions in the image: direction x and direction y, which is arranged perpendicular to direction x. Along direction x, the prediction block includes an extension W with samples ranging from 0 to W-1. Along direction y, the prediction block includes an extension H with samples ranging from 0 to H-1.

[0040] A reference region with index i may indicate a distance between the respective reference samples, i.e. the nearest reference sample with index i=0, i.e. the reference samples located directly adjacent and the extended reference sample is spaced from the prediction block 1066 by at least the nearest reference sample 1062. For example, the reference region index i may indicate an extension of the distance between the prediction block 1066 and the respective reference sample 1062 or 1064. As an example, increasing a parameter x along a direction x may be referred to as moving to the right, and conversely, decreasing x may be referred to as moving to the left.

[0041] Alternatively, or in addition, decreasing the index i along the negative direction y can be referred to as moving towards the top or upper part of the image, and with increasing parameter y, moving towards the bottom or lower part of the image can be denoted. Terms such as top, bottom, left and right are used to simplify the understanding of the present invention. According to other embodiments, such terms can be modified, changed or replaced in any other direction without limiting the scope of the present embodiment. By way of example, the reference samples 1062 and / or 1064 located to the left from the prediction block, i.e. with x<0, can be referred to as left reference samples. Assuming that the top left corner of the prediction block 1066 has the position 0,0, yThe reference samples 1062 and / or 1064 positioned to have a .intg.<0 may be referred to as top reference samples. The identified reference samples, the left reference samples, and the top reference samples may be referred to as corner reference samples. Thus, the reference samples beyond the extension W along the x-direction may be referred to as right reference samples, and the reference samples beyond the extension H of the prediction block 1066 may be referred to as bottom reference samples.

[0042] To indicate which reference sample to use for prediction, each line of reference samples is associated with a reference region index i. The nearest reference sample is given index i=0, the next line of extended reference samples i=1, etc. Using the notation in Figure 3, the reference samples above are The left reference sample is written as TIFF0007601639000001.tif312, with y ranging from 0 to N. The parameter M for the horizontal extension of the reference sample and the parameter N for the vertical extension depend on the intra-prediction. For example, as explained in relation to Fig. 4e, if the top right diagonal prediction is used as the maximum clockwise angle, the top reference sample is TIFF0007601639000003.tif219 samples need to be expanded. For example, as illustrated in Figure 4a, if we use the bottom-left diagonal prediction as the maximum angle in the counterclockwise direction, the left reference sample is expanded vertically. TIFF0007601639000004.tif219 samples need to be expanded. As shown in Figure 3, W represents the width of the prediction block and H represents its height.

[0043] A video encoder according to an embodiment, such as the video encoder 1000, may be configured to encode an image of a video into encoded data by block-based predictive coding, the block-based predictive coding including intra-image prediction. The video encoder may use multiple extended reference samples of an image in the intra-image prediction to encode a predictive block of the image, each extended reference sample of the multiple extended reference samples being separated from the predictive block by at least one nearest reference sample of the multiple reference samples that is directly adjacent to the predictive block. The video encoder may sequentially determine availability or unavailability of each of the multiple nearest reference samples, and may replace the nearest reference sample determined to be unavailable by a replacement sample. The video encoder may use the replacement sample for the intra-image prediction.

[0044] To determine availability or unavailability, the video encoder may sequentially check the samples according to the sequence and determine the replacement sample as a copy of the last extended reference sample determined to be available in the sequence, and / or may determine the replacement sample as a copy of the next extended reference sample determined to be available in the sequence.

[0045] The video encoder may further determine availability or unavailability sequentially according to the sequence, and determine a replacement sample based on a combination of extended reference samples that are determined to be available and the reference sample is determined to be unavailable, are placed in the sequence before the extended reference sample is determined to be available, and are placed in sequence after the reference sample is determined to be unavailable.

[0046] Alternatively, or in addition, the video encoder may be configured to use a plurality of closest reference samples and a plurality of extended reference samples of an image directly adjacent to the predictive block to encode a predictive block of the image for intra-image prediction, each extended reference sample of the plurality of extended reference samples being separated from the predictive block by at least one closest reference sample of the plurality of reference samples, and to determine availability or unavailability of each of the plurality of extended reference samples. The video encoder may signal the use of the plurality of extended reference samples if a portion of the available extended reference samples of the plurality of extended reference samples is equal to or greater than a predetermined threshold, and may skip signaling the use of the plurality of extended reference samples if a portion of the available extended reference samples of the plurality of extended reference samples is below the predetermined threshold.

[0047] Thus, the video decoder 1008 or a respective decoder, such as a video decoder for generating a video stream again, may be configured to decode an image encoded with the encoding data into a video by block-based predictive decoding including intra-picture prediction, in which a plurality of extended reference samples of the image are used to encode a predictive block of the image, each extended reference sample of the plurality of extended reference samples being separated from the predictive block by at least one nearest reference sample of the plurality of reference samples directly adjacent to the predictive block, and to sequentially determine the availability or unavailability of each of the plurality of extended reference samples. The video decoder may replace the extended reference sample determined to be unavailable by a replacement sample and use the replacement sample for the intra-picture prediction.

[0048] The video decoder may further be configured to determine availability or unavailability in order according to the sequence and to determine the replacement sample as a copy of the last extended reference sample determined to be available in the sequence and / or to determine the replacement sample as a copy of the next extended reference sample determined to be available in the sequence.

[0049] Further, the video decoder may be configured to determine availability or unavailability in order according to the sequence, and to determine a replacement sample based on a combination of extended reference samples, where a reference sample is determined to be available and where a reference sample is determined to be unavailable, and where an extended reference sample is placed in the sequence before the extended reference sample is determined to be available, and where a reference sample is determined to be unavailable, and where a replacement sample is placed in the sequence after the reference sample is determined to be unavailable.

[0050] The video decoder may alternatively or additionally be configured to use a plurality of closest reference samples and a plurality of extended reference samples of an image directly adjacent to the predictive block to decode a predictive block of an image for intra-image prediction, each extended reference sample of the plurality of extended reference samples being separated from the predictive block by at least one closest reference sample of the plurality of reference samples, determine availability or unavailability of each of the plurality of extended reference samples, notify received information indicating use of the plurality of extended reference samples if a portion of the available extended reference samples of the plurality of extended reference samples is equal to or greater than a predetermined threshold and is use of the plurality of extended reference samples, and skip use of the plurality of extended reference samples in the absence of the information, i.e., if a portion of the available extended reference samples of the plurality of extended reference samples is below a predetermined threshold.

[0051] If adjacent reference samples are unavailable, then according to an embodiment, an extended reference sample replacement can be performed, for example, by replacing the unavailable sample with the nearest available adjacent sample, the combination of the two nearest neighbors, or, for example, a combination of two nearest neighbors if no adjacent samples are available. bitdepth-1 For example, reference samples are unavailable if they are outside the boundaries of a picture, slice, or tile, or if a constrained intra prediction is used that does not allow using samples from an inter-picture prediction region as reference for an intra-picture prediction region.

[0052] For example, if the current block to be predicted is at the left image border, the left and top left corner reference samples are unavailable. In this case, the left and top left corner reference samples are replaced with the first available top reference sample. This first available top reference sample is the first sample, i.e. For TIFF0007601639000005.tif312, the replacement of nearest (i=0) and extended (i>0) reference samples can be formulated as follows: TIFF0007601639000006.tif346TIFF0007601639000007.tif355The parameter i can be any maximum value greater than 0 within the encoder / decoder system, for example 1, 2, 3, 4 or more, such as 5 or more or 10 or more.

[0053] When using constrained intra prediction, it may happen that one or more neighboring blocks are not available because they were coded using inter-picture prediction. For example, the H samples on the left TIFF0007601639000008.tif312 and TIFF0007601639000009.tif313 is unavailable because it is within an inter-picture prediction block and constrained intra prediction is enabled.

[0054] In one embodiment, the availability check process for each reference sample is performed sequentially, e.g., from bottom left to top right or vice versa, and the first unavailable sample along this direction is replaced by the last available sample. If there is no previous available sample, the unavailable sample is replaced by the next available sample. In an embodiment starting from the bottom right, the W bottom right sample TIFF0007601639000010.tif312 and TIFF0007601639000011.tif314 is available. Therefore, the sample on the left is replaced with the following: In another embodiment, unavailable reference samples between available reference samples can be generated by linearly interpolating between the two nearest samples, one from each side.

[0055] If it is determined that most of the extended reference area samples are unavailable, then using the extended reference sample provides no advantage over using the closest reference sample. Thus, notification of the reference area index may be saved and may be limited to blocks where at least half of the extended reference samples are available.

[0056] A video encoder according to an embodiment, such as video encoder 1000, may be configured to encode an image of a video into encoded data by block-based predictive coding including intra-image prediction, where in the intra-image prediction, use a plurality of extended reference samples of the image to encode a predictive block of the image, where each extended reference sample of the plurality of extended reference samples is separated from the predictive block by at least one nearest reference sample of the plurality of reference samples directly neighboring the predictive block, filter at least a subset of the plurality of extended reference samples using a bilateral filter to obtain a plurality of filtered extended reference samples, and use the plurality of filtered extended reference samples for the intra-image prediction.

[0057] A video encoder according to this embodiment may be configured to combine a plurality of filtered extended reference samples with a plurality of unfiltered extended reference samples to obtain a plurality of combined reference values, and the video encoder is configured to use the plurality of combined reference values ​​for intra-image prediction.

[0058] Alternatively, or in addition, the video encoder may be configured to filter the multiple enhanced reference samples using one of a 3-tap filter, a 5-tap filter, and a 7-tap filter.

[0059] The video encoder may further be configured to select to predict the prediction block using an angular prediction mode, where the 3-tap filter, the 5-tap filter, and the 7-tap filter are configured as bilateral filters, and the video encoder may be configured to select to use one of the 3-tap filter, the 5-tap filter, and the 7-tap filter based on an angle used for the angular prediction, where the angle is disposed between the horizontal or vertical direction of the angular prediction mode, and / or the video decoder may be configured to select to use one of the 3-tap filter, the 5-tap filter, and the 7-tap filter based on a block size of the prediction block. As shown in FIG. 4f, the angle ε may represent an angle of the direction of the angular prediction used to predict the prediction block 1066 with respect to the horizontal boundary 1072 and / or the vertical boundary 1074 of the prediction block 1066, measured toward the diagonal 1076 between the horizontal and vertical directions. That is, the angle of the angular prediction is at most 45°. The larger the angle ε, the more taps can be used in the filter. Alternatively, or in addition, the block size may define a basis or dependency for selecting the filter.

[0060] A corresponding video decoder may be configured to decode an image encoded with the encoding data into video by block-based predictive decoding including intra-image prediction, where in the intra-image prediction, use a plurality of extended reference samples of the image to decode a predictive block of the image, each extended reference sample of the plurality of extended reference samples being separated from the predictive block by at least one nearest reference sample of the plurality of reference samples directly neighboring the predictive block, filter at least a subset of the plurality of extended reference samples using a bilateral filter to obtain a plurality of filtered extended reference samples, and use the plurality of filtered extended reference samples for the intra-image prediction.

[0061] The video decoder can be further configured to combine the plurality of filtered enhanced reference samples with the plurality of unfiltered enhanced reference samples to obtain a plurality of combined reference values, and the video decoder is configured to use the plurality of combined reference values ​​for intra-picture prediction.

[0062] Alternatively, or in addition, the video decoder may be configured to filter the multiple enhanced reference samples using one of a 3-tap filter, a 5-tap filter, and a 7-tap filter. As described for the encoder, the 3-tap filter, the 5-tap filter, and the 7-tap filter may be configured as a bilateral filter, and the video decoder may be configured to predict the prediction block using an angle prediction mode and select to use one of the 3-tap filter, the 5-tap filter, and the 7-tap filter based on an angle used for the angle prediction, the angle being disposed between the horizontal direction or the vertical direction of the angle prediction mode, and / or the video decoder may be configured to select to use one of the 3-tap filter, the 5-tap filter, and the 7-tap filter based on a block size of the prediction block.

[0063] For example, instead of a bilateral filter, a 3-tap FIR filter can be used, which makes it possible to filter only the nearest reference samples (even if no bilateral filter is used) and leave the extended reference samples unfiltered.

[0064] When the sample area is large, discontinuities in the reference samples can occur, distorting the prediction. The state-of-the-art solution to this is to apply a linear smoothing filter to the reference samples. In the case of discontinuities, which can be detected for example by comparison with a predefined threshold, stronger smoothing can be applied. This usually involves generating reference samples by interpolating between corner reference samples.

[0065] However, the linear smoothing filter can also remove edge structures that need to be preserved. Applying a bilateral filter to the extended reference samples according to the reference sample filtering embodiment can prevent undesired smoothing of sharp edges. Because bilateral filtering is more efficient for large blocks and intra prediction angles that deviate from the exact horizontal and exact vertical directions, the decision of whether to apply the filter and the length of the filter can depend on the block size and / or prediction mode. An example design can incorporate a dependency as shown in FIG. 4g, which shows a dependency for block sizes smaller than 64×64 and equal to or larger than 64×64 in terms of W×H, and FIG. 4h shows a different dependency for block sizes smaller than 64×64 and equal to or larger than 64×64 in terms of W×H. In the embodiment of FIG. 4g, the intra prediction mode can be, for example, one of a planar mode, a DC mode, a near horizontal mode, a near vertical mode, or a different angle of an angle mode. In the embodiment of FIG. 4h, the angles identified as more horizontal and more vertical can be further selected, for example, the angle ε shown in FIG. 4f has a larger value when compared to the near horizontal or near vertical. As can be seen, a larger block size can result in a larger number of taps to facilitate filtering of a larger amount of data, and furthermore, an increase in ε can also result in an increase in taps. According to FIG. 4h, an embodiment can apply a small 3-tap filter to the near-horizontal and near-vertical modes, and increase the filter length as the distance from the horizontal and vertical directions increases. Although shown as depending on both the prediction mode and the block size, the choice of filter or at least the number of taps can instead depend only on one of both and / or additional parameters.

[0066] In another embodiment of reference sample filtering, the intra-picture prediction using the filtered reference sample can be combined with the unfiltered reference sample using position-dependent weighting, as described in connection with the combination of position-dependent prediction. In this case, the reference sample of the prediction using the filtered reference sample can use a different reference sample filtering than in the case of uncombined prediction. For example, the filtering can be selected from a set of 3-tap, 5-tap, and 7-tap filters.

[0067] A video encoder according to an embodiment, such as video encoder 1000, may be configured to encode an image of a video into encoded data by block-based predictive coding including intra-image prediction, where in the intra-image prediction, use a plurality of extended reference samples of the image to encode a predictive block of the image, where each extended reference sample of the plurality of extended reference samples is separated from the predictive block by at least one nearest reference sample of the plurality of reference samples directly neighboring the predictive block, where the plurality of nearest reference samples are arranged along a first image direction of the predictive block and along a second image direction of the predictive block, where at least a portion of the nearest reference samples arranged along the second direction are mapped to extended reference samples arranged along the first direction such that the mapped reference samples exceed an extension of the predictive block along the first image direction, and where the mapped extended reference samples are used for prediction.

[0068] The video encoder can be further configured to map the portion of the nearest reference sample according to a prediction mode used to predict the predictive block. The video encoder can be configured to map the portion of the nearest reference sample according to a direction used in the prediction mode for predicting the predictive block.

[0069] A corresponding video decoder decodes an image encoded with the encoding data into video by block-based predictive decoding including intra-image prediction, in which a plurality of extended reference samples of the image are used to decode a predictive block of the image, each extended reference sample of the plurality of extended reference samples being separated from the predictive block by at least one nearest reference sample of the plurality of reference samples directly adjacent to the predictive block, and the nearest reference samples are arranged along a first image direction of the predictive block and along a second image direction of the predictive block; The method may be configured to map at least a portion of the nearest reference samples arranged along the second direction to extended reference samples arranged along the first direction such that the mapped reference samples exceed an extension of the prediction block along the first image direction, and to use the mapped extended reference samples for prediction.

[0070] The video decoder can be configured to map portions of the nearest reference samples according to a prediction mode used to predict the predictive block. The video decoder can be configured to map portions of the nearest reference samples according to a direction used in the prediction mode for predicting the predictive block.

[0071] In principle, all intra-picture predictions that use directly neighboring reference samples can be adapted to use extended reference samples. Three predictions have been employed in the literature:

[0072] ·plane DC ·angle In the following, in order to explain the embodiments of the present invention, each prediction is explained in detail for the extended reference samples.

[0073] Planar prediction is a bilinear interpolation of W × H prediction samples from the boundaries shown in Figure 3. Since the right and bottom boundaries have not yet been reconstructed, the right boundary samples are calculated using the sample in the top right corner. TIFF0007601639000013.tif39, and the bottom border sample is set equal to the bottom left corner sample Set equal to TIFF0007601639000014.tif39. TIFF0007601639000015.tif679TIFF0007601639000016.tif353TIFF0007601639000017.tif354For ​​extended reference samples, vertical and horizontal predictions TIFF0007601639000018.tif24 and TIFF0007601639000019.tif34 depends on the reference region index i. TIFF0007601639000020.tif361TIFF0007601639000021.tif362One embodiment for improving the extended reference samples is to use vertical and horizontal prediction as follows: TIFF0007601639000022.tif25 and In TIFF0007601639000023.tif35, the closest upper right corner sample TIFF0007601639000024.tif39 and the nearest bottom left corner sample The solution is to use TIFF0007601639000025.tif39. Another embodiment defines the inclusion of corner reference samples in the horizontal and vertical predictions with a weighting that depends on the distance. The further the extended reference region is, i.e. the larger the parameter i, the more samples of the top left corner can be used for prediction. For i=0, one sample TIFF0007601639000028.tif310, i.e., i vertical corner samples adjacent to the predicted block or with distance 0, for i>0 TIFF0007601639000029.tif312 and TIFF0007601639000030.tif315, i horizontal corner samples TIFF0007601639000031.tif312 and TIFF0007601639000032.tif215 and corner sample One embodiment of the present invention averages the horizontal and vertical corner samples (both including the corner samples) and uses it in the vertical and horizontal predictions as follows: TIFF0007601639000034.tif34 and Includes distance-dependent weighting according to TIFF0007601639000035.tif34. TIFF0007601639000036.tif681TIFF0007601639000037.tif333TIFF0007601639000038.tif682TIFF0007601639000039.tif334Another embodiment is a combination of the two previous embodiments using the nearest top right sample, the nearest bottom left sample, and the extended corner sample.

[0074] DC prediction is done by calculating the average of the reference sample value DC and all predicted samples in the W × H block. Set TIFF0007601639000040.tif37 equal to mean. TIFF0007601639000041.tif345 For mean value DC, only N samples can be used on the left side of the prediction block and M samples on the right side, where N can be set to the height H and M can be set to the width W. TIFF0007601639000042.tif843 For extended reference samples, the mean value DC depends on the reference region index i. TIFF0007601639000043.tif850 In one embodiment, the top left corner sample of the predicted block may also be used in the average. TIFF0007601639000044.tif862Angular prediction calculates predicted samples by deriving predicted samples along a specified direction or angle. The simplest prediction directions are horizontal, vertical, and three diagonals, as shown in Figure 4a to Figure 4e, which show examples of intra-picture prediction angles for five angles for a 4x2 block of predicted samples. It can be seen that the reference samples along each direction always point to a perfect (integer) sample position. It is easy to modify the extended reference samples (green) described in the literature, just by extending the direction. The size of the block of predicted samples is 4x2, but intra-angle prediction can be extended to any block size.

[0075] According to an embodiment, finer prediction angle granularity can be used so that the directions can point to non-integer sample positions. These can be generated by interpolation using the nearest integer sample positions. Interpolation can be performed using a simple bilinear filter or a more advanced filter, for example a cubic or Gaussian 4-tap filter.

[0076] For prediction angles between horizontal and vertical, e.g., the top left angle of the diagonal and all angles in between, a prediction is generated using reference samples from the left and top. To facilitate such generation, reference sample projections can be used. For all apex angles between the top left and vertical of the diagonal, the top reference sample can be displayed as the primary reference and the left sample as the secondary reference. For all horizontal angles between the top left horizontal line and the diagonal, the left reference sample can be displayed as the primary reference and the top reference sample as the secondary reference. To simplify the calculations, i.e., to not switch between primary and secondary reference sample calculations, the secondary reference samples are projected along the prediction angle to extend the line of the primary reference sample. Left of the top primary reference TIFF0007601639000045.tif38 or each reference sample above the main reference on the left TIFF0007601639000046.tif38 is projected to the corresponding sub-reference sample according to the inverse prediction angle. Depending on the prediction angle, the projection can point to a sub-sample position.

[0077] 5a-5c show embodiments relating to angle prediction using a definition of angle parameter A given at 1 / 32 sample accuracy for the top left diagonal prediction angle and two other vertical angles. The following embodiments assume 1 / 32 sample accuracy and a vertical prediction angle between vertical and top left diagonal, any other value can be implemented. The 33 prediction angle ranges are defined by the angle parameters A ranging from 0 (vertical) to 32 (top left diagonal) as shown in FIG. 5a-5c. TIFF0007601639000047.tif22. A can be referred to as the distance along the second image direction for a given first image extension, such as 32 samples. In Fig. 5a, A is 32 samples, i.e. 32, resulting in an angle of 45° based on the image extension of 32 samples along the image direction perpendicular to A. In Fig. 5b, A=17, and in Fig. 5c, A=1.

[0078] Angle parameters Each main reference sample based on TIFF0007601639000048.tif22 Vertical offset to left reference column of TIFF0007601639000049.tif38 TIFF0007601639000050.tif314 can be calculated as follows: TIFF0007601639000051.tif656TIFF0007601639000052.tif372TIFF0007601639000053.tif527Parameter Vertical offset based on TIFF0007601639000054.tif22 TIFF0007601639000055.tif314 and maximum horizontal offset The derivation of TIFF0007601639000056.tif312 is shown in Figures 6a to 6c and 7a to 7c, where Figures 6a to 6c show the calculation of the vertical offset given horizontal displacements x for top-left horizontal prediction angles A=32, 17, and 1 as shown in Figures 5a to 5c.

[0079] 7a to 7c show the height of the prediction block for the top-left horizontal prediction angles A=32, 17, and 1. Maximum horizontal offset given TIFF0007601639000057.tif22 The calculation for TIFF0007601639000058.tif312 is shown.

[0080] In this embodiment, the nearest integer reference sample is used instead of the subsample position by rounding the vertical offset, which simplifies the calculation, but it is also possible to project the reference sample on the interpolated subsample side.

[0081] Figure 8 shows the diagonal upper left corner ( TIFF0007601639000059.tif27) and the closest reference sample. In the case of TIFF0007601639000060.tif26), four ( It can be seen that the left reference sample of TIFF0007601639000061.tif316) can be projected diagonally to extend the main reference sample line, i.e., the nearest top reference sample. According to an embodiment, the illustrated sizes of images, blocks, slices, etc. are given by way of example only and may include any other values.

[0082] FIG. 8 shows an example of a projection of the nearest left reference sample as a side reference next to the nearest top reference sample as a primary reference for a left-top horizontal direction with angle parameter A=17 according to an embodiment.

[0083] For an extended reference sample of reference region index i, the projection can be fitted as follows: TIFF0007601639000062.tif656TIFF0007601639000063.tif382TIFF0007601639000064.tif636Figure 9 shows The diagonal upper left corner ( TIFF0007601639000066.tif27) and height 8 samples ( TIFF0007601639000067.tif26 is an example of a file. Here, two ( TIFF0007601639000068.tif316)The left reference sample can be projected diagonally to extend the main reference sample line, i.e. the nearest top reference sample.

[0084] FIG. 9 shows an example of projection of an extended left reference sample as a secondary reference next to an extended top reference sample as a primary reference in the case of top-left diagonal prediction with angle parameter A=17 according to an embodiment.

[0085] When using reference samples from the left and top, the extended reference sample allows combining the closest reference sample with the extended reference sample. This embodiment of the simple approach from FIG. 9 allows utilizing extended reference samples along the main direction, i.e. the main reference, where the longer distance of the reference sample may be beneficial in case of occlusion or edges on the closest reference sample. However, in case of secondary references, the correlation between the predicted sample and the closest reference sample may be higher than between the extended reference sample and the predicted sample.

[0086] For example, the expanded main reference sample TIFF0007601639000069.tif312 to nearest subreference sample One way to combine with TIFF0007601639000070.tif38 can be formulated as follows: TIFF0007601639000071.tif656 TIFF0007601639000072.tif353 TIFF0007601639000073.tif351 TIFF0007601639000074.tif632 This combination is shown in FIG. 10 based on the same embodiment as described in relation to the previous figure, where the extended reference samples 1064 arranged in a line 3 is used for the primary reference sample (top) in the prediction direction (vertical), and the nearest reference sample 1062 arranged in a column can be used for the secondary reference sample that is projected onto an extended reference sample line (top row) along the prediction direction to extend the primary reference sample line.

[0087] Figure 10 shows an example of projection of the nearest left reference sample as a secondary reference next to the extended top reference sample as a primary reference in the case of left-top diagonal prediction with angle parameter A=17. According to the embodiment of Figure 10, the nearest reference sample is used as a source for generating the extended reference sample, i.e., the nearest reference sample is mapped to the extended reference sample. Alternatively, the extended reference sample can also be mapped to the extended reference sample. Please refer to Figure 9.

[0088] A video encoder according to an embodiment, such as video encoder 1000, may be configured to encode an image of a video into encoded data by block-based predictive coding including intra-image prediction, and for intra-image prediction, to use a plurality of nearest reference samples of images directly neighboring the predictive block and a plurality of extended reference samples to encode a predictive block of the image, where each extended reference sample of the plurality of extended reference samples is separated from the predictive block by at least one nearest reference sample of the plurality of reference samples. The video decoder is configured to perform boundary filtering in a mode in which the extended samples are not used, and to not use boundary filtering or to border filter at least a subset of the nearest reference samples and not use boundary filtering for the extended samples when the extended samples are used.

[0089] A corresponding video decoder is configured to decode an image encoded by encoding data into video by block-based predictive decoding including intra-image prediction, and for the intra-image prediction, to use a plurality of nearest reference samples of an image directly neighboring the predictive block and a plurality of extended reference samples to decode a predictive block of the image, each extended reference sample of the plurality of extended reference samples being separated from the predictive block by at least one nearest reference sample of the plurality of reference samples. The video decoder is configured to perform boundary filtering in a mode in which the extended samples are not used, and to not use boundary filtering when the extended samples are used, or to border filter at least a subset of the plurality of nearest reference samples and not use boundary filtering for the extended samples.

[0090] Since the extended reference samples are not directly adjacent to the predicted samples, discontinuities at a particular block boundary may not be as severe as if the nearest reference sample were used. It is therefore beneficial to perform predictive filtering according to: Do not perform boundary filtering operations if extended reference samples are used, or · Modify boundary smoothing by using the nearest reference sample instead of the extended reference sample.

[0091] With extended prediction, predictions from the nearest reference sample and the extended reference sample can be combined to obtain a combined prediction. In the literature, a fixed combination of a prediction using the nearest reference sample and a prediction using the extended reference sample with a certain weight is described. In this case, both predictions use the same prediction mode for all reference regions, and when the mode is signaled, the prediction combination is also signaled. On the other hand, this reduces the signaling overhead for indicating the reference sample region, but also loses the flexibility to combine two different prediction modes with two different reference sample regions. Possible combinations to increase flexibility are described in detail below.

[0092] A video encoder according to an embodiment, such as video encoder 1000, may be configured to encode an image of a video into encoded data by block-based predictive coding including intra-image prediction, where in the intra-image prediction, to encode a predictive block of the image, determine a plurality of closest reference samples and a plurality of extended reference samples of an image directly neighboring the predictive block, each extended reference sample of the plurality of extended reference samples being separated from the predictive block by at least one closest reference sample of the plurality of reference samples, determine a prediction of the predictive block using the extended reference samples, filter the extended reference samples, and obtain a plurality of filtered extended reference samples. The video encoder is configured to combine the prediction and the filtered extended reference samples to obtain a combined prediction of the predictive block.

[0093] The video encoder can be configured to combine prediction samples and enhanced reference samples that are located on a main or sub-diagonal of samples with respect to the predictive block.

[0094] The video encoder may be configured to combine the prediction samples and the enhanced reference samples based on the following decision rules: TIFF0007601639000075.tif888, where p c (x,y) denotes a combined prediction of coordinates x and y in a prediction block, and p(x,y) denotes a prediction of coordinates x and y in a prediction block. TIFF0007601639000076.tif411 is the prediction weight, d x is a parameter set to 1 or 2 depending on the size of the prediction block in dimension x, and d y is a parameter set to 1 or 2 depending on the size of the prediction block in dimension y, r(x,-1-i)i>0 is the extended top reference sample at horizontal position x, r(-1-i,y)i>0 is the extended left reference sample at vertical position y, r(-1-i,-1-i)i>0 is the extended corner reference sample of the multiple extended reference samples with respect to the boundary of the prediction block, and b(x,y) denotes a normalization factor.

[0095] The normalization factor can be determined based on a determination rule. The TIFF0007601639000077.tif942 video encoder may be configured to filter the extended reference samples using either a 3-tap filter, a 5-tap filter, or a 7-tap filter to obtain filtered extended reference samples (r(x,-1-i), r(-1-i,y), r(-1-i,-1-i) for i>0) (combination), and use the filtered extended reference samples for prediction.

[0096] The video encoder may be configured to use a combination of extended corner reference samples of a predictive block and extended reference samples (r(-1-i,-1-i)) located in corner regions of the reference samples.

[0097] The video encoder may be configured to obtain the combined prediction based on the following decision rules: TIFF0007601639000078.tif992 where p c (x,y) denotes a combined prediction of coordinates x and y in a prediction block, and p(x,y) denotes a prediction of coordinates x and y in a prediction block. TIFF0007601639000079.tif411 is the prediction weight, d x is a parameter set to 1 or 2 depending on the size of the prediction block in dimension x, and d y is a parameter set to 1 or 2 depending on the size of the prediction block in dimension y, r(x,-1-i) for i>0 is the extended top reference sample at horizontal position x, r(-1-i,y) for i>0 is the extended left reference sample at vertical position y, and TIFF0007601639000080.tif515 is the extended corner reference sample combined with respect to the boundary of the prediction block, and b(x,y) denotes the normalization factor.

[0098] The video encoder may be configured to obtain the prediction p(x,y) based on intra-picture prediction.

[0099] The video encoder can be configured to use only planar prediction as intra-picture prediction.

[0100] The video encoder can be configured to determine, for each encoded video block, a parameter set that identifies a combination of a prediction and a filtered extended reference sample. The video encoder can be configured to determine a parameter set that identifies a combination of a prediction and a filtered extended reference sample using a lookup table that includes a set of different block sizes of prediction blocks.

[0101] A corresponding video decoder may be configured to decode an image encoded by encoding data into video by block-based predictive decoding including intra-image prediction, and in the intra-image prediction, to decode a predictive block of the image, determine a plurality of nearest reference samples and a plurality of extended reference samples of the image directly adjacent to the predictive block, each extended reference sample of the plurality of extended reference samples being separated from the predictive block by at least one nearest reference sample of the plurality of reference samples, determine a prediction of the predictive block using the extended reference samples, filter the extended reference samples to obtain a plurality of filtered extended reference samples, and combine the prediction and the filtered extended reference samples to obtain a combined prediction of the predictive block.

[0102] The video decoder may be configured to combine prediction samples and enhanced reference samples that are located on a main or sub-diagonal of samples with respect to the predictive block.

[0103] The video encoder may be configured to combine the prediction samples and the enhanced reference samples based on the following decision rules: TIFF0007601639000081.tif888, where p c (x,y) denotes a combined prediction of coordinates x and y in a prediction block, and p(x,y) denotes a prediction of coordinates x and y in a prediction block. TIFF0007601639000082.tif411 is the prediction weight, d x is a parameter set to 1 or 2 depending on the size of the prediction block in dimension x, and d y is a parameter set to 1 or 2 depending on the size of the prediction block in dimension y, r(x,-1-i)i>0 is the extended top reference sample at horizontal position x, r(-1-i,y)i>0 is the extended left reference sample at vertical position y, r(-1-i,-1-i)i>0 is the extended corner reference sample of the multiple extended reference samples with respect to the boundary of the prediction block, and b(x,y) denotes a normalization factor.

[0104] The normalization factor can be determined based on a determination rule. The TIFF0007601639000083.tif942 video decoder can be configured to filter the extended reference samples using either a 3-tap filter, a 5-tap filter, or a 7-tap filter to obtain filtered extended reference samples (r(x,-1-i), r(-1-i,y), r(-1-i,-1-i) for i>0) (combination), and use the filtered extended reference samples for prediction.

[0105] The video decoder may be configured to use a combination of extended corner reference samples of a predictive block and extended reference samples located in corner regions of the reference samples (r(-1-i,-1-i)).

[0106] The video decoder may be configured to obtain the combined prediction based on the following decision rules: TIFF0007601639000084.tif992 where p c (x,y) denotes a combined prediction of coordinates x and y in a prediction block, and p(x,y) denotes a prediction of coordinates x and y in a prediction block. TIFF0007601639000085.tif411 is the prediction weight, d x is a parameter set to 1 or 2 depending on the size of the prediction block in dimension x, and d y is a parameter set to 1 or 2 depending on the size of the prediction block in dimension y, r(x,-1-i) for i>0 is the extended top reference sample at horizontal position x, r(-1-i,y) for i>0 is the extended left reference sample at vertical position y, and TIFF0007601639000086.tif515 is the extended corner reference sample combined with respect to the boundary of the prediction block, and b(x,y) denotes the normalization factor.

[0107] The video decoder may be configured to obtain the prediction p(x,y) based on intra-picture prediction. The video decoder may, for example, only use planar prediction as intra-picture prediction.

[0108] The video encoder may be configured to determine, for each decoded video block, a parameter set that identifies a combination of prediction and filtered enhanced reference samples.

[0109] The video decoder may be configured to determine a parameter set that identifies a combination of prediction and filtered enhanced reference samples using a lookup table that includes a set of different block sizes of prediction blocks.

[0110] When filtering reference samples, predictions using the filtered samples can be combined with unfiltered reference samples based on the position of the respective samples to obtain position-dependent prediction combinations. Let TIFF0007601639000087.tif37 be the prediction using the filtered reference sample, Let TIFF0007601639000088.tif37 be the unfiltered reference sample. Combined prediction TIFF0007601639000089.tif38 can be calculated as follows: TIFF0007601639000090.tif888 coefficients TIFF0007601639000091.tif411 stores the prediction weights scaled by the position-dependent terms, for blocks with width 16 or less TIFF0007601639000092.tif37 and the block width is greater than 16 TIFF0007601639000093.tif37, for blocks with height less than 16 TIFF0007601639000094.tif37 and for blocks with height greater than 16 TIFF0007601639000095.tif37. Normalization factor TIFF0007601639000096.tif37 can be derived as follows: As already described in connection with TIFF0007601639000097.tif749 reference sample filtering, the reference sample filtering of this combination may differ from that of normal prediction. For example, one of three or more predefined low-pass filters may be used to smooth the boundary samples. The three predefined low-pass filters may include one 3-tap, one 5-tap, and one 7-tap filter. To adapt the smoothing to the content, the selection of the smoothing filter may be based on the block size and the intra prediction mode. k Define as the impulse response of filter k, and an additional stored weighting parameter a, and we obtain the filtered reference TIFF0007601639000098.tif32 is the unfiltered reference It is calculated from TIFF0007601639000099.tif32 as follows, where "*" indicates convolution. TIFF0007601639000100.tif326 Considering again the different characteristics of different block sizes, we can choose one fixed set of prediction parameters ( TIFF0007601639000101.tif411, a and filter index k).

[0111] This known combination of nearest reference samples can be improved according to the embodiment by using extended reference samples for the filtered and unfiltered reference samples. The following shows a combination of predictions using extended reference samples according to the embodiment, in which the nearest corner sample r(-1,-1) is replaced by the extended corner sample r(-1,-1) as follows: Replaced with TIFF0007601639000102.tif317. Another embodiment of the extended reference sample is to convert the nearest corner sample r(-1,-1) into a closed extended vertical corner sample as follows: TIFF0007601639000104.tif313 and Horizontal Corner Samples This refers to replacing it with TIFF0007601639000105.tif313. TIFF0007601639000106.tif788 Another embodiment of the extended reference sample defines replacing the nearest corner sample r(-1,-1) with the average of the horizontal and vertical corner samples (both inclusive) as follows: TIFF0007601639000107.tif992 In one embodiment, the combined predictions TIFF0007601639000108.tif37 is a planar prediction. In another embodiment, this combination can also be applied to other intra-picture predictions such as DC or angle.

[0112] Alternatively, or in addition, different enhanced prediction modes can be combined to combine predictions with samples.

[0113] A video encoder according to an embodiment, such as video encoder 1000, is configured to encode an image of a video into encoded data by block-based predictive coding including intra-image prediction, and for the intra-image prediction, to encode a predictive block of the image using a plurality of nearest reference samples and / or a plurality of extended reference samples of images directly neighboring the predictive block, each extended reference sample of the plurality of extended reference samples being separated from the predictive block by at least one nearest reference sample of the plurality of reference samples, to determine a first prediction of the predictive block using a first prediction mode of a set of prediction modes, the first set of prediction modes including a prediction mode using the plurality of nearest reference samples in the absence of an extended reference sample, to determine a second prediction of the predictive block using a second prediction mode of a second set of prediction modes, the second set of prediction modes including a subset of the prediction modes of the first set being associated with the plurality of extended reference samples. The video encoder is configured to: 0 (x, y)) and the second prediction (p i (x, y)) and weighted (w 0 ;w i ) to obtain a combined prediction (p(x,y)) as a prediction for a predictive block of encoded data.

[0114] The video encoder may be configured to use the first prediction and the second prediction according to a predetermined combination that is a subset of possible combinations of valid first prediction modes and valid second prediction modes.

[0115] The video encoder may be configured to signal either the first prediction mode or the second prediction mode without signaling the other prediction mode. For example, the first mode may be derived from the parameters i based on additional implicit information, such as a particular prediction mode that may only be used in conjunction with a particular index i or index m.

[0116] As an example of such implicit information, the video encoder may be configured to exclusively use a planar prediction mode as one of the first prediction mode and the second prediction mode.

[0117] The video encoder may be configured to adapt a first weight applied to the first prediction of the joint prediction and a second weight applied to the second prediction of the joint prediction based on a block size of the prediction block, and / or to adapt the first weight based on the first prediction mode or the second weight based on the second prediction mode.

[0118] The video encoder may be configured to adapt a first weight applied to a first prediction of the joint prediction and a second weight applied to a second prediction of the joint prediction based on a position and / or distance within the prediction block.

[0119] The video encoder may be configured to adapt the first weight and the second weight based on the following decision rule: TIFF0007601639000109.tif345Where, 0 (x,y) is the first weight based on the position x,y in the prediction block, and w i is the second weight based on the position x,y in the prediction block, and p 0 (x,y) is the first prediction for location x,y, and p i (x,y) is the second prediction at position x,y, and i denotes the enhanced reference sample used for the second prediction.

[0120] A corresponding video decoder decodes an image encoded by encoding data into video by block-based predictive decoding including intra-image prediction, and for intra-image prediction, uses a plurality of nearest reference samples and / or a plurality of extended reference samples of an image directly adjacent to the predictive block to decode a predictive block of the image, each extended reference sample of the plurality of extended reference samples being separated from the predictive block by at least one nearest reference sample of the plurality of reference samples; The video decoder may be configured to determine a first prediction of the predictive block using a first prediction mode of a set of prediction modes, the first set of prediction modes including a prediction mode that uses multiple nearest reference samples in the absence of an extended reference sample, and to determine a second prediction of the predictive block using a second prediction mode of a second set of prediction modes, the second set of prediction modes including a subset of the prediction modes of the first set, the subset being associated with multiple extended reference samples. 0 (x, y)) and the second prediction (p i (x, y)) and weighted (w 0 ;w i ) to obtain a combined prediction (p(x,y)) as a prediction for a predictive block of encoded data.

[0121] The video decoder may be configured to use the first prediction and the second prediction according to a predetermined combination that is a part of possible combinations of the valid first prediction mode and the valid second prediction mode, which allows obtaining a low-load notification.

[0122] The video decoder may be configured to receive a signal indicating the second prediction mode without receiving a signal indicating the first prediction mode, and to derive the first prediction mode from the second prediction mode or the parameter information (i).

[0123] The video decoder may be configured to exclusively use the planar prediction mode as one of the first prediction mode and the second prediction mode.

[0124] The video decoder may be configured to adapt a first weight applied to a first prediction of the joint prediction and a second weight applied to a second prediction of the joint prediction based on a block size of the prediction block, and / or to adapt the first weight based on the first prediction mode or the second weight based on the second prediction mode.

[0125] The video decoder may be configured to adapt a first weight applied to a first prediction of the joint prediction and a second weight applied to a second prediction of the joint prediction based on a position and / or distance within the prediction block.

[0126] The video decoder may be configured to adapt the first weight and the second weight based on the following decision rule: TIFF0007601639000110.tif345Where, 0 (x,y) is the first weight based on the position x,y in the prediction block, and w i is the second weight based on the position x,y in the prediction block, and p 0 (x,y) is the first prediction for location x,y, and p i (x,y) is the second prediction of position x,y, and i indicates the extended reference sample used for the second prediction. For varying positions x,y within the prediction block, the extended reference sample i may include varying distances. Alternatively, or in addition, different sets of a given extended reference sample, indicated by area index i and / or mode m, may include different distances to the prediction block.

[0127] When using different reference sample regions, the prediction using the nearest reference sample mode is as follows: Prediction using TIFF0007601639000111.tif38 and extended reference sample i It can be combined with TIFF0007601639000112.tif37. One known way to combine predictions is to fix the weights and use the same prediction mode for both predictions, which avoids the need to signal the second prediction mode.

[0128] To relax this strict restriction, an embodiment defines that only certain combinations of modes are allowed. This requires the signaling of the second mode, but limits the number of modes to signal compared to the first mode. Details of prediction mode signaling are provided in relation to mode and reference signaling. One promising combination according to an embodiment is to use only planar as the second mode, so that additional signaling of intra modes becomes obsolete. For example, any intra mode as the first part of the weighted sum can be used. You can use TIFF0007601639000114.tif37 and the second part is generated using the nearest reference sample and planar mode. Another possibility is to switch the order, i.e. planar mode is used for the extended reference sample.

[0129] To adapt the weights to the prediction size and mode, TIFF0007601639000115.tif22 can be specified separately for each block size and for each prediction mode. Furthermore, weights can be signaled at sequence, picture or slice level to allow the combination to be adapted to the video content. Another embodiment is to give different weights to different reference sample predictions for different regions within the prediction block. This can be achieved by weighting the position dependent weights as follows: TIFF0007601639000116.tif37 can be yielded: TIFF0007601639000117.tif345 When using one prediction and extended reference sample, it is state of the art to signal the prediction mode and the reference region index i to be used, i.e., to perform mode and reference region signaling. In the following, an embodiment of prediction mode coding when using one and / or multiple predictions is presented, followed by an embodiment of extended reference region index coding.

[0130] A video encoder according to an embodiment, such as video encoder 1000, may be configured to encode an image of a video into encoded data by block-based predictive coding including intra-image prediction, and for intra-image prediction, to encode a predictive block of the image using a plurality of nearest reference samples and / or a plurality of extended reference samples of images directly adjacent to the predictive block, each extended reference sample of the plurality of extended reference samples being separated from the predictive block by at least one nearest reference sample of the plurality of reference samples, and to use a prediction mode that is one of a first set of prediction modes for predicting the predictive block using the nearest reference samples, or to use a prediction mode that is one of a second set of prediction modes for predicting the predictive block using the extended reference samples, the second set of prediction modes being a subset of the first set of prediction modes, the second subset being determined by the encoder, and the subset may also include a match of both sets. The video encoder may be configured to signal mode information (m) indicating a prediction mode used to predict the predictive block, and then, if the prediction mode is included in a second set of prediction modes, signal parameter information (i) indicating a subset of extended reference samples used for the prediction mode, and skip signaling the parameter information if the prediction mode used is not included in the second set of prediction modes, thereby enabling a conclusion that parameter i has a predetermined value, such as 0.

[0131] The video encoder may be configured to skip reporting the parameter information if the mode information indicates a DC mode or a planar mode.

[0132] A corresponding video decoder may be configured to decode an image encoded with the encoding data by block-based predictive coding including intra-image prediction into video, and for intra-image prediction, to decode a predictive block of the image, using a plurality of nearest reference samples and / or a plurality of extended reference samples of an image directly adjacent to the predictive block, each extended reference sample of the plurality of extended reference samples being separated from the predictive block by at least one nearest reference sample of the plurality of reference samples, using a prediction mode that is one of a first set of prediction modes for predicting the predictive block using the nearest reference samples, or using a prediction mode that is one of a second set of prediction modes for predicting the predictive block using the extended reference samples, the second set of prediction modes being a subset of the first set of prediction modes. The second set of prediction modes is a subset of the first set of prediction modes and may be determined and / or signaled by the encoder. Being a subset may include a match of both sets. The video encoder may be configured to receive mode information (m) indicating a prediction mode used to predict the predictive block, then receive parameter information (i) indicating a subset of extended reference samples used for the prediction mode, thereby indicating that the prediction mode is included in a second set of prediction modes, and if no parameter information is received, determine that the used prediction mode is not included in the second set of prediction modes and determine the use of the nearest reference sample for prediction.

[0133] The video decoder may be configured to determine the mode information as indicating the use of DC mode or planar mode when no parameter information is received.

[0134] If the set of allowed intra prediction modes of the extended reference sample is restricted, i.e. a subset of the allowed intra prediction modes of the nearest reference sample, e.g., the subset is called the constrained prediction modes of the extended reference sample, there are two ways to signal the mode m and index i: 1. Signal mode m before index i, such that the index signaling can depend on mode m as follows: a. If mode m is not in the set of allowed modes of the extended reference sample, the notification of index i is skipped and the predicted mode m is applied to the nearest reference sample (i=0). b. Otherwise, i is signaled and prediction mode m is applied to the reference sample indicated by i.

[0135] 2. Signal index i before mode m, such that the mode signal can depend on index i as follows: If ai indicates that an extended reference sample (i>0) is used, then the set of modes m that can be advertised is the same as the restricted one.

[0136] b. Otherwise (i=0), the set of advertised allowed modes m is equal to the set of unrestricted modes.

[0137] For example, if mode m is signaled using Most Probable Mode (MPM) coding using an index into the MPM list, modes that are not in the set of allowed modes are not included in the MPM list.

[0138] That is, according to a second option, which may be alternatively or additionally implemented, a video encoder according to an embodiment, such as video encoder 1000, encodes an image of a video into encoded data by block-based predictive coding including intra-image prediction, and for the intra-image prediction, uses a plurality of reference samples and a plurality of extended reference samples, including nearest reference samples of images directly adjacent to the predictive block, for encoding a predictive block of the image, each extended reference sample of the plurality of extended reference samples being separated from the predictive block by at least one nearest reference sample of the plurality of reference samples, and a first set of prediction modes for predicting the predictive block using the nearest reference samples. the prediction mode being one of a first set of prediction modes, or a second set of prediction modes for predicting the prediction block using the extended reference samples, the second set of prediction modes being a subset of the first set of prediction modes, signaling parameter information (i) indicating a subset of the plurality of reference samples to be used for the prediction mode, the subset of the plurality of reference samples including only the nearest reference sample or the extended reference sample, then signaling mode information (m) indicating a prediction mode to be used to predict the prediction block, the mode information indicating a prediction mode from the subset of modes, the subset being restricted to a set of prediction modes allowed according to the parameter information (i).

[0139] For both options, the video decoder is adapted such that, in addition to the nearest reference sample, extended reference samples of the modes included in the second set of prediction modes are used.

[0140] Further, the video encoder may be adapted such that a first set of prediction modes describes prediction modes that are capable of being used with the nearest reference sample, and a second set of prediction modes describes prediction modes of the first set of prediction modes that are also capable of being used by the extended reference sample.

[0141] The range of values ​​of the parameter information, i.e., the domain of values ​​that can be represented by the area index i, can cover the use of only the nearest reference value and the use of different subsets of extended reference values. As described in connection with FIG. 11, i can represent the use of only the nearest reference value (i=0), a specific set of extended reference values, or a combination of the set of extended reference values ​​(e.g., lines and / or columns or distances).

[0142] According to an embodiment, different parts of the extended reference samples include different distances to the prediction block.

[0143] The video encoder may be configured to set the parameter information to one of a predetermined number of values, which indicates the number and distance of reference samples to be used for the prediction mode.

[0144] The video encoder may be configured to determine the first set of prediction modes and / or the second set of prediction modes based on a most probable mode coding.

[0145] A corresponding decoder of a second option is configured to decode an image encoded by encoding data into video by block-based predictive decoding including intra-image prediction, and for intra-image prediction, to use a plurality of reference samples and a plurality of extended reference samples including a nearest reference sample of an image directly adjacent to the predictive block to decode a predictive block of the image, each extended reference sample of the plurality of extended reference samples being separated from the predictive block by at least one nearest reference sample of the plurality of reference samples, to use a prediction mode that is one of a first set of prediction modes for predicting the predictive block using the nearest reference sample or to use a prediction mode that is one of a second set of prediction modes for predicting the predictive block using the extended reference samples, the second set of prediction modes being a subset of the first set of prediction modes, wherein the subset of the plurality of reference samples includes only the nearest reference sample or the at least one extended reference sample, and then to receive mode information (m) indicating a prediction mode to be used to predict the predictive block, wherein the mode information indicates a prediction mode from the subset of modes, the subset being restricted to a set of allowed prediction modes according to the parameter information (i).

[0146] A decoder according to the first option and / or the second option can be adapted such that, in addition to the nearest reference sample, extended reference samples of modes included in the second set of prediction modes are used, e.g. by combining predictions with samples and / or combinations of predictions.

[0147] The first set of prediction modes may describe prediction modes that are allowed for use with the nearest reference sample, and the second set of prediction modes may describe prediction modes of the first set of prediction modes that are also allowed for use with the extended reference sample.

[0148] As described for the encoder, the range of values ​​for the parameter information covers the use of only the nearest reference value and the use of a different subset of the extended reference values.

[0149] Different portions of the extended reference samples may contain different distances to the prediction block.

[0150] The video decoder may be configured to set the parameter information to one of a predetermined number of values, which indicates the number and distance of reference samples used for the prediction mode.

[0151] The video decoder may be configured to determine a first set of prediction modes and / or a second set of prediction modes based on Most Probable Mode (MPM) coding, i.e., generate respective lists, which may be adapted to include only modes that are allowed to be used for the respective sample.

[0152] Alternatively, or in addition, embodiments relating to combining predictions from nearest and extended reference samples can be implemented.

[0153] A video encoder according to an embodiment, such as video encoder 1000, may be configured to encode an image of a video into encoded data by block-based predictive coding including intra-image prediction, and for the intra-image prediction, use a plurality of nearest reference samples and / or a plurality of extended reference samples of images directly neighboring the predictive block to encode a predictive block of the image, where each extended reference sample of the plurality of extended reference samples is separated from the predictive block by at least one nearest reference sample of the plurality of reference samples, determine a first prediction of the predictive block using a first prediction mode of a set of prediction modes, the first set of prediction modes including a prediction mode that uses the plurality of nearest reference samples in the absence of an extended reference sample, determine a second prediction of the predictive block using a second prediction mode of a second set of prediction modes, the second set of prediction modes including a subset of the first set of prediction modes associated with the plurality of extended reference samples, and combine the first prediction and the second prediction to obtain a combined prediction as a prediction of the predictive block in the encoded data.

[0154] The predictive block may be a first predictive block, and the video encoder may be configured to predict a second predictive block of the video using a plurality of nearest reference samples associated with the second predictive block in the absence of a plurality of extended reference samples associated with the second predictive block. The video encoder may be configured to signal combination information, such as a bipred flag, indicating that a prediction in the encoded data is based on a combination of predictions or based on a prediction using a plurality of extended reference samples in the absence of a plurality of nearest reference samples.

[0155] The video encoder may be configured to use the first prediction mode as the predetermined prediction mode.

[0156] The video encoder may be configured to select the first prediction mode as the same mode as the second prediction mode and to use the nearest reference sample in the absence of an extended reference sample, or to use the first prediction mode as a pre-set prediction mode, such as a planar prediction mode.

[0157] A corresponding video decoder may be configured to decode an image encoded by encoding the data into video by block-based predictive decoding including intra-image prediction, and for intra-image prediction, to use a plurality of nearest reference samples and / or a plurality of extended reference samples of images directly neighboring the predictive block to decode a predictive block of the image, each extended reference sample of the plurality of extended reference samples being separated from the predictive block by at least one nearest reference sample of the plurality of reference samples, to determine a first prediction of the predictive block using a first prediction mode of a set of prediction modes, the first set of prediction modes including a prediction mode that uses the plurality of nearest reference samples in the absence of an extended reference sample, and to determine a second prediction of the predictive block using a second prediction mode of a second set of prediction modes, the second set of prediction modes including a subset of the prediction modes of the first set, the subset being associated with the plurality of extended reference samples. The video decoder may be configured to combine the first prediction and the second prediction to obtain a combined prediction as a prediction of the predictive block in the encoded data.

[0158] The predictive block may be a first predictive block, and the video decoder may be configured to predict a second predictive block of the video using a plurality of nearest reference samples associated with the second predictive block in the absence of a plurality of extended reference samples associated with the second predictive block. The video decoder may be further configured to receive combination information indicating that a prediction in the encoded data is based on a combination of predictions or based on a prediction using a plurality of extended reference samples in the absence of a plurality of nearest reference samples, and to decode the encoded data accordingly.

[0159] The video decoder may be configured to use the first prediction mode as the predetermined prediction mode.

[0160] The video decoder may be configured to select the first prediction mode as the same mode as the second prediction mode and to use the nearest reference sample in the absence of an extended reference sample, or to use the first prediction mode as a preset prediction mode, such as a planar mode.

[0161] If the reference region index i indicates the use of an extended reference sample (i>0), a respective piece of information, such as a binary information or a flag, hereinafter referred to as a bi-prediction flag (bipred flag), is used to signal whether a prediction using the extended reference sample is combined with a prediction using the nearest reference sample.

[0162] If the bipred flag indicates a combination of predictions from the nearest (i=0) and extended reference samples (i>0), then the extended reference sample m i The mode of is signaled before or after the reference region index i, as described here. 0 The mode of is fixed, e.g., always set to a specific mode such as planar, or to the same mode as the extended reference sample.

[0163] As described below, a video encoder according to an embodiment encodes an image of a video into encoded data by block-based predictive coding including intra-image prediction, in which a plurality of extended reference samples of the image are used to encode a predictive block of the image, each extended reference sample of the plurality of extended reference samples being separated from the predictive block by at least one nearest reference sample among the plurality of reference samples directly adjacent to the predictive block, and the plurality of extended reference samples are used according to a predetermined set of the plurality of extended reference samples, i.e., a list of area indices can be generated and / or used, and the list can be configured to indicate a particular subset of reference samples, such as only the nearest reference sample, at least one distance of the extended reference samples, and / or a combination of different distances of the extended reference samples.

[0164] The video encoder may be configured to determine multiple predetermined sets of extended reference samples such that multiples in the sets differ from each other by the number or combination of lines and / or rows of samples of the image used as reference samples.

[0165] The video encoder may be configured to determine a predetermined set of the multiple enhanced reference samples based on a block size of the predictive block and / or a prediction mode used to predict the predictive block.

[0166] The video encoder may be configured to determine a set of extended reference samples for which a block size of the predictive block is at least a predetermined threshold, and to skip reporting of the set of extended reference samples if the block size is below the predetermined threshold.

[0167] The pre-defined threshold may be a pre-defined number of samples along the width or height of the prediction block and / or a pre-defined aspect ratio of the prediction block along the width and height.

[0168] The predetermined number of samples may be any number, but is preferably 8. Alternatively, or in addition, the aspect ratio may be greater than 1 / 4 and less than 4, possibly based on the number of 8 samples that define a quotient of the aspect ratio.

[0169] The video encoder may be configured to predict a predictive block as a first predictive block using a plurality of extended reference samples and to predict a second predictive block (which may be part of the same or a different image) without using the extended reference samples, and the video encoder may be configured to signal a predetermined set of the plurality of extended reference samples associated with the first predictive block and not signal a predetermined set of the extended reference samples associated with the second predictive block. The predetermined set may be indicated, for example, by a reference region index i.

[0170] The video encoder may be configured to signal, for each predictive block, information indicating one of a particular set of extension reference samples, and the use of the nearest reference sample only before information indicating the intra-picture prediction mode.

[0171] The video encoder can be configured to signal information indicating intra-picture prediction, thereby indicating a prediction mode according to an indicated particular number of sets of extended reference samples, or according to an indicated use of only the nearest reference sample.

[0172] A corresponding video decoder is configured to decode an image encoded by encoding data into video by block-based predictive decoding including intra-image prediction, and in the intra-image prediction, to use a plurality of extended reference samples of the image to decode a predictive block of the image, each extended reference sample of the plurality of extended reference samples being separated from the predictive block by at least one nearest reference sample among the plurality of reference samples directly adjacent to the predictive block, and to use the plurality of extended reference samples according to a predetermined set of the plurality of extended reference samples.

[0173] The video decoder may be configured to determine multiple predetermined sets of extended reference samples such that multiples in the sets differ from each other by the number or combination of rows and / or rows of samples of the image used as reference samples.

[0174] The video decoder may be configured to determine a predetermined set of the multiple enhanced reference samples based on a block size of the predictive block and / or a prediction mode used to predict the predictive block.

[0175] The video encoder may be configured to determine a set of extended reference samples for which a block size of the predictive block is at least a predetermined threshold, and to skip using the set of extended reference samples if the block size is below the predetermined threshold.

[0176] For example, the notification of the set (which can use parameter information i or a similar syntax) can be skipped if the top-left sample of the current block at position (x0,y0) is in the first row in the coding tree block (CTB). The CTB can be seen as the fundamental processing unit into which the image is divided and which is the root of further subdivisions of blocks.

[0177] This can be done by checking if the vertical y coordinate y0 is not a multiple of the CTB size. For example, if the CTB size is 64 luma samples, then in the first CTB row the above blocks in the CTB have y0=0, in the second CTB row they have y0=64, etc. Thus, if all blocks in the CTB are at the upper limit of this CTB (which can be checked for example by the following modulo operation: y0%CtbSizeY==0), then the notification of parameter i or a similar syntax can be skipped.

[0178] Thus, the pre-defined threshold may be a pre-defined number of samples along the width or height of the prediction block and / or a pre-defined aspect ratio of the prediction block along the width and height.

[0179] Thus, the predetermined number of samples may be eight, and / or the aspect ratio may need to be greater than 1 / 4 and less than four.

[0180] The video decoder may be configured to predict a predictive block as a first predictive block using a plurality of extended reference samples and to predict a second predictive block without using the extended reference samples, wherein the video decoder is configured to receive information indicating a predetermined set of a plurality of extended reference samples associated with the first predictive block, for example using an area index i, and to determine the predetermined set of extended reference samples associated with the second predictive block in the absence of a respective signal.

[0181] The video decoder can be configured to receive, for each predictive block, information indicating one of a particular set of enhanced reference samples and the use of the nearest reference sample only before the information indicating the intra-picture prediction mode.

[0182] The video decoder may thereby be configured to receive information indicating intra-picture prediction to indicate a prediction mode according to an indicated particular number of a plurality of sets of a plurality of extended reference samples, or according to an indicated use of only the nearest reference sample.

[0183] Since the reference area index can change per block, the index can be transmitted in the bitstream for all prediction blocks to which it applies. An embodiment relates to an extended reference sample area signaling to allow the decoder to use the correct sample. This area can correspond to a parameter i indicating the extended reference sample or the nearest reference sample. To trade off the signaling and the use of further away extended reference samples, a predefined set of extended reference sample lines can be used. For example, only two additional extended reference lines can be used, the lines with indexes 1 and 3. A set according to an embodiment can be I={0,1,3}, with |I|=3. This allows the reference sample area to be extended up to three rows, but only two signals are needed. The set can also be extended to four rows, for example I={0,1,3,4}, and only the first MaxNumRefAreaIdx element is used, but MaxNumRefAreaIdx can be fixed or signaled at sequence, picture or slice level. The index n into set I can be signaled in the bitstream using entropy coding and a truncated single code, as shown in the table shown in FIG. 11, which shows an example of a truncated single code for a particular reference area set of MaxNumRefAreaIdx equal to 3 and 4.

[0184] To account for different spatial characteristics of different block sizes, the set of reference sample lines can also depend on the prediction block size and / or intra prediction mode. In another embodiment, the set includes only one additional row with index 2 for small blocks and only two rows for large blocks with index 1 and 3. An embodiment of the set selection depending on the intra prediction mode is to have different sets of reference sample lines for prediction directions between horizontal and vertical directions.

[0185] The notification of the reference region index can also be restricted to larger block sizes by selecting an empty set for small block sizes that do not require notification. This can essentially disable the use of extended reference samples for small block sizes. In one embodiment, the use of extended reference samples can be restricted to blocks with both width W and height H equal to or greater than 8. In addition, blocks with one side less than or equal to one-quarter of the other, such as 32x8 blocks, or 8x32 blocks based on the symmetry mentioned above, can also be excluded from using extended reference samples. Figures 12a and 12b show this embodiment, where some block sizes are already excluded for intra prediction in general (Figure 12a), as indicated by reference number 1102, and the blocks are shaded accordingly. In this embodiment, it is assumed that intra and inter prediction slices allow for different block size combinations. Sample 1104 and the blocks shaded accordingly are not allowed for extended reference samples, i.e., i>0. 12a and 12b show examples of restrictions on the extended reference samples of intra-predicted slices and inter-predicted slices. As can be seen, the allowance or restriction of blocks depending on the block size, especially the aspect ratio of the block, can be symmetric with respect to the quotients W / H and H / W.

[0186] If there are other intra prediction modes that do not use extended reference samples, the reference area index is signaled only if a prediction mode that uses extended reference samples is signaled. Another method is to signal the reference area index before all other intra mode information is signaled. If the reference area index i indicates an extended reference sample (i>0), the signaling of mode information that does not use extended reference samples (e.g., template matching, trained predictors, etc.) can be skipped. This can also skip signaling information of certain transformations that are not applied to the prediction residual of predictions that use extended reference samples.

[0187] In the following, reference is made to embodiments that refer to parallel encoding considerations.

[0188] A video encoder according to an embodiment may be configured to encode a plurality of predictive blocks into encoded data by block-based predictive coding including intra-image prediction, and to use a plurality of extended reference samples of an image for encoding a predictive block of the plurality of predictive blocks for intra-image prediction, wherein each extended reference sample of the plurality of extended reference samples is separated from the predictive block by at least one nearest reference sample of the plurality of reference samples directly adjacent to the predictive block, and the video encoder may be configured to determine the extended reference sample to be at least a partial part of a neighboring predictive block of the plurality of predictive blocks, determine that the neighboring predictive block has not yet been predicted, and notify information indicating the extended prediction sample associated with the predictive block and located as an unavailable sample in the neighboring predictive block.

[0189] The video encoder can be configured to encode an image by parallel coding lines of blocks according to a wavefront approach and predict a prediction block based on an angle prediction, and the video encoder is configured to determine an extended reference sample used to predict the prediction block to be placed in an already predicted block of the image. According to the wavefront approach, the encoding or decoding of the second line can track the decoding of the first line separated by one block, for example. For example, in a 45° vertical angle mode, when starting from the second line, a maximum of one block in the upper right corner can be decoded and encoded.

[0190] The video encoder can be configured to inform adjacent predictive blocks of extended prediction samples associated with the predictive block and arranged differently as unavailable or available samples at a sequence level, i.e., a sequence of pictures, a picture level, or a slice level, where a slice is a portion of a picture.

[0191] The video encoder may be configured to signal information associated with the predictive block and indicative of parallel coding of the image together with information indicative of extended predictive samples located in adjacent predictive blocks as unavailable samples.

[0192] A corresponding decoder can be configured to decode images encoded with the encoding data into video by block-based predictive decoding including intra-image prediction, where for each image, a plurality of predictive blocks are decoded, and for intra-image prediction, to use a plurality of extended reference samples of the image to decode a predictive block of the plurality of predictive blocks, where each extended reference sample of the plurality of extended reference samples is separated from the predictive block by at least one nearest reference sample of the plurality of reference samples directly adjacent to the predictive block, and the video decoder is configured to determine the extended reference sample to be at least a partial part of a neighboring predictive block of the plurality of predictive blocks, determine that the neighboring predictive block has not yet been predicted, and receive information indicating the extended prediction sample associated with the predictive block and located as an unavailable sample in the neighboring predictive block.

[0193] The video decoder can be configured to decode an image by parallel decoding lines of blocks according to a wavefront approach and predict a predictive block based on an angular prediction, and the video decoder is configured to determine extended reference samples used to predict the predictive block to be located in an already predicted block of the image.

[0194] The video decoder can be configured to receive information associated with a predictive block and indicating extended prediction samples that are atypically arranged in neighboring predictive blocks as unavailable or available samples at a sequence level, image level, or slice level.

[0195] The video decoder may be configured to receive information associated with the predictive block and indicative of parallel decoding of the image together with information indicative of extended predictive samples located in adjacent predictive blocks as unavailable samples.

[0196] In angular intra prediction, the reference samples are copied into the current prediction area along a specified direction. If this direction points to the top right, the required reference sample area also shifts to the right the greater the distance to the prediction area boundary. Figure 13 shows an embodiment of vertical angular prediction at a 45 degree angle for a WxH prediction block. It can be seen that the closest reference sample area (blue) extends H samples to the top right of the current WxH block. When extending the prediction to a more distant reference sample area, the extended reference samples (green) extend H+1, H+2, ... samples to the top right of the current WxH block.

[0197] For example, if a square coding tree unit (CTU) is used as the basic processing unit, the maximum intra prediction block size can be equal to the maximum block size, i.e., the CTU block size N×N. Figures 14a and 14b show the embodiment from Figure 13 where the W×H prediction block is equal to the CTU block size of N×N. Correspondingly, the extended reference sample 1064 1 and / or 1064 2 spans the top right CTU (CTU2) and has at least one sample reaching the next CTU (CTU3), i.e., the extended reference sample 1064 for 45° vertical angle prediction 1 and / or 1064 2 A portion of the samples may be placed in unprocessed CTU3. If CTU3 and CTU5 are processed in parallel, these samples may be unavailable.

[0198] 14a and 14b show examples of nearest reference samples 1062 and extended reference samples 1064 required for diagonal vertical intra prediction.

[0199] In Fig. 14b, two CTU lines can be seen. If two lines are coded in parallel using a wavefront-like approach, once CTU2 is coded and coding of CTU3 starts, coding of the second CTU line can start with CTU5. In this case, 45 degree vertical angle prediction with extended reference samples cannot be used for CTU5, since some reference samples 1104 are located inside CTU3. The following approach can solve this problem: 1. As described in relation to the present embodiment, mark the extended references in the next basic processing domain as unavailable so that they can be treated like other unavailable reference samples, for example at image, slice or tile boundaries or when constrained intra prediction is used which does not allow the use of samples from inter-image prediction domains as references for intra-image prediction domains.

[0200] 2. Allow the use of extended reference samples only for regional and intra-picture prediction modes, so that the reference samples are not extended to multiple elementary processing units to the top right of the current region.

[0201] Both approaches can be made switchable by a high-level flag at the sequence, picture, or slice level. In this way, the encoder can inform the decoder to apply the limitation if required by the encoder's parallel processing scheme. If not required, the encoder can inform the decoder that the limitation is not applied.

[0202] An alternative is to combine the signaling of both limitations with the signaling of the parallel coding scheme, e.g., if wavefront parallelism is enabled and signaled in the bitstream, the extended reference sample limitation also applies.

[0203] In the following, some advantageous embodiments are described.

[0204] 1. In one embodiment, intra prediction uses two additional reference lines with reference sample area indexes i=1 and i=3 (see FIG. 3). These extended reference sample lines allow only angular prediction as described herein. The signaling of the reference sample area index i is performed as described in connection with FIG. 11 with MaxNumRefAreaIdx=3 and is signaled before signaling the intra prediction mode. If the reference area index is not equal to 0, i.e., if the extended reference lines are used, DC and intra-plane prediction modes are not used. To avoid unnecessary signaling, DC and planar modes are excluded from the signaling of intra prediction modes. The intra prediction modes can be coded using an index into a list of most probable modes (MPM). The MPM list contains a fixed number of prediction mode candidates derived from neighboring blocks. If the neighboring block is coded using the closest reference line (i=0), DC or planar modes can be used, so the derivation process of the MPM list is modified to not include DC and planar modes. If removed, the DC and planar modes can be replaced with horizontal, vertical and bottom left diagonal modes to fill the list (see Figures 4a to 4e). This is done in a way that avoids redundancy, e.g. if the first candidate mode derived from a neighboring block is vertical and the second candidate mode is DC, the DC mode is replaced with a horizontal mode instead of a vertical mode since it is not yet in the list. Another way to prevent unnecessary signaling of DC and planar modes when i>0 is to signal the reference region index i after the prediction mode and coordinate the signaling of i in the prediction mode. If the intra prediction mode is equal to DC or planar, the index i is not signaled. However, if the intra prediction mode is signaled using an index into the MPM list, this method introduces a parsing dependency, so the aforementioned method is preferred over this method. In order to be able to parse i, it is necessary to reconstruct the intra prediction mode, which requires the derivation of the MPM list. On the other hand, the derivation of the MPM list refers to the prediction modes of neighboring blocks, which need to be reconstructed before parsing i.This undesirable parsing dependency is resolved by signaling i before the prediction mode and modifying the derivation of the MPM list accordingly as described above. The video encoder can be configured to determine a list of the most probable prediction modes based on the use of the nearest reference samples or the use of the extended reference samples for the prediction mode, where the video encoder is configured to replace the prediction modes that are restricted for the used reference samples by the modes allowed in the prediction mode. A corresponding video decoder can be configured to determine a list of the most probable prediction modes based on the use of the nearest reference samples or the use of the extended reference samples for the prediction mode, where the video decoder is configured to replace the prediction modes that are restricted for the used reference samples by the modes allowed in the prediction mode.

[0205] 2. In another embodiment, the intra prediction of the extended reference samples is further restricted to be applied only to luma samples. That is, the video encoder can be configured to apply prediction using the extended reference samples to images that include only luma information. Thus, the video decoder can be configured to apply prediction using the extended reference samples to images that include only luma information.

[0206] 3. In another embodiment, extended reference samples (i>0) (see W+H in FIG. 3) that exceed the width and height of the nearest reference sample (i=0) are not generated by using already reconstructed samples (if available), but by padding from the last sample, e.g., r(23,-1-i) for the top right sample in FIG. 3 and r(-1-i,23) for the bottom left sample. This reduces memory accesses for extended reference sample lines. That is, the video encoder can be configured to generate extended reference samples that exceed the width and / or height of the nearest reference sample along the first and second image directions by padding from the nearest extended reference sample. Thus, the video decoder can be configured to generate extended reference samples that exceed the width and / or height of the nearest reference sample along the first and second image directions by padding from the nearest extended reference sample.

[0207] 4. In another embodiment, only the per second angle modes can be used for the extended reference samples. As a result, the derivation of the MPM list is modified to exclude these modes in addition to the DC and planar modes for i>0. That is, the video encoder can be configured to predict the prediction using an angle prediction mode that uses only a subset of angles from the possible angles of the angle prediction mode and exclude the unused angles from the notification of the coding information to the decoder. Thus, the video decoder can be configured to predict the prediction using an angle prediction mode that uses only a subset of angles from the possible angles of the angle prediction mode and exclude the unused angles from the prediction.

[0208] 5. In another embodiment, the number of additional reference sample lines is increased to 3 (see FIG. 11 by MaxNumRefAreaIdx=4). That is, the extended reference sample can be arranged at least 2 lines and rows in addition to the nearest reference sample, preferably at least 3 lines and rows. Such a configuration can be applied to a video encoder and a decoder. A video encoder can be configured to use a specific set of extended reference samples to predict a prediction block, and the video encoder is configured to select a specific set from the set to include the lowest similarity of image content when compared to the nearest reference samples extended by the set, i.e., to use related reference samples having the same reference area index, for example. Thus, a video decoder can be configured to use a specific set of extended reference samples to predict a prediction block, and the video decoder is configured to select a specific set from the set to include the lowest similarity of image content when compared to the nearest reference samples extended by the set.

[0209] 6. In another embodiment, instead of index i, a flag is signaled indicating whether extended reference samples are used. If the flag indicates that extended reference samples are used, index i (i>0) is derived by calculating the similarity between reference line i>0 and reference line i=0 (e.g., using the sum of absolute differences). The index i that results in the lowest similarity is selected. The idea behind it is that the higher the correlation between the extended reference sample (i>0) and the normal reference sample (i=0), the higher the correlation of the resulting prediction, so there is no additional benefit in using extended reference samples in prediction. That is, the video encoder can be configured to signal the use of extended reference samples using a flag or other, possibly binary, information. Thus, the video decoder can be configured to receive information indicating the use of extended reference samples by such a flag.

[0210] 7. In another embodiment, a secondary transform such as a non-separable secondary transform (NSST) may be applied after the first transform of the intra-prediction residual. For the extended reference sample line, the secondary transform is not performed, and all notifications related to the NSST are disabled when i>0. That is, the video encoder may be configured to selectively use only the extended reference sample or the nearest reference sample, and the video encoder may be configured to transform the residual obtained by predicting the prediction block using a first transform procedure to obtain a first transform result, and to transform the first transform result using a second transform procedure to obtain a second transform result when the extended reference sample is not used to predict the prediction block. This may also affect the notification of whether the second transform is used. That is, if the use of the second transform needs to be notified, the notification may be skipped when the extended reference sample is used. The video encoder may be configured to notify the use of the secondary transform, or to implicitly notify the non-use of the secondary transform when indicating the use of the extended reference sample, and not include information related to the result of the secondary transform in the encoded data.

[0211] Thus, the video decoder can be configured to selectively use only the extended reference samples or the nearest reference samples, and the video decoder is configured to transform a residual obtained by predicting the predictive block using a first transformation procedure to obtain a first transformation result and to transform the first transformation result using a second transformation procedure to obtain a second transformation result when the extended reference samples are not used to predict the predictive block. The video decoder can be configured to receive information indicating the use of a secondary transformation or to derive non-use of the secondary transformation when indicating the use of the extended reference samples and not to receive information related to a result of the secondary transformation of the encoded data.

[0212] 8. In another embodiment, prediction using extended reference samples (i>0) is combined with planar prediction using the nearest reference sample (i=0). As outlined in relation to the combination of different extended prediction modes, i.e., using extended reference samples, weighting can be fixed (e.g., 0.5 and 0.5), dependent on block size, or signaled at slice, image, or sequence level. When extended reference samples are used (i>0), an additional flag indicates whether combined prediction is applied. That is, the video encoder can be configured. The video decoder can be configured accordingly.

[0213] 9. In another embodiment, in order to reduce the overhead of notification, the notification of the combined prediction from above is omitted. Instead, the decision of whether to apply the combined prediction is derived based on the analysis of the nearest reference sample (i=0). One possible analysis can be the flatness of the nearest reference sample. If the nearest reference sample signal is flat (no edges), the combination is applied, and if it contains high frequencies and edges, the combination is not applied. That is, the video encoder can be configured. The video decoder can be configured accordingly.

[0214] While some aspects have been described in the context of an apparatus, it will be apparent that these aspects also represent a description of a corresponding method, where a block or apparatus corresponds to a method step or feature of a method step, and similarly, aspects described in the context of a method step also represent a description of a corresponding block or item or function of a corresponding apparatus.

[0215] Depending on specific implementation requirements, embodiments of the present invention can be implemented in hardware or software, for example using a digital storage medium such as a floppy disk, DVD, CD, ROM, PROM, EPROM, EEPROM or flash memory, on which electronically readable control signals are stored and which cooperate (or can cooperate) with a programmable computer system to perform the respective method.

[0216] Some embodiments of the present invention comprise a data carrier having electronically readable control signals capable of cooperating with a programmable computer system to cause one of the methods described herein to be performed.

[0217] Generally, embodiments of the present invention can be implemented as a computer program product comprising program code which operates to perform one of the methods when the computer program product is run on a computer. The program code may for example be stored on a machine readable carrier.

[0218] Other embodiments comprise the computer program for performing one of the methods described herein, stored on a machine readable carrier.

[0219] In other words, an embodiment of the inventive method is, therefore, a computer program having a program code for performing one of the methods described herein, when the computer program runs on a computer.

[0220] A further embodiment of the inventive method is, therefore, a data carrier (or digital storage medium, or computer readable medium) having recorded thereon the computer program for performing one of the methods described herein.

[0221] A further embodiment of the inventive method is therefore a data stream or a sequence of signals representing the computer program for performing one of the methods described herein, the data stream or the sequence of signals may be adapted to be transferred via a data communication connection, such as for example the Internet.

[0222] A further embodiment comprises a processing means, for example a computer, or a programmable logic device, configured to or adapted to perform one of the methods described herein.

[0223] A further embodiment comprises a computer having installed thereon the computer program for performing one of the methods described herein.

[0224] In some embodiments, a programmable logic device (e.g., a field programmable gate array) may be used to perform some or all of the functions of the methods described herein. In some embodiments, a field programmable gate array may cooperate with a microprocessor to perform one of the methods described herein. In general, the methods are preferably performed by any hardware apparatus.

[0225] The above-described embodiments merely exemplify the principles of the present invention. It is understood that modifications and variations of the configurations and details described herein will be apparent to those skilled in the art. It is therefore intended to be limited only by the scope of the appended claims and not by the specific details presented as illustrative and explanatory of the embodiments herein.

Claims

1. A video encoder (1000), comprising: a plurality of extended reference samples for a predictive block (1066) of an image, each extended reference sample of the plurality of extended reference samples being separated from the predictive block by at least one nearest reference sample (1062) that is directly adjacent to the predictive block; determining the availability or unavailability of each extended reference sample of the plurality of extended reference samples in sequence; Replace the extended reference samples that are determined to be unavailable with replacement samples. and configured to generate the generating one or more extended reference samples in a reference region by copying a nearest extended reference sample of the plurality of extended reference samples, wherein a position of each of the one or more extended reference samples in the reference region exceeds an extended reference sample position of the plurality of extended reference samples along a first image direction and a second image direction. and configuring to encode an image of a video into encoded data by block-based intra-picture predictive coding, the block-based intra-picture predictive coding of the predictive block using the plurality of extended reference samples and the one or more extended reference samples in the reference region. The replacement sample is determined as a copy of the last extended reference sample determined to be available in the sequence; and / or The replacement sample is determined to be a copy of a next extended reference sample determined to be available in the sequence.

2. The video encoder, determining the replacement sample based on a combination of an extended reference sample determined to be available and located in a sequence preceding an extended reference sample determined to be unavailable, and an extended reference sample determined to be available and located in a sequence following an extended reference sample determined to be unavailable; The video encoder of claim 1 , configured to:

3. A video decoder (1008), a plurality of extended reference samples for a predictive block (1066) of an image, each extended reference sample of the plurality of extended reference samples being separated from the predictive block by at least one nearest reference sample (1062) that is directly adjacent to the predictive block; determining the availability or unavailability of each extended reference sample of the plurality of extended reference samples in sequence; Replace the extended reference samples that are determined to be unavailable with replacement samples. and configured to generate the generating one or more extended reference samples in a reference region by copying a nearest extended reference sample of the plurality of extended reference samples, wherein a position of each of the one or more extended reference samples in the reference region exceeds an extended reference sample position of the plurality of extended reference samples along a first image direction and a second image direction. and decoding an image encoded in the encoded data by block-based intra-picture predictive coding to generate video, the block-based intra-picture predictive coding of the predictive block using the plurality of extended reference samples and the one or more extended reference samples in the reference region. The replacement sample is determined as a copy of the last extended reference sample determined to be available in the sequence; and / or The replacement sample is determined to be a copy of a next enhanced reference sample determined to be available in the sequence.

4. The video decoder comprises: determining the replacement sample based on a combination of an extended reference sample determined to be available and located in a sequence preceding an extended reference sample determined to be unavailable, and an extended reference sample determined to be available and located in a sequence following an extended reference sample determined to be unavailable; 4. A video decoder according to claim 3, configured to:

5. 1. A method for encoding video, comprising the steps of: a plurality of extended reference samples for a predictive block (1066) of an image, each extended reference sample of the plurality of extended reference samples being separated from the predictive block by at least one nearest reference sample (1062) that is directly adjacent to the predictive block; determining the availability or unavailability of each extended reference sample of the plurality of extended reference samples in sequence; Replace the extended reference samples that are determined to be unavailable with replacement samples. and generating one or more extended reference samples in a reference region by copying a nearest extended reference sample of the plurality of extended reference samples, and a position of each of the one or more extended reference samples in the reference region exceeds an extended reference sample position of the plurality of extended reference samples along a first image direction and a second image direction; encoding an image of a video into encoded data by block-based intra-predictive coding, wherein the block-based intra-predictive coding of the predictive block uses the plurality of extended reference samples and the one or more extended reference samples in the reference region; The replacement sample is determined as a copy of the last extended reference sample determined to be available in the sequence; and / or A method of encoding video, wherein the replacement sample is determined as a copy of a next extended reference sample determined to be available in the sequence.

6. 1. A method for decoding a video, comprising: a plurality of extended reference samples for a predictive block (1066) of an image, each extended reference sample of the plurality of extended reference samples being separated from the predictive block by at least one nearest reference sample (1062) that is directly adjacent to the predictive block; determining the availability or unavailability of each extended reference sample of the plurality of extended reference samples in sequence; Replace the extended reference samples that are determined to be unavailable with replacement samples. and generating one or more extended reference samples in a reference region by copying a nearest extended reference sample of the plurality of extended reference samples, and a position of each of the one or more extended reference samples in the reference region exceeds an extended reference sample position of the plurality of extended reference samples along a first image direction and a second image direction; decoding an image encoded in encoded data by block-based intra-picture predictive coding to generate video; and the block-based intra-picture predictive coding of the predictive block uses the plurality of extended reference samples and the one or more extended reference samples in the reference region; The replacement sample is determined as a copy of the last extended reference sample determined to be available in the sequence; and / or A method of decoding video, wherein the replacement sample is determined to be a copy of a next enhanced reference sample determined to be available in the sequence.

7. A non-transitory storage medium storing a computer program having a program code for performing the method according to claim 5 or 6, when the program is executed on a computer.

Images