Extended reference intra-picture prediction

Abstract

A video encoder is configured to encode an image of a video into encoded data by block-based predictive coding, where the block-based predictive coding includes intra-image prediction. The video encoder is configured to encode a prediction block of an image using a plurality of closest reference samples of the image directly adjacent to the prediction block and a plurality of extended reference samples for the intra prediction, each extended reference sample of the plurality of extended reference samples is separated from the prediction block by at least one closest reference sample of a plurality of reference samples. The video encoder is further configured to sequentially determine availability or non-availability of each of the plurality of closest reference samples, and replace the closest reference sample determined to be unavailable with an alternative sample. The video encoder is configured to perform the intra-image prediction using the alternative samples.

Description

[0001] This application is a divisional application of the patent application with application number 201980057073.5, application date June 28, 2019, and name “Extended Reference Image Intra-Prediction”. Technical Field

[0002] The present invention relates to video coding, in particular to hybrid video coding including intra-image prediction. The present invention further relates to a video encoder, a video decoder and methods for video encoding and decoding respectively. Background Art

[0003] H.265 / HEVC is a video codec that has provided tools for improving or even enabling parallel processing at the encoder and / or decoder. For example, HEVC supports subdividing an image into an array of tiles that are encoded independently of each other. Another concept supported by HEVC is about WPP, according to which CTU rows or CTU lines of an image can be processed in parallel from left to right, i.e., in strips, with the constraint that a certain minimum CTU offset is observed when processing consecutive CTU lines. However, it would be advantageous to have a video codec that would have the ability to support parallel processing of a video encoder and / or video decoder even more efficiently. Summary of the invention

[0004] It is therefore an object of the present invention to provide a video codec that enables more efficient processing at an encoder and / or decoder with respect to reference samples used for predicting a prediction block.

[0005] This object is achieved through the subject matter of the independent technical solution of this application.

[0006] According to an embodiment, a video encoder is configured to encode an image of a video into encoded data by block-based predictive coding, wherein the block-based predictive coding includes intra-image prediction. The video encoder is configured to use a plurality of extended reference samples of the image for the intra-image prediction to encode a prediction block of the image, each of the plurality of extended reference samples being separated from the prediction block by at least one closest reference sample of a plurality of reference samples that is directly adjacent to the prediction block. The video encoder is further configured to sequentially determine the availability or unavailability of each of the plurality of closest reference samples, and replace the closest reference sample determined to be unavailable with a replacement sample. The video encoder is configured to use the replacement sample for the intra-image prediction. This allows the use of a prediction concept using the closest reference sample, even if such a sample is not available, which may occur, for example, when there is a line or row of samples in a buffer / memory, but there is actually no column of samples in the memory that is therefore unavailable.

[0007] According to an embodiment, a video encoder is configured to encode an image of a video into encoded data by block-based predictive coding, wherein the block-based predictive coding includes intra-image prediction. The video encoder is configured to use multiple extended reference samples of the image to encode a prediction block of the image in the intra-image prediction, each of the multiple extended reference samples being separated from the prediction block by at least one closest reference sample of multiple reference samples that is directly adjacent to the prediction block. The video encoder is further configured to filter at least a subset of the multiple extended reference samples using a bilateral filter to obtain multiple filtered extended reference samples; and use the multiple filtered extended reference samples to perform the intra-image prediction.

[0008] According to an embodiment, a video encoder is configured to encode an image of a video into encoded data by block-based predictive coding, wherein the block-based predictive coding includes intra-image prediction. The video encoder is configured to use a plurality of extended reference samples of an image to encode a prediction block of the image in the intra-image prediction, each of the plurality of extended reference samples being separated from the prediction block by at least one closest reference sample directly adjacent to the prediction block among a plurality of reference samples, wherein the plurality of closest reference samples are arranged along a first image direction of the prediction block and along a second image direction of the prediction block; and at least a portion of the closest reference samples arranged along the second direction are mapped to the extended reference samples arranged along the first direction, so that the mapped reference samples exceed the extension of the prediction block along the first image direction. The video encoder is configured to use the mapped extended reference samples for the prediction.

[0009] According to an embodiment, a video encoder is configured to encode an image of a video into encoded data by block-based predictive coding, wherein the block-based predictive coding includes intra-image prediction. The video encoder is configured to use a plurality of closest reference samples of the image directly adjacent to the prediction block and a plurality of extended reference samples to encode the prediction block of the image for the intra-image prediction, wherein each of the plurality of extended reference samples is separated from the prediction block by at least one closest reference sample of a plurality of reference samples, wherein the video encoder is configured to perform boundary filtering in a mode not using extended samples; and not use boundary filtering when using extended samples; or wherein the video encoder is configured to perform boundary filtering on at least a subset of the plurality of closest reference samples, and not use boundary filtering on the extended samples.

[0010] According to an embodiment, a video encoder is configured to encode an image of a video into encoded data by block-based predictive coding, wherein the block-based predictive coding includes intra-image prediction. The video encoder is configured to determine, in the intra-image prediction, a plurality of closest reference samples of the image that are directly adjacent to a prediction block and a plurality of extended reference samples to encode the prediction block of the image, wherein each of the plurality of extended reference samples is separated from the prediction block by at least one of the closest reference samples of a plurality of reference samples. The video encoder is further configured to use the extended reference samples to determine a prediction for the prediction block, and to filter the extended reference samples to obtain a plurality of filtered extended reference samples; and to combine the prediction with the filtered extended reference samples to obtain a combined prediction for the prediction block.

[0011] According to an embodiment, a video encoder is configured to encode an image of a video into encoded data by block-based predictive coding, wherein the block-based predictive coding includes intra-image prediction. The video encoder is configured to use a plurality of closest reference samples and / or a plurality of extended reference samples of the image directly adjacent to the prediction block to encode the prediction block of the image for the intra-image prediction, each of the plurality of extended reference samples being separated from the prediction block by at least one of the closest reference samples of a plurality of reference samples; determine a first prediction for the prediction block using a first prediction mode in a prediction mode set, the first prediction mode set including a prediction mode using the plurality of closest reference samples in the absence of the extended reference sample; and determine a second prediction for the prediction block using a second prediction mode in a second prediction mode set, the second prediction mode set including a subset of the prediction modes in the first set, the subset being associated with the plurality of extended reference samples. The video encoder is configured to weight the prediction mode in a weighted manner (w0; w i ) combines the first prediction (p0(x,y)) with the second prediction (p i (x, y)) in order to obtain a combined prediction (p(x, y)) as a prediction for the prediction block in the coded data.

[0012] According to an embodiment, a video encoder is configured to encode an image of a video into encoded data by block-based predictive coding, wherein the block-based predictive coding includes intra-image prediction; for the intra-image prediction, a plurality of closest reference samples and / or a plurality of extended reference samples of the image that are directly adjacent to the prediction block are used to encode the prediction block of the image, each of the plurality of extended reference samples being separated from the prediction block by at least one closest reference sample of a plurality of reference samples; a prediction mode is used, the prediction mode being one of a first prediction mode set that uses these closest reference samples to predict the prediction block, for example, in the absence of extended reference samples; or a prediction mode for using the extended reference samples to predict the prediction block. wherein the second prediction mode set is a subset of the first prediction mode set; signaling mode information (m) indicating the prediction mode used to predict the prediction block; and if the prediction mode is included in the second prediction mode set, then signaling parameter information (i) indicating a subset of extended reference samples used for the prediction mode; and when the prediction mode used is not included in the second prediction mode set, skipping signaling the parameter information, thereby allowing a conclusion to be drawn at the decoder that a specific value of the parameter has been selected or chosen or determined, the predefined property allowing skipping of signaling, i.e., the absence of signaling is given an informative meaning. For example, the absence may indicate that the closest reference sample must be used.

[0013] According to an embodiment, a video encoder is configured to encode an image of a video into encoded data by block-based predictive coding, wherein the block-based predictive coding includes intra-image prediction; for the intra-image prediction, a plurality of reference samples including the closest reference sample of the image directly adjacent to the prediction block and a plurality of extended reference samples are used to encode the prediction block of the image, each of the plurality of extended reference samples being separated from the prediction block by at least one of the closest reference samples of the plurality of reference samples; a prediction mode is used, the prediction mode is one of a first prediction mode set for predicting the prediction block using the closest reference sample; or one of a second prediction mode set for predicting the prediction block using the extended reference sample; wherein the second prediction mode set is a subset of the first prediction mode set. The video encoder may use available reference data to generate a first group and / or a second group and / or may use information derived from the image to determine the group. The second group as a subset of the first group includes a situation where the two groups are equal. The video encoder is configured to signal parameter information indicating a subset of a plurality of reference samples for the prediction mode, the subset of the plurality of reference samples comprising only the closest reference sample or the extended reference sample; and subsequently signal mode information (m) indicating a prediction mode for predicting the prediction block, wherein the mode information indicates a prediction mode from a subset of modes, the subset being restricted to a set of allowed prediction modes according to the parameter information (i). Based on the association of the reference (i.e., closest or extended) samples used, identification of the restricted set is possible because only those prediction modes associated with the indicated reference samples are applicable.

[0014] According to an embodiment, a video encoder is configured to encode an image of a video into encoded data by block-based predictive coding, wherein the block-based predictive coding includes intra-image prediction; for the intra-image prediction, a plurality of closest reference samples and / or a plurality of extended reference samples of the image directly adjacent to the prediction block are used to encode the prediction block of the image, each of the plurality of extended reference samples being separated from the prediction block by at least one closest reference sample of a plurality of reference samples; a first prediction mode in a prediction mode set is used to determine a first prediction for the prediction block, the first prediction mode set including a prediction mode using the plurality of closest reference samples in the absence of the extended reference samples; and a second prediction mode in a second prediction mode set is used to determine a second prediction for the prediction block, the second prediction mode set including a subset of the prediction modes in the first set, the subset being associated with the plurality of extended reference samples. The video encoder is configured to combine the first prediction with the second prediction so as to obtain a combined prediction as a prediction for the prediction block in the encoded data.

[0015] According to an embodiment, a video encoder is configured to encode an image of a video into encoded data by block-based predictive coding, wherein the block-based predictive coding includes intra-image prediction; in the intra-image prediction, a plurality of extended reference samples of the image are used to encode a prediction block of the image, each of the plurality of extended reference samples being separated from the prediction block by at least one closest reference sample of a plurality of reference samples that is directly adjacent to the prediction block; and the plurality of extended reference samples are used according to a predefined set of the plurality of extended reference samples. For example, the plurality of extended reference samples may be included in a region index list, for example, identified by an identifier.

[0016] According to an embodiment, a video encoder is configured to encode an image of a video into encoded data by performing block-based predictive encoding on multiple prediction blocks, wherein the block-based predictive encoding includes intra-image prediction; a prediction block in the multiple prediction blocks is encoded using multiple extended reference samples of the image for the intra-image prediction, each of the multiple extended reference samples being separated from the prediction block by at least one closest reference sample in multiple reference samples that is directly adjacent to the prediction block. The video encoder is configured to determine the extended reference sample so as to at least partially become part of a neighboring prediction block in the multiple prediction blocks, and determine that the neighboring prediction block has not been predicted; and signal information indicating an extended prediction sample associated with the prediction block and arranged in the neighboring prediction block as an unavailable sample.

[0017] According to an embodiment, a video decoder is configured to decode an image encoded as encoded data into a video through block-based predictive decoding, wherein the block-based predictive decoding includes intra-image prediction; in the intra-image prediction, multiple extended reference samples of the image are used to encode a prediction block of the image, each of the multiple extended reference samples is separated from the prediction block by at least one closest reference sample directly adjacent to the prediction block among multiple reference samples; the availability or unavailability of each of the multiple closest reference samples is sequentially determined; the closest reference sample determined to be unavailable is replaced with a replacement sample; and the intra-image prediction is performed using the replacement sample.

[0018] According to an embodiment, a video decoder is configured to decode an image encoded as encoded data into a video through block-based predictive decoding, wherein the block-based predictive decoding includes intra-image prediction; in the intra-image prediction, multiple extended reference samples of the image are used to decode a prediction block of the image, each of the multiple extended reference samples is separated from the prediction block by at least one closest reference sample directly adjacent to the prediction block among multiple reference samples; at least a subset of the multiple extended reference samples is filtered using a bilateral filter to obtain multiple filtered extended reference samples; and the intra-image prediction is performed using the multiple filtered extended reference samples.

[0019] According to an embodiment, a video decoder is configured to decode an image encoded as encoded data into a video through block-based predictive decoding, wherein the block-based predictive decoding includes intra-image prediction; in the intra-image prediction, multiple extended reference samples of the image are used to decode a prediction block of the image, each of the multiple extended reference samples is separated from the prediction block by at least one closest reference sample directly adjacent to the prediction block among multiple reference samples, wherein the multiple closest reference samples are arranged along a first image direction of the prediction block and along a second image direction of the prediction block; at least a portion of the closest reference samples arranged along the second direction are mapped to extended reference samples arranged along the first direction, so that the mapped reference samples exceed the extension of the prediction block along the first image direction; and the prediction is performed using the mapped extended reference samples.

[0020] According to an embodiment, a video decoder is configured to decode an image encoded as encoded data into a video by block-based predictive decoding, wherein the block-based predictive decoding includes intra-image prediction; for the intra-image prediction, a plurality of closest reference samples of the image directly adjacent to the prediction block and a plurality of extended reference samples are used to decode the prediction block of the image, each of the plurality of extended reference samples being separated from the prediction block by at least one closest reference sample of a plurality of reference samples. The video decoder is configured to perform boundary filtering in a mode not using extended samples; and not use boundary filtering when using extended samples; or to perform boundary filtering on at least a subset of the plurality of closest reference samples, and not use boundary filtering on the extended samples.

[0021] According to an embodiment, a video decoder is configured to decode an image encoded as encoded data into a video through block-based predictive decoding, wherein the block-based predictive decoding includes intra-image prediction; in the intra-image prediction, multiple closest reference samples of the image directly adjacent to a prediction block and multiple extended reference samples are determined to decode the prediction block of the image, each of the multiple extended reference samples being separated from the prediction block by at least one closest reference sample of a plurality of reference samples; the extended reference samples are used to determine a prediction for the prediction block, and the extended reference samples are filtered to obtain multiple filtered extended reference samples; and the prediction is combined with the filtered extended reference samples to obtain a combined prediction for the prediction block.

[0022] According to an embodiment, a video decoder is configured to decode an image encoded as coded data into a video by block-based predictive decoding, wherein the block-based predictive decoding includes intra-image prediction; for the intra-image prediction, a plurality of closest reference samples and / or a plurality of extended reference samples of the image directly adjacent to the prediction block are used to decode the prediction block of the image, each of the plurality of extended reference samples being separated from the prediction block by at least one closest reference sample of a plurality of reference samples; a first prediction mode in a prediction mode set is used to determine a first prediction for the prediction block, the first prediction mode set comprising a prediction mode using the plurality of closest reference samples in the absence of the extended reference samples; and a second prediction mode in a second prediction mode set is used to determine a second prediction for the prediction block, the second prediction mode set comprising a subset of the prediction modes in the first set, the subset being associated with the plurality of extended reference samples. The video decoder is configured to combine the first prediction with the second prediction in a weighted manner so as to obtain a combined prediction as a prediction for the prediction block in the coded data.

[0023] According to an embodiment, a video decoder is configured to decode an image encoded as encoded data into a video by block-based predictive decoding, wherein the block-based predictive decoding includes intra-image prediction; for the intra-image prediction, using a plurality of closest reference samples and / or a plurality of extended reference samples of the image directly adjacent to the prediction block to decode the prediction block of the image, each of the plurality of extended reference samples being separated from the prediction block by at least one closest reference sample of a plurality of reference samples; using a prediction mode, the prediction mode being a first prediction mode for predicting the prediction block using the closest reference samples one of a set of prediction modes; or one of a second prediction mode set for predicting the prediction block using the extended reference sample; wherein the second prediction mode set is a subset of the first prediction mode set; receiving mode information (m) indicating the prediction mode used to predict the prediction block; and subsequently receiving parameter information (i) indicating a subset of extended reference samples used for the prediction mode, thereby indicating that the prediction mode is included in the second prediction mode set; and when the parameter information is not received, determining that the prediction mode used is not included in the second prediction mode set, and determining the use of the closest reference sample for the prediction.

[0024] According to an embodiment, a video decoder is configured to decode an image encoded as encoded data into a video by block-based predictive decoding, wherein the block-based predictive decoding includes intra-image prediction; for the intra-image prediction, using a plurality of reference samples including a closest reference sample of the image directly adjacent to the prediction block and a plurality of extended reference samples to decode the prediction block of the image, each of the plurality of extended reference samples being separated from the prediction block by at least one closest reference sample of a plurality of reference samples; using a prediction mode, the prediction mode being a first prediction mode for predicting the prediction block using the closest reference sample one of a set of prediction modes; or one of a second set of prediction modes for predicting the prediction block using the extended reference sample; wherein the second set of prediction modes is a subset of the first set of prediction modes; receiving parameter information (i) indicating a subset of the plurality of reference samples for the prediction mode, the subset of the plurality of reference samples including only the closest reference sample or at least one extended reference sample; and subsequently receiving mode information (m) indicating the prediction mode for predicting the prediction block; wherein the mode information indicates a prediction mode from a subset of modes, the subset being restricted to a set of allowed prediction modes according to the parameter information (i).

[0025] According to an embodiment, a video decoder is configured to decode an image encoded as coded data into a video by block-based predictive decoding, wherein the block-based predictive decoding includes intra-image prediction; for the intra-image prediction, use a plurality of closest reference samples and / or a plurality of extended reference samples of the image directly adjacent to the prediction block to decode the prediction block of the image, each of the plurality of extended reference samples being separated from the prediction block by at least one closest reference sample of a plurality of reference samples; determine a first prediction for the prediction block using a first prediction mode in a prediction mode set, the first prediction mode set including prediction modes using the plurality of closest reference samples in the absence of the extended reference samples; and determine a second prediction for the prediction block using a second prediction mode in a second prediction mode set, the second prediction mode set including a subset of the prediction modes in the first set, the subset being associated with the plurality of extended reference samples. The video decoder is configured to combine the first prediction with the second prediction so as to obtain a combined prediction as a prediction for the prediction block in the coded data.

[0026] According to an embodiment, a video decoder is configured to decode an image encoded as encoded data into a video by block-based predictive decoding, wherein the block-based predictive decoding includes intra-image prediction; in the intra-image prediction, multiple extended reference samples of the image are used to decode a prediction block of the image, each of the multiple extended reference samples is separated from the prediction block by at least one closest reference sample of multiple reference samples that is directly adjacent to the prediction block; and the multiple extended reference samples are used according to a predefined set of multiple extended reference samples.

[0027] According to an embodiment, a video decoder is configured to decode an image encoded as encoded data into a video by block-based predictive decoding, wherein a plurality of prediction blocks are decoded for each image, wherein the block-based predictive decoding includes intra-image prediction; for the intra-image prediction, a plurality of extended reference samples of the image are used to decode a prediction block in the plurality of prediction blocks, each of the plurality of extended reference samples being separated from the prediction block by at least one closest reference sample in a plurality of reference samples that is directly adjacent to the prediction block. The video decoder is configured to determine the extended reference sample so as to at least partially become part of a neighboring prediction block in the plurality of prediction blocks, and to determine that the neighboring prediction block has not been predicted; and to receive information indicating an extended prediction sample associated with the prediction block and arranged in the neighboring prediction block as an unavailable sample.

[0028] Other embodiments relate to methods for encoding and decoding video, and to a computer program product.

[0029] Regarding the aforementioned embodiments of the present application, it should be noted that these embodiments can be combined such that more than one of the aforementioned embodiments, such as all embodiments, are implemented in a video codec at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In addition, advantageous embodiments of the present application are the subject of the patent scope of the present application and are described below with reference to the accompanying drawings, in which preferred embodiments of the present application are described.

[0031] Figure 1 shows a schematic block diagram of a video encoder according to an embodiment comprising a decoder according to an embodiment;

[0032] Figure 2 A schematic flow chart showing a method for encoding a video stream according to an embodiment;

[0033] Figure 3 An example of directly adjacent (closest) reference samples and extended reference samples used in the embodiments is shown;

[0034] Figures 4a to 4e An example of five angle intra-image prediction angles for a 4×2 block of prediction samples according to an embodiment is shown;

[0035] Figure 4f A schematic diagram showing the direction of angle prediction used in the embodiment is shown;

[0036] Figure 4g to Figure 4h shows a table illustrating an example dependency of the number of taps used in the filter, the number depending on the block size of the prediction block and on the prediction mode;

[0037] Figure 5a to Figure 5c An embodiment of angle prediction in conjunction with the use of a definition of an angle parameter is shown;

[0038] Figures 6a to 6c shows the derivation of vertical offsets in conjunction with mapped reference samples according to an embodiment;

[0039] Figures 7a to 7c shows the derivation of the horizontal offset according to an embodiment;

[0040] Figure 8 An embodiment showing the use of a diagonal upper left angle and a closest reference sample according to an embodiment;

[0041] Fig. 9 shows an example projection of an extended left reference sample as a side reference next to an extended top reference sample as a main reference in the case of top left diagonal prediction according to an embodiment;

[0042] Fig.10 shows an example projection of a closest left reference sample as a side reference next to an extended top reference sample according to an embodiment;

[0043] Fig.11 shows an example truncated unary code for a particular set of reference regions according to an embodiment;

[0044] Figure 12a to Figure 12b shows a schematic illustration of available block sizes according to an embodiment;

[0045] Fig.13 A schematic diagram according to an embodiment showing vertical angle prediction using a prediction block at a 45 degree angle; and

[0046] Figure 14a to Figure 14b An example of required closest reference samples and extended reference samples in the case of diagonal vertical intra-image prediction and according to an embodiment is shown. DETAILED DESCRIPTION

[0047] Even if the same or equivalent one or more components having the same or equivalent functionality appear in different drawings, the one or more components are denoted by the same or equivalent reference numerals in the following description.

[0048] In the following description, a number of details are set forth to provide a more thorough explanation of embodiments of the present invention. However, it will be apparent to one skilled in the art that embodiments of the present invention may be practiced without these specific details. In other cases, well-known structures and devices are shown in block diagram form rather than in detail to avoid obscuring embodiments of the present invention. In addition, unless otherwise specifically noted, the features of the different embodiments described below may be combined with each other.

[0049] In hybrid video coding, intra-image prediction is used to encode a region of image samples by generating a prediction signal from available neighboring samples (i.e., reference samples). The prediction signal is subtracted from the original video data to obtain a residual signal. This residual signal or prediction error is further transformed, scaled, quantized, and entropy coded, such as Figure 1 As described in Figure 1A schematic block diagram of a video encoder 1000 according to an embodiment is shown, the video encoder being, for example, a hybrid video encoder with an intra-image prediction block 1001. The video encoder 1000 is configured to receive an input video signal 1002 comprising a plurality of images, a sequence of images forming a video. The video encoder 1000 comprises a block 1004 configured to divide the signal 1002 into sample regions, i.e. to form blocks from the input video signal 1002. A controller 1006 of the video encoder 1000 is configured to control the block 1004 and to control a decoder 1008 which may be part of the encoder 1000. A decoder for receiving and decoding an output bitstream 1012 and a generated output video signal 1014 (i.e., encoded data) may be implemented accordingly. Specifically, the transform, scaling, and quantization block 1016 together with the block 1018 for motion estimation of the signal 1022 can provide information about the quantized transform coefficients and motion information to achieve entropy encoding of the output bitstream 1012, where the signal 1022 is the input video signal 1002 divided into blocks by the block 1004.

[0050] The quantized transform coefficients are then scaled and inversely transformed to generate a reconstructed residual signal before potential in-loop filtering operations. This signal can again be added to the prediction signal to obtain a reconstruction that is also available at the decoder. The reconstructed signal can be used to predict subsequent samples in the coding order within the same picture.

[0051] Figure 2 Intra-image prediction is further described in detail in . First, in block 1042, reference samples for prediction are generated based on the reconstructed samples. This stage also includes, for example, replacing unavailable adjacent samples at image, slice or block boundaries. Second, in block 1044, the reference samples can be filtered to eliminate discontinuities in the reference signal. Third, in block 1046, the reference samples are used to calculate the prediction samples according to the intra-prediction mode. The prediction mode describes how to generate the prediction signal from the reference samples, for example by averaging the reference samples in DC mode or by copying the reference samples along a prediction angle in angular prediction mode. The encoder must decide which intra-prediction mode to select, and signal the selected intra-prediction mode to the decoder in the bitstream by entropy encoding. On the decoder side, the intra-prediction mode is extracted from the bitstream by entropy decoding. Fourth and possibly last, in block 1048, the prediction samples can also be filtered to smooth the signal. In other words, Figure 2A flow chart showing an intra-image prediction process or method. In general, the correlation between samples in an image decreases as the distance increases. Therefore, directly adjacent samples are usually more suitable as reference samples to predict a sample region. However, there are situations where directly adjacent reference samples represent edges or objects in a uniform area (occlusion). In these situations, the correlation between the sample used for prediction (uniform or textured area) and the directly adjacent reference sample (edge) will be low. Extended reference intra-image prediction solves this problem by incorporating more distant reference samples that are not directly adjacent. The concept of extending the closest reference samples is known, but several novel improvements to all parts of the intra-image prediction process and signal notification are defined in embodiments of the present invention and described below.

[0052] Extended reference intra-image prediction allows the use of extended reference samples to generate prediction signals for sample regions. Extended reference samples are available reference samples that are not directly adjacent. In the following, improved reference sample generation, filtering, prediction, and prediction filtering using extended reference samples according to embodiments are described in further detail. Special cases of combining prediction using extended reference samples with prediction using directly adjacent samples or unfiltered reference samples are then covered. Thereafter, various methods according to embodiments are described to improve prediction modes for extended reference samples and extended reference region signaling. Additionally, embodiments are described to facilitate parallel coding using extended reference samples.

[0053] For reference sample generation, current video coding standards use direct neighboring samples to predict the current block. In the literature, it is proposed to use multiple reference lines in addition to the closest direct neighboring samples. The additional reference lines to be used for intra-image prediction are further referred to as extended reference samples, which are described in detail below. Hereinafter, an improved method for replacing non-available extended reference samples according to an embodiment is described.

[0054] An example diagram showing one closest reference sample line and three extended reference sample lines for a 16×8 block to be predicted is shown in Figure 3 middle, Figure 3 Examples of directly adjacent (closest) reference sample 1062 and extended reference samples 10641 , 10642 , and 10643 are shown.

[0055] The closest reference sample 1062 and the extended reference samples 10641, 10642 and 10643 are arranged adjacent to the prediction block 1066 to be predicted and arranged along two directions of the image, namely direction x and direction y arranged perpendicular to direction x. Along direction x, the prediction block includes an extension W of samples in the range of 0 to W-1. Along direction y, the prediction block includes an extension H of samples in the range of 0 to H-1.

[0056] The reference region with index i may indicate the distance between each reference sample (i.e., index i=0, i.e., the closest reference sample arranged directly adjacent) and the extended reference sample spaced apart from at least the closest reference sample 1062 from the prediction block 1066. For example, the reference region index i may indicate the extension of the distance between the prediction block 1066 and each reference sample 1062 or 1064. By way of example, an increasing parameter x along the direction x may be referred to as moving to the right, and conversely, a decreasing parameter x may be referred to as moving to the left.

[0057] Alternatively or additionally, decreasing the index i in the negative direction y may be referred to as moving upward or toward the top of the image, wherein as the parameter y increases, a movement downward or toward the bottom of the image may be indicated. Terms such as top, bottom, left, and right are used to simplify the understanding of the present invention. According to other embodiments, these terms may be changed, altered, or replaced in any other direction without limiting the scope of the embodiments of the present invention. By way of example, reference samples 1062 and / or 1064 arranged to the left (i.e., with x<0) from the prediction block may be referred to as left reference samples. Assuming that the upper left edge of the prediction block 1066 has a position of 0,0, the reference samples 1062 and / or 1064 arranged so as to have y<0 may be referred to as top reference samples. Reference samples identified as left reference samples and top reference samples may be referred to as corner reference samples. Therefore, reference samples beyond the extension W in the x direction may be referred to as right reference samples, wherein reference samples beyond the extension H of the prediction block 1066 may be referred to as bottom reference samples.

[0058] To indicate which reference samples are to be used for prediction, each line of reference samples is associated with a reference region index i. The closest reference sample is given index i=0, the next line of extended reference samples is given i=1, and so on. Figure 3 The notation of , the top reference sample can be described by r(x, -1-i), where x is in the range of 0 to M, and the left reference sample can be described by r(-1-i, y), where y is in the range of 0 to N. The parameter M for the horizontal extension of the reference sample and the parameter N for the vertical extension depend on the intra-image prediction. In the case where the top right diagonal prediction is used as the maximum angle clockwise, for example, as combined Figure 4e As described above, the top reference sample needs to be extended by M=W+H-1+i samples in the horizontal direction. In the case where the bottom left diagonal prediction is used as the maximum angle counterclockwise, for example, as combined with Figure 4a As described above, the left reference sample needs to be extended by M=H+W-1+i samples in the vertical direction. Figure 3 In , W describes the width of the prediction block and H describes its height.

[0059] According to an embodiment of the present invention, a video encoder, such as the video encoder 1000, may be configured to encode an image of a video into encoded data by block-based predictive coding, wherein the block-based predictive coding includes intra-image prediction. The video encoder may use a plurality of extended reference samples of an image in intra-image prediction to encode a prediction block of the image, each of the plurality of extended reference samples being separated from the prediction block by at least one closest reference sample of a plurality of reference samples that is directly adjacent to the prediction block. The video encoder may sequentially determine the availability or unavailability of each of the plurality of closest reference samples, and may replace the closest reference sample determined to be unavailable with a replacement sample. The video encoder may use the replacement sample for intra-image prediction.

[0060] To determine availability or unavailability, the video encoder may sequentially examine samples according to a sequence, determine the replacement sample to be a copy of the last extended reference sample determined to be available in the sequence; and / or determine the replacement sample to be a copy of the next extended reference sample determined to be available in the sequence.

[0061] The video encoder may further determine the availability or unavailability sequentially according to a sequence, and determine the replacement sample based on a combination of an extended reference sample determined to be available and arranged before the reference sample determined to be unavailable in the sequence, and an extended reference sample determined to be available and arranged after the reference sample determined to be unavailable in the sequence.

[0062] Alternatively or additionally, the video encoder may be configured to encode the prediction block of the image using a plurality of closest reference samples of the image directly adjacent to the prediction block and a plurality of extended reference samples for intra-image prediction, each of the plurality of extended reference samples being separated from the prediction block by at least one of the closest reference samples of the plurality of reference samples; and determine the availability or unavailability of each of the plurality of extended reference samples. When a portion of the available extended reference samples in the plurality of extended reference samples is greater than or equal to a predetermined threshold, the video encoder may signal the use of the plurality of extended reference samples; and when the portion of the available extended reference samples in the plurality of extended reference samples is lower than the predetermined threshold, skip signaling the use of the plurality of extended reference samples.

[0063] Each decoder such as the video decoder 1008 or a video decoder for regenerating a video stream can be configured accordingly to decode an image encoded as encoded data into a video by block-based predictive decoding, wherein the block-based predictive decoding includes intra-image prediction; encoding a prediction block of the image using a plurality of extended reference samples of the image in the intra-image prediction, each of the plurality of extended reference samples being separated from the prediction block by at least one closest reference sample directly adjacent to the prediction block among a plurality of reference samples; and sequentially determining the availability or unavailability of each of the plurality of extended reference samples. The video decoder can replace the extended reference sample determined to be unavailable with a replacement sample; and use the replacement sample for intra-image prediction.

[0064] The video decoder may be further configured to determine the availability or unavailability sequentially according to a sequence; determine the replacement sample as a copy of the last extended reference sample determined to be available in the sequence; and / or determine the replacement sample as a copy of the next extended reference sample determined to be available in the sequence.

[0065] In addition, the video decoder can be configured to determine the availability or unavailability sequentially according to a sequence, and determine the replacement sample based on a combination of an extended reference sample determined to be available and arranged before the reference sample determined to be unavailable in the sequence, and an extended reference sample determined to be available and arranged after the reference sample determined to be unavailable in the sequence.

[0066] Alternatively or additionally, the video decoder may be configured to, for intra-image prediction, decode a prediction block of the image using a plurality of closest reference samples of the image that are directly adjacent to the prediction block and a plurality of extended reference samples, each of the plurality of extended reference samples being separated from the prediction block by at least one closest reference sample of the plurality of reference samples; determine the availability or unavailability of each of the plurality of extended reference samples; signal receiving information indicating the use of the plurality of extended reference samples when a portion of the available extended reference samples in the plurality of extended reference samples is greater than or equal to a predetermined threshold, and use the plurality of extended reference samples; and in the absence of the information, i.e., when the portion of the available extended reference samples in the plurality of extended reference samples is below the predetermined threshold, skip using the plurality of extended reference samples.

[0067] When the adjacent reference sample is not available, extended reference sample replacement may be performed according to an embodiment. For example, the unavailable sample may be replaced with the nearest available adjacent sample, or with a combination of the nearest two samples or a predetermined value, such as 2 bitdepth-1For example, reference samples are not available when they are outside a picture, slice or tile boundary or when restricted intra prediction is used which does not allow using samples from inter-picture predicted areas as references for intra-picture predicted areas.

[0068] For example, if the current block to be predicted is located at the left image boundary, the left and top left corner reference samples are not available. In this case, the left and top left corner reference samples are replaced with the first available top reference sample. When this first available top reference sample is the first reference sample, i.e., r(0, -1-i), the replacement of the closest reference sample (i=0) and the extended reference sample (i>0) can be formulated as follows:

[0069] r(-1-i,y)=r(0,-1-i), where y=-1-i..M

[0070] r(x,-1-i)=r(0,-1-i), where y=-1-i..M, for i>0

[0071] The parameter i may have any maximum value greater than 0 within the encoder / decoder system, for example 1, 2, 3, 4 or more, such as 5 or more, or 10 or more.

[0072] When restricted intra prediction is used, it may happen that one or more neighboring blocks may not be available because they are coded using inter-image prediction. For example, the left H samples r(-1-i,y), where y=0..H-1, are inside the inter-image predicted block and are not available because restricted intra prediction is enabled.

[0073] In one embodiment, the availability check process for each reference sample is performed sequentially, e.g., from the bottom left to the top right reference sample or vice versa, and the first sample that is unavailable in this direction is replaced by the last available sample, if one exists. If there was no available sample before, the unavailable sample is replaced by the next available sample. In the embodiment starting from the bottom right, W bottom right samples r(-1-i,y) are available, where y=H..W-1. Therefore, the left sample is replaced as follows:

[0074] r(-1-i,y)=r(-1-i,M), where y=0..M-1

[0075] In another embodiment, unavailable reference samples between available reference samples may be generated by linear interpolation between the two closest samples, one from each side.

[0076] In case it is determined that most of the extended reference region samples are not available, using the extended reference samples does not bring any benefit compared to using the closest reference samples. Therefore, the signaling of the reference region index can be omitted and its signaling can be limited to blocks where at least half of the extended reference samples are available.

[0077] According to an embodiment, a video encoder, such as video encoder 1000, can be configured to encode an image of a video into encoded data through block-based predictive coding, wherein the block-based predictive coding includes intra-image prediction; encoding a prediction block of the image using multiple extended reference samples of the image in the intra-image prediction, each of the multiple extended reference samples being separated from the prediction block by at least one closest reference sample of the multiple reference samples that is directly adjacent to the prediction block; filtering at least a subset of the multiple extended reference samples using a bilateral filter to obtain multiple filtered extended reference samples; and using the multiple filtered extended reference samples for intra-image prediction.

[0078] The video encoder according to this embodiment may be configured to combine a plurality of filtered extended reference samples with a plurality of unfiltered extended reference samples to obtain a plurality of combined reference values, wherein the video encoder is configured to perform intra-image prediction using the plurality of combined reference values.

[0079] Alternatively or additionally, the video encoder may be configured to filter the plurality of extended reference samples using one of a 3-tap filter, a 5-tap filter, and a 7-tap filter.

[0080] The video encoder may be further configured to select to use an angle prediction mode to predict a prediction block; wherein the 3-tap filter, the 5-tap filter, and the 7-tap filter are configured as bilateral filters, wherein the video encoder is configured to select to use one of the 3-tap filter, the 5-tap filter, and the 7-tap filter based on the angle used for angle prediction, the angle being arranged between the horizontal direction or the vertical direction of the angle prediction mode; and / or wherein the video decoder is configured to select to use one of the 3-tap filter, the 5-tap filter, and the 7-tap filter based on the block size of the prediction block. Figure 4f As shown in , the angle ε can describe the angle of the direction of the angular prediction used to predict the prediction block 1066 relative to the horizontal boundary 1072 and / or the vertical boundary 1074 of the prediction block 1066, and is measured towards the diagonal 1076 between the horizontal direction and the vertical direction, that is, the angle of the angular prediction is at most 45°. As the angle ε increases, an increasing number of taps can be used in the filter. Alternatively or additionally, the block size can define the basis or dependency for selecting the filter.

[0081] The corresponding video decoder can be configured to decode an image encoded as encoded data into a video through block-based predictive decoding, wherein the block-based predictive decoding includes intra-image prediction; using multiple extended reference samples of the image in the intra-image prediction to decode a prediction block of the image, each of the multiple extended reference samples is separated from the prediction block by at least one closest reference sample directly adjacent to the prediction block among multiple reference samples; filtering at least a subset of the multiple extended reference samples using a bilateral filter to obtain multiple filtered extended reference samples; and using the multiple filtered extended reference samples for intra-image prediction.

[0082] The video decoder may be further configured to combine the plurality of filtered extended reference samples with the plurality of unfiltered extended reference samples to obtain a plurality of combined reference values, wherein the video decoder is configured to use the plurality of combined reference values ​​for intra-image prediction.

[0083] Alternatively or additionally, the video decoder may be configured to filter the plurality of extended reference samples using one of a 3-tap filter, a 5-tap filter, and a 7-tap filter. As described for the encoder, the 3-tap filter, the 5-tap filter, and the 7-tap filter are configured as bilateral filters, wherein the video decoder is configured to predict the prediction block using an angle prediction mode, and selects to use one of the 3-tap filter, the 5-tap filter, and the 7-tap filter based on an angle for angle prediction, the angle being arranged between the horizontal direction or the vertical direction of the angle prediction mode; and / or wherein the video decoder is configured to select to use one of the 3-tap filter, the 5-tap filter, and the 7-tap filter based on a block size of the prediction block.

[0084] For example, instead of a bilateral filter, a 3-tap FIR filter may be used. This may allow filtering only the closest reference samples (although not using a bilateral filter), and not filtering the extended reference samples.

[0085] For larger sample areas, discontinuities in the reference samples may occur that distort the prediction. A prior art solution for this is to apply a linear smoothing filter to the reference samples. In the case of discontinuities that can be detected, for example, by comparison with a predetermined threshold, a strong smoothing can be applied. This is typically done by interpolating between corner reference samples to generate reference samples.

[0086] However, a linear smoothing filter may also remove edge structures that need to be preserved. Applying a bilateral filter to the extended reference samples according to an embodiment for reference sample filtering can prevent undesired smoothing of sharp edges. Since bilateral filtering is more efficient for larger blocks and for intra-prediction angles that deviate from exact horizontal and exact vertical directions, the decision whether to apply the filter and the length of the filter can depend on the block size and / or prediction mode. An example design can incorporate dependencies, such as the dependency shown with respect to W×H for block sizes less than 64×64 and greater than or equal to 64×64 Figure 4g , and shows the different dependencies of W×H for block sizes less than 64×64 and greater than or equal to 64×64 Figure 4h As shown in Figure 4g In the embodiment of , the intra prediction mode may be, for example, a planar mode, a DC mode, a near horizontal mode, a near vertical mode, or one of different angles of an angular mode. Figure 4h In an embodiment of the present invention, an angle identified as being farther horizontal and farther vertical may be additionally selected, for example, an angle having a value greater than that of being closer to horizontal or closer to vertical. Figure 4f As can be seen, a larger block size can result in a higher number of taps to facilitate filtering of a larger amount of data, wherein additionally, an increase in ε can also result in an increase in taps. Figure 4h , embodiments may apply a small 3-tap filter to near horizontal and near vertical modes, and increase the filter length as the distance from the horizontal and vertical directions increases. Although illustrated as depending on both the prediction mode and the block size, the choice of filter or at least the number of taps may alternatively depend on only one of the two and / or on additional parameters.

[0087] In another embodiment for reference sample filtering, intra-image prediction using filtered reference samples can be combined with unfiltered reference samples using position-dependent weighting, as described in conjunction with position-dependent prediction combination. In this case, reference samples for prediction using filtered reference samples can be filtered using reference samples different from non-combined prediction. For example, filtering can be selected from a set of 3-tap filters, 5-tap filters, and 7-tap filters.

[0088] According to an embodiment, a video encoder, such as video encoder 1000, can be configured to encode an image of a video into encoded data through block-based predictive coding, wherein the block-based predictive coding includes intra-image prediction; in the intra-image prediction, multiple extended reference samples of the image are used to encode a prediction block of the image, each of the multiple extended reference samples is separated from the prediction block by at least one closest reference sample directly adjacent to the prediction block among multiple reference samples, wherein the multiple closest reference samples are arranged along a first image direction of the prediction block and along a second image direction of the prediction block; at least a portion of the closest reference samples arranged along the second direction are mapped to extended reference samples arranged along the first direction, so that the mapped reference samples exceed the extension of the prediction block along the first image direction; and; and the prediction is performed using the mapped extended reference samples.

[0089] The video encoder may be further configured to map the portion closest to the reference sample according to a prediction mode used to predict the prediction block. The video encoder may be configured to map the portion closest to the reference sample according to a direction used in the prediction mode used to predict the prediction block.

[0090] The corresponding video decoder may be configured to decode an image encoded as encoded data into a video by block-based predictive decoding, wherein the block-based predictive decoding comprises intra-image prediction; in the intra-image prediction, a plurality of extended reference samples of the image are used to decode a prediction block of the image, each of the plurality of extended reference samples being separated from the prediction block by at least one closest reference sample of a plurality of reference samples that is directly adjacent to the prediction block, wherein the plurality of closest reference samples are arranged along a first image direction of the prediction block and along a second image direction of the prediction block;

[0091] At least a portion of the closest reference samples arranged along the second direction are mapped to extended reference samples arranged along the first direction, so that the mapped reference samples exceed the extension of the prediction block along the first image direction; and the prediction is performed using the mapped extended reference samples.

[0092] The video decoder may be configured to map the portion closest to the reference sample according to a prediction mode used to predict the prediction block. The video decoder may be configured to map the portion closest to the reference sample according to a direction used in the prediction mode used to predict the prediction block.

[0093] In principle, per-image intra prediction using directly neighboring reference samples can be adapted to use extended reference samples. In the literature, the following three predictions have been adapted:

[0094] ·flat

[0095] DC

[0096] ·angle

[0097] In the following, each prediction will be described in detail for the extended reference samples in order to thereby describe embodiments of the present invention.

[0098] Plane prediction is Figure 3 Bilinear interpolation of W×H prediction samples from the boundaries as illustrated in . Since the right and bottom boundaries are not reconstructed, the right boundary sample is set equal to the top-right sample r(W,–1), and the bottom boundary sample is set equal to the bottom-left sample r(–1,H):

[0099] Where x=0..W-1, y=0..H-1

[0100] p ver (x,y)=(H-1-y)·r(x,-1)+(y+1)·r(-1,H)

[0101] p hor (x,y)=(W-1-x)·r(-1,y)+(x+1)·r(W,-1)

[0102] For the extended reference samples, the vertical and horizontal prediction P ver and p hor Depends on the reference region index i:

[0103] p ver，i (x, y)=(H-1-y)·r(x,-1-i)+(y+1)·r(-1-i,H)

[0104] p hor，i (xy)=(W-1-x)·r(-1-i,y)+(x+1)·r(W,-1-i)

[0105] One embodiment for improving the extended reference samples is to use the vertical and horizontal prediction p ver，i and p hor，i The sample closest to the upper right corner r(W, –1) and the sample closest to the lower left corner r(–1, H) are used as follows:

[0106] p ver，i (x, y)=(H-1-y)·r(x,-1-i)+(y+1)·r(-1,H)

[0107] p hor，i(x,y)=(W-1-x)·r(-1-i,y)+(x+1)·r(W,-1)

[0108] Another embodiment defines including corner reference samples in horizontal and vertical prediction and distance-dependent weighting. The farther the reference area is extended, i.e., the larger the parameter i, the more upper left corner samples are available for prediction. For i=0, it is a sample r(-1,-1), i.e., adjacent to the prediction block or at a distance of 0, and for i>0, there are i vertical corner samples r(-1-i, y), where y=-1..-1-i; i horizontal corner samples r(x, -1-i), where x=-1..-1-i; and corner samples r(-1-i, -1-i). One implementation method according to an embodiment defines averaging horizontal and vertical corner samples (both of which include corner samples) in vertical and horizontal prediction, and including them using distance-dependent weighting performed by a() and b(), as follows:

[0109]

[0110] Where a(y)+b(y)=(H-1-y)

[0111]

[0112] where a(x)+b(x)=(W-1-x)

[0113] Another embodiment is a combination of the two previous embodiments using the closest top right sample, the closest bottom left sample and extended corner samples.

[0114] DC prediction calculates the average of the reference sample values ​​DC and sets each prediction sample p(x, y) inside the W×H block equal to the average:

[0115] p(x,y)=DC, where x=0..W-1, y=0..H-1

[0116] For the mean value DC, only N samples to the left and M samples to the right of the prediction block may be used, where N may be set to the height H and M may be set to the width W:

[0117]

[0118] For the extended reference sample, the mean value DC depends on the reference region index i:

[0119]

[0120] In one embodiment, the top left corner sample of the prediction block may also be used for averaging:

[0121]

[0122] Angular prediction calculates prediction samples by deriving them along a specified direction angle. The simplest prediction directions are horizontal, vertical, and the three diagonal directions, which are described in Figures 4a to 4e In Figure 1, these figures show examples of five angular intra-image prediction angles for a 4×2 block of prediction samples. It can be seen that the reference samples along each direction always point to a full (integer) sample position. The modification of extending the reference samples (in green) as described in the literature is straightforward and only extends the direction. Although the block size of the prediction samples is 4×2, the angular intra prediction can be extended to arbitrary block sizes.

[0123] According to an embodiment, a finer prediction angle granularity may be used so that directions may point to non-integer sample positions. These may be generated by interpolation using the closest integer sample positions. Interpolation may be performed using a simple bilinear filter or a higher order filter (e.g., a cubic or Gaussian 4-tap filter).

[0124] For prediction angles between horizontal and vertical (e.g., diagonal upper left angle and all angles between the two), reference samples from the left and top are used to generate predictions. Reference sample projection can be used to facilitate such generation. For all vertical angles between the diagonal upper left and vertical, the top reference sample can be considered as the main reference and the left sample can be considered as the side reference. For all horizontal angles between the horizontal and the diagonal upper left, the left reference sample can be considered as the main reference and the top reference sample can be considered as the side reference. In order to simplify the calculation, that is, in order not to switch between the main reference sample calculation and the side reference sample calculation, the side reference sample is projected along the prediction angle to extend the main reference sample of the line. Each reference sample r(x,-1) to the left of the top main reference or r(-1,y) above the left main reference is projected onto the corresponding side reference sample according to the inverse prediction angle. Depending on the prediction angle, the projection can point to the subsample position.

[0125] Figure 5a to Figure 5c An embodiment incorporating angle prediction is shown that uses a definition of an angle parameter A for the top left diagonal prediction angle and two other vertical angles given with 1 / 32 sample accuracy. The following embodiment uses 1 / 32 sample accuracy and vertical prediction angles between vertical and top left diagonal, where any other variations may be implemented. The range of 33 prediction angles may be described by an angle parameter A in the range of 0 (vertical) to 32 (top left diagonal), such as Figure 5a to Figure 5c A can be referred to as the distance along the second image direction relative to a given first image extension of, for example, 32 samples. Figure 5a In , A is 32 samples, that is, 32 at an angle of 45° are generated based on the image expansion of 32 samples along the image direction perpendicular to A. Figure 5bIn, A=17, and in Figure 5c , A=1.

[0126] Based on the angle parameter A, the vertical offset verOffset(x) in the left reference line entering each primary reference sample r(x,-1) can be calculated as follows:

[0127]

[0128] r(-1-x,-1)=r(-1,-1+verOffset(x)), where x=1…maxOffset-1

[0129]

[0130] Figures 6a to 6c as well as Figures 7a to 7c The vertical offset verOffset(x) and the maximum horizontal offset maxOffset are derived based on parameter A, where Figures 6a to 6c shows the calculation of the vertical offset for a given horizontal displacement x for the upper left horizontal prediction angle, where according to Figure 5a to Figure 5c , A=32,17,1.

[0131] Figures 7a to 7c The calculation of the maximum horizontal offset maxOffset for a given prediction block height H for the top left horizontal prediction angle is shown, where A=32, 17, 1.

[0132] In an embodiment, the sub-sample position is replaced by the nearest integer reference sample by rounding the vertical offset. This facilitates the calculation. However, it is also possible to project interpolated sub-sample side reference samples.

[0133] Figure 8 An embodiment of the diagonal top left angle (A=17) and the closest reference sample is shown. For a height of eight samples (H=8), it can be seen that four (maxOffset-1) left reference samples can be projected in the diagonal direction to extend the main reference sample line, i.e., the closest top reference sample. Depending on the embodiment, the illustrated sizes of images, blocks, slices, etc. are given only as examples and may include any other values.

[0134] Figure 8 An example projection of a closest left reference sample as a side reference next to a closest top reference sample as a primary reference in the case of an upper left horizontal direction with an angle parameter A=17 according to an embodiment is shown.

[0135] For an extended reference sample with reference region index i, the projection may be adapted as follows:

[0136]

[0137] r(-1-ix,-1-i)=r(-1-i,-1+verOffset(x)), where x=1…maxOffset-1

[0138]

[0139] Fig. 9 An embodiment using a diagonal top left angle (A=17) of an extended reference line with i=3 and a height of eight samples (H=8) is shown. Here, two (maxOffset-1) left reference samples can be projected diagonally to extend the main reference sample line, i.e., closest to the top reference sample.

[0140] Fig. 9 An example projection of an extended left reference sample as a side reference next to an extended top reference sample as a main reference is shown in the case of top left diagonal prediction with angle parameter A=17 according to an embodiment.

[0141] When using reference samples starting from the left and top, extended reference samples allow combining the closest reference sample with the extended reference sample. Fig. 9 This embodiment of the direct method allows to utilize extended reference samples along the main direction, i.e. the main reference, where a larger distance of the reference samples may be beneficial in cases where there are occlusions or edges on the closest reference samples. However, for the side references, the correlation between the sample to be predicted and the closest reference sample may be higher than the correlation between the extended reference samples and the sample to be predicted.

[0142] For example, one way to combine the extended main reference sample r(x, -1-i) with the closest side reference sample r(-1, y) can be formulated as follows:

[0143]

[0144] r(-1-ix,-1-i)=r(-1-i,-1+verOffset(x))

[0145] where x=1…maxOffset-1 and verOffset(x)>i

[0146]

[0147] exist Fig.10This combination is described in , based on the same embodiment as described in conjunction with the previous figures. Here, the extended reference samples 10643 arranged in a line are used for the main reference samples (top) in the prediction direction (vertical), and the closest reference samples 1062 arranged in a column can be used for the side reference samples projected to the extended reference sample line (top row) along the prediction direction to extend the main reference sample line.

[0148] Fig.10 An example projection of the closest left reference sample as a side reference next to an extended top reference sample as a main reference is shown in the case of an upper left diagonal prediction with angle parameter A=17. Fig.10 In an embodiment of the invention, the closest reference sample is used as a source for generating the extended reference sample, i.e., the closest reference sample is mapped to the extended reference sample. Alternatively, the extended reference sample may be mapped to the extended reference sample, see Fig. 9 .

[0149] According to an embodiment, a video encoder, such as the video encoder 1000, may be configured to encode an image of a video into encoded data by block-based predictive coding, wherein the block-based predictive coding comprises intra-image prediction; for the intra-image prediction, a plurality of closest reference samples of the image directly adjacent to the prediction block and a plurality of extended reference samples are used to encode the prediction block of the image, each of the plurality of extended reference samples being separated from the prediction block by at least one closest reference sample of the plurality of reference samples. The video encoder is configured to perform boundary filtering in a mode not using extended samples; and not use boundary filtering when using extended samples; or to perform boundary filtering on at least a subset of the plurality of closest reference samples, and not use boundary filtering on the extended samples.

[0150] The corresponding video decoder can be configured to decode an image encoded as encoded data into a video by block-based predictive decoding, wherein the block-based predictive decoding includes intra-image prediction; for the intra-image prediction, a plurality of closest reference samples of the image directly adjacent to the prediction block and a plurality of extended reference samples are used to decode the prediction block of the image, each of the plurality of extended reference samples being separated from the prediction block by at least one reference sample of the plurality of reference samples. The video decoder is configured to perform boundary filtering in a mode not using extended samples; and not use boundary filtering when using extended samples; or perform boundary filtering on at least a subset of the plurality of closest reference samples, and not use boundary filtering on the extended samples.

[0151] Since the extended reference samples are not directly adjacent to the prediction samples, discontinuities on certain block boundaries may not be as severe as when the closest reference samples are used. Therefore, it is beneficial to perform prediction filtering according to each of the following:

[0152] · not performing boundary filtering operations in the case of using extended reference samples, or

[0153] • Change boundary smoothing by using the closest reference sample instead of the extended reference sample.

[0154] By extending the prediction, it is possible to combine the predictions from the closest and the extended reference samples in order to obtain a combined prediction. In the literature, a fixed combination of prediction using the closest reference sample and prediction using the extended reference samples with predetermined weights is described. In this case, both predictions use the same prediction mode for all reference regions, and signaling the mode also signals the combination of predictions. On the one hand, this reduces the signaling overhead to indicate the reference sample region, but it also removes the flexibility of combining two different prediction modes with two different reference sample regions. Possible combinations that allow more flexibility are detailed below.

[0155] According to an embodiment of the present invention, a video encoder, such as the video encoder 1000, may be configured to encode an image of a video into encoded data by block-based predictive coding, wherein the block-based predictive coding includes intra-image prediction; in the intra-image prediction, a plurality of closest reference samples of the image directly adjacent to a prediction block and a plurality of extended reference samples are determined to encode the prediction block of the image, each of the plurality of extended reference samples being separated from the prediction block by at least one closest reference sample of a plurality of reference samples; using the extended reference samples to determine a prediction for the prediction block; filtering the extended reference samples to obtain a plurality of filtered extended reference samples. The video encoder is configured to combine the prediction with the filtered extended reference samples to obtain a combined prediction for the prediction block.

[0156] The video encoder may be configured to combine the prediction with extended reference samples arranged in a main diagonal or a sub-diagonal of samples relative to the prediction block.

[0157] The video encoder may be configured to combine the prediction with the extended reference sample based on the following determination rule:

[0158]

[0159] where p c (x, y) represents the combined prediction for the coordinates x and y in the prediction block, p(x, y) represents the prediction for the coordinates x and y in the prediction block, is the prediction weight, d x is a parameter set to 1 or 2 depending on the size of the prediction block in dimension x, d y is a parameter set to 1 or 2 depending on the size of the prediction block in dimension y, r(x,-1-i) for i>0 is the extended top reference sample at the horizontal position x, r(-1-i,y) for i>0 is the extended left reference sample at the vertical position y, and r(-1-i,-1-i) for i>0 is the extended corner reference sample of the multiple extended reference samples relative to the boundary of the prediction block, and b(x,y) represents a normalization factor.

[0160] The regularization factor may be determined based on the following determination rule:

[0161]

[0162] The video encoder can be configured to filter the extended reference samples to obtain filtered extended reference samples (r(x, -1-i), r(-1-i,y), r(-1-i, -1-i), for i>0) (combined) using one of a 3-tap filter, a 5-tap filter and a 7-tap filter, and use the filtered extended reference samples for the prediction.

[0163] The video encoder may be configured to use a combination of an extended corner reference sample of the prediction block and an extended reference sample arranged in a corner region of a reference sample (r(-1-i, -1-i)).

[0164] The video encoder may be configured to obtain the combined prediction based on the following determination rule:

[0165]

[0166] where p c (x, y) represents the combined prediction for the coordinates x and y in the prediction block, p(x, y) represents the prediction for the coordinates x and y in the prediction block, is the prediction weight, d x is a parameter set to 1 or 2 depending on the size of the prediction block in dimension x, d y is a parameter set to 1 or 2 depending on the size of the prediction block in dimension y, r(x,-1-i) for i>0 is the extended top reference sample at horizontal position x, r(-1-i,y) for i>0 is the extended left reference sample at vertical position y, and is the combined extended corner reference samples relative to the boundary of the prediction block, and b(x, y) represents the normalization factor.

[0167] The video encoder may be configured to obtain the prediction p(x,y) based on intra-image prediction.

[0168] The video encoder may be configured to use only planar prediction as intra-image prediction.

[0169] The video encoder may be configured to determine, for each encoded video block, a set of parameters identifying the combination of the prediction and the filtered extended reference samples. The video encoder may be configured to determine the set of parameters identifying the combination of the prediction and the filtered extended reference samples using a lookup table containing a set of different block sizes for the prediction block.

[0170] The corresponding video decoder can be configured to decode an image encoded as encoded data into a video through block-based predictive decoding, wherein the block-based predictive decoding includes intra-image prediction; in the intra-image prediction, multiple closest reference samples of the image directly adjacent to the prediction block and multiple extended reference samples are determined to decode the prediction block of the image, each of the multiple extended reference samples is separated from the prediction block by at least one closest reference sample of multiple reference samples; the extended reference samples are used to determine a prediction for the prediction block, and the extended reference samples are filtered to obtain multiple filtered extended reference samples; and the prediction is combined with the filtered extended reference samples to obtain a combined prediction for the prediction block.

[0171] The video decoder may be configured to combine the prediction with extended reference samples arranged in a main diagonal or a sub-diagonal of samples relative to the prediction block.

[0172] The video decoder may be configured to combine the prediction with the extended reference sample based on the following determination rule:

[0173]

[0174] where p c (x, y) represents the combined prediction for the coordinates x and y in the prediction block, p(x, y) represents the prediction for the coordinates x and y in the prediction block, is the prediction weight, d x is a parameter set to 1 or 2 depending on the size of the prediction block in dimension x, d yis a parameter set to 1 or 2 depending on the size of the prediction block in dimension y, r(x,-1-i) for i>0 is the extended top reference sample at the horizontal position x, r(-1-i,y) for i>0 is the extended left reference sample at the vertical position y, and r(-1-i,-1-i) for i>0 is the extended corner reference sample of the multiple extended reference samples relative to the boundary of the prediction block, and b(x,y) represents a normalization factor.

[0175] The regularization factor may be determined based on the following determination rule:

[0176]

[0177] The video decoder can be configured to filter the extended reference samples to obtain filtered extended reference samples (r(x, -1-i), r(-1-i, y), r(-1-i, -1-i), for i>0) (combined) using one of a 3-tap filter, a 5-tap filter and a 7-tap filter, and use the filtered extended reference samples for the prediction.

[0178] The video decoder may be configured to use a combination of extended corner reference samples of the prediction block and extended reference samples, which are arranged in a corner region of the reference sample (r(-1-i, -1-i)).

[0179] The video decoder may be configured to obtain the combined prediction based on the following determination rule:

[0180]

[0181] where p c (x, y) represents the combined prediction for the coordinates x and y in the prediction block, p(x, y) represents the prediction for the coordinates x and y in the prediction block, is the prediction weight, d x is a parameter set to 1 or 2 depending on the size of the prediction block in dimension x, d y is a parameter set to 1 or 2 depending on the size of the prediction block in dimension y, r(x,-1-i) for i>0 is the extended top reference sample at horizontal position x, r(-1-i,y) for i>0 is the extended left reference sample at vertical position y, and is the combined extended corner reference samples relative to the boundary of the prediction block, and b(x, y) represents the normalization factor.

[0182] The video decoder may be configured to obtain the prediction p(x,y) based on intra-image prediction.The video decoder may use, for example, only planar prediction as intra-image prediction.

[0183] The video decoder may be configured to determine, for each decoded video block, a set of parameters identifying the combination of the prediction and the filtered extended reference sample.

[0184] The video decoder may be configured to determine the set of parameters identifying the combination of the prediction and the filtered extended reference sample using a lookup table containing a set of different block sizes for the prediction block.

[0185] When filtering reference samples, it is possible to combine predictions using filtered samples with unfiltered reference samples based on the positions of the individual samples in order to obtain a position-dependent prediction combination. According to an example embodiment, let p(x,y) be the prediction using the filtered reference samples, and let r(x,y) be the unfiltered reference samples. The combined prediction p c (x,y) can be calculated as follows:

[0186]

[0187] coefficient is the stored prediction weights scaled by the position dependency, where for blocks with width less than or equal to 16, d x = 1, and for blocks with width greater than 16, d x = 2, for blocks with height less than or equal to 16, d y = 1, and for blocks with height greater than 16, d y = 2. The normalization factor b(x,y) can be derived as follows:

[0188]

[0189] As already described in conjunction with reference sample filtering, the reference sample filtering for this combination may be different from the filtering for conventional prediction. For example, one of three or fewer or more predefined low-pass filters may be used to smooth the boundary samples. The three predefined low-pass filters may include a 3-tap filter, a 5-tap filter, and a 7-tap filter. In order to adapt the smoothing to the content, the selection of the smoothing filter may be based on the block size and the intra prediction mode. By defining h k is the impulse response of filter k, and additionally the stored weighting parameters a, the filtered reference r' is calculated from the unfiltered reference r as follows, where the "*" indicates convolution:

[0190] r=ar+(1-a)(h k *r)

[0191] Again, different features are counted for different block sizes, and a fixed set of prediction parameters ( a and filter index k) can be defined per block size.

[0192] This known combination for the closest reference sample can be improved by an embodiment by using extended reference samples of the filtered as well as unfiltered reference samples. In the following, a prediction combination according to an embodiment and with extended reference samples is shown, where the closest corner sample r(-1,-1) is replaced by the extended corner sample r(-1-i,-1-i) as follows.

[0193]

[0194] Another embodiment of extending the reference sample refers to replacing the closest corner sample r(-1,-1) with the closed extended vertical corner sample r(-1-i,-1) and the horizontal corner sample r(-1,-1-i) as follows:

[0195]

[0196] Another embodiment of the extended reference sample is defined as follows: the nearest corner sample r(-1, -1) is replaced by the average value of the horizontal corner sample and the vertical corner sample (both including the corner sample).

[0197]

[0198] In one embodiment, the prediction p(x,y) to be combined is a planar prediction. In another embodiment, the combination may also be applied to other intra-image predictions, such as DC prediction or angular prediction.

[0199] Alternatively or additionally, different extended prediction modes may be combined in order to combine predictions with samples.

[0200] According to an embodiment of the present invention, a video encoder, such as video encoder 1000, may be configured to encode an image of a video into encoded data by block-based predictive coding, wherein the block-based predictive coding comprises intra-image prediction; for the intra-image prediction, a plurality of closest reference samples and / or a plurality of extended reference samples of the image directly adjacent to the prediction block are used to encode the prediction block of the image, each of the plurality of extended reference samples being separated from the prediction block by at least one closest reference sample of a plurality of reference samples; a first prediction mode in a prediction mode set is used to determine a first prediction for the prediction block, the first prediction mode set comprising a prediction mode using the plurality of closest reference samples in the absence of the extended reference sample; and a second prediction mode in a second prediction mode set is used to determine a second prediction for the prediction block, the second prediction mode set comprising a subset of the prediction modes in the first set, the subset being associated with the plurality of extended reference samples. The video encoder may be configured to weight the prediction mode in a weighted manner (w0; w i ) combines the first prediction (p0(x,y)) with the second prediction (p i (x, y)) in order to obtain a combined prediction (p(x, y)) as a prediction for the prediction block in the coded data.

[0201] The video encoder may be configured to use the first prediction and the second prediction according to a predefined combination, the predefined combination being part of possible combinations of the enabled first prediction mode and the enabled second prediction mode.

[0202] The video encoder may be configured to signal the first prediction mode or the second prediction mode without signaling the other prediction mode. For example, the first mode may be derived from parameter i based on additional implicit information, such as a specific prediction mode that can only be used in conjunction with a specific index i or index m.

[0203] As an example of such implicit information, the video encoder may be configured to exclusively use planar prediction mode as one of the first prediction mode and the second prediction mode.

[0204] The video encoder may be configured to adapt a first weight applied to a first prediction in the combined prediction and a second weight applied to a second prediction in the combined prediction based on a block size of the prediction block; and / or to adapt the first weight based on the first prediction mode, or to adapt the second weight based on the second prediction mode.

[0205] The video encoder may be configured to adapt a first weight applied to a first prediction in the combined prediction and a second weight applied to a second prediction in the combined prediction based on a position and / or a distance in the prediction block.

[0206] The video encoder may be configured to adapt the first weight and the second weight based on the following determination rule:

[0207] p(x,y)=w0(x,y)p0(x,y)+w i (x,y)p i (x,y)

[0208] Where w0(x,y) is the first weight based on the position x,y in the prediction block, w i is a second weight based on the position x, y in the prediction block, p0(x, y) is the first prediction for the position x, y, and p i (x, y) is the second prediction for position x, y, and i indicates the extended reference sample to be used for the second prediction.

[0209] The corresponding video decoder may be configured to decode an image encoded as encoded data into a video by block-based predictive decoding, wherein the block-based predictive decoding comprises intra-image prediction; for the intra-image prediction, a plurality of closest reference samples and / or a plurality of extended reference samples of the image directly adjacent to the prediction block are used to decode the prediction block of the image, each of the plurality of extended reference samples being separated from the prediction block by at least one closest reference sample of the plurality of reference samples;

[0210] Determine a first prediction for the prediction block using a first prediction mode in a prediction mode set, the first prediction mode set including a prediction mode using the plurality of closest reference samples in the absence of the extended reference samples; and determine a second prediction for the prediction block using a second prediction mode in a second prediction mode set, the second prediction mode set including a subset of the prediction modes in the first set, the subset being associated with the plurality of extended reference samples. The video decoder may be configured to weight the prediction mode in a weighted manner (w0; w i ) combines the first prediction (p0(x,y)) with the second prediction (p i (x, y)) in order to obtain a combined prediction (p(x, y)) as a prediction for the prediction block in the coded data.

[0211] The video decoder may be configured to use the first prediction and the second prediction according to a predefined combination, the predefined combination being part of the possible combinations of the enabled first prediction mode and the enabled second prediction mode. This allows to obtain a low load of signaling.

[0212] The video decoder may be configured to receive a signal indicating the second prediction mode without receiving a signal indicating the first prediction mode, and to derive the first prediction mode from the second prediction mode or parameter information (i).

[0213] The video decoder may be configured to exclusively use planar prediction mode as one of the first prediction mode and the second prediction mode.

[0214] The video decoder can be configured to adapt a first weight applied to a first prediction in the combined prediction and a second weight applied to a second prediction in the combined prediction based on a block size of the prediction block; and / or adapt the first weight based on the first prediction mode, or adapt the second weight based on the second prediction mode.

[0215] The video decoder may be configured to adapt a first weight applied to a first prediction in the combined prediction and a second weight applied to a second prediction in the combined prediction based on a position and / or a distance in the prediction block.

[0216] The video decoder may be configured to adapt the first weight and the second weight based on the following determination rule:

[0217] p(x,y)=w0(x,y)p0(x,y)+w i (x,y)p i (x,y)

[0218] Where w0(x,y) is the first weight based on the position x,y in the prediction block, w i is a second weight based on the position x, y in the prediction block, p0(x, y) is the first prediction for the position x, y, and p i (x, y) is a second prediction for position x, y, and i indicates an extended reference sample to be used for the second prediction. For different positions x, y in the prediction block, the extended reference sample i may include different distances. Alternatively or additionally, different sets of predefined extended reference samples indicated by region index i and / or by mode m may include different distances from the prediction block.

[0219] When using different reference sample regions, it is possible to combine the prediction using the closest reference sample pattern p0(x,y) with the prediction p using the extended reference sample i as followsi (x,y):

[0220] p(x,y)=w0 p0(x,y)+w i p i (x,y)

[0221] One known way to combine predictions is to use fixed weights and use the same prediction mode for both predictions. This eliminates the need to signal the second prediction mode.

[0222] An embodiment that relaxes this strict restriction defines that only certain combinations of modes are allowed. This still requires signaling the second mode, but it limits the number of modes to be signaled compared to the first mode. More details on prediction mode signaling are given in conjunction with mode and reference signaling. A promising combination according to an embodiment is to use only planes as the second mode, making the additional signaling of intra-frame modes obsolete. For example, p with arbitrary intra-frame modes can be used i (x,y) as the first part of the weighted sum and the second part is generated by using the closest reference sample and the planar mode. Another possibility is to swap the order, ie use the planar mode for the extended reference sample.

[0223] In order to adapt the weighting to the prediction size and mode, the weights w can be specified separately per block size and per intra prediction mode. Furthermore, the weights can be signaled at the sequence, image or slice level so that the combination can be adapted to the video content. Another embodiment is to weight different reference sample predictions differently for different areas inside the prediction block. This can generate position-dependent weights w(x,y) as follows:

[0224] p(x,y)=w0(x,y)p0(x,y)+w i (x,y)p i (x,y)

[0225] In the case of using one prediction and extended reference samples, the prior art is to signal the prediction mode and the reference region index i used, i.e., perform mode and reference region signaling. In the following, an embodiment of prediction mode encoding in the case of using one and / or more predictions is presented, followed by an embodiment of extended reference region index encoding.

[0226] According to an embodiment, a video encoder, such as video encoder 1000, can be configured to encode an image of a video into encoded data by block-based predictive coding, wherein the block-based predictive coding includes intra-image prediction; for the intra-image prediction, multiple closest reference samples and / or multiple extended reference samples of the image directly adjacent to the prediction block are used to encode the prediction block of the image, and each of the multiple extended reference samples is separated from the prediction block by at least one closest reference sample of a plurality of reference samples; a prediction mode is used, and the prediction mode is one of a first prediction mode set for predicting the prediction block using the closest reference sample, for example, only using the closest sample or at least in the absence of an extended reference sample; or one of a second prediction mode set for predicting the prediction block using the extended reference sample; wherein the second prediction mode set is a subset of the first prediction mode set, and the second subset can be determined by the encoder; wherein the subset can also include overlap of two groups. The video encoder may be configured to signal mode information (m) indicating a prediction mode used to predict the prediction block; and if the prediction mode is included in the second prediction mode set, then signal parameter information (i) indicating a subset of extended reference samples used for the prediction mode; and when the prediction mode used is not included in the second prediction mode set, skip signaling the parameter information, thereby allowing a conclusion to be drawn that the parameter i has a predefined value such as 0.

[0227] The video encoder may be configured to skip signaling the parameter information when the mode information indicates a DC mode or a planar mode.

[0228] The corresponding video decoder may be configured to decode an image encoded as encoded data into a video by block-based predictive decoding, wherein the block-based predictive decoding includes intra-image prediction; for the intra-image prediction, using a plurality of closest reference samples and / or a plurality of extended reference samples of the image directly adjacent to the prediction block to decode the prediction block of the image, each of the plurality of extended reference samples being separated from the prediction block by at least one closest reference sample of a plurality of reference samples; using a prediction mode, the prediction mode being one of a first set of prediction modes for predicting the prediction block using the closest reference sample, for example using only the closest sample or at least in the absence of an extended reference sample; or one of a second set of prediction modes for predicting the prediction block using the extended reference sample. The second set of prediction modes is a subset of the first set of prediction modes and may be determined and / or signaled by the encoder. Being a subset may include an overlap of two groups. The video encoder may be configured to receive mode information (m) indicating the prediction mode used to predict the prediction block; and subsequently receive parameter information (i) indicating a subset of extended reference samples used for the prediction mode, thereby indicating that the prediction mode is included in the second prediction mode set; and when the parameter information is not received, determine that the prediction mode used is not included in the second prediction mode set, and determine the use of the closest reference sample for the prediction.

[0229] The video decoder may be configured to determine the mode information as indicating use of the DC mode or the planar mode when the parameter information is not received.

[0230] When the set of allowed intra prediction modes for extended reference samples is restricted, i.e. is a subset of the allowed intra prediction modes for the closest reference samples, there are two different ways to signal the mode m and the index i, e.g. when the subset is called restricted prediction modes for extended reference samples:

[0231] 1. Signal mode m before index i, so that the index signaling may depend on mode m as follows:

[0232] a. If mode m is not in the set of allowed modes for extending the reference sample, signaling of index i is skipped and prediction mode m is applied to the closest reference sample (i=0).

[0233] b. Otherwise, i is signaled and prediction mode m is applied to the reference sample indicated by i.

[0234] 2. Signal index i before mode m, so that mode signaling may depend on index i as follows:

[0235] a. If i indicates the use of extended reference samples (i>0), the set of allowed modes m to be signaled is equal to the restricted set.

[0236] b. Otherwise (i=0), the set m of allowed modes to be signaled is equal to the unrestricted set of modes.

[0237] For example, if most probable mode (MPM) encoding is used to signal mode m using an index pointing to the MPM list, modes that are not in the set of allowed modes are not included in the MPM list.

[0238] That is, according to a second option that can be implemented alternatively or additionally, a video encoder according to an embodiment, such as the video encoder 1000, can be configured to encode an image of a video into encoded data by block-based predictive encoding, wherein the block-based predictive encoding includes intra-image prediction; for the intra-image prediction, using a plurality of reference samples including a closest reference sample of the image directly adjacent to the prediction block and a plurality of extended reference samples to encode the prediction block of the image, each of the plurality of extended reference samples being separated from the prediction block by at least one closest reference sample of the plurality of reference samples; using a prediction mode, the prediction mode being for using the closest reference sample to encode the prediction block; one of a first prediction mode set for predicting the prediction block using the extended reference sample; or one of a second prediction mode set for predicting the prediction block using the extended reference sample; wherein the second prediction mode set is a subset of the first prediction mode set; signaling parameter information (i) indicating a subset of a plurality of reference samples for the prediction mode, the subset of the plurality of reference samples comprising only the closest reference sample or the extended reference sample; and subsequently signaling mode information (m) indicating the prediction mode for predicting the prediction block, wherein the mode information indicates a prediction mode from a subset of modes, the subset being restricted to a set of allowed prediction modes according to the parameter information (i).

[0239] For both options, the video decoder is adapted such that in addition to the closest reference samples, extended reference samples in modes comprised in the second set of prediction modes are used.

[0240] Furthermore, the video encoder may be adapted such that a first prediction mode set describes prediction modes allowed for use with the closest reference sample, wherein the second prediction mode set describes prediction modes in the first prediction mode set that are also allowed for use with the extended reference sample.

[0241] The value range of the parameter information, i.e., the value domain that can be represented by the region index i, can cover the use of only the closest reference value, as well as the use of different subsets of extended reference values. Fig.11 As described, i may represent the use of only the closest reference value (i=0), a specific set of extended reference values, or a combination of sets of extended reference values ​​(eg, lines and / or columns or distances).

[0242] According to an embodiment, different parts of the extended reference samples comprise different distances from the prediction block.

[0243] The video encoder may be configured to set the parameter information to one of a predefined number of values ​​indicating the number and distance of reference samples used for the prediction mode.

[0244] The video encoder may be configured to determine the first prediction mode set and / or the second prediction mode set based on a most probable mode encoding.

[0245] The corresponding decoder of the second option is configured to decode an image encoded as encoded data into a video by block-based predictive decoding, wherein the block-based predictive decoding includes intra-image prediction; for the intra-image prediction, use a plurality of reference samples including a closest reference sample of the image directly adjacent to the prediction block and a plurality of extended reference samples to decode the prediction block of the image, each of the plurality of extended reference samples being separated from the prediction block by at least one closest reference sample of the plurality of reference samples; use a prediction mode, the prediction mode being a first prediction mode for predicting the prediction block using the closest reference sample one of a set of prediction modes; or one of a second set of prediction modes for predicting the prediction block using the extended reference sample; wherein the second set of prediction modes is a subset of the first set of prediction modes; receiving parameter information (i) indicating a subset of the plurality of reference samples for the prediction mode, the subset of the plurality of reference samples including only the closest reference sample or the extended reference sample; and subsequently receiving mode information (m) indicating the prediction mode for predicting the prediction block, wherein the mode information indicates a prediction mode from a subset of modes, the subset being restricted to a set of allowed prediction modes according to the parameter information (i).

[0246] The decoder according to the first option and / or the second option may be adapted such that in addition to the closest reference samples, extended reference samples in modes contained in the second set of prediction modes are used, for example by combining prediction with samples and / or combining prediction.

[0247] The first prediction mode set may describe prediction modes that are allowed for use with the closest reference sample, wherein the second prediction mode set may describe prediction modes in the first prediction mode set that are also allowed for use with the extended reference sample.

[0248] As described for the encoder, the value range of the parameter information covers the use of only the closest reference value as well as the use of different subsets of extended reference values.

[0249] Different portions of the extended reference samples may comprise different distances from the prediction block.

[0250] The video decoder may be configured to set the parameter information to one of a predefined number of values ​​indicating the number and distance of reference samples used for the prediction mode.

[0251] The video decoder may be configured to determine the first set of prediction modes and / or the second set of prediction modes based on a Most Probable Mode (MPM) encoding, i.e., the MPM encoding may generate respective lists, wherein the lists may be adapted to contain only such modes that allow the use of respective samples.

[0252] Alternatively or additionally, embodiments relating to combined prediction from closest and extended reference samples may be implemented.

[0253] According to an embodiment, a video encoder, such as video encoder 1000, can be configured to encode an image of a video into encoded data by block-based predictive coding, wherein the block-based predictive coding includes intra-image prediction; for the intra-image prediction, use multiple closest reference samples and / or multiple extended reference samples of the image that are directly adjacent to the prediction block to encode the prediction block of the image, each of the multiple extended reference samples being separated from the prediction block by at least one closest reference sample of a plurality of reference samples; determine a first prediction for the prediction block using a first prediction mode in a prediction mode set, the first prediction mode set including a prediction mode using the multiple closest reference samples in the absence of the extended reference samples; determine a second prediction for the prediction block using a second prediction mode in a second prediction mode set, the second prediction mode set including a subset of the prediction modes in the first set, the subset being associated with the multiple extended reference samples, and combining the first prediction with the second prediction so as to obtain a combined prediction as a prediction for the prediction block in the encoded data.

[0254] The prediction block may be a first prediction block, wherein the video encoder may be configured to predict the second prediction block using a plurality of closest reference samples associated with the second prediction block in the absence of a plurality of extended reference samples associated with the second prediction block of the video. The video encoder may be configured to signal combination information such as a bipred flag, the combination information indicating that the prediction in the encoded data is based on a combination of predictions or is based on prediction using the plurality of extended reference samples in the absence of the plurality of closest reference samples.

[0255] The video encoder may be configured to use the first prediction mode as a predefined prediction mode.

[0256] The video encoder may be configured to select the first prediction mode as the same mode as the second prediction mode and use the closest reference sample if the extended reference sample does not exist; or use the first prediction mode as a preset prediction mode, such as a planar prediction mode.

[0257] The corresponding video decoder may be configured to decode an image encoded as coded data into a video by block-based predictive decoding, wherein the block-based predictive decoding includes intra-image prediction; for the intra-image prediction, use a plurality of closest reference samples and / or a plurality of extended reference samples of the image directly adjacent to the prediction block to decode the prediction block of the image, each of the plurality of extended reference samples being separated from the prediction block by at least one closest reference sample of a plurality of reference samples; determine a first prediction for the prediction block using a first prediction mode in a prediction mode set, the first prediction mode set including prediction modes using the plurality of closest reference samples in the absence of the extended reference samples; and determine a second prediction for the prediction block using a second prediction mode in a second prediction mode set, the second prediction mode set including a subset of the prediction modes in the first set, the subset being associated with the plurality of extended reference samples. The video decoder may be configured to combine the first prediction with the second prediction so as to obtain a combined prediction as a prediction for the prediction block in the coded data.

[0258] The prediction block may be a first prediction block, wherein the video decoder may be configured to predict the second prediction block using a plurality of closest reference samples associated with the second prediction block in the absence of a plurality of extended reference samples associated with the second prediction block of the video. The video decoder may be further configured to receive information indicating whether the prediction in the encoded data is based on a combination of predictions or based on a combination of predictions using the plurality of extended reference samples in the absence of the plurality of closest reference samples, and decode the encoded data accordingly.

[0259] The video decoder may be configured to use the first prediction mode as a predefined prediction mode.

[0260] The video decoder may be configured to select the first prediction mode as the same mode as the second prediction mode and use the closest reference sample if the extended reference sample does not exist; or use the first prediction mode as a preset prediction mode, such as a planar mode.

[0261] If the reference region index i indicates the use of an extended reference sample (i>0), various information such as binary information, or a flag referred to hereinafter as a bidirectional prediction flag (bipred flag), may be used to signal whether the prediction using the extended reference sample is combined with the prediction using the closest reference sample.

[0262] If the bipred flag indicates a combination of predictions from the closest (i=0) and extended reference samples (i>0), the mode m for extending the reference samples is signaled before the reference region index i, or after the reference region index i as described herein. i The mode m0 used for the closest reference sample prediction is fixed, for example, always set to a specific mode such as the planar mode, or set to the same mode as the mode used for the extended reference sample.

[0263] As will be described below, a video encoder according to an embodiment may be configured to encode an image of a video into encoded data by block-based predictive coding, wherein the block-based predictive coding includes intra-image prediction; in the intra-image prediction, a plurality of extended reference samples of the image are used to encode a prediction block of the image, each of the plurality of extended reference samples being separated from the prediction block by at least one closest reference sample of a plurality of reference samples that is directly adjacent to the prediction block; and the plurality of extended reference samples are used according to a predefined set of the plurality of extended reference samples, that is, a region index list may be generated and / or used, the list indicating a specific subset of reference samples, such as only the closest reference samples, at least one distance of the extended reference samples, and / or a combination of different distances of the extended reference samples.

[0264] The video encoder may be configured to determine the predefined set of multiple extended reference samples such that multiple extended reference samples in the set differ from each other by multiple lines and / or rows or combinations of samples of the image to be used as reference samples.

[0265] The video encoder may be configured to determine the predefined set of a plurality of extended reference samples based on a block size of the prediction block and / or a prediction mode used to predict the prediction block.

[0266] The video encoder may be configured to determine the set of multiple extended reference samples for a block size of the prediction block being at least a predefined threshold, and to skip signaling the set of multiple extended reference samples when the block size is below the predefined threshold.

[0267] The predefined threshold may be a predefined number of samples along the width or height of the prediction block and / or a predefined aspect ratio of the prediction block along the width and the height.

[0268] The predefined number of samples may be any number, but is preferably 8. Alternatively or additionally, the aspect ratio may be greater than 1 / 4 and less than 4, possibly a number of 8 samples based on a quotient defining the aspect ratio.

[0269] The video encoder may be configured to predict the prediction block as a first prediction block using the plurality of extended reference samples, and predict a second prediction block (which may be part of the same or a different picture) without using the extended reference samples, wherein the video encoder is configured to signal a predefined set of the plurality of extended reference samples associated with the first prediction block, and not signal a predefined set of extended reference samples associated with the second prediction block. For example, the predefined set may be indicated by a reference region index i.

[0270] The video encoder may be configured to signal, for each prediction block, information indicating use of a specific plurality of extended reference samples in a set of a plurality of extended reference samples and only a closest reference sample before the information indicating the intra-image prediction mode.

[0271] The video encoder may be configured to signal the information indicative of the intra-image prediction so as to thereby indicate whether the prediction mode is based on an indicated specific plurality of extended reference samples from the set of multiple extended reference samples, or based on an indicated use of only closest reference samples.

[0272] The corresponding video decoder can be configured to decode an image encoded as encoded data into a video through block-based predictive decoding, wherein the block-based predictive decoding includes intra-image prediction; in the intra-image prediction, multiple extended reference samples of the image are used to decode a prediction block of the image, each of the multiple extended reference samples is separated from the prediction block by at least one closest reference sample of multiple reference samples that is directly adjacent to the prediction block; and the multiple extended reference samples are used according to a predefined set of the multiple extended reference samples.

[0273] The video decoder may be configured to determine the predefined set of multiple extended reference samples such that multiple extended reference samples in the set differ from each other by multiple lines and / or rows or combinations of samples of the image to be used as reference samples.

[0274] The video decoder may be configured to determine the predefined set of a plurality of extended reference samples based on a block size of the prediction block and / or a prediction mode used to predict the prediction block.

[0275] The video decoder may be configured to determine the set of multiple extended reference samples for a block size of the prediction block being at least a predefined threshold, and to skip using the set of multiple extended reference samples when the block size is below the predefined threshold.

[0276] For example, if the top left sample of the current block at position (x0, y0) is located in the first row inside a coding tree block (CTB), the signaling of the set (which may use parameter information i or similar syntax) may be skipped. A CTB may be viewed as a basic processing unit into which an image is partitioned, and the basic processing unit is the root of another block sub-partition.

[0277] This can be done by checking if the vertical y coordinate y0 is not a multiple of the CTB size. For example, when the CTB size is 64 luma samples, in the first CTB row, the upper blocks inside the CTB have y0=0, in the second CTB row, they have y0=64, etc. Therefore, for all blocks inside the CTB at the upper boundary of the CTB (which can be checked, for example, by a modulo operation as follows: y0%CtbSizeY==0), the signaling of parameter i or similar syntax can be skipped.

[0278] Thus, the predefined threshold may be a predefined number of samples along the width or height of the prediction block and / or a predefined aspect ratio of the prediction block along the width and the height.

[0279] Thus, the predefined number of samples may be 8, and / or the aspect ratio may need to be greater than 1 / 4 and less than 4.

[0280] The video decoder may be configured to predict the prediction block as a first prediction block using the plurality of extended reference samples, and to predict a second prediction block without using the extended reference samples, wherein the video decoder is configured to receive information indicating a predefined set of multiple extended reference samples associated with the first prediction block, for example using a region index i, and to determine a predefined set of extended reference samples associated with the second prediction block in the absence of a corresponding signal.

[0281] The video decoder may be configured to receive, for each prediction block, information indicating a specific plurality of extended reference samples in a set of a plurality of extended reference samples and using only one of the closest reference samples before the information indicating the intra-image prediction mode.

[0282] The video decoder may be configured to receive the information indicating the intra-image prediction to thereby indicate whether the prediction mode is according to an indicated specific plurality of extended reference samples in the set of multiple extended reference samples, or according to an indicated use of only closest reference samples.

[0283] The reference area index may be changed per block and, therefore, the index may be transmitted in the bitstream for each prediction block to which the index applies. An embodiment relates to extended reference sample area signaling to allow the decoder to use the correct sample. The area may correspond to a parameter i indicating an extended reference sample or a closest reference sample. In order to compromise between signaling and the use of other distant extended reference samples, a predetermined set of extended reference sample lines may be used. For example, only two additional extended reference lines may be used, which are lines with indices 1 and 3. The set according to an embodiment may be I={0,1,3}, where |I|=3. This allows the reference sample area to be extended to up to three additional lines, but only two lines need to be signaled. The set may also be extended to four lines, for example I={0,1,3,4}, and only the first MaxNumRefAreaIdx element is used, while MaxNumRefAreaIdx may be fixed or signaled at the sequence, image or slice level. Such a set may be used. Fig.11 The entropy encoding shown in the table illustrated in FIG. 1 and the truncated unary code, which shows an example truncated unary code for a specific set of reference areas with MaxNumRefAreaIdx equal to 3 and 4, are signaled in the bitstream to an index n in set I.

[0284] In order to take into account different spatial characteristics of different block sizes, the set of reference sample lines may also depend on the prediction block size and / or the intra prediction mode. In another embodiment, the set contains only one additional line with index 2 for small blocks and two lines with indexes 1 and 3 for larger blocks. An embodiment for intra prediction mode dependent set selection would have different sets of reference sample lines for prediction directions between horizontal and vertical directions.

[0285] The signaling of the reference region index may also be limited to larger block sizes by selecting an empty set for smaller block sizes that do not need signaling. This may essentially disable the use of extended reference samples for smaller block sizes. In one embodiment, the use of extended reference samples may be limited to blocks with both width W and height H greater than or equal to 8. In addition to this, blocks with one side less than or equal to one quarter of the other side may also be excluded from using extended reference samples, e.g. 32×8 blocks, or based on the mentioned symmetry, 8×32 blocks. Fig.12a and Figure 12b This embodiment is illustrated in which some block sizes have been excluded for intra-image prediction ( Fig.12a ), generally as indicated by reference numeral 1102 and the corresponding shaded blocks. In this embodiment, it is assumed that different block size combinations are allowed for intra-image prediction slices and inter-image prediction slices. Sample 1104 and the corresponding shaded blocks are not allowed for extended reference samples, i.e., i>0. Fig.12a and Figure 12b Example restrictions of extended reference samples for intra-image prediction slices and inter-image prediction slices are shown.As can be seen, the block allowance or block restriction depending on the block size, in particular on the aspect ratio of the block, may be symmetric with respect to the quotients W / H and H / W.

[0286] In the presence of other intra prediction modes that do not use extended reference samples, the reference region index is signaled only when the prediction mode using extended reference samples is signaled. Another approach may be to signal the reference region before signaling all other intra-frame mode information. In the case where the reference region index i indicates an extended reference sample (i>0), the signaling of the mode information that does not use extended reference samples (e.g., template matched or trained predictor) may be skipped. This may also skip signaling information of a specific transform that is not applied to the prediction residual for prediction using extended reference samples.

[0287] In the following, reference is made to embodiments mentioning considerations for parallel encoding.

[0288] According to an embodiment, a video encoder may be configured to encode an image of a video into encoded data by performing block-based predictive encoding on a plurality of prediction blocks, wherein the block-based predictive encoding comprises intra-image prediction; for the intra-image prediction, a plurality of extended reference samples of the image are used to encode a prediction block in the plurality of prediction blocks, each of the plurality of extended reference samples being separated from the prediction block by at least one closest reference sample in a plurality of reference samples that is directly adjacent to the prediction block; wherein the video encoder may be configured to determine the extended reference sample so as to at least partially become a part of a neighboring prediction block in the plurality of prediction blocks, and determine that the neighboring prediction block has not been predicted; and signal information indicating an extended prediction sample that is associated with the prediction block and is arranged in the neighboring prediction block as an unavailable sample.

[0289] The video encoder may be configured to encode the image by encoding the lines of the block in parallel according to a wavefront method, and predicting the prediction block based on an angular prediction, wherein the video encoder is configured to determine that the extended reference sample is used to predict the prediction block so as to arrange it in the predicted block of the image. According to the wavefront method, the encoding or decoding of the second line may track the decoding of the first line separated by, for example, one block. For example, in a vertical angle mode of 45°, encoding or decoding may be performed to the right and above at most one block starting from the second line.

[0290] The video encoder may be configured to signal extended prediction samples, which are associated with the prediction block and are variably arranged in adjacent prediction blocks as unavailable samples or available samples at a sequence level (i.e., a sequence of images), a picture level, or a slice level, the slice being a part of a picture.

[0291] The video encoder may be configured to signal information indicating extended prediction samples, which are associated with the prediction block and arranged in the neighboring prediction blocks as unavailable samples, and information indicating parallel encoding of the image.

[0292] The corresponding decoder can be configured to decode an image encoded as encoded data into a video through block-based predictive decoding, wherein multiple prediction blocks are decoded for each image, wherein the block-based predictive decoding includes intra-image prediction; for the intra-image prediction, multiple extended reference samples of the image are used to decode a prediction block in the multiple prediction blocks, each of the multiple extended reference samples is separated from the prediction block by at least one closest reference sample in multiple reference samples that is directly adjacent to the prediction block; wherein the video decoder is configured to determine the extended reference sample so as to at least partially become a part of a neighboring prediction block in the multiple prediction blocks, and determine that the neighboring prediction block has not been predicted; and receive information indicating an extended prediction sample associated with the prediction block and arranged in the neighboring prediction block as an unavailable sample.

[0293] The video decoder may be configured to decode the image by decoding lines of the block in parallel according to a wavefront method, and predict the prediction block based on angular prediction, wherein the video decoder is configured to determine that the extended reference sample is to be used to predict the prediction block and is thereby arranged in a predicted block of the image.

[0294] The video decoder may be configured to receive information indicating the extended prediction samples, which are associated with the prediction block and are variably arranged in the neighboring prediction blocks as unavailable samples or available samples at a sequence level, a picture level, or a slice level.

[0295] The video decoder may be configured to receive the information indicating the extended prediction sample, which is associated with the prediction block and arranged in the neighboring prediction block as an unavailable sample, and the information indicating parallel decoding of the image.

[0296] In angular intra-image prediction, reference samples are copied into the current prediction region along a specified direction. In the case where this direction points to the upper right, the required reference sample region is also shifted to the right, making the distance from the prediction region boundary larger. Fig.13 An embodiment of vertical angle prediction using a W×H prediction block at a 45 degree angle is shown. It can be seen that the closest reference sample area (blue) extends H samples to the upper right of the current W×H block. By extending the prediction to a more distant reference sample area, the extended reference sample (green) extends H+1, H+2, ... samples to the upper right of the current W×H block.

[0297] When, for example, a square coding tree unit (CTU) is used as a basic processing unit, the maximum intra prediction block size may be equal to the maximum block size, ie, the CTU block size N×N. Fig.14a as well as Fig.14b The diagram shows Fig.13 , where the W×H prediction block is equal to the N×NCTU block size. Correspondingly, the extended reference samples 10641 and / or 10642 span the upper right CTU (CTU 2) and make at least one sample enter the next CTU (CTU 3). That is, a portion of the extended reference samples 10641 and / or 10642 for 45° vertical angle prediction may be arranged in the unprocessed CTU 3. If CTU3 and CTU5 are processed in parallel, these samples may not be available.

[0298] Fig.14a and Fig.14b An example of required closest reference samples 1062 and extended reference samples 1064 in the case of diagonal vertical intra-image prediction is shown.

[0299] exist Fig.14b , two CTU lines can be seen. When encoding the two lines in parallel using a wavefront-like approach, once CTU 2 has been encoded and encoding of CTU 3 begins, encoding of the second CTU line can start with CTU 5. In this case, 45 degree vertical angle prediction with extended reference samples cannot be used in CTU 5, since some reference samples 1104 are located inside CTU 3. The following method can solve this problem:

[0300] 1. Mark the extended reference inside the next basic processing region as unavailable, as described in conjunction with the embodiments of the present invention.

[0301] In this way, the samples may be treated like other unavailable reference samples, for example at picture, slice or tile boundaries, or when restricted intra prediction is used which does not allow using samples from inter-picture prediction areas as reference for intra-picture prediction areas.

[0302] 2. It is allowed to use extended reference samples only for regions and intra-image prediction modes, so the reference samples do not extend more than one basic processing unit to the upper right of the current region.

[0303] Both approaches can be made switchable by a high-level flag at the sequence, picture or slice level. In this way, the encoder can signal the decoder to apply restrictions in situations where the encoder's parallel processing scheme requires it. In situations where it is not required, the encoder can signal the decoder not to apply restrictions.

[0304] Another way is to combine the signaling for both restrictions with the signaling of the parallel coding scheme. For example, when wavefront parallel processing is enabled and signaled in the bitstream, the restriction on extended reference samples also applies.

[0305] In the following, some advantageous embodiments are described.

[0306] 1. In one embodiment, intra prediction uses two additional reference lines, where reference sample region index i=1 and i=3 (see Figure 3 ). For these extended reference sample lines, only angular predictions as described herein are allowed. Fig.11 As described, signaling of a reference sample area index i is performed, where MaxNumRefAreaIdx=3, and the index is signaled before the intra-prediction mode is signaled. When the reference area index is not equal to 0, that is, when an extended reference line is used, the DC and plane intra-prediction modes are not used. In order to avoid unnecessary signaling, DC and plane modes are excluded from the intra-prediction mode signaling. The intra-prediction mode can be encoded to the most probable mode (MPM) using an index to a list. The MPM list contains a fixed number of prediction mode candidates derived from neighboring blocks. When a neighboring block is encoded using the closest reference line (i=0), the neighboring block can use DC or plane mode, and therefore, the MPM list derivation process is modified to not include DC and plane mode. If removed, the DC and plane modes can be replaced by horizontal, vertical, and lower left diagonal angle modes to fill the list (see Figures 4a to 4e). This operation is performed in a way that avoids redundancy, for example, when the first candidate mode derived from the neighboring block is vertical and the second candidate mode is DC, the DC mode is not replaced by the vertical mode but by the horizontal mode, because the vertical mode is not yet in the list. Another way to prevent unnecessary signaling of DC and planar modes in the case of i>0 is to signal the reference area index i after the prediction mode and to adjust the signaling of i in the prediction mode. If the intra-prediction mode is equal to DC or planar, the index i is not signaled. However, the previously described method is superior to this method because it introduces a syntax dependency in the case where the intra-prediction mode is signaled to the MPM list using an index. In order to be able to perform syntax analysis on i, the intra-prediction mode must be reconstructed, which requires an MPM list export. On the other hand, the MPM list export refers to the prediction mode of the neighboring block, which also needs to be reconstructed before performing syntax analysis on i. This undesired syntax dependency is solved by signaling i before the prediction mode and modifying the MPM list export accordingly, as described above. A video encoder may be configured to determine a list of most likely prediction modes based on the use of the plurality of closest reference samples or the use of the plurality of extended reference samples for the prediction mode, wherein the video encoder is configured to replace the prediction mode restricted for the used reference samples by the mode allowed for the prediction mode. A corresponding video decoder may be configured to determine a list of most likely prediction modes based on the use of the plurality of closest reference samples or the use of the plurality of extended reference samples for the prediction mode, wherein the video decoder is configured to replace the prediction mode restricted for the used reference samples by the mode allowed for the prediction mode.

[0307] 2. In another embodiment, the intra prediction of the extended reference samples is additionally restricted to be applied only to luma samples. That is, the video encoder can be configured to apply the prediction using the extended reference samples to images that only include luma information. Correspondingly, the video decoder can be configured to apply the prediction using the extended reference samples to images that only include luma information.

[0308] 3. In another embodiment, the width and height of the closest reference sample (i=0) are exceeded (see Figure 3 The extended reference samples (i>0) of W+H in are not generated by using reconstructed samples (if available), but by padding from the last sample, such as Figure 3r(23,-1-i) of the upper right sample or r(-1-i,23) of the lower left sample in . This reduces the memory access of the extended reference sample line. That is, the video encoder can be configured to generate an extended reference sample that exceeds the width and / or height of the closest reference sample along the first image direction and the second image direction by padding from the nearest extended reference sample. Accordingly, the video decoder can be configured to generate an extended reference sample that exceeds the width and / or height of the closest reference sample along the first image direction and the second image direction by padding from the nearest extended reference sample.

[0309] 4. In another embodiment, only every second angle mode may be used for extended reference samples. Therefore, the MPM list export is modified to exclude these modes in addition to DC and planar modes in the case where i>0. That is, the video encoder can be configured to use an angle prediction mode to predict the prediction, the angle prediction mode only uses a subset of angles from the possible angles of the angle prediction mode, and excludes unused angles from the operation of signaling the encoding information to the decoder. Accordingly, the video decoder can be configured to use an angle prediction mode to predict the prediction, the angle prediction mode only uses a subset of angles from the possible angles of the angle prediction mode, and excludes unused angles from the prediction.

[0310] 5. In another embodiment, the number of additional reference sample lines is increased to 3 (see Fig.11 , where MaxNumRefAreaIdx=4). That is, in addition to the closest reference sample, these extended reference samples can also be arranged in at least 2 lines and rows, preferably at least 3 lines and rows. This configuration can be applied to video encoders and decoders. The video encoder can be configured to use a specific multiple extended reference samples from a set of multiple extended reference samples to predict the prediction block, wherein the video encoder is configured to select the specific multiple extended reference samples from the set of multiple extended reference samples so as to include the lowest similarity of image content when compared with the multiple closest reference samples extended by the set, that is, to use associated reference samples, for example, with the same reference area index. Therefore, a video decoder can be configured to predict the prediction block using a specific multiple extended reference samples from a set of multiple extended reference samples, wherein the video decoder is configured to select the specific multiple extended reference samples from the set of multiple extended reference samples so as to include the lowest similarity of image content when compared with the multiple closest reference samples extended by the set.

[0311] 6. In another embodiment, a flag indicating whether extended reference samples are used is signaled instead of index i. If the flag indicates the use of extended reference samples, the index i (i>0) is derived by calculating the similarity between reference line i>0 and reference line i=0 (e.g., using the sum of absolute differences). The index i that generates the lowest similarity is selected. The idea behind this is that the more the extended reference samples (i>0) are correlated with the regular reference samples (i=0), the more correlated the resulting predictions are, and therefore there is no additional benefit in using the extended reference samples in the prediction. That is, the video encoder can be configured to signal the use of extended reference samples using a flag or other (possibly binary) information. Accordingly, the video decoder can be configured to receive information indicating a use of extended reference samples through such a flag.

[0312] 7. In another embodiment, a secondary transform, such as a non-separable secondary transform (NSST), may be applied after the first transform of the intra-prediction residual. In the case of an extended reference sample line, the secondary transform is not performed, and all signal notifications related to NSST are disabled in the case of i>0. That is, the video encoder may be configured to selectively use the extended reference samples or only the closest reference samples, wherein the video encoder is configured to transform the residual obtained by predicting the prediction block using a first transform procedure to obtain a first transform result when the extended reference samples are not used to predict the prediction block, and to transform the first transform result using a second transform procedure to obtain a second transform result. This may also affect the signal notification of whether the second transform is used, that is, if the use of the second transform must be signaled, the signal notification may be skipped when the extended reference samples are used. The video encoder may be configured to signal the use of the secondary transform; or implicitly signal the non-use of the secondary transform when the use of the extended reference samples is indicated, and not include information related to the result of the secondary transform in the encoded data.

[0313] Accordingly, the video decoder may be configured to selectively use the extended reference sample or only the closest reference sample, wherein the video decoder is configured to, when the extended reference sample is not used to predict the prediction block, transform the residual obtained by predicting the prediction block using a first transform procedure to obtain a first transform result, and transform the first transform result using a second transform procedure to obtain a second transform result. The video decoder may be configured to receive information indicating the use of the secondary transform; or derive the non-use of the secondary transform when the use of the extended reference sample is indicated, and not receive information related to the result of the secondary transform in the encoded data.

[0314] 8. In another embodiment, prediction using extended reference samples (i>0) is combined with planar prediction using the closest reference sample (i=0). As outlined in conjunction with combining different extended prediction modes (i.e., using extended reference samples), the weighting may be fixed (e.g., 0.5 and 0.5), may be block size dependent, or may be signaled at the slice, picture, or sequence level. In the case of using extended reference samples (i>0), an additional flag indicates whether combined prediction is applied. That is, the video encoder may be configured to. Accordingly, the video decoder may be configured to.

[0315] 9. In another embodiment, the signaling of the combined prediction from above is omitted to reduce the signaling overhead. Alternatively, the decision whether to apply the combined prediction is derived based on an analysis of the closest reference sample (i=0). One possible analysis may be the flatness of the closest reference sample. In the case where the closest reference sample signal is flat (no edges), the combination is applied, and if the closest reference sample signal contains high frequencies and edges, the combination is not applied. That is, the video encoder may be configured. Accordingly, the video decoder may be configured.

[0316] Although some aspects have been described in the context of an apparatus, it is apparent that these aspects also represent the description of the corresponding method, where a block or device corresponds to a method step or a feature of a method step. Similarly, the aspects described in the context of a method step also represent the description of the features of the corresponding block or item or corresponding apparatus.

[0317] Depending on certain implementation requirements, embodiments of the present invention may be implemented in hardware or software. The implementation may be performed using a digital storage medium, such as a floppy disk, DVD, CD, ROM, PROM, EPROM, EEPROM or flash memory, on which electronically readable control signals are stored, which cooperate (or can cooperate) with a programmable computer system to cause the execution of the respective method.

[0318] Some embodiments according to the invention comprise a data carrier having electronically readable control signals, which are capable of cooperating with a programmable computer system, such that one of the methods described herein is performed.

[0319] Generally speaking, embodiments of the present invention can be implemented as a computer program product with a program code, when the computer program product runs on a computer, the program code is operative for performing one of the methods. The program code may, for example, be stored on a machine readable carrier.

[0320] Other embodiments comprise the computer program for performing one of the methods described herein, stored on a machine readable carrier.

[0321] In other words, an embodiment of the inventive method is, therefore, a computer program having a program code for performing one of the methods described herein, when the computer program runs on a computer.

[0322] A further embodiment of the inventive methods is, therefore, a data carrier (or a digital storage medium, or a computer-readable medium) comprising, recorded thereon, the computer program for performing one of the methods described herein.

[0323] Therefore, a further embodiment of the method of the present invention is a data stream or a sequence of signals representing a computer program for executing one of the methods described herein. The data stream or the sequence of signals may, for example, be configured to be transmitted via a data communication connection (for example, via the Internet).

[0324] A further embodiment comprises a processing means, for example a computer or a programmable logic device, configured to or adapted to perform one of the methods described herein.

[0325] A further embodiment comprises a computer having installed thereon the computer program for performing one of the methods described herein.

[0326] In some embodiments, a programmable logic device (e.g., a field programmable gate array) may be used to perform some or all of the functionality of the methods described herein. In some embodiments, the field programmable gate array may cooperate with a microprocessor to perform one of the methods described herein. Generally, these methods are preferably performed by any hardware device.

[0327] The above embodiments are merely illustrative of the principles of the present invention. It should be understood that modifications and variations to the arrangements and details described herein will be apparent to those skilled in the art. Therefore, it is intended that the invention be limited only by the scope of the following claims, and not by the specific details presented by the description of the embodiments herein.

Claims

1. A method for video decoding, the method comprising: determining whether a reference sample at a first position within a reference line is available, wherein the reference line is spatially separated from the prediction block; In response to determining that the reference sample at the first position is not available, sequentially determining whether each reference sample included in the reference line is available, starting from a reference sample located adjacent to the reference sample at the first position, until determining that a reference sample at a second position is available; setting the value of the reference sample at the first position to the value of the reference sample at the second position; generating at least one additional reference sample having a value of the reference sample positioned at an end of the reference line; as well as The prediction block is decoded using the value of the reference sample of the reference line and the value of the at least one additional reference sample.

2. The method according to claim 1, further comprising: After setting the value of the reference sample at the first position, the value of at least one reference sample located between the first position and the second position is set to the value of the reference sample at the first position.

3. The method according to claim 1, further comprising: The reference line is identified from a plurality of reference lines based on a syntax element in a bitstream of encoded video data.

4. The method according to claim 1, wherein: The reference line is an extended reference line; as well as The method further comprises: determining whether a neighboring reference line directly adjacent to the prediction block is available, and In response to determining that the adjacent reference line is available, it is determined whether the reference sample at the first position within the extended reference line is available.

5. The method according to claim 1, further comprising: filtering the reference samples of the reference line; as well as The prediction block is decoded using the filtered values ​​of the reference samples of the reference line and the at least one additional reference sample.

6. The method of claim 1, wherein decoding the prediction block further comprises: The prediction block is decoded using at least one angular prediction mode.

7. The method of claim 1, wherein decoding the prediction block further comprises: Based on the prediction mode used to predict at least one reference sample within the reference line, a list of most likely prediction modes is determined.

8. The method of claim 1, wherein the reference line is spatially separated from the prediction block by at least one sample position.

9. A decoder for decoding a video, the decoder being configured to: determining whether a reference sample at a first position within a reference line is available, wherein the reference line is spatially separated from the prediction block; In response to determining that the reference sample at the first position is not available, sequentially determining whether each reference sample included in the reference line is available, starting from a reference sample located adjacent to the reference sample at the first position, until determining that a reference sample at a second position is available; setting the value of the reference sample at the first position to the value of the reference sample at the second position; generating at least one additional reference sample having a value of the reference sample positioned at an end of the reference line; as well as The prediction block is decoded using the value of the reference sample of the reference line and the value of the at least one additional reference sample.

10. The decoder according to claim 9, further configured to: After setting the value of the reference sample at the first position, the value of at least one reference sample located between the first position and the second position is set to the value of the reference sample at the first position.

11. The decoder according to claim 9, further configured to: The reference line is identified from a plurality of reference lines based on a syntax element in a bitstream of encoded video data.

12. The decoder of claim 9, wherein: The reference line is an extended reference line; and The decoder is further configured to: determining whether a neighboring reference line directly adjacent to the prediction block is available, and In response to determining that the adjacent reference line is available, it is determined whether the reference sample at the first position within the extended reference line is available.

13. The decoder according to claim 9, configured to: filtering the reference samples of the reference line; and The prediction block is decoded using the filtered values ​​of the reference samples of the reference line and the at least one additional reference sample.

14. The decoder according to claim 9, wherein in order to decode the prediction block, the decoder is configured to: The prediction block is decoded using at least one angular prediction mode.

15. The decoder according to claim 9, wherein in order to decode the prediction block, the decoder is further configured to: Based on the prediction mode used to predict at least one reference sample within the reference line, a list of most likely prediction modes is determined.

16. The decoder of claim 9, wherein the reference line is spatially separated from the prediction block by at least one sample position.

17. A non-transitory computer readable medium comprising instructions that, when executed, cause at least one processor to: determining whether a reference sample at a first position within a reference line is available, wherein the reference line is spatially separated from the prediction block; In response to determining that the reference sample at the first position is not available, sequentially determining whether each reference sample included in the reference line is available, starting from a reference sample located adjacent to the reference sample at the first position, until determining that a reference sample at a second position is available; setting the value of the reference sample at the first position to the value of the reference sample at the second position; generating at least one additional reference sample having a value of the reference sample positioned at an end of the reference line; as well as The prediction block is decoded using the value of the reference sample of the reference line and the value of the at least one additional reference sample.

18. The non-transitory computer readable medium of claim 17, further comprising instructions that, when executed, cause the at least one processor to: After setting the value of the reference sample at the first position, the value of at least one reference sample located between the first position and the second position is set to the value of the reference sample at the first position.

19. The non-transitory computer readable medium of claim 17, further comprising instructions that, when executed, cause the at least one processor to: The reference line is identified from a plurality of reference lines based on a syntax element in a bitstream of encoded video data.

20. The non-transitory computer readable medium of claim 17, wherein: The reference line is an extended reference line; and The instructions, when executed, further cause the at least one processor to: determining whether a neighboring reference line directly adjacent to the prediction block is available, and In response to determining that the adjacent reference line is available, it is determined whether the reference sample at the first position within the extended reference line is available.

21. The non-transitory computer readable medium of claim 17, further comprising instructions that, when executed, cause the at least one processor to: filtering the reference samples of the reference line; and The prediction block is decoded using the filtered values ​​of the reference samples of the reference line and the at least one additional reference sample.

22. The non-transitory computer-readable medium of claim 17, wherein the instructions, when executed, cause the at least one processor to decode the prediction block, comprising instructions that, when executed, cause the at least one processor to: The prediction block is decoded using at least one angular prediction mode.

23. The non-transitory computer-readable medium of claim 17, wherein the instructions, when executed, cause the at least one processor to decode the prediction block, comprising instructions that, when executed, cause the at least one processor to: Based on the prediction mode used to predict at least one reference sample within the reference line, a list of most likely prediction modes is determined.

24. The non-transitory computer-readable medium of claim 17, wherein the reference line is spatially separated from the prediction block by at least one sample position.

Images